Skip to content

feat(bwrap): N2C token checks for service accounts, and a stricter session-directory guard - #35

Merged
Dan Allan (danielballan) merged 6 commits into
NSLS2:mainfrom
stuwilkins:feat/bwrap-n2c-system-creds
Nov 4, 2026
Merged

Dan Allan (danielballan) merged 6 commits into
NSLS2:mainfrom
stuwilkins:feat/bwrap-n2c-system-creds

Conversation

@stuwilkins

@stuwilkins Stuart B. Wilkins (stuwilkins) commented Nov 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Two N2C changes to the sandbox launcher, in one PR because both are about how N2C tokens reach the sandbox (this also contains what was #34):

  1. Let it mount N2C tokens that a root service keeps up to date, for service accounts that have no n2c-token session, with one set of checks for tokens owned by you or by root.
  2. Make the last-resort guard that keeps the n2c-token session directory (the refresh token) out of the sandbox catch symlinked and nested mount sources.

Description

Part 1: tokens for service accounts

--n2c-token only accepted the layout n2c-token makes (a credentials directory owned by you, mode 0700). A service account (for example the account that runs a beamline's coding agent) has no n2c-token session: a root service on the host (a timer that gets app-only tokens with a certificate) writes the same creds/NAME/token files and the account only reads them. Those root-owned directories failed the check, so the launch stopped.

The launcher now refuses tokens that someone else could read or replace, the way ssh refuses a private key. The same checks apply whether you or root own the credentials directory. Each stops the launch:

  • N2SN_CREDS_DIR is an absolute path with no symlink, ., .. or doubled / in it (a symlink could be changed between the check and the mount, which resolves the path again).
  • Every directory above it, up to /, is owned by you or by root and not writable by its group or others (otherwise a parent could be renamed and replaced with an unchecked token before the mount resolves the path). A directory others can write to, such as /tmp, is refused.
  • The creds directory is owned by you or by root, and each token directory and each token has the same owner: a tree is yours throughout (n2c-token) or root's throughout (a service account's tokens).
  • The creds directory, each token directory and each token are not writable by their group, and have no permission for others. 0750, 0700, 0640, 0600 and 0440 pass; 0755, 0775 and 0644 do not. Group read (and enter, for a directory) is allowed: that is how a service account reads tokens that root writes.
  • The directories are real directories, you can enter a token directory and read its token, and a token is a regular file, not a symlink.

Other changes:

  • An empty token is mounted, with a warning, for both owners. A token is filled in place, so a mounted file shows it as soon as it is written. A token with no file yet is skipped with a warning.
  • --n2c-token all skips token directories you cannot enter (another service's tokens in the same directory); naming one you cannot enter is an error.
  • The warning hints depend on who owns the creds directory: "Run: n2c-token login" for yours, "kept by a root service" for root's.

_n2c_owner_mode and _n2c_my_uid are small functions so the checks can be tested without root-owned files. docs/bw-wrappers.md has a "Checks on the credentials" section and a "Service accounts" section with what a service must do when it writes tokens (root:GROUP, directories 0750, files 0440, rewritten in place).

Behaviour change for existing n2c-token users: a creds directory that others can read (for example 0755) is refused as before, but a group-readable 0750 one now passes. n2c-token itself creates 0700 and 0600, so nothing changes in practice.

Part 2: session-directory guard (was #34, follow-up to #32)

verify_n2c_state_not_exposed runs after every mount is built and stops the launch if a bind mount would expose the N2C session directory (it holds the MSAL cache with the refresh token). It had two gaps:

  • It compared the mount source as written. A source that is a symlink to a directory holding the session directory was not caught, for example ~/.npm linked to a directory that contains N2C_STATE_DIR.
  • It only tested "the source contains the session directory". A mount whose source lies inside the session directory was not caught.

Now each source is compared as written and with symlinks resolved (readlink -f) against the session directory as written and resolved, in both directions: the source is the session directory, contains it or lies inside it. The error message is unchanged. docs/bw-wrappers.md describes the wider check with the symlinked ~/.npm example. This is commit 47f1865 from #34, unchanged.

The two parts do not interact: a service account has no session directory, and the guard does nothing while that directory does not exist.

Testing

  • pixi run -e dev lint-files lib/bwrap_sandbox_lib.sh docs/bw-wrappers.md: all hooks pass (shfmt, shellcheck, codespell, markdownlint, gitleaks, mypy).
  • A throwaway harness with 33 checks, all passing:
    • owned by you (real file ownership): 0700 and 0600 mount and set N2SN_CREDS_DIR; 0755 and a 0775 creds directory, and 0644 and 0660 tokens are refused; 0750 and 0640 pass; an empty token is mounted with the n2c-token hint; a symlinked creds directory is refused.
    • owned by root (ownership faked, because the sandbox cannot create root-owned files): 0750 and 0440 mount, and all finds all three; 0755 creds directory, a group-writable token directory, a world-readable token, a token or directory owned by a third user, an unreadable token and a symlinked token are refused; an empty token is mounted with the root-service hint; all skips a directory it cannot enter; a missing token file only warns.
    • same owner: a token owned by the service account in a root tree, a token directory owned by another user, a root token in your tree, and a creds directory owned by a third user are refused. Parents: a group-writable parent, a 1777 (/tmp-like) parent and a parent owned by a third user are refused; a 0755 root-owned and a 0700 own parent pass. These fail on the earlier version.
    • the path as written: a symlink in a parent, a .. and a relative path are refused, and the same tree by its real path mounts. These fail on the earlier version.
  • validate_extra_path accepts a token path from a root-owned tree.
  • Session-directory guard (part 2), 9 checks, all passing on this branch; on main the 4 gap cases fail, as expected: the state directory itself, a directory that contains it, a directory inside it, a symlink to a directory that contains it (~/.npm linked), a symlink to the state directory, a symlink to a directory inside it, the state given through a symlink with the real parent as source (all refused), and an unrelated directory and a sibling that only shares a prefix (allowed).
  • Not yet tested with real root-owned directories: N2SN_CREDS_DIR=/run/nsls2/agent/creds bwopencode2 --n2c-token all --dry-run on a beamline workstation, then a real launch with one skill call, and a token refresh seen from inside the sandbox.

Checklist

  • Edit the sections above with relevant information
  • Update in-repo documentation, if applicable
  • Run pixi run -e dev lint locally (or wait for CI) — see .github/workflows/lint.yml
  • Test the changes on a representative managed host (or runansible deployment)

When the credentials directory is owned by root, accept it as system-managed tokens kept up to date by a root service, for an account that has no n2c-token session. Every directory from / down must be owned by root and not writable by group or others, and each token must be a regular, root-owned file that others cannot write and the account can read. An empty token is mounted, because the service rewrites it in place, and --n2c-token all skips token directories the account cannot enter. A directory owned by the current user keeps the existing 0700 checks. Documented in docs/bw-wrappers.md.

Assisted-by: opencode:claude-sonnet-5-5
Copilot AI balanced review requested due to automatic review settings November 1, 2026 12:39

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A critical path-validation flaw lets an attacker replace an unchecked symlink alias after validation and redirect a mount to a token file that never passed ownership checks.

1 open issue
What changed in this PR

Adds support for root-managed N2C tokens so service accounts can use the sandbox launchers without an n2c-token session.

Changes:

  • Selects user or system credentials checks based on directory ownership.
  • Adds system-token handling for permissions, empty files, inaccessible directories, and warnings.
  • Documents service-account setup and in-place token refresh requirements.
File Description
lib/​bwrap_sandbox_lib.sh Adds system-managed token validation and mount handling.
docs/​bw-wrappers.md Documents the system layout, required permissions, and token update behavior.

Review effort: Balanced

Comment thread lib/bwrap_sandbox_lib.sh Resolved
The ancestors of the system-managed credentials directory were checked after resolving it, so a symlink or '..' in the path as given was never examined, and the mount resolves the path again later. Require an absolute path with no symlink, '.', '..' or doubled slash in it, then check every directory above it as before.

Assisted-by: opencode:claude-sonnet-5-5
Copilot AI balanced review requested due to automatic review settings November 1, 2026 12:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Reported tests mock root ownership, leaving service-account permissions, sandbox token readability, and in-place refresh visibility unverified in a real managed-host bwrap launch.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Replace the two credential layouts with one set of checks that applies whether you or root own the credentials directory. N2SN_CREDS_DIR must be an absolute path with no symlink in it. The credentials directory, each token directory and each token must be owned by you or by root, not writable by their group, and have no permission for others (so 0750, 0700, 0640, 0600 and 0440 pass, and 0755 and 0644 do not). A token directory must be enterable and a token readable. An empty token is mounted with a warning for both owners, and --n2c-token all skips token directories that cannot be entered. The parent directories are no longer walked.

Assisted-by: opencode:claude-sonnet-5-5
Copilot AI balanced review requested due to automatic review settings November 1, 2026 13:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The credentials-path validation permits ancestor replacement that can substitute an unchecked token and fails to enforce the promised owner-specific validation policies.

2 open issues

Review effort: Balanced

Comment thread lib/bwrap_sandbox_lib.sh Resolved
Comment thread lib/bwrap_sandbox_lib.sh Resolved
…ne owner

Walk the directories above the credentials directory up to /, and refuse the launch unless each is owned by you or by root and not writable by its group or others, so a parent cannot be renamed and replaced with an unchecked token before the mount resolves the path. Require each token directory and token to have the same owner as the credentials directory, so a tree is yours throughout or root's throughout.

Assisted-by: opencode:claude-sonnet-5-5
Copilot AI balanced review requested due to automatic review settings November 1, 2026 13:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed to weigh an unpublished documentation caveat that setting N2SN_CREDS_DIR does not grant service accounts access through the account checker or agent-install ACLs.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

The check for mounts that expose the N2C session directory compared
the source as written, so a symlinked source that leads to a directory
holding it was missed. Compare the source as written and resolved
against the session directory as written and resolved, in both
directions (contains or lies inside).

Assisted-by: opencode:claude-sonnet-5-5

(cherry picked from commit 47f1865)

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive credential path has only mocked ownership coverage, leaving sandbox group access and in-place token refresh unverified on a representative managed host.

No open issues 🎉

Review effort: Balanced

@stuwilkins Stuart B. Wilkins (stuwilkins) changed the title feat(bwrap): mount system-managed N2C tokens for service accounts feat(bwrap): N2C token checks for service accounts, and a stricter session-directory guard Nov 1, 2026
…lugin

A service account that sends requests to an AIFAPIM agents product needs an Entra ID token as well as its key. The token is mounted with --n2c-token aifapim like any other, and the aifapim-entra OpenCode 2 plugin reads the file named by AIFAPIM_TOKEN_FILE on every request. The sandbox environment is cleared, so the variable has to be passed through. bwopencode does not forward it: OpenCode 1 does not load the plugin.

Assisted-by: opencode:claude-sonnet-5-5
Copilot AI balanced review requested due to automatic review settings November 4, 2026 12:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Mocked ownership tests leave the security-sensitive credential flow’s real root-owned token access, sandbox launch, in-place refresh, and deployed account-eligibility policy unverified on a representative managed host.

No open issues 🎉

Review effort: Balanced

@danielballan
Dan Allan (danielballan) merged commit acd7f7e into NSLS2:main Nov 4, 2026
1 check passed
@stuwilkins
Stuart B. Wilkins (stuwilkins) deleted the feat/bwrap-n2c-system-creds branch November 4, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants