Skip to content

feat(kubernetes): configurable resources for the supervisor and sandbox init containers #3930

Description

@rain-sicoreai

User Story

As a platform operator running OpenShell on a Kubernetes cluster that requires CPU and memory limits on every container (an admission policy such as "require resource limits", common in multi-tenant clusters),
I want to set resources for the containers OpenShell creates itself,
so that sandboxes can start on such clusters without exempting OpenShell's pods from the policy.

Problem Statement

With the Kubernetes driver, OpenShell creates containers that carry no resources and offers no setting for them:

  • the supervisor container of the supervisor pod (supervisor_pod in crates/openshell-driver-kubernetes/src/sandbox_runtime.rs),
  • the workload pod's init containers openshell-sandbox-bootstrap and workspace-init.

SandboxTemplate.resources reaches only the workload (agent) container. The chart's supervisor values have no resources block, and the driver configuration has no equivalent. This is unchanged on main as of 2026-09-30.

Impact / Why This Matters

  • On a cluster whose admission rejects containers without limits, sandbox creation fails: pods "os-supervisor-..." is forbidden: ... memory and cpu limits are required (container supervisor), and the workload pod is refused for openshell-sandbox-bootstrap. The sandbox never starts.
  • Workarounds are unattractive: exempting OpenShell's pods from the policy (the supervisor then has unbounded CPU and memory in tenant namespaces), a namespace LimitRange (which silently defaults every tenant pod in that namespace), or a mutating webhook written just to add limits. We used a mutating webhook matched on openshell.ai/boundary-role.
  • Unbounded supervisor containers also make it harder to size nodes and namespaces for many sandboxes.

Proposed Design

Operators set requests and limits for OpenShell-created containers in the gateway configuration (and chart values), with defaults suitable for most workloads:

  • supervisor container (supervisor pod),
  • workload-pod init containers (openshell-sandbox-bootstrap, workspace-init).

For example [openshell.drivers.kubernetes.supervisor_resources] / init_container_resources, or supervisor.resources in the chart. The values apply to every sandbox the gateway creates. Callers keep setting the workload container's resources through SandboxTemplate.resources as today.

Acceptance Criteria

  • An operator can set CPU/memory requests and limits for the supervisor container and the workload pod's init containers, through the gateway configuration and the Helm chart.
  • With those set, a sandbox starts on a cluster whose admission requires limits on every container, without exemptions, LimitRange or mutation.
  • Documentation lists the containers OpenShell creates and how their resources are configured.

Alternatives Considered

  • Namespace LimitRange defaults: these also apply to every other pod in the tenant namespace, changing a "reject pods without limits" policy into "silently default them".
  • Admission exemption for OpenShell pods: the supervisor then runs without bounds in tenant namespaces.
  • A site-specific mutating webhook (what we run today): it works, but it duplicates OpenShell's pod knowledge (labels, container names) outside OpenShell, and breaks if those change.
  • Deriving the supervisor's resources from SandboxTemplate.resources: couples control-plane overhead to the workload's size. A separate operator setting seems clearer.

Agent Investigation

In v0.1.0 and on main, the supervisor Container in sandbox_runtime.rs is built without resources. container_resources() in driver.rs applies SandboxTemplate.resources (and containers.agent.resources from driver config) only to the agent container. The bootstrap init container is added in driver.rs without resources.

Checklist

  • I've reviewed existing issues and the published docs
  • This is a design proposal, not a "please build this" request

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions