Skip to content

refactor(env): run stock MCR images, prepare the sandbox on the fly - #202

Merged
Bala (0xba1a) merged 1 commit into
mainfrom
bala/env-distro-refactor
Oct 28, 2026
Merged

Bala (0xba1a) merged 1 commit into
mainfrom
bala/env-distro-refactor

Conversation

@0xba1a

@0xba1a Bala (0xba1a) commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Summary

Stop building and publishing a custom shell-server image. Sandboxes now run an unmodified image from the Microsoft Container Registry, and Microbots prepares it when the environment starts.

  • The default image is mcr.microsoft.com/devcontainers/python:3.11, the base of the old custom image, so bots get the same toolset as before.
  • Callers keep choosing the sandbox with image. The sandbox's distribution is never configured; later PRs read it from the container's /etc/os-release when they need it.

This is the first PR in a stack, based on main.

How a stock image becomes a sandbox

LocalDockerEnvironment.start() now:

  1. Runs the image idle (sleep infinity under init, so docker stop is prompt).
  2. Copies local_docker/shell_server/ into /opt/microbots, owned by root so the bot cannot patch its own server.
  3. Runs shell_server/prepare.sh as root over the docker exec control plane. The script is idempotent and:
    • installs only what's missing (python3/venv, useradd/usermod/groupmod, mount, git) with apt-get, tdnf, dnf or yum;
    • removes accounts holding the agent's uid/gid, plus the devcontainer's passwordless-sudo vscode user (the old Dockerfile did the same);
    • creates the unprivileged agent account, /var/log/microbots, and git's safe.directory;
    • pip-installs fastapi, uvicorn and pydantic into a root-owned venv at /opt/microbots/venv, isolated from the agent's Python.
  4. Starts dockerShell.py as root from that venv. It still drops to agent before running any bot command.
  5. Waits until the server answers on the published port, replacing the old fixed sleep(2).

If any setup step fails, the container is stopped and the error includes the script output or the server log.

Changes

  • shell_server/: dockerShell.py and ShellCommunicator.py are moved unchanged from image_builder/ (including Wrap commands to avoid entire bash failing #195's command wrapping), and prepare.sh is new.
  • images.py: DEBIAN_IMAGE and DEFAULT_IMAGE constants.
  • Removed: image_builder/Dockerfile, the Docker Hub publish workflow (dockerBuildPush.yml), and the CI buildx/cache/build steps. CI now pulls the stock image. The remaining workflow actions keep Pin GitHub Actions to full-length commit SHAs #193's pinned SHAs.
  • Docs: mkdocs nav and API reference pages (shell_server, images), plus the logging example.

Trade-off

Each start now does network work: about 9 seconds on the default image for the pip install of the server's dependencies. In exchange, no image has to be built or published, and any MCR image can be used.

Testing

  • Unit tests: startup order, failure cleanup, the root-owned archive, the prepare/launch commands, readiness and timeout, and prepare.sh syntax. 680 passed. The 6 failures in test_microbot.py::TestMicrobotUnit and 4 errors in test_memory_tool_integration.py also occur on main and come from my local environment.
  • Docker integration tests (test/environment/local_docker) on the stock default image: 10 passed.
  • Manual check: I also ran mcr.microsoft.com/azurelinux/base/core:3.0 with READ_ONLY and READ_WRITE mounts. The shell runs as agent, it cannot use sudo or modify /opt/microbots, and files it writes on the host belong to the host user.
  • prepare.sh is included in the built wheel.

Upcoming PRs in this stack

  1. This PR: stock MCR images prepared on the fly.
  2. Distro-aware tool YAML commands, resolved from the sandbox's /etc/os-release.
  3. Azure Linux 3 sandbox (AZURELINUX_IMAGE = mcr.microsoft.com/azurelinux/base/core:3.0), with an image option on the bots.
  4. Optional root_access for the sandbox shell.
  5. rpm / mock / azldev tool definitions.
  6. CopilotBot session_options and skill tracking, so it can use a repository's agent skills (for example Azure Linux's azldev-*).

@0xba1a
Bala (0xba1a) force-pushed the bala/env-distro-refactor branch 2 times, most recently from 0c4c69f to fab2f7a Compare October 22, 2026 10:32
@0xba1a Bala (0xba1a) changed the title refactor(env): split shell server from per-distro sandbox images refactor(env): run stock MCR images, prepare the sandbox on the fly Oct 22, 2026
Base automatically changed from bala/native_tool_calling to main October 27, 2026 10:40
@0xba1a
Bala (0xba1a) force-pushed the bala/env-distro-refactor branch from fab2f7a to 3c593f9 Compare October 27, 2026 10:45
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.24561% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 76.48%. Comparing base (d9f40f5) to head (731c3e5).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...environment/local_docker/LocalDockerEnvironment.py 98.18% 1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (98.24%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #202      +/-   ##
==========================================
+ Coverage   76.12%   76.48%   +0.36%     
==========================================
  Files          45       46       +1     
  Lines        3057     3113      +56     
==========================================
+ Hits         2327     2381      +54     
- Misses        730      732       +2     
Flag Coverage Δ
ghcp 31.70% <85.96%> (+0.98%) ⬆️
integration 41.37% <85.96%> (+0.81%) ⬆️
ollama_local 33.27% <85.96%> (+0.96%) ⬆️
slow-browser 18.02% <26.31%> (+0.16%) ⬆️
slow-other 38.64% <85.96%> (+0.86%) ⬆️
unit 73.43% <98.24%> (+0.81%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/microbots/environment/local_docker/images.py 100.00% <100.00%> (ø)
...ent/local_docker/shell_server/ShellCommunicator.py 72.96% <ø> (ø)
...vironment/local_docker/shell_server/dockerShell.py 95.34% <ø> (ø)
...environment/local_docker/LocalDockerEnvironment.py 87.80% <98.18%> (+2.39%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Stop building and publishing a custom shell-server image. Sandboxes now
run an unmodified image from the Microsoft Container Registry
(default: mcr.microsoft.com/devcontainers/python:3.11, the old image's
base) and are prepared when the environment starts:

- shell_server/: the distro-agnostic server (dockerShell.py,
  ShellCommunicator.py, moved unchanged from image_builder/) plus
  prepare.sh, which installs missing prerequisites with apt-get, tdnf,
  dnf or yum, removes accounts holding the agent's uid/gid and the
  devcontainer 'vscode' sudo user, creates the unprivileged `agent`
  account, and pip-installs fastapi/uvicorn/pydantic into a root-owned
  venv at /opt/microbots/venv.
- LocalDockerEnvironment starts the image idle (`sleep infinity` under
  init), copies shell_server/ into /opt/microbots (root-owned) over the
  root control plane, runs prepare.sh, launches dockerShell.py as root
  (it still drops to `agent`), and waits until the server answers
  instead of sleeping. Setup failures stop the container.
- images.py: DEBIAN_IMAGE / DEFAULT_IMAGE constants.
- Remove the Dockerfile, the Docker Hub publish workflow and the CI
  image build; CI pulls the stock image instead.

Preparation takes ~9s on the default image (pip install). Any image with
sh, bash and python3 or a supported package manager works.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@0xba1a
Bala (0xba1a) force-pushed the bala/env-distro-refactor branch from 3c593f9 to 731c3e5 Compare October 28, 2026 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants