Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,9 @@ jobs:
- name: Pull the stock sandbox image for integration tests
if: matrix.test-type != 'unit'
# Sandboxes run this unmodified MCR image and are prepared on the fly.
run: docker pull mcr.microsoft.com/devcontainers/python:3.11
run: |
docker pull mcr.microsoft.com/devcontainers/python:3.11
docker pull mcr.microsoft.com/azurelinux/base/core:3.0

- name: Check disk space before ollama installation
if: matrix.test-type == 'ollama_local'
Expand Down
47 changes: 47 additions & 0 deletions docs/advanced/sandbox-images.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Sandbox Images

Every bot runs its commands inside a Docker sandbox created by `LocalDockerEnvironment`. The sandbox runs an **unmodified image from the Microsoft Container Registry**, and you choose it by passing `image`. Two images have constants in `microbots.environment.local_docker.images`:

| Constant | Image | Use it for |
|----------|-------|------------|
| `DEBIAN_IMAGE` (default) | `mcr.microsoft.com/devcontainers/python:3.11` | General work. Debian with Python and a full developer toolset. |
| `AZURELINUX_IMAGE` | `mcr.microsoft.com/azurelinux/base/core:3.0` | Azure Linux and RPM work. Its `tdnf` repos package `mock`, `rpm-build`, and `golang`. |

## How a stock image becomes a sandbox

There is no custom image to build. When an environment starts, Microbots:

1. Starts the image idle.
2. Copies its shell server (`local_docker/shell_server`) into `/opt/microbots`, owned by root.
3. Runs `shell_server/prepare.sh` as root. It installs anything missing (Python, `useradd`, `mount`, `git`) with `apt-get`, `tdnf`, `dnf`, or `yum`. It also creates the unprivileged `agent` account, removes the devcontainer's passwordless-sudo `vscode` user, and installs the server's dependencies (FastAPI, uvicorn, pydantic) into a root-owned venv.
4. Starts the shell server, which drops to the `agent` account before running any bot command.

Preparation needs network access for package and PyPI downloads. It takes about 10 seconds on `DEBIAN_IMAGE` and about 40 seconds on `AZURELINUX_IMAGE`, which also installs Python and the user tools.

## Choose an image

Pass `image` to any bot or to `LocalDockerEnvironment`:

```python
from microbots import WritingBot
from microbots.environment.local_docker.images import AZURELINUX_IMAGE

bot = WritingBot(
model="azure-openai/your-deployment",
folder_to_mount="/path/to/repo",
image=AZURELINUX_IMAGE,
)
```

```python
from microbots.environment.local_docker.LocalDockerEnvironment import LocalDockerEnvironment

env = LocalDockerEnvironment(port=8080, image="mcr.microsoft.com/devcontainers/base:debian")
```

- Any image works if it has `sh` and `bash`, plus either `python3` or one of the supported package managers. Images that already include the prerequisites start faster.
- A bot's `image` only applies to the environment the bot creates. If you pass your own `environment`, choose the image when you create it; passing both raises `ValueError`.

## Tools and distros

You never declare the sandbox's distribution. Tools read it from the running container's `/etc/os-release` and pick their [distro-specific commands](tools/tesseract_ocr_tool_use.md#distro-specific-commands) from it. A tool that only defines `debian` commands, such as `cscope` or `browser-use`, fails to install on the Azure Linux image with `UnsupportedDistroError`.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ nav:
- Advanced:
- Bots:
- CopilotBot: advanced/bots/copilot-bot.md
- "Sandbox Images": advanced/sandbox-images.md
- "Authentication Setup": advanced/authentication.md
- "Azure Managed Identity & Service Connection Setup": advanced/azure-managed-identity-setup.md
- "Azure Pipelines Log Analyzer": advanced/azure-pipelines-log-analyzer.md
Expand Down
22 changes: 22 additions & 0 deletions src/microbots/MicroBot.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
from microbots.llm.llm import ToolCall, llm_output_format_str
from microbots.tools.native_tools import OPENAI_NATIVE_TOOLS, RUN_COMMAND_TOOL, run_file_tool
from microbots.tools.tool import ToolAbstract, get_tool_from_call
from microbots.environment.local_docker.images import DEFAULT_IMAGE
from microbots.extras.mount import Mount, MountType
from microbots.utils.logger import LogLevelEmoji
from microbots.utils.network import get_free_port
Expand Down Expand Up @@ -154,6 +155,9 @@ class MicroBot:
A callable that returns a bearer token for Azure AD authentication.
If not provided, it may be auto-created from environment variables.
Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (``DEFAULT_IMAGE``).
"""

def __init__(
Expand All @@ -165,6 +169,7 @@ def __init__(
additional_tools: Optional[list[ToolAbstract]] = None,
folder_to_mount: Optional[Mount] = None,
token_provider: Optional[any] = None,
image: Optional[str] = None,
):
"""
Init function for MicroBot class.
Expand Down Expand Up @@ -196,8 +201,24 @@ def __init__(
A callable that returns a bearer token for Azure AD authentication.
If not provided, it may be auto-created from environment variables.
Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (``DEFAULT_IMAGE``).
Cannot be combined with ``environment``.

Raises
------
ValueError
If both ``image`` and ``environment`` are provided.
"""

if image is not None and environment is not None:
raise ValueError(
"Pass either `image` or `environment`, not both: the image of a "
"provided environment is chosen when that environment is created."
)
self.image = image or DEFAULT_IMAGE

self.folder_to_mount = folder_to_mount

# TODO : Need to check on the purpose of variable `mounted`
Expand Down Expand Up @@ -488,6 +509,7 @@ def _create_environment(self, folder_to_mount: Optional[Mount]):
self.environment = LocalDockerEnvironment(
port=free_port,
folder_to_mount=folder_to_mount,
image=self.image,
)

def _create_llm(self):
Expand Down
8 changes: 8 additions & 0 deletions src/microbots/bot/AgentBoss.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ class AgentBoss(MicroBot):
is created automatically when *None*.
additional_tools : Optional[list[ToolAbstract]]
Extra tools to install alongside the built-in ``microbot_sub`` tool.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (Debian image).

Workflow
--------
Expand All @@ -45,6 +48,7 @@ def __init__(
folder_to_mount: str,
environment: Optional[any] = None,
additional_tools: Optional[list[ToolAbstract]] = None,
image: Optional[str] = None,
):
"""
Create the bot with the ``microbot_sub`` tool and the repository mounted read-write.
Expand All @@ -60,6 +64,9 @@ def __init__(
is created automatically when *None*.
additional_tools : Optional[list[ToolAbstract]]
Extra tools to install alongside the built-in ``microbot_sub`` tool.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (Debian image).
"""
if additional_tools is None:
additional_tools = []
Expand Down Expand Up @@ -115,6 +122,7 @@ def __init__(
environment=environment,
additional_tools=additional_tools,
folder_to_mount=folder_mount_info,
image=image,
)

def run(
Expand Down
21 changes: 21 additions & 0 deletions src/microbots/bot/BrowsingBot.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,33 @@


class BrowsingBot(MicroBot):
"""
A bot that searches the web with the ``browser`` tool.

Parameters
----------
model : str
The model to use, in the format ``<provider>/<model_name>``.
environment : Optional[Environment]
The execution environment for the bot. If not provided, a default
environment is created. Defaults to None.
additional_tools : Optional[list[ToolAbstract]]
Extra tools to install alongside the browser tool. Defaults to None.
token_provider : Optional[any]
A callable that returns a bearer token for Azure AD authentication.
Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment. The browser tool
only supports Debian-based images (the default). Defaults to None.
"""

def __init__(
self,
model: str,
environment: Optional[Environment] = None,
additional_tools: Optional[list[ToolAbstract]] = None,
token_provider: Optional[any] = None,
image: Optional[str] = None,
):
# validate init values before assigning
bot_type = BotType.BROWSING_BOT
Expand All @@ -32,6 +52,7 @@ def __init__(
environment=environment,
additional_tools=(additional_tools or []) + [BROWSER_USE_TOOL],
token_provider=token_provider,
image=image,
)

def run(self, task, timeout_in_seconds=200) -> BotRunResult:
Expand Down
8 changes: 8 additions & 0 deletions src/microbots/bot/LogAnalysisBot.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ class LogAnalysisBot(MicroBot):
token_provider : Optional[any]
An optional callable that supplies authentication tokens for the
model provider. Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (Debian image).

Attributes
----------
Expand Down Expand Up @@ -76,6 +79,7 @@ def __init__(
environment: Optional[any] = None,
additional_tools: Optional[list[ToolAbstract]] = None,
token_provider: Optional[any] = None,
image: Optional[str] = None,
):
"""
Initialize a LogAnalysisBot instance.
Expand All @@ -97,6 +101,9 @@ def __init__(
token_provider : Optional[any]
An optional callable that supplies authentication tokens for the
model provider. Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (Debian image).
"""
# validate init values before assigning
bot_type = BotType.LOG_ANALYSIS_BOT
Expand Down Expand Up @@ -124,6 +131,7 @@ def __init__(
additional_tools=additional_tools or [],
folder_to_mount=folder_mount_info,
token_provider=token_provider,
image=image,
)

def run(
Expand Down
8 changes: 8 additions & 0 deletions src/microbots/bot/ReadingBot.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ class ReadingBot(MicroBot):
token_provider : Optional[any]
A callable that returns a bearer token for Azure AD authentication.
Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (Debian image).
"""

def __init__(
Expand All @@ -37,6 +40,7 @@ def __init__(
environment: Optional[any] = None,
additional_tools: Optional[list[ToolAbstract]] = None,
token_provider: Optional[any] = None,
image: Optional[str] = None,
):
"""
Create the bot with the repository mounted read-only.
Expand All @@ -55,6 +59,9 @@ def __init__(
token_provider : Optional[any]
A callable that returns a bearer token for Azure AD authentication.
Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (Debian image).
"""
# validate init values before assigning
bot_type = BotType.READING_BOT
Expand Down Expand Up @@ -82,4 +89,5 @@ def __init__(
additional_tools=additional_tools or [],
folder_to_mount=folder_mount_info,
token_provider=token_provider,
image=image,
)
8 changes: 8 additions & 0 deletions src/microbots/bot/WritingBot.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ class WritingBot(MicroBot):
token_provider : Optional[any]
A callable that returns a bearer token for Azure AD authentication.
Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (Debian image).
"""

def __init__(
Expand All @@ -37,6 +40,7 @@ def __init__(
environment: Optional[any] = None,
additional_tools: Optional[list[ToolAbstract]] = None,
token_provider: Optional[any] = None,
image: Optional[str] = None,
):
"""
Create the bot with the repository mounted read-write.
Expand All @@ -55,6 +59,9 @@ def __init__(
token_provider : Optional[any]
A callable that returns a bearer token for Azure AD authentication.
Defaults to None.
image : Optional[str]
Shell-server image of the auto-created environment, for example
``images.AZURELINUX_IMAGE``. Defaults to None (Debian image).
"""
# validate init values before assigning
bot_type = BotType.WRITING_BOT
Expand Down Expand Up @@ -87,4 +94,5 @@ def __init__(
additional_tools=additional_tools or [],
folder_to_mount=folder_mount_info,
token_provider=token_provider,
image=image,
)
4 changes: 4 additions & 0 deletions src/microbots/environment/local_docker/images.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,8 @@
# Debian with Python and a full developer toolset (git, curl, build tools).
DEBIAN_IMAGE = "mcr.microsoft.com/devcontainers/python:3.11"

# Azure Linux 3. Minimal: Python and the user tools are installed on the fly
# (with tdnf), and its repos package RPM tooling such as mock and rpm-build.
AZURELINUX_IMAGE = "mcr.microsoft.com/azurelinux/base/core:3.0"

DEFAULT_IMAGE = DEBIAN_IMAGE
46 changes: 46 additions & 0 deletions test/bot/test_microbot.py
Original file line number Diff line number Diff line change
Expand Up @@ -1074,3 +1074,49 @@ def test_llm_finds_symbols_using_cscope(self, cscope_tool, c_code_repo, openai_m
f"LLM should have found that main() calls add_numbers. Got: {response.result}"
finally:
del bot


@pytest.mark.unit
class TestMicroBotImage:
"""image selects the shell-server image of the auto-created environment."""

@pytest.mark.parametrize("image, expected", [
(None, "mcr.microsoft.com/devcontainers/python:3.11"),
("mcr.microsoft.com/azurelinux/base/core:3.0", "mcr.microsoft.com/azurelinux/base/core:3.0"),
])
def test_forwarded_to_auto_created_environment(self, image, expected):
with patch("microbots.MicroBot.LocalDockerEnvironment") as lde, \
patch("microbots.MicroBot.get_free_port", return_value=4242), \
patch.object(MicroBot, "_create_llm"):
bot = MicroBot(model="azure-openai/test-deploy", image=image)

assert bot.image == expected
lde.assert_called_once_with(port=4242, folder_to_mount=None, image=expected)

def test_rejects_image_with_provided_environment(self):
with patch.object(MicroBot, "_create_llm"), \
pytest.raises(ValueError, match="Pass either `image` or `environment`"):
MicroBot(model="azure-openai/test-deploy", environment=MagicMock(), image="custom/shell:dev")

def test_provided_environment_is_used_as_is(self):
env = MagicMock()
with patch.object(MicroBot, "_create_environment") as create_env, \
patch.object(MicroBot, "_create_llm"):
bot = MicroBot(model="azure-openai/test-deploy", environment=env)
assert bot.environment is env
create_env.assert_not_called()

@pytest.mark.parametrize("bot_name, kwargs", [
("ReadingBot", {"folder_to_mount": "/tmp/repo"}),
("WritingBot", {"folder_to_mount": "/tmp/repo"}),
("LogAnalysisBot", {"folder_to_mount": "/tmp/repo"}),
("BrowsingBot", {}),
("AgentBoss", {"folder_to_mount": "/tmp/repo"}),
])
@pytest.mark.parametrize("image", [None, "custom/shell:dev"])
def test_subclasses_forward_image(self, bot_name, kwargs, image):
import microbots.bot as bots

with patch.object(MicroBot, "__init__", return_value=None) as base_init:
getattr(bots, bot_name)(model="azure-openai/test-deploy", image=image, **kwargs)
assert base_init.call_args.kwargs["image"] == image
Loading
Loading