Technical Report · Website · Documentation · Discord
What · Why · v1.1.1 · Install · Quickstart · Examples · Documentation
v1.1.1 · Skill development, execution reliability and the Opper gateway
PhyAgentOS is a Recursive Self-Improvement (RSI) framework for embodied agents. It connects cognitive planning, Physical Execution, and task verification in a runtime-wide feedback loop, improving Skills and Lessons from verified experience for future tasks.
| Capability | What it enables |
|---|---|
| One framework, multiple control modes | Combine general-model reasoning, action models, or hybrid control through the capabilities exposed by an installed Skill. |
| Verified task outcomes | Evaluate goals against observations and execution facts, with bounded replanning when recovery is needed. |
| Experience that improves future tasks | Accumulate reusable workflows and scoped Lessons from verified experience, with guarded promotion and revision history. |
| Reusable physical capabilities | Package environment-specific tools and runtimes as versioned Skills, keeping cognitive planning separate from robot integration. |
Skill development, execution reliability and the Opper gateway. Changes since v1.1.0:
- Opper gateway: OpenAI-compatible routing with provider selection, environment credentials and unchanged model identifiers.
- Local Skill development: locally edited workflow documents and Node executables can run with warnings, while bindings and experience retain the activated workflow hash. Explicit Node verification and install-cache acceptance remain strict.
- Action rejection handling: structured pre-dispatch Gateway rejections are recorded as failed and do not block a later retry or successful task finalization. Timeouts and uncertain remote execution retain their existing guards.
- Piper X development example: bilingual Track B guides and an independent simulation Skill example cover model conversion, original Node locks, packaging and recorded validation. See the Piper X guide for its original runtime baseline.
- Clearer documentation: consolidated bilingual manuals and contribution guidance, a MuJoCo Quickstart with live GIFs, and retained benchmark documentation links.
See the full changelog and changes since v1.1.0.
Start with a model-backed CLI conversation, then follow the MuJoCo Quickstart below. Prerequisites: Python 3.11 or 3.12, Git, and an API key for your model provider.
git clone --branch v1.1.1 https://github.com/PhyAgentOS/PhyAgentOS-core.git
cd PhyAgentOS-core
python -m venv .venv
source .venv/bin/activate
python -m pip install -e .
paos onboardOn Windows PowerShell, replace the activation command with .venv\Scripts\Activate.ps1.
Configure credentials with hidden input, then select the default:
paos provider configure openrouter
paos provider use openrouter --model anthropic/claude-sonnet-4See the provider CLI guide for
Docker/secret input, process overrides and session commands (/provider, /model, /effort, /status).
Session switches affect subsequent turns only and leave running tasks and other sessions unchanged.
Full configuration reference (v1.1.1)
The configuration file is serialized in camelCase; snake_case keys are also accepted.
{
"agents": {
"defaults": {
"workspace": "~/.PhyAgentOS/workspace",
"model": "openrouter/openai/gpt-4o-mini",
"provider": "openrouter"
},
"verification": {
"serviceEnabled": true,
"evidenceRetention": "failed",
"maxReplansPerEpisode": 2,
"maxVerifierCallsPerRun": 50
},
"evolution": {
"enabled": true,
"scope": "verified_forge_lineage",
"promotionMode": "guarded_auto",
"minSuccessfulEpisodes": 3,
"minLessonEpisodes": 3,
"maxLessonsPerSkill": 8,
"maxEvolutionCallsPerRun": 20
}
},
"providers": {
"openrouter": {
"apiKey": "YOUR_API_KEY"
}
},
"forge": {
"requestTimeoutS": 10,
"pollIntervalS": 0.5,
"executionTimeoutS": 300,
"evidence": {
"requiredImageSources": ["front"],
"captureTimeoutS": 5,
"postCaptureTimeoutS": 5,
"connectionTimeoutS": 2,
"maxArtifactBytes": 8388608,
"associationQuality": "best_effort"
}
},
"resourceRegistry": {
"url": "https://paos-resource-manager.dev.x-era.com"
}
}The front source is only an example. resourceRegistry.url selects a generic package registry;
it may be empty when all artifacts are installed from local bundles or a supplied static index.
PAOS connects only to the Gateway URL in the manifest of the explicitly started, healthy Skill
Runtime. It never starts or downloads a concrete Skill merely because the Agent starts.
paos status
paos agent -m "Hello! Introduce yourself."
paos agentYou should receive a model response; paos agent opens an interactive conversation. This checks the core Agent and provider connection. Physical execution additionally needs a running Skill Runtime.
Need another provider or deployment option? See the user manual · Docker.
Tip
Developers: bring your own robot — Piper X customization demo. Follow the complete model-to-Skill example for joint and end-effector control. Use the Skill development manual to choose between configuration changes, a new profile and Node development.
Use robot-control-sim to control a simulated Piper arm through joint targets or
end-effector poses. Complete the installation and model configuration above first.
The prebuilt simulation Nodes used here target Linux x86_64 and require a working
graphical display or virtual display for MuJoCo and the image viewer.
v1.1.1 retains the Dora CLI 0.4.1 (dora-message 0.7.0) compatibility baseline:
curl --proto '=https' --tlsv1.2 -LsSf \
https://github.com/dora-rs/dora/releases/download/v0.4.1/dora-cli-installer.sh | sh
dora --versionDora is required to start the simulation Runtime. See the Dora installation reference for other installation methods.
Set resourceRegistry.url in your config, or PAOS_RESOURCE_REGISTRY_URL, to a deployment
that provides robot-control-sim and its locked Node artifacts. Then run:
paos skill search robot-control-sim
paos skill install robot-control-sim
paos skill inspect robot-control-siminspect shows the installed version, profiles and dependencies. This example uses
mujoco-piper; mujoco-franka and mujoco-so101 select other robots. For a separately
obtained Bundle, use paos skill install /path/to/robot-control-sim.tar.gz --local;
its locked Nodes must also be available. Core does not bundle these simulation assets.
paos skill start robot-control-sim --profile mujoco-piper
paos skill status robot-control-simWait for the Runtime to report healthy and the simulation camera viewer to open. If startup
fails, inspect paos skill logs robot-control-sim before sending a motion request.
The Agent reads sim.get_robot_info and live Tool context to determine joints, limits, frames
and units, then follows the active Skill workflow. The absolute targets below are for
mujoco-piper (arm_base / tcp); use the selected robot's frames and reachable targets when switching profiles.
| Control mode | Tool | Target |
|---|---|---|
| Joint targets | sim.set_joint_positions |
Absolute arm joint positions in radians |
| End-effector pose | sim.move_eef |
Absolute position in meters and quaternion orientation; the motion server solves IK |
These motion demos use verification.mode=off, so success is derived from Tool execution results. For semantic task verification and evidence policies, see the Core developer manual.
Try a joint-space request:
paos agent -m "Use robot-control-sim (mujoco-piper). Inspect robot info and Tool context. For this \
motion demo use verification.mode=off. Move joint1-joint6 to [0.35, 0.55, -0.9, 0.15, \
0.25, -0.1] rad via sim.set_joint_positions, with velocity_scale=0.2 and \
acceleration_scale=0.2. Read the action result, finalize the task and report briefly. Do \
not edit files or use shell tools; report any failure."
Then try a Cartesian request:
paos agent -m "Use robot-control-sim (mujoco-piper). Inspect Tool context. For this motion demo use \
verification.mode=off. Call sim.move_eef for TCP in arm_base: position (0.284045, \
0.007878, 0.478706) m, quaternion xyzw (-0.002904, 0.693239, 0.008319, 0.720654), \
position_tolerance_m=0.01, orientation_tolerance_rad=0.05, velocity_scale=0.2, \
acceleration_scale=0.2. Let the motion server solve IK. Read the action result, finalize \
the task and report briefly. Do not edit files or use shell tools; report any failure."
Live recordings of the command and MuJoCo simulation; the captured command panel is held visible and model-wait pauses are shortened.
You can also run paos agent for interactive control. Motion should be visible in the simulation;
read the returned execution result and final state rather than treating admission as completion.
After the motion task has finished:
paos skill stop robot-control-sim- TrackB framework guide: architecture, repository catalog and Node protocols.
- Skill development manual: configure a profile, add a profile, or develop Nodes and assemble a Skill.
- Piper X development demo: adapt
robot-control-simto another arm, including model conversion and both control modes. - Bundle and Tool integration reference: manifest fields, packaging and local installation.
| Platform | Coverage | Guide |
|---|---|---|
| Piper robot arm | Skill installation, CAN setup, hardware startup, and tool readiness | Piper quickstart (Chinese) |
| LIBERO-10 simulation benchmark | Skill/Node installation, π0.5 weights, model API setup, and a 50-episode run (π0.5-only or GPT-6 supervised) | LIBERO quickstart (Chinese) |
| I want to… | Start here |
|---|---|
| Install, configure, and run PhyAgentOS | User manual |
| Build a Skill or connect an environment | TrackB development guide |
| Understand the Physical Execution API | Physical Execution Tool API |
| Extend robot-control-sim to another arm | Piper X development demo |
| Develop and test Core | Core developer manual |
| Inspect benchmark results and evaluation context | Benchmark documentation |
| Browse all documentation | Documentation index |
2026-10-23 · v1.1.1 released: Opper gateway support, local Skill development and Action rejection fixes, plus bilingual Track B and Piper X guides. Release notes and packages.
2026-10-10 · GOSIM Shenzhen 2026: PhyAgentOS has been selected for Spotlight and will present in the Open Source Robotics track. Read the announcement and event details.
| Version | Date | Update |
|---|---|---|
| v1.1.1 | 2026-10-23 | Opper gateway, local Skill development and Action rejection fixes, bilingual Track B and Piper X guides, and a MuJoCo Quickstart. |
| v1.1.0 | 2026-10-09 | Centralized provider management, direct OpenAI Responses and new gateways, Skill Runtime packaging and startup improvements, and an opt-in governed interaction foundation. |
| v1.0.0 | 2026-08-30 | Initial stable release of PhyAgentOS. |
| v0.2.3 | 2026-08-27 | Physical Execution Skills can be installed and managed independently, activated into immutable AgentTask bindings, and used through governed Query, Action, and Session Tool API lifecycles with recovery and version-scoped experience. |
| v0.2.2 | 2026-08-21 | Unified Physical Execution execution on the Query/Action Tool API and added AgentTask aggregation, a verifiable Skill Runtime, Resource Registry integration, and the move-arm-by-ee Skill while retaining Agent verification and evolution. |
See the full changelog.
Start with the contribution guide and the development manuals. We recommend focusing on TrackB Nodes, Skills and robot/environment integrations. External resource-platform submissions and changes to existing registered repositories are not yet open; they require completion and opening of the approval service. Local development and validation can proceed; see the guide for the current scope and source-review process.
Development checks
python -m pip install -e ".[dev]"
pytest
ruff check PhyAgentOS tests
python -m compileall -q PhyAgentOS testsDiscord · X · Bilibili · LinkedIn · Xiaohongshu
If PhyAgentOS helps your research, please cite our technical report.
@article{liu2026phyagentos,
title={PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution},
author={Liu, Yang and Chen, Weixing and Song, Xinshuai and Pu, Tao and Mo, Siwen and Bai, Yongjie and Chen, Zihao and Sun, Qianran and Zhong, Liruo and Shen, Ying and others},
journal={arXiv preprint arXiv:2607.16636},
year={2026}
}Thanks to the maintainers of MuJoCo, ROS, the open benchmark suites, and every contributor to PhyAgentOS.
Jointly developed by Sun Yat-sen University HCP Lab, Peng Cheng Laboratory, and X-Era Lab.
MIT License · Copyright © 2025–2026 PhyAgentOS