Skip to content

fix(codex): copy skills into .agents/skills instead of symlinking - #225

Merged
bai-uipath merged 3 commits into
mainfrom
bai/codex-copy-skills
Oct 15, 2026
Merged

bai-uipath merged 3 commits into
mainfrom
bai/codex-copy-skills

Conversation

@bai-uipath

@bai-uipath bai-uipath commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

The codex harness symlinks each skill into .agents/skills/, and those symlinks can leave a codex agent without its skill: codex lists the skills under the plugin's namespace, models sometimes open a path that doesn't exist, and the search they fall back on can't see symlinks.

Changes

  • Copy skills instead of symlinking them. The symlinks cause two problems that compound:
    • Codex follows a symlinked skill to its source, finds the skills repo's .claude-plugin/plugin.json there ("name": "uipath"), and lists the skill as uipath:<skill>. Models sometimes turn that name into the folder .agents/skills/uipath/<skill>/.
    • The search that follows (rg --files .agents/skills, find .agents/skills -name SKILL.md) skips symlinks and returns nothing, so the agent continues without the skill.
  • Copy before each turn, remove after it. The copies are also removed in stop() and kill_sync(). Removing them only in stop() is not enough, because grading runs before agent.stop(): 31 skill dirs (about 18 MB, 1,800 files) would sit in front of criteria and end up in run artifacts. OpenCode avoids writing skills into the sandbox for the same reason.
  • Workspace content is left alone. A skill dir the task workspace already ships is not overwritten or removed, and files the agent writes under .agents/ stay.
  • _setup_skills is split into three methods. _collect_skills resolves the skill dirs once at start(), and _install_skills / _remove_skills run around each turn.
  • Docs updated. docs/agents/CODEX.md, docs/agents/ANTIGRAVITY.md, the skills section of .claude/notes/agents.md, and the wording in the task guide and test docstrings.
  • Sample task tasks/agents/codex_skills_test.yaml rewritten. It asserted .agents/skills exists at grading time, which per-turn removal makes false. It now has the agent write the output of find .agents/skills -name SKILL.md | wc -l to a file, and checks that .agents is gone before grading.

Validation

Codex 0.156.1 (the version on main) debug prompt-input, run on the Docker sandbox layout with the real skills repo:

Check Symlinks (main) Copies (this PR)
Skill name codex shows the model uipath:<skill> <skill>
Skill path codex shows r1/<skill>/SKILL.md r1/<skill>/SKILL.md
find .agents/skills -name SKILL.md 0 files 31 files
Workspace after the turn .agents/skills/ with 31 symlinks empty

Live runs of tasks/agents/codex_skills_test.yaml (codex gpt-5.5, PLUGIN_PATH = the skills repo, 31 skills):

Code Driver find count the agent wrote .agents in artifacts Result
main (3ffb2ef) tempdir 0 yes 0/1
this PR tempdir 31 no 1/1
this PR docker (agent image built from this branch, skills copied from /work/plugins/0) 31 no 1/1
Tests Result
New TestSkillCopies (5 tests) pass; the os.walk test fails when _install_skills symlinks
test_codex_agent.py + test_codex_agent_unit.py (codex extra installed) 176 passed
make verify (format, ruff, pyright, CE lint rules, docs budget, tests) 6,330 passed, 93% coverage; the 6 local failures (test_stats_nonfinite, one test_agent_context_window, two test_docker_runner_mounts) fail the same way on a clean main checkout
CI Live Integration Tests (Codex) pass

🤖 Generated with Claude Code

bai-uipath and others added 2 commits October 15, 2026 11:49
Codex follows a symlinked skill to its source, finds the skills repo's
plugin manifest and lists the skill as `uipath:<skill>`; models sometimes
turn that into a nonexistent `.agents/skills/uipath/<skill>/` path. The
fallback search (`rg --files`, `find`) skips symlinks and finds nothing.
In the 2026-10-15 codex nightly, 41 of 471 Linux tasks opened the wrong
path and 15 never read their skill.

Skills are now copied in before each turn and removed after it (and on
stop/kill), so criteria and run artifacts never see them. Skill dirs the
workspace already ships are left alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…numbers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sample task asserted .agents/skills still exists at grading time,
which per-turn removal makes false. It now has the agent count SKILL.md
files with find (zero through symlinks) and checks .agents is gone
before grading. Also drops the last "symlinks" mention in the
Antigravity doc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bai-uipath
bai-uipath marked this pull request as ready for review October 15, 2026 19:03
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Claude finished @bai-uipath's task in 1m 43s —— View job


🔍 Code Review in Progress

Task List:

  • Read .github/code_review.md for review standards and format
  • Read CLAUDE.md for project conventions
  • Get full diff with git diff origin/main...HEAD
  • Read each changed file in full for context
  • Perform cross-file consistency checks
  • Analyze what's missing
  • Provide structured review feedback

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Claude finished @bai-uipath's task in 1m 30s —— View job


Code Review in Progress

Todo List:

  • Read code review guidelines (.github/code_review.md)
  • Read project conventions (CLAUDE.md)
  • Examine the full diff (git diff origin/main...HEAD)
  • Review each changed file with full context
  • Perform cross-file consistency checks
  • Analyze what's missing
  • Provide design-level scrutiny
  • Post comprehensive review feedback

Starting review...

@bai-uipath
bai-uipath merged commit 1895a2c into main Oct 15, 2026
31 of 33 checks passed
@bai-uipath
bai-uipath deleted the bai/codex-copy-skills branch October 15, 2026 19:25
bai-uipath added a commit to UiPath/skills that referenced this pull request Oct 15, 2026
Picks up UiPath/coder_eval#225 (codex copies skills instead of
symlinking them). 0.12.14 must be released before this merges.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants