Skip to content

Fix vendored Pipenv re-vendor to a newer patch (#769) - #825

Open
Mikola Lysenko (mikolalysenko) wants to merge 4 commits into
mainfrom
agent/fix-pipenv-vendored-revendor
Open

Mikola Lysenko (mikolalysenko) wants to merge 4 commits into
mainfrom
agent/fix-pipenv-vendored-revendor

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

LLM Description written by Claude Code:claude-opus-5-5

Fixes #769

Summary

Before this change, a Pipenv project vendored at patch A could never move
to a newer patch B for the same package. scan --mode vendored and
get <B> --mode vendored downloaded B, reported it as replacing A, and
then exited 1, while --dry-run previewed would_revendor:

  • lock-only checkout: failed pypi_pipenv_source_already_exists
  • venv installed from A's vendored wheel (pipenv sync): skipped package_not_installed ("no installed package found on disk"), which
    is false

Now both shapes re-vendor to B: Pipfile.lock is rewired to B's wheel, A's
artifact is swept (vendor_stale_artifact_removed), the ledger moves to
B, and vendor --revert of B restores the original registry pin.

Root cause

Two gaps, both about socket-patch's own wiring at an older patch uuid:

  1. Core, Pipfile.lock guard. check_target_guards in
    vendor/pypi_pipenv.rs refused the "ours, but a stale patch
    generation" entry outright, because wiring over it would lose the only
    recorded registry original. The original is not lost: the ledger entry
    for the older uuid holds it.
  2. CLI, installed-variant probe. In commands/vendor.rs, a PyPI
    install is hashed against the new record's beforeHash. A venv
    installed from A's wheel holds A's patched bytes, so the probe dropped
    the package, and it fell through to package_not_installed.

Fix

  • pypi_prelude looks up the ledger entry that vendored this package at
    another uuid and passes it to the new
    check_target_guards_superseding / wire_pipenv_superseding. An entry
    routed through that older uuid's wheel is rewired in place only when the
    ledger records that exact entry (section:key, unchanged since
    vendoring) with a pre-vendor original, and the wheel names the same
    release. The new record carries that original forward. With no ledger
    record, after an edit, or for another release it still refuses, and
    the message now says which.
  • superseded_install in the vendor loop (and the service download plan,
    which mirrors the loop): when the probe fails but the ledger holds exactly
    this package at an older uuid, the candidate gets the same pristine source
    path a lock-only checkout uses, instead of being skipped. This is limited
    to a sole candidate, or a ledger key equal to the candidate, so it never
    picks among sibling release variants.

This is the Pipenv lane of the #765 family (requirements.txt: #766;
uv/Hatch hosted: #743). It is kept separate so #766, which is ready for
review, doesn't grow. Follow-up (not in scope): pypi_poetry.rs and
pypi_pdm.rs have the same "STALE patch generation" refusal arm. No issue
has been filed for those yet.

No wrapper changes: npm/, pypi/ and gem/ only dispatch to the binary.

Tests (red → green)

Issue case Test Without fix With fix
#769 lock-only re-vendor vendor::pypi::tests::pipenv_superseding_uuid_revendors_in_place (core) FAIL (Refused) pass
#769 lock-only, end to end mode_migration_pypi::pipenv_revendors_to_a_superseding_patch (lock-only lane) FAIL pass
#769 venv installed from A mode_migration_pypi::pipenv_revendors_to_a_superseding_patch (venv lane) FAIL (package_not_installed, as reported) pass
Guard: no ledger record still refuses pipenv_superseding_uuid_without_ledger_refuses (new wording) pass
Guard: drifted entry still refuses pipenv_superseding_uuid_drifted_entry_refuses (new wording) pass

The red runs were done by applying the tests to the pre-fix sources: the
three core tests failed on main code, and the venv lane failed with
only the core half applied.

Commands run locally (Linux, root):

  • cargo clippy --workspace --all-features -- -D warnings: clean
  • cargo test -p socket-patch-core --all-features --lib: 4845 passed.
    4 failed, all chmod/permission simulations that can't fail as root, and
    none in touched code (copy_tree, vlt_heal, poetry/requirements
    write-failure tests).
  • cargo test -p socket-patch-cli --all-features --lib: 834 passed
  • CLI suites mode_migration_pypi, in_process_vendor,
    in_process_redirect_pipenv, in_process_python_envs,
    e2e_vendor_pypi_build, e2e_vendored_production, e2e_vex_vendor,
    vendor_group_commit_e2e, vendor_ledger_schema_e2e,
    scan_requirements_lock_only, covgap_commands_get: all pass.
    covgap_commands_vendor has 3 failures, the state-write-failure tests,
    which also need a non-root chmod.
  • Real Pipenv: SOCKET_PATCH_PIPENV_E2E_REQUIRED=1 SOCKET_PATCH_PIPENV_E2E_VERSIONS=2026.8.0 cargo test -p socket-patch-cli --all-features --test e2e_vex_build -- pipenv:: --ignored: pass
  • The full cargo test --workspace ran out of this session's disk
    allowance while building every test binary, so CI is the full run.
  • cargo fmt: the touched code is formatted. main itself isn't
    rustfmt-clean under 1.93.1, so the unrelated reformat churn was left out.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VSXCFoPbraq7rNKJpXEP2n


Generated by Claude Code

Assisted-by: Claude Code:claude-opus-5-5
A Pipenv project vendored at one patch never moved to a newer patch
for the same package: the re-vendor refused with
pypi_pipenv_source_already_exists and the run exited 1, although the
dry run previewed would_revendor.

When the vendor ledger records the Pipfile.lock entry the older patch
wrote, and that entry is unchanged, it is now rewired in place to the
new wheel. The record carries the older entry's pre-vendor registry
original forward, so vendor --revert still restores the user's pin.
Without that record, or after an edit, it still refuses as before.

Refs #769

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) force-pushed the agent/fix-pipenv-vendored-revendor branch from 6483c64 to 4fc3896 Compare October 5, 2026 04:45
When a venv was installed from the vendored wheel of an older patch
(pipenv sync after vendoring), re-vendoring to a newer patch skipped
the package as package_not_installed and exited 1: the installed
files are the old patch's bytes, so they failed the new patch's
installed-variant check.

When the vendor ledger holds exactly this package at an older patch
uuid, such an install is now treated like a lock-only checkout: the
pristine wheel comes from the lock, registry or patch service, and the
package is re-vendored. The service download plan makes the same call.

Fixes #769

Assisted-by: Claude Code:claude-opus-5-5
check_target_guards and wire_pipenv now have no production caller
(the vendor flow passes the ledger through the _superseding
variants), so clippy flagged them as dead code. Compile them for
tests only and point the docs at the variants production uses.

Refs #769

Assisted-by: Claude Code:claude-opus-5-5
@mikolalysenko
Mikola Lysenko (mikolalysenko) marked this pull request as ready for review October 5, 2026 05:02
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

BugBot review


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 41dcd42. Configure here.

@mikolalysenko Mikola Lysenko (mikolalysenko) added the Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review label Oct 5, 2026
@mikolalysenko

Copy link
Copy Markdown
Collaborator Author

Burn-down agent: labeled Ready for review.

  • Head: 41dcd4258284c3e0c59e67e9223236bd4a633caa
  • CI: 485/485 green (6 skipped) on this head; branch not behind its base, no conflicts
  • Bugbot: reviewed 41dcd42, no new issues; all review threads resolved
  • Reviewer note: Nothing special; branch is current with main.

Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Ready for review Agent-verified: mergeable, CI green, Bugbot clean — awaiting human review

Projects

None yet

2 participants