Skip to content

fix(caching): preserve missing results when restoring in-memory caches - #1826

Open
JustinBinber wants to merge 1 commit into
apache:mainfrom
JustinBinber:codex/fix-memory-cache-misses
Open

JustinBinber wants to merge 1 commit into
apache:mainfrom
JustinBinber:codex/fix-memory-cache-misses

Conversation

@JustinBinber

Copy link
Copy Markdown

Restoring a partial result cache into memory can turn missing values into successful None cache hits, so the Driver returns incorrect results instead of recomputing them. Closes #1825.

Changes

  • Import a None result only if its version exists in the source store; non-None results retain a single read.
  • Document that restoration skips missing versions while preserving stored None values.
  • Cover disk and memory sources, explicit versions, metadata and their union, legitimate falsey values, and public Driver recomputation followed by a normal cache hit.

How I tested this

  • Collected the seven new offline regression cases before execution.
  • Before the fix: 7 failed. The Driver returned {'answer': None} instead of {'answer': 42}; all six store/selection cases incorrectly reported a missing result as present.
  • After the fix: 7 passed.
  • python -m pytest -q tests/caching/result_store tests/caching/metadata_store tests/caching/test_integration.py tests/caching/test_adapter.py: 88 passed, including the existing threading/multiprocessing cases. One existing warning concerns the trusted pickle test fixture.
  • Configured prek hooks pass for all three files, including Ruff 0.16.9, formatting, whitespace, AST and license checks. git diff --check passes.
  • Validated on Python 3.11.7, Windows, against the current checkout's Python sources.

Notes

No API signatures, dependencies or cache serialization formats change. Loading remains a non-atomic operation if the source is concurrently modified.

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal
  • Code passed the configured pre-commit checks
  • Changed behavior is tested
  • Existing method documentation describes the restored behavior
  • No placeholder code or additional deferred implementation

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

InMemoryResultStore.load_from turns missing results into None cache hits

1 participant