Skip to content

fix(python-sdk): validate observe response shape - #503

Merged
loveRhythm1990 merged 2 commits into
matrixorigin:mainfrom
loveRhythm1990:fix/python-sdk-observe-response-shape
Nov 17, 2026
Merged

loveRhythm1990 merged 2 commits into
matrixorigin:mainfrom
loveRhythm1990:fix/python-sdk-observe-response-shape

Conversation

@loveRhythm1990

@loveRhythm1990 loveRhythm1990 commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #498

Problem

observe() and observe_deduplicated() (sync and async, 4 call sites in client.py) passed the 2xx body to ObserveResult.from_dict(data or {}):

  • ["mem_1"], "text", 42 raised a raw AttributeError from .get().
  • [], false, null, 204 and an empty 200 were collapsed to {} by data or {}, so the SDK reported "no memories extracted".
  • {"memories": null} raised TypeError: 'NoneType' object is not iterable.

Change

  • client.py: pass data straight to ObserveResult.from_dict, removing data or {}.
  • models.py: ObserveResult.from_dict requires a JSON object and raises MemoriaResponseError otherwise. An explicit "memories": null is read as []. A memories value that is neither an array nor null (for example a string, which list() would split into characters) is rejected.
  • tests/unit/test_observe_response_shape.py: sync and async tests for both methods, covering every body shape in the issue plus the empty forms, the null memories case and valid bodies.
  • CHANGELOG entry.

The server's /v1/observe handlers always return an object with a memories array, so an empty body is a protocol violation, not a valid "nothing extracted" answer.

Relationship to #502

#502 (fix for #499) makes the other resource parsers raise MemoriaResponseError for empty bodies. This PR does not use the _require_object helper that #502 adds, so the code does not depend on it.

The two PRs touch sdk/python/CHANGELOG.md and models.py. This PR's changelog entry sits at the end of the Fixed section, away from #502's entry, and a trial merge of both heads is conflict-free. If main changes before they land, rebase whichever lands second and keep both entries. A later cleanup could switch ObserveResult.from_dict to the shared helper.

Verification

  • pytest tests/unit: 1055 passed.
  • The json-null test case now sends a literal null body (httpx.Response(200, json=None) sends an empty one).
  • With the src changes stashed, the new test file has 44 failures. The 3 valid-body cases pass either way.
  • ruff check and mypy src are clean.

observe() and observe_deduplicated() passed the 2xx body to
ObserveResult.from_dict(data or {}). Arrays, strings and numbers raised a raw
AttributeError, and falsy or empty bodies were turned into a fabricated empty
result.

ObserveResult.from_dict now requires a JSON object and raises
MemoriaResponseError otherwise. An explicit null memories field is read as an
empty list, and a non-array memories field is rejected.
@mergify

mergify Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

This pull request is merged.

@loveRhythm1990 loveRhythm1990 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for the code at b47f54d.

I independently checked all four changed files, all four observe call sites, and the server's shared observe response construction. Removing data or {} prevents falsy malformed responses from masquerading as successful empty extraction. The object/array checks fix the reported exceptions while retaining the explicitly requested null-memories compatibility.

The new tests produce 44 failures on base; all 1,055 SDK unit tests pass on this head. Eight additional sync/async wire-body checks pass, including literal JSON null.

Non-blocking test correction: httpx.Response(200, json=None) produces an empty body, so json-null currently duplicates empty-200. Use content=b"null" with application/json. I checked the literal-null path separately and it correctly raises MemoriaResponseError.

Integration note: the source changes compose with #502, but the two heads conflict in sdk/python/CHANGELOG.md. Whichever lands second needs that conflict resolved while preserving both entries; the PR description's "either order" should not imply a conflict-free merge.

Coverage: 4/4 changed files reviewed, 100%, none skipped. Current CI is green and there is no conflict with current main.

httpx.Response(200, json=None) produces an empty body, so the case duplicated
empty-200. Use content=b'null' with a JSON content type to exercise JSON
decoding. Also move the changelog entry to the end of the Fixed section so it
does not collide with the neighbouring matrixorigin#502 entry.
@loveRhythm1990

Copy link
Copy Markdown
Collaborator Author

Addressed in the latest push: the json-null case now sends a literal null body, the changelog entry moved to the end of the Fixed section, and the description no longer claims either-order merging without noting the changelog. A trial merge with #502 is conflict-free.

@loveRhythm1990 loveRhythm1990 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM at 4e1db47.

The literal-null fixture and changelog conflict are addressed. I reviewed all four current changed files and confirmed the observe response validation remains correct for sync/async calls. All 1,055 SDK unit tests pass on this head.

A local trial merge with #502's current head is conflict-free, preserves both changelog entries, and passes all 1,125 SDK unit tests. No additional findings.

Coverage: 4/4 changed files reviewed, 100%, none skipped. The head has no conflict with current main. DB Tests are still running; this is a code LGTM, pending completion of CI.

@loveRhythm1990
loveRhythm1990 merged commit 70680f0 into matrixorigin:main Nov 17, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(python-sdk): observe() 2xx bodies with valid non-object JSON crash raw or fabricate an empty ObserveResult

2 participants