Skip to content

fix(core): flatten $defs/manual before schema code generation - #230

Open
LukasGold wants to merge 1 commit into
stagefrom
fix/flatten-manual-defs
Open

LukasGold wants to merge 1 commit into
stagefrom
fix/flatten-manual-defs

Conversation

@LukasGold

Copy link
Copy Markdown
Contributor

Closes #229

Changes

  • src/osw/core.py: new helper _flatten_manual_definitions. fetch_schema calls it on the result of merge_generated_definitions, before code generation.
    • It removes every root reference to #/$defs/manual: the root $ref and all matching allOf entries. An allOf that becomes empty is removed.
    • It merges the content of $defs/manual into the root with merge_deep, so existing root keys win, as for $defs/generated.
    • It removes $defs/manual, and $defs if that leaves it empty.
    • An unreferenced $defs/manual leaves the schema unchanged. A "$ref": "#" inside the manual schema is kept.
    • Limit, as for $defs/generated: pointers below #/$defs/manual/... are not rewritten.
  • tests/test_fetch_schema_manual_defs.py: 11 tests.
    • 3 run fetch_schema offline: the minimal two-page case from the issue, the hand-authored Characteristic shape ($defs/generated pointing to $defs/manual), and an object-rooted manual schema used as an allOf mixin.
    • 8 test the helper directly, including a root $ref plus an allOf entry, and two allOf entries.

Rationale

Hand-authored Characteristic pages keep their schema in $defs/manual. A reference to such a page with a key beside its $ref, such as title*, makes datamodel-code-generator fail with A Parser can not resolve classes. fetch_schema already merges $defs/generated into the root, so this change does the same for $defs/manual. The fix is in osw-python, not in oold, because $defs/manual is an OSW convention.

Tests

  • Without the helper call, two of the three fetch_schema tests fail with the error from the issue; the object-rooted mixin test passes either way. With it, all 11 tests pass.
  • make check passes, and make test reports 1004 passed.

@github-actions

Copy link
Copy Markdown
Contributor

Release preview

Merging this PR would release v2.10.0 (current: v2.9.1).

Changelog preview (truncated)
## v2.10.0 (2026-11-13)

### Bug Fixes

- Botfilter for user sync
  ([`4d6f440`](https://github.com/OpenSemanticLab/osw-python/commit/4d6f4401c63a148c924bef89963b8f18bc8d4b0c))

- Follow properties and items when collecting category templates
  ([#220](https://github.com/OpenSemanticLab/osw-python/pull/220),
  [`55e500e`](https://github.com/OpenSemanticLab/osw-python/commit/55e500efb896f0a3d6bde98409b3a6f43a652bdc))

- Keep the stored jsonschema when store_entity regenerates it
  ([`9b76a8f`](https://github.com/OpenSemanticLab/osw-python/commit/9b76a8fbeb813b34c4881b54d329e9e837b08f95))

- Select category templates from the schema allOf like the wiki
  ([#220](https://github.com/OpenSemanticLab/osw-python/pull/220),
  [`9f606f4`](https://github.com/OpenSemanticLab/osw-python/commit/9f606f48fefd640cca48e6718f1a92eca118da6b))

- **core**: Flatten $defs/manual before schema code generation
  ([#229](https://github.com/OpenSemanticLab/osw-python/pull/229),
  [`51fa791`](https://github.com/OpenSemanticLab/osw-python/commit/51fa791e427e5e53a65ee6bd0f65530848436b66))

- **user-sync**: Actually remove pruned fields via empty relations and remove_empty
  ([`87cf85b`](https://github.com/OpenSemanticLab/osw-python/commit/87cf85b31cbfec6b80e2e30b92852c359586133c))

- **user-sync**: Align with stage and the tools docs
  ([`bc6b0d6`](https://github.com/OpenSemanticLab/osw-python/commit/bc6b0d642f5697ca8048b4be05fd26c9ad2162f6))

- **user-sync**: Count and redirect only stored items
  ([#224](https://github.com/OpenSemanticLab/osw-python/pull/224),
  [`0b8dc97`](https://github.com/OpenSemanticLab/osw-python/commit/0b8dc974e972e8adaa7a2bb03bd3ae4396589f40))

- **user-sync**: Drop python-dotenv, use default accounts.pwd.yaml
  ([`02570c8`](https://github.com/OpenSemanticLab/osw-python/commit/02570c8f88d6d27c58e1e595bc8bb13e90354fd4))

- **user-sync**: Prune a single-valued key stored with an empty value
  ([`ebda514`](https://github.com/OpenSemanticLab/osw-python/commit/ebda514c731718ab20a084b93288491b3b944627))

- **user-sync**: Prune by field type and decide from stored jsondata
  ([`9b36e4e`](https://github.com/OpenSemanticLab/osw-python/commit/9b36e4ef8417b8847ba304535a95c0eedaf628b9))

### Chores

- **user-sync**: Scaffold module, example and tracking
  ([`217b101`](https://github.com/OpenSemanticLab/osw-python/commit/217b1017b46295340bc79cee050a91c45c0d379d))

### Continuous Integration

- Run Main on pushes to stage and Integration on PRs into stage
  ([`1b55f02`](https://github.com/OpenSemanticLab/osw-python/commit/1b55f02c90cae646d690f83d6208c01f0e26e62b))

### Documentation

- State which schema references the template walk follows
  ([#220](https://github.com/OpenSemanticLab/osw-python/pull/220),
  [`799bc07`](https://github.com/OpenSemanticLab/osw-python/commit/799bc07952aac7e6ab579bc127c5cb3000710491))

- **user-sync**: Add user sync guide and finalize example
  ([`830d120`](https://github.com/OpenSemanticLab/osw-python/commit/830d12072f9acfcbbaa265e5b75e6db38ca0a0d1))

- **user-sync**: Document prune and opt-in extras
  ([`1899d7f`](https://github.com/OpenSemanticLab/osw-python/commit/1899d7f2c070a3bee614602509109bc97ea19f79))

### Features

- **user-sync**: Add interactive preview and resolution
  ([`e0c54db`](https://github.com/OpenSemanticLab/osw-python/commit/e0c54db9845447a62de544c4651e7d512a6ead74))

- **user-sync**: Add orcid public api client
  ([`5e40525`](https://github.com/OpenSemanticLab/osw-python/commit/5e40525b826713d37af47602d0542f957fba0e94))

- **user-sync**: Add reconciliation engine
  ([`54711e9`](https://github.com/OpenSemanticLab/osw-python/commit/54711e96f36c74ac125a4411b7b2cf7b33d23d04))

- **user-sync**: Enumerate and classify mediawiki users
  ([`90bf3da`](https://github.com/OpenSemanticLab/osw-python/commit/90bf3daf68639af2a1f133472ced70d0c8d2af71))

- **user-sync**: Exclude system accounts and complete idempotent sync
  ([`f2d011d`](https://github.com/OpenSemanticLab/osw-python/commit/f2d011d61bf4dfe1ea445cdfc78bd3f6b69af520))

Preview via python-semantic-release and conventional commits.

@LukasGold LukasGold left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Manually reviewed the code. Looks fine to me

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant