Repository navigation
Mapping-set selection and rc.2 x-oold-context transform - #154
simontaurus wants to merge 19 commits into
Conversation
Release previewMerging this PR would release v2.4.0 (current: Changelog preview (truncated)## v2.4.0 (2026-11-16)
### Bug Fixes
- **locator**: Confine resolution and raise on an unreadable document
([`31722bd`](https://github.com/OO-LD/oold-python/commit/31722bd247c701274a3678e1edd1eac22a9cc258))
- **mappings**: Keep a promoted term's scoped @context
([`e87f580`](https://github.com/OO-LD/oold-python/commit/e87f580df7256a014bde51e60047e9d6afc63664))
- **mappings**: Reject a fragment-only scoped @context
([`6803978`](https://github.com/OO-LD/oold-python/commit/680397823fb0d219a93531e6ff03af979c796605))
- **transform**: Warn from the deprecated alias-notation helpers
([`a082163`](https://github.com/OO-LD/oold-python/commit/a0821634bd43babcb0b98df194d1b58a16d2c167))
- **utils**: Make oold.utils a package
([`22fc60d`](https://github.com/OO-LD/oold-python/commit/22fc60ddaf1720f25126326ca5eae0f0b81823da))
### Documentation
- Add the ported modules to the API reference
([`7e84880`](https://github.com/OO-LD/oold-python/commit/7e8488008d3f2c23fa0d0addcc631e9965f9be5b))
### Features
- **generate**: Build an instance from authored examples and defaults
([`e23d4d3`](https://github.com/OO-LD/oold-python/commit/e23d4d31eed58dbad0ed43b48652a5ade1e047cc))
- **labels**: Harvest labels from pinned ontology snapshots
([`f1a7b67`](https://github.com/OO-LD/oold-python/commit/f1a7b67086e8a18aa52ced1dfc5871df6ff6558c))
- **locator**: Resolve a published IRI in the working tree
([`e085814`](https://github.com/OO-LD/oold-python/commit/e085814b68c0021b2b7e3fae4f3ab3fd9cf9adde))
- **mappings**: Inline a scoped @context under the selected set
([`bc6a90d`](https://github.com/OO-LD/oold-python/commit/bc6a90d96221c107739020381a0c8faf549aed9b))
- **readings**: Render an instance as RDF per mapping set
([`d6009ab`](https://github.com/OO-LD/oold-python/commit/d6009ab4013feff9151ae3ae14954946a5c08cf1))
- **sssom**: Derive mapping sets from the schemas that carry them
([`73e88db`](https://github.com/OO-LD/oold-python/commit/73e88db9e6cf96d04f1da8ca4211fae8b9fe6a03))
- **utils**: Mapping-set selection and rc.2 x-oold-context transform
([`dfeb43f`](https://github.com/OO-LD/oold-python/commit/dfeb43febec2377b7dc3be65115fdebab023b80d))
- **utils**: Rewrite synonyms in JSON-LD rather than in RDF
([`5b1f25d`](https://github.com/OO-LD/oold-python/commit/5b1f25d0c2020d948e16b759e816bc0331fffc90))
- **validation**: Register the mapping-set round-trip check id
([`2a7c9a7`](https://github.com/OO-LD/oold-python/commit/2a7c9a71e5c76eecfe7b04b9f9469356e3ea22e0))
- **validation**: Round-trip an instance under every mapping set
([`cd3f8a5`](https://github.com/OO-LD/oold-python/commit/cd3f8a551e450620a3e7d991da0508c90dc39c09))
### Refactoring
- **locator**: Read the term graph from @context, not from $ref
([`0c8fec3`](https://github.com/OO-LD/oold-python/commit/0c8fec38355aa90be77b1d659a1b57b8215676f0))
### Testing
- Cover every branch of the ported modules
([`6ecc2cf`](https://github.com/OO-LD/oold-python/commit/6ecc2cf9fff6c4f8a421666624e562bea24dfd21))
- Cover the error paths of the ported modules
([`c9d208e`](https://github.com/OO-LD/oold-python/commit/c9d208ead3b95655fef4a2514a77a8273761ade3))
Preview via python-semantic-release and conventional commits. |
📊 Benchmark ResultsClick to see benchmark comparisonThreshold: 1.3x (30% slower triggers a regression warning) Note: Benchmarks are informational only and won't fail the build. 💡 Tip: Download the |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
Reviewed at 156b9b0 merged with current Blocking: silent, total data loss. schema = {"@context": CTX, "x-oold-instance-rdf-type": ["ex:Person"], "properties": {...}}
inst = {"id": "https://ex.org/p1", "name": "Jane", "address": {...}}
transform(inst, [schema], [schema])
# -> {'@context': {...}} every triple gone, no error
The RDF detour is not needed, and the reason is not cost. Measured over the schemas.oo-ld.org cases and synthetic graphs, all three candidates agree on every corpus case and separate only on constructed input: So rewriting the dataset dict is the faster of the two viable options, and it still cannot be made correct. An RDF dataset carries no term definitions, so nothing in it separates a Separately, Correctness
Process
|
📊 Benchmark ResultsClick to see benchmark comparisonThreshold: 1.3x (30% slower triggers a regression warning) Note: Benchmarks are informational only and won't fail the build. 💡 Tip: Download the |
- add oold.utils.mappings: promote, mapping_sets, chain, context_of, synonym_entries; the selection OOLD-EXT-4966 describes, complementing validation's promoted_terms which deliberately never chooses - replace the name* alias notation in oold.utils.transform with to_rdf/from_rdf/transform over x-oold-context: exactMatch rewriting, @reverse inversion, @graph framing, @nest shape bridging under a named set (OO-LD/oold-schema#135) - re-encode the alias-notation test cases as x-oold-context, same inputs and expected outputs
- bridge terms carry the rewrite: compact under a context where the term means the synonym IRI, redefine it to the primary IRI, flatten, recompact - a dataset carries no term definitions, so rewriting RDF cannot tell a @vocab-coerced value from a reference to a node - from_rdf reads a JSON-LD document as it stands instead of sending it through RDF and back, keeping @index and @direction - keep jsonld_to_jsonld and json_to_json, deprecated, with their tests - Dataset.default_union, else triples() drops every quad in a named graph
- what a term scopes is a property of the shape, not of the vocabulary - a @reverse synonym lost it, so an embedded object fell back to the consensus terms under a selected set
- inline_scoped resolves a term's scoped @context reference and promotes it with the same mapping set, so an embedded object speaks the vocabulary of the tree it hangs in - ScopeSource states the resolution step as a protocol, keeping the module free of any directory layout - a cyclic reference drops the scoping instead of recursing
- MirrorRule states the IRI-to-path layout as data, so a different tree is a rule rather than a patch - term_chain walks allOf, @context and $ref, which is what a reading needs and wider than the inheritance chain - loader() feeds mappings.chain; a reference the tree cannot reach is skipped, and a cycle terminates
- effective_views promotes each declared set in turn and reports lossless or lossy per reading, which the declared-context checks cannot see - a reading that only changes the document shape is compared as a graph (URDNA2015) rather than as a tree - view_checks reports them as roundtrip.mapping_set records
- readings() returns Turtle plus a Mermaid diagram for the consensus reading and every declared set, and reports a mapping a processor rejects instead of failing the run - blank nodes are skolemised from their URDNA2015 labels, so the same input serialises to the same bytes on every run - named_namespaces names what a serializer would call ns1, ns2
- synth_instance fills a schema from its own examples, defaults and enums only, so a schema with no committed instance still has one to show without inventing a value - a schema whose required properties cannot be filled yields nothing - the inheritance walk it needs is mappings.chain over a Locator
- collect() groups the inline mappings by set, with the consensus ones emitted as a set of their own - crosswalk() chains two sets through the terms they share, and only where one side is an exactMatch - render_set() writes the SSSOM preamble and rows, sorted, so the same mappings give the same file
- a term is labelled by the ontology that owns it, not by one that merely imports it, decided on the local name prefix where there is one - only the IRIs the schemas map are kept, read from the schemas rather than listed - the network lookup is a separate opt-in step, since it is the one part that is neither offline nor reproducible
d57172b to
f1a7b67
Compare
📊 Benchmark ResultsClick to see benchmark comparisonThreshold: 1.3x (30% slower triggers a regression warning) Note: Benchmarks are informational only and won't fail the build. 💡 Tip: Download the |
- a reference is input, so ../../secret.json and a .. buried in an IRI segment are refused; the boundary is the root, or the referring document's directory when no root is given - a document that cannot be read is raised rather than skipped, which shortened the chain and reported success over a reading that lost data
- a processor ignores the fragment and applies the named document's top-level @context, so the embedded object takes the enclosing vocabulary; normalising it away hid that - record all three differences from the source generators in one place, including the @reverse IRI taken from the entry key - note where chain and build_docs.inherited part, which the corpus the two were compared on does not reach
- roundtrip.mapping_set was public but described nowhere, so no report could name it and no --only could filter on it - the drift corpus now calls effective_views directly, so the registry is held to it in both directions instead of being exempted
- a comment banner reaches whoever opens the file, and a caller still on the old notation is by definition not reading it - both bodies ran in no test at all, the only caller being skipped for flakiness, so "kept working" was a claim nothing checked - name the parameter the signature actually has
It imported as an implicit namespace package, but static tooling could not see it as a member of oold, so no module in it could be documented.
- six public modules had no entry; mkdocstrings collects them now that oold.utils is a package - a skolem IRI is minted under whoever publishes the document, so say that the default genid is nobody's host in particular and that byte identity with a site generator needs the same authority
- the label lookup service, stubbed rather than called - a context that cannot be assembled, and each view_checks outcome - the scoped-context and label-prefix paths in readings
📊 Benchmark ResultsClick to see benchmark comparisonThreshold: 1.3x (30% slower triggers a regression warning) Note: Benchmarks are informational only and won't fail the build. 💡 Tip: Download the |
- a context is never resolved through $ref: a scoped @context names the same document the $ref does, and a processor resolves that natively - context_chain walks the context declarations; applicable_schemas adds the allOf chain, which is the only carrier of x-oold-context synonyms - undefined_terms follows a property scope, so an embedded object's terms stay where they belong instead of being hoisted
- locator, mappings, readings, sssom, labels, transform and effective_views reach 100% statement and branch coverage - the deprecated alias notation is exercised in each spelling it accepts: plain IRI, reverse alias, list-valued document context - a reverse-only primary term is asserted on the report it makes, not on the output, which is wrong there
📊 Benchmark ResultsClick to see benchmark comparisonThreshold: 1.3x (30% slower triggers a regression warning) Note: Benchmarks are informational only and won't fail the build. 💡 Tip: Download the |
Changes
oold.utils.mappings(new):promote,mapping_sets,chain,context_of,synonym_entries,inline_scoped. This is the profile-based selectionOOLD-EXT-4966describes; it complementsvalidation.context_resolution.promoted_terms, which deliberately never chooses a synonym (OOLD-EXT-8f62). Ported fromoold-reference-schemas/scripts/{_shared,build_docs}.pyand verified equivalent against that implementation on every module, all schemas, all three mapping sets plus consensus. Three deliberate differences from that source, listed once in the module docstring:mapping_set_idmay be a list,chaintakes a resolver callable rather than doingPatharithmetic, and a promoted@reversetakes the synonym IRI from the entry's key, which is whatOOLD-EXT-4966andOOLD-EXT-7c5drequire of it.oold.utils.transform(replaced):to_rdf/from_rdf/transformoverx-oold-context. Export promotes a named set; import rewrites everyexactMatchsynonym onto the primary IRI (no set needed), inverts@reverserelations, frames@graphinput viavalidation.frame.schema_to_frame, and bridges document shapes when a promoted fragment carries@nestand a set is named. Readings A/B/C/E of Mapping-set selection can bridge document shapes via @nest, and the spec should say so oold-schema#135 are reproduced exactly and pinned as regression tests, including the boundary that vocabulary is bridged without a selection and shape is not.tests/test_transform.pyrewritten: thename*alias cases are re-encoded asx-oold-contextwith the same inputs and expected outputs, so the change of notation provably does not change what the transformation produces.promotekept a term's scoped@context. A@reversesynonym dropped it, so an embedded object fell back to the consensus terms under a selected set. Direction does not change which terms the object at the end of an edge may use.Notation
x-oold-contextreplaces thename*alias notation as whattransformreads.That notation predates the specification and was never part of it. Nothing in the spec surface changes. The commit stays
feat:.jsonld_to_jsonldandjson_to_jsonare kept and now raise aDeprecationWarning, scheduled for removal at the next major. Each has a live test, the one that existed having been skipped for flakiness.Generic tools ported from oold-reference-schemas
Nothing ported assumes a directory layout, a module name or a docs pipeline.
mappings.inline_scopedand theScopeSourceprotocol: resolves a term's scoped@contextfrom the tree and promotes it with the same mapping set. Identical tobuild_docs.inline_scopedon 58 schema and mapping-set combinations. A fragment-only reference is rejected rather than normalised. A processor ignores the fragment and applies the named document's top-level@context, handing the embedded object the enclosing vocabulary.oold.utils.locator:LocatorplusMirrorRule, which states the published-IRI-to-path layout as data.context_chainreads the term dependency graph from the@contextdeclarations and leaves a$refalone, since a context is never resolved through one.applicable_schemasadds theallOfchain, the only carrier ofx-oold-contextsynonyms. Resolution is confined to the root, or to the referring document's directory where there is none, and a cycle terminates instead of recursing.oold.validation.effective_views: the round trip per mapping set, with the URDNA2015 fallback for a reading that legitimately changes the document shape. Same verdict and triple count asscripts/effective_views.pyon 26 readings.validation/compliance.pyonly ever exercises the declared context, so a defect confined to one set was invisible until now. The id it emits,roundtrip.mapping_set, is registered and documented; the pipeline and the CLI do not run it.oold.utils.readings: Turtle and Mermaid per mapping set, returned rather than written. 54 readings byte-identical tobuild_docs.write_rdf, withgenid=set to the generator's authority. A skolem IRI is minted under whoever publishes the document. The default here is nobody's host in particular.validation.generate.synth_instance: an instance built from a schema's own examples and defaults. Identical on all 15 schemas and to the committed generated instances.inheritedneeded no port,mappings.chainover aLocatorreproduces it exactly.oold.utils.sssom: SSSOM sets and the crosswalk between them. 10 sets identical tobuild_mappings.py, except 51 rows where the source holds a literal0x01byte in place of a back-reference (build_mappings.pyline 209) and emits a relative subject. The port writes what the comment beside it says.oold.utils.labels: label harvest from pinned ontology snapshots.mapped_irisidentical at 121 IRIs, and the harvest over the 16 pinned snapshots reproducesgenerated/labels.jsonbyte for byte.Left out:
scripts/level.py, because OO-LD/oold-reference-schemas#19 changessurface()andcompare(), the two functions a port would consist of.Verification
main, against 804, 10, 1 onmainitself.make checkandmake docs-testgreen.mainafter fix(frame): reverse IRI grouping, $ref embeds and default type anchors #179. The tests pass against the newschema_to_frameoutput rather than around it.schema-playgroundin https://github.com/OO-LD/oold-playgrounds/tree/feat/schema-playground vendors exactly these modules pending this PR; once released they can be deleted there.Closes the mappings/transform part of the migration listed in https://github.com/OO-LD/oold-playgrounds/blob/feat/schema-playground/schema-playground/README.md#pending-upstream-migration