Skip to content

Respect explicitly declared constraintProperties on StatVars - #795

Merged
dwnoble merged 4 commits into
masterfrom
respect-declared-constraint-properties
Oct 16, 2026
Merged

dwnoble merged 4 commits into
masterfrom
respect-declared-constraint-properties

Conversation

@dwnoble

@dwnoble dwnoble commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

When a user defines a StatisticalVariable with an explicit constraintProperties list alongside extra metadata properties:

Node: MyVariable
typeOf: dcid:StatisticalVariable
myConstraintProperty: value1
myNonConstraintProperty: value2
constraintProperties: myConstraintProperty

ingestion previously ignored the explicitly set constraintProperties: myConstraintProperty and overwrote it with every property on the node outside the hardcoded exclusion list:

Node: MyVariable
typeOf: dcid:StatisticalVariable
myConstraintProperty: value1
myNonConstraintProperty: value2
constraintProperties: myConstraintProperty, myNonConstraintProperty

As a result, any custom metadata on a StatVar (such as sourceIndicatorCode: "NGDP") was treated as a constraint property and included in the auto-generated name and definition.

Solution

Respect constraintProperties when it is explicitly set by the user, and only derive the list when none is declared on the node.

Technical Details

  • McfMutator.mutateNode:
    • Checks whether the StatVar node already declares a non-empty constraintProperties list.
    • When declared: strips namespaces (dcid:, dcs:, schema:), de-duplicates the declared property names, skips any property in Vocabulary.NON_CONSTRAINT_STAT_VAR_PROPERTIES (logging Mutator_InvalidConstraintProperty) or missing a value on the node (logging Mutator_MissingConstraintPropertyValue), and writes the validated constraintProperties back to the node.
    • When not declared: preserves existing behavior and derives constraintProperties from all properties outside Vocabulary.NON_CONSTRAINT_STAT_VAR_PROPERTIES.
    • Uses the resolved constraintPvs for both the Base DC definition and fallback name.
    • Applies across all McfMutator callers: Platform Spanner ingestion (GraphTransformer), the dc-import CLI (tool/Processor), and server/Processor.
  • StatVarNameGenerator:
    • Adds generateName(PropertyValues pvs, Set<String> constraintProps) so McfMutator can pass the resolved constraint property set when generating fallback StatVar names.
    • Updates the 1-arg generateName(PropertyValues pvs) path to also respect constraintProperties when present on pvs before falling back to Vocabulary.NON_CONSTRAINT_STAT_VAR_PROPERTIES.

Testing

  • McfMutatorTest: Added unit tests covering:
    • Explicit constraintProperties alongside extra metadata (sourceIndicatorCode: "NGDP")
    • Unset constraintProperties falling back to derivation
    • Multiple declared constraints with mixed namespace prefixes (dcs:, schema:)
    • Declared constraint missing a property value on the node (warning logged, stripped from constraintProperties, excluded from name/definition)
    • Declared non-constraint properties (populationType, measuredProperty) filtered from constraintProperties while still preserved as core properties in the generated name and definition
    • Duplicate entries in constraintProperties (both in MCF input and proto TypedValues)
    • Empty constraintProperties falling back to derivation
    • ComplexValue expansion ([dcs:Year 18]) with declared constraintProperties
    • Explicit name and definition preserved alongside declared constraintProperties
  • StatVarNameGeneratorTest: Added tests for explicit, empty, and null constraint sets, plus 1-arg generateName(pvs) when constraintProperties is present on the node.
  • GraphTransformerTest: Updated testStatVarTransformationWithExistingConstraintProperties (which previously pinned the overwrite behavior) and added testStatVarTransformationPreservesDeclaredConstraintPropertiesWithMetadata.

@dwnoble
dwnoble requested a review from ajaits October 14, 2026 22:28
@codacy-production

codacy-production Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 34 complexity

Metric Results
Complexity 34

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the MCF mutation and name generation logic to respect explicitly declared constraintProperties on StatisticalVariable nodes, ensuring that other properties are treated as metadata and not incorrectly included in generated names or definitions. The feedback recommends improving the robustness of these changes by performing namespace-insensitive lookups when retrieving constraint properties and their values from the node's property-value map.

Comment thread util/src/main/java/org/datacommons/util/McfMutator.java
Comment thread util/src/main/java/org/datacommons/util/McfMutator.java
When a user defines a StatisticalVariable with an explicit
constraintProperties list alongside extra metadata properties:

  Node: MyVariable
  typeOf: dcid:StatisticalVariable
  myConstraintProperty: value1
  myNonConstraintProperty: value2
  constraintProperties: myConstraintProperty

ingestion previously ignored the explicitly set
"constraintProperties: myConstraintProperty" and overwrote it with every
property on the node outside the hardcoded exclusion list:

  Node: MyVariable
  typeOf: dcid:StatisticalVariable
  myConstraintProperty: value1
  myNonConstraintProperty: value2
  constraintProperties: myConstraintProperty, myNonConstraintProperty

As a result, any custom metadata on a StatVar (such as
sourceIndicatorCode: "NGDP") was treated as a constraint property and
included in the auto-generated name and definition.

This change respects constraintProperties when explicitly set by the user:
- Keep the declared constraintProperties list as-is instead of overwriting it.
- Only derive constraintProperties when none is declared on the node.
- Use only the declared constraint properties when generating fallback
  name and definition values.
- Log a warning (Mutator_MissingConstraintPropertyValue) if a declared
  constraint property has no value on the node.
@dwnoble
dwnoble force-pushed the respect-declared-constraint-properties branch from b942a21 to 172482d Compare October 14, 2026 22:32
Comment thread util/src/main/java/org/datacommons/util/McfMutator.java
Comment thread util/src/main/java/org/datacommons/util/McfMutator.java
Comment thread util/src/main/java/org/datacommons/util/McfMutator.java Outdated
Comment thread util/src/test/java/org/datacommons/util/McfMutatorTest.java Outdated
Comment thread util/src/test/java/org/datacommons/util/McfMutatorTest.java
@dwnoble
dwnoble force-pushed the respect-declared-constraint-properties branch from 9208ca7 to bd465bf Compare October 15, 2026 23:39
@dwnoble
dwnoble merged commit fbec009 into master Oct 16, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants