Skip to content

fix(query): preserve count and predicate semantics (2/3) - #3262

Open
contrueCT wants to merge 60 commits into
apache:masterfrom
hugegraph:task/tp381-2-query-semantics
Open

contrueCT wants to merge 60 commits into
apache:masterfrom
hugegraph:task/tp381-2-query-semantics

Conversation

@contrueCT

@contrueCT contrueCT commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Purpose of the PR

Part 2 of a three-PR upgrade series targeting apache/hugegraph:master. Preserve transaction count and predicate semantics while retaining TinkerPop 3.5.1.

Merge order: apache#3261 (Java 17 foundation) → apache#3262 (query semantics) → apache#3263 (TinkerPop upgrade). All three PRs target master directly. Merge this PR after apache#3261. Until the prerequisite is merged, the Files changed tab includes its changes; the part-2-only diff is available in community #232.

Source: hugegraph/hugegraph#232, submitted directly from hugegraph:task/tp381-2-query-semantics. The local validation statements below are carried over from that source PR; CI on this ASF PR head remains the merge gate.

Relationship to other work

  • This three-part series is the split delivery path for the modernization tracked in apache#3117 and issue #3069. BREAKING CHANGE(server): upgrade Java17 + TP3.7 + Groovy4 #3117's current code also targets TinkerPop 3.8.1; the series is intended to replace that monolithic delivery, rather than add another independent upgrade.

  • Generic label/ID/SEARCH condition resolution and selective predicate pushdown remain with apache#2994, issue #3201 and candidate-index coverage in apache#3243. This series retains whole-step local filtering and transaction/count fixes; it adds no generalized partial extraction or optimizer-time index-coverage heuristic. Existing special handling around match() and connective label filters is inherited from master.

  • community #260 relocates shared query/model types, and community #261 changes transaction lifecycle. Their overlapping engine files need integration coordination; those migrations are not folded into this series.

  • Single-ID query fast paths (apache#3175, apache#2859) and adjacency-query optimization (apache#2864) remain separate performance work. Cypher parameter binding (community #238) and error mapping (apache#3259, apache#3241) remain separate behavior changes; part 3 only adapts upgraded transport APIs and result normalization.

  • The Java 17 launcher checks also cover the older Java 11 startup-check goal in apache#2846. Configurable distribution paths (apache#3253) and restart diagnostics (apache#3258) touch the same scripts and remain separate integration work.

Main Changes

  • Count uncommitted vertex/edge changes through the transaction query path and always close fallback iterators. Continue rejecting limit, offset, paging and unsupported aggregates with uncommitted changes.
  • For ordinary GraphStep/VertexStep extraction, keep the whole HasStep local when any sibling condition is unsupported. Retain the existing special handling around match() and connective label filters from master. Count optimization does not skip filtering barriers.
  • Reset optimized count execution state and exclude result iterators from query-step equality. This fixes the existing count equality regression.

TinkerPop-specific NotP, serializer and step-API changes remain in part 3. This PR still compiles and runs against TinkerPop 3.5.1.

Verifying these changes

  • Already covered by existing tests: CountStrategyCoreTest, TraversalUtilOptimizeTest, GraphTransactionTest, and QueryListTest.
  • Local validation: Formatting and all-module clean compile passed. 82 focused core/optimizer tests passed on each of Memory and RocksDB against TinkerPop 3.5.1, plus 17 GraphTransaction/QueryList unit tests in their separate unit-test profile.

Does this PR potentially affect the following parts?

  • Dependencies
  • Modify configurations
  • The public API
  • Other affects: transaction count and traversal optimization behavior
  • Nope

Documentation Status

  • Doc - TODO
  • Doc - Done
  • Doc - No Need

Documentation: docs/query-semantics.md.

Review follow-up

  • Pending removals take precedence over added/updated records, so update-then-remove yields zero in both lists and counts, including self-loops. Same-ID re-adds remain visible.
  • Document the existing dirty-index boundary: conditions using an index reject uncommitted index changes; native label scans, global counts and direct-ID queries retain their supported behavior. Real Memory and RocksDB regressions cover these paths.
  • Immutable label collections such as List.of(...) and Set.of(...) are inspected by iteration; predicates that actually contain null labels retain local filtering. Optimizer and query regressions cover both cases.
  • Unsupported mixed native ID/text predicates stay together in the local HasStep. Regression checks cover matching and nonmatching results, counts, pagination and plan retention with the HugeGraph strategies enabled and disabled.
  • Preserve transaction-visible counts, self-loop multiplicity, backend system counts and filtering barriers. Ordinary ID equality remains covered.
  • Inherit the Java 11 bytecode target for published Commons/RPC libraries and Ivy 2.6.0 from the foundation; build and service runtimes remain Java 17.
  • Java 17 formatting and all-module clean compile pass. The CountStrategy/optimizer regressions pass against RocksDB; runtime Grape loading is validated in the foundation.

CI on the updated PR head remains separate verification.

contrueCT and others added 12 commits October 3, 2026 00:09
- Poll the Raft readiness endpoint before client tests
- Accept only HTTP 200 as a ready PD response
- Prevent authentication failures from passing startup checks
- Inherit the PD readiness wait from the foundation branch
- Keep the query semantics changes unchanged
- Preserve existing stack ancestry for downstream pull requests
- Close committed fallback iterators on success and failure
- Assert that count optimization retains the ordering barrier
- Cover cleanup through the primary-key count fallback path
- limit backend count to traversal start scans
- cover repeated scans, input bulk and explicit ids
- document intermediate scan count semantics
@codecov

codecov Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.96491% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 37.15%. Comparing base (0a3e4ae) to head (1a53ed4).

Files with missing lines Patch % Lines
...rg/apache/hugegraph/store/meta/GraphIdManager.java 85.96% 8 Missing and 8 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #3262      +/-   ##
============================================
- Coverage     41.60%   37.15%   -4.46%     
+ Complexity     7320     1741    -5579     
============================================
  Files           794      255     -539     
  Lines         69127    17546   -51581     
  Branches       9258     1894    -7364     
============================================
- Hits          28761     6519   -22242     
+ Misses        37091    10495   -26596     
+ Partials       3275      532    -2743     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- remove Commons from required checks
- require the Java 17 memory Server check
- retain analysis and license checks
- avoid claiming TinkerPop 3.8.1 in the foundation
- describe shared TinkerPop and Kryo access needs
- retain the test and runtime permission boundary
- update both Store dependencies to Fastjson 1.2.84
- align the release license and dependency inventory
- retain the Fastjson 1.x dependency contract
- inherit the Fastjson security patch
- align required checks with Java 17
- retain consistent test JVM guidance
- override the legacy client transitive dependency
- retain the existing client and production versions
- remove obsolete release inventory entries
- inherit the cluster Commons Text override
- retain the query behavior and client version
- align release dependency metadata
imbajin and others added 7 commits October 4, 2026 16:11
Move existing controller cases into the Store test module.
Select the controller class in the core test profile.
Retain the existing validation assertions.
Inherit the controller test relocation.
Select validation cases in the Store core test profile.
Preserve the query semantics changes.
Use the existing Node JUnit Platform provider.
Keep controller tests alongside the Node implementation.
Remove the cross-module core test include.
contrueCT and others added 30 commits October 4, 2026 21:39
Adopt the Java-independent memory test gate.
Use the shared runtime definition with the Java 17 baseline.
Preserve foundation validation and platform contracts.
Carry the version-independent memory test gate.
Keep the Java 17 query validation baseline.
Preserve the reviewed predicate and count fixes.
Cover native ID equality with local text predicates.
Verify counts and pagination with strategies on and off.
Keep ordinary ID lookup routes covered.
Set the Commons and RPC compiler release to 11.
Keep Java 17 as the build and service runtime baseline.
Document the published library compatibility contract.
Manage Ivy consistently across the upgrade series.
Update bundled dependency and licensing metadata.
Verify Gremlin and fresh Grape class loading.
Keep Commons and RPC bytecode compatible with Java 11.
Use patched Ivy with matching release metadata.
Retain the conservative query regression coverage.
Inspect label elements without probing collections for null.
Retain local filtering for predicates containing null labels.
Cover immutable and nullable collections in query regressions.
Filter removed IDs before matching transaction records.
Cover vertex, edge and self-loop reads and re-adds.
Document pending index-update query boundaries.
- Keep one active run per workflow and PR
- Preserve push and manual workflow executions
- Cover CI workflows on this PR target branch
- Keep one active run per workflow and PR
- Preserve push and manual workflow executions
- Cover CI workflows on this PR target branch
- incorporate the upstream Helm deployment chart
- retain the foundation build and dependency changes
- preserve existing Java sources and POM files
- merge the remote PR concurrency updates
- keep the upstream master synchronization
- preserve the validated Java and POM content
- incorporate the upstream Helm deployment chart
- preserve the query and concurrent CI updates
- retain the existing Java sources and POM files
- Cover the newly added Helm workflow
- Keep the latest Helm checks per PR
- Preserve concurrent branch changes
- Cover the newly added Helm workflow
- Keep the latest Helm checks per PR
- Preserve concurrent branch changes
Quote JVM option files and the process build directory.
Load RPC test configuration through a decoded file URI.
Enable Commons tests in the local build script.
Merge validated JVM argument quoting from foundation.
Retain decoded RPC fixture resource paths.
Include the Commons test-enabled local entry point.
Preserve the independently added Helm concurrency rule.
Keep validated test path and resource fixes.
Retain a normal fast-forward push history.
Merge the finalized foundation test setup.
Preserve its independent Helm concurrency update.
Retain validated query behavior and branch history.
Keep the parallel query branch CI commit.
Preserve the finalized test setup and query fixes.
Maintain a normal fast-forward push history.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants