[ColinLeeo]

Description

The Java-backed tests in #16795 exercise dispatch and argument/result handling, but cannot detect failures in real library loading, symbol lookup, or the FFM calling convention. This adds the small native-integration follow-up discussed in #15840, based directly on current main and the native contract from #16812. It can be reviewed independently of #16795.

A scalar C fixture implements six representative functions: byte dot product and cosine, float dot product/distance/cosine, and in-place normalization. The build produces both lucene_-prefixed and unprefixed variants. Native counters and argument observations verify that calls really reached C, including element counts and the normalization flag. The fixture checks lengths before accessing arrays, so an incorrect length fails a Java assertion instead of reading outside a heap array.

Five tests each launch a fresh JVM to cover:

• Real calls through array and static MemorySegment entry points, using heap and off-heap byte segments, empty inputs, and boundary lengths; fractional cosine results and normalization's null-pointer/exception behavior.
• Fallback for missing symbols, and rejection of unprefixed symbols even when the library loads successfully.
• Missing optional and required libraries, and strict-mode failure on a missing prefixed symbol.

-Ptests.nativevector.run=true builds the fixtures with cc on Linux/macOS and enables the tests. -Plucene.tool.cc=<compiler> selects another compiler. Ordinary test runs skip this suite and do not invoke a C compiler. A dedicated workflow runs the integration suite on Linux x86-64. Pull-request and push runs are limited to vectorization code/tests, the native fixtures and C API, and their build/CI dependencies; manual dispatch remains available. The fixture uses scalar C implementations, so CI uses a single architecture.

Production Java and C implementations are unchanged. This is an initial real-library test harness, not complete native coverage: the remaining 16 functions, strict-mode success with a complete library, and Windows fixture builds remain follow-up work. Relates to #15840.

Validation

On macOS AArch64, Temurin 25.0.5+8, with Clang:

./gradlew :lucene:core:test \
  --tests 'org.apache.lucene.internal.vectorization.*' \
  --tests 'org.apache.lucene.util.TestVectorUtil' \
  :lucene:core:check \
  -Ptests.nativevector.run=true \
  -Ptests.defaultvectorization=true -Pvalidation.errorprone=true

• 369 tests, no failures, 17 existing skips; all five integration tests ran. Seed: A6D22205733C9209.
• Default mode also passed with lucene.tool.cc set to a nonexistent compiler: 369 tests, 22 skips (the five integration tests are disabled), with no native compilation tasks run.
• Core checks passed with Error Prone enabled. Repository checks on the changed files passed, including actionlint, zizmor, ast-grep, and spelling/whitespace checks. Repository-wide EditorConfig validation (eclint v0.5.1) and license checks also passed.
• Isolated mutation checks confirmed failures for byte counts passed to float operations, an incorrect byte-cosine return descriptor, ignored null normalization results, missing symbol prefixes, and an incorrect normalization flag. These experiments did not modify the repository's production sources.
• All five child-JVM scenarios also passed when compiled against the implementation from Add Java-backed tests for NativeVectorUtilSupport #16795, confirming compatibility with its lazy native holder.

Workflow validation passed after narrowing the path filters and switching to a single Linux runner, including actionlint, zizmor, EditorConfig, and representative path-inclusion/exclusion checks. The full repository test suite has not been run locally.

Review status

Prepared with Codex assistance under the author's direction. This draft is for review of the native-test approach and its CI/build integration.

[rmuir]

I don't think we should add 2 github actions jobs to every PR/push build for this. Can we please scope it to only relevant globs

[rmuir]

what is the benefit of running these tests on both architectures?

[rmuir]

would use ubuntu-26.04-arm when we don't care. its faster and has significantly less queue time than ubuntu-latest or any of the x86 runners.