SOLR-18457: Migrate V2 /update endpoints to using JAX-RS by epugh · Pull Request #4177 · apache/solr

epugh · 2026-03-02T18:11:07Z

https://issues.apache.org/jira/browse/SOLR-18457

Summary

Migrate the V2 update APIs to JAX-RS while preserving the legacy update handler's indexing behavior. The new API contract exposes the supported update formats, query parameters, request bodies, and typed versioned responses.

API changes

Add the JAX-RS UpdateApi contract and Jersey UpdateAPI implementation.
Support the generic V2 update endpoint with content-type driven format detection for JSON, XML, CSV, Javabin, and CBOR.
Add explicit V2 format paths:
- /api/collections/{collection}/update/json for document-only JSON indexing
- /api/collections/{collection}/update/xml for XML updates
- /api/collections/{collection}/update/csv for CSV updates
- /api/collections/{collection}/update/javabin for Javabin updates
Declare the supported query parameters in the OpenAPI contract, including commit, commitWithin, overwrite, softCommit, and versions.
Define the request bodies as streamed update content selected by the Content-Type header.
Return a strongly typed V2 response with typed adds, deletes, and deleteByQuery version entries.
Preserve response decoration and replication metadata while avoiding duplicate response headers.
Preserve the /c/{collection} alias when routing requests through JAX-RS.

The implementation delegates update processing to the existing UpdateRequestHandler, so V1 and V2 continue to share the same loaders, update processors, routing, and commit behavior. The generic /update endpoint accepts JSON update commands; the /update/json path is the document-only JSON convenience endpoint.

Documentation and Admin UI

Consolidate V1 and V2 update documentation in indexing-with-update-handlers.adoc, using V1/V2 examples for JSON, XML, CSV, Javabin, versioned responses, and common update parameters.
Update partial-document updates and custom JSON transformation examples for both APIs.
Document the typed V2 version response and the versions=true behavior.
Update the classic Admin UI Documents screen to submit JSON through the V2 update API and simplify its raw command flow.
Update the sample techproducts schema for nested documents and in-place updates.

Testing

Replace the old mapping test with UpdateAPITest, an integration test using SolrJettyTestRule.
Verify V1/V2 parity across supported update formats.
Verify custom JSON transformation and indexing through V2.
Verify assigned versions, typed versioned responses, XML response headers, and the generic update endpoint.
Run the reference guide build and the repository check excluding tests.

@operation

- Add UpdateApi JAX-RS interface in solr/api module with @path, @post, @operation annotations for /update, /update/json, /update/xml, /update/csv, /update/bin endpoints - Rewrite UpdateAPI as a JerseyResource implementing UpdateApi; delegates to UpdateRequestHandler; adds UpdateRequestHandlerConfig inner class (APIConfigProvider.APIConfig) for injection - Update V2UpdateRequestHandler to implement APIConfigProvider and use getJerseyResources() instead of getApis() - Replace mock-based V2UpdateAPIMappingTest with UpdateAPITest integration test using SolrJettyTestRule Co-authored-by: epugh <22395+epugh@users.noreply.github.com>

Resolves failing test: NodeHealthAPITest2.testLegacyMode_WithoutMaxGenerationLagReturnsOk The test verifies that the V2 GET /api/node/health endpoint returns status=OK in legacy (standalone, non-ZooKeeper) mode when maxGenerationLag is not specified in the request. This exercises the null-maxGenerationLag code path in HealthCheckHandler.healthCheckLegacyMode() which immediately returns OK without checking replication lag. Test uses SolrJettyTestRule with no ZooKeeper to run in legacy mode. Co-authored-by: epugh <22395+epugh@users.noreply.github.com>

…n V2Request update/json The V2Request("/c/{collection}/update/json") call was broken after the UpdateAPI migration to JAX-RS: the JAX-RS path regex "cores|collections" doesn't match the "/c/" alias. Replace with the proper SolrJ cloudClient.add() (which goes through the v1 /update path) and update span name/collection assertions to match. Co-authored-by: epugh <22395+epugh@users.noreply.github.com>

Normalize /c/ → /collections/ in ContainerRequestUtils.getRequestUri() before Jersey sees the URI, fixing JAX-RS path matching for all endpoints that use INDEX_PATH_PREFIX (update, select, schema, etc.) with the /c/ shorthand. The tracing span name correctly preserves /c/{collection}/... since it is computed from V2HttpCall.pathSegments which use the original request path. Co-authored-by: epugh <22395+epugh@users.noreply.github.com>

…o UpdateRequestHandler DistributedZkUpdateProcessor.handleReplicationFactor() calls rsp.getResponseHeader().add(...), which requires the response header to be initialized by SolrCore.preDecorateResponse(). In the JAX-RS path, this was never called (only the old Api path calls it in executeCoreRequest). Add pre/postDecorateResponse calls around the updateRequestHandler.handleRequest() call in UpdateAPI.handleUpdate(), mirroring what SolrCore.execute() does for v1. Co-authored-by: epugh <22395+epugh@users.noreply.github.com>

Co-authored-by: epugh <22395+epugh@users.noreply.github.com>

Copilot

Pull request overview

This PR adds Ref Guide documentation for Solr’s V2 update endpoints and (in the same change set) migrates the V2 /update implementation to a JAX-RS resource, including URI normalization to support the /c/ collection shorthand in Jersey routing.

Changes:

Added a new Ref Guide page documenting V2 update endpoints and their behavior vs V1 update handlers.
Updated indexing guide navigation to include and cross-reference the new V2 update documentation.
Switched V2 update handling to a Jersey resource + added /c/ → /collections/ URI normalization, along with updated tests and a new OpenAPI endpoint interface.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
solr/solr-ref-guide/modules/indexing-guide/pages/indexing-with-v2-apis.adoc	New V2 update API documentation and V1↔V2 comparison.
solr/solr-ref-guide/modules/indexing-guide/pages/indexing-with-update-handlers.adoc	Cross-reference note + page-children update for new V2 page.
solr/solr-ref-guide/modules/indexing-guide/indexing-nav.adoc	Adds the new V2 page to the indexing sidebar nav.
solr/core/src/java/org/apache/solr/jersey/container/ContainerRequestUtils.java	Normalizes `/c/` collection shorthand for Jersey path matching.
solr/core/src/java/org/apache/solr/handler/admin/api/UpdateAPI.java	New Jersey-based V2 update resource delegating to `UpdateRequestHandler`.
solr/core/src/java/org/apache/solr/handler/V2UpdateRequestHandler.java	Registers `UpdateAPI` as a Jersey resource and provides config injection.
solr/api/src/java/org/apache/solr/client/api/endpoint/UpdateApi.java	New OpenAPI/JAX-RS endpoint interface for v2 update paths.
solr/core/src/test/org/apache/solr/handler/V2UpdateAPIMappingTest.java	Removes old unit test for v2→v1 update path rewriting.
solr/core/src/test/org/apache/solr/handler.admin.api/UpdateAPITest.java	Adds integration tests covering V2 update endpoints.

Comments suppressed due to low confidence (1)

solr/solr-ref-guide/modules/indexing-guide/pages/indexing-with-v2-apis.adoc:136

Capitalization of “Javabin” here (and in the table / comparison section) is inconsistent with the rest of the ref guide, which uses “JavaBin”. Please standardize on “JavaBin” (while keeping the MIME type application/javabin as-is).

== Javabin Document Indexing

The v2 `/update/bin` endpoint accepts documents in Javabin format, which is the native binary format used by SolrJ.
This endpoint is primarily intended for use by SolrJ clients.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…-v2-apis.adoc Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

epugh

This guy is tough!

janhoy · 2026-09-11T22:52:19Z

I hit an issue trying to do atomic update (JSON format) using v2 /update api. Does not work, and SOLR-11265 has been open for 9 years. Then I found this PR which looks promising in making some decent decisions for how the V2 JAX-RS update APIs should work.

I'm leaning towards making the base /api/collections/{collection}/update endpoint support multiple input formats (much like for v1). The HTTP standard supports this through Content-Type already. That way clients can be explicit in what they send and solr never need to doubt. We can also make extension content-types for solr e.g. for the "commands" JSON syntax:

Content-Type	Behavior
`application/json`, `text/csv`, `application/xml`	Straightforward, same as today's sub-paths
`application/javabin` or `application/vnd.solr.javabin`	SolrJ's native binary format, already used internally, just exposed as a `Content-Type` on the main endpoint instead of only `/update/bin`
`application/x-ndjson`	One JSON doc per line, for streaming bulk loads without a top-level array
`application/vnd.solr.commands+json`	Top-level JSON parsed as the v1-style command envelope (`add`/`delete`/`commit`), restoring what this PR's table marks as unsupported in v2
`application/vnd.solr.atomic+json`	Tells Solr that `{"set": ...}`, `{"add": ...}`, `{"inc": ...}` are atomic-update operators, not literal field structure — directly fixes SOLR-11265 by making intent explicit instead of guessing

Example:

curl -X POST 'http://localhost:8983/api/collections/mycollection/update?commit=true' \
  -H 'Content-Type: application/vnd.solr.atomic+json' \
  --data-binary '{"id": "mydoc", "price": {"set": 99}}'

Not entirely sure whether atomic-syntax should be supported in normal application/json, I think it is problematic since we also need to support nested child docs and that gets ambigous. So an explicit application/vnd.solr.atomic+json would let Solr know what to expect. But then if you want atomic update of a deeply nested JSON -- well, what to do? Time to namespace the atomic qualifiers to avoid confusion, e.g. inc becomes _inc_, set becomes _set_ or similar.

epugh · 2026-09-12T10:12:25Z

Not entirely sure whether atomic-syntax should be supported in normal application/json, I think it is problematic since we also need to support nested child docs and that gets ambigous. So an explicit application/vnd.solr.atomic+json would let Solr know what to expect. But then if you want atomic update of a deeply nested JSON -- well, what to do? Time to namespace the atomic qualifiers to avoid confusion, e.g. inc becomes _inc_, set becomes _set_ or similar.

I think with the V2 we shoudn't be constrained to V1 ideas.. I would extend that to the syntax we use in the payload... If there is a better way of structuring the JSON (or other payloads being sent) then we should do it. Now is the time to make a big change to our indexing patterns. Having said that, this PR ended up starting to be a big job, so that is why it's DRAFT on hold. But I would love any improvements you make to this PR!

janhoy · 2026-09-12T13:35:22Z

I think with the V2 we shoudn't be constrained to V1 ideas.

That's a good point, in V2 we can also change the atomic update contract to make it NOT ambigoous. Maybe there is a genuinely better way, maybe something refencing a field with jsonPath and the operation with some operation object?

janhoy · 2026-09-13T23:41:58Z

I did #4910 to test the ND-JSON idea. No atomic support, keep it simple.

epugh · 2026-09-25T16:03:23Z

@janhoy I am going to address your feedback, and try to do them as cohesive commits, that way, if we want to pull any of them into brand NEW PR's so that this current PR can be mergable, we can. I know I struggle with scope creep a lot. Especially wihtout a good sense of "can I record "and this doesn't work"" and still make this a legit migration.

Also, if when you see my next steps, you think "Lets break this up into X PR's" we can... If we want the js changes for example to be in their own PR.

epugh · 2026-09-25T16:27:47Z

Two asks before merge:

partial-document-updates.adoc should say explicitly that atomic syntax works on /update only, and that /update/json will silently destroy the document.

Worth a follow-up JIRA to make /update/json reject atomic-operator maps rather than index them.

The single-object-vs-array asymmetry on /update is also worth smoothing over eventually — {"id":...} failing with Unknown command 'id' is a rough first experience.

I dealt with this in ac83841

epugh · 2026-09-25T16:32:05Z

Or we keep the adds, deletes, deleteByQuery:

My friend Claude has this to say about touched_docs vs keeping the three: I'd keep adds / deletes / deleteByQuery rather than collapsing to touched_docs. They aren't redundant: a delete-by-query key is a query, not an id, and merging them loses that distinction — which is exactly why the PR needed both VersionedDocument and VersionedQuery.

Love it! Thanks for thinking about it.

…gh/solr into copilot/migrate-update-api-to-jax-rs

…ong typed responses. We may need to think about how update model objects have some sort of "escape" for when random third party Update Processors ADD to the repsonse. Rihgt now in this PR, we actually add strong typing.... Which may be fine, but what if we have processors dynamically loaded?

…out everything but metrics Committing as single chunk for furthur reveiw and feedback.

janhoy · 2026-09-26T00:20:27Z

If we want the js changes for example to be in their own PR.

I'd say yes to this. When we approach completion, ask LLM to extract all frontend edits into its own branch/PR, stacked on top of this one, and commit separately.

That said, I do appreciate using UI changes as a test bench for actually testing the generated JS client and vetting the API changes, that is how it should be done.

epugh · 2026-09-27T17:38:59Z

If we want the js changes for example to be in their own PR.

I'd say yes to this. When we approach completion, ask LLM to extract all frontend edits into its own branch/PR, stacked on top of this one, and commit separately.

That said, I do appreciate using UI changes as a test bench for actually testing the generated JS client and vetting the API changes, that is how it should be done.

in the vein, we could also pull out the use of techproducts in the examples into their own PR as well...

Or you give a +1 and we merge this ;-)

janhoy · 2026-11-10T19:39:33Z

@epugh I'm going to push a commit bringing this PR up to date with main, so we can continue working on it. I will likely merge #4910 first since that is ready, but the conflict is not that complex. Also, there are 10 unresolved review comments in this PR, do you want to close the ones that are truly resolved?

# Conflicts: # solr/solr-ref-guide/modules/indexing-guide/pages/indexing-with-update-handlers.adoc # solr/solr-ref-guide/modules/indexing-guide/pages/partial-document-updates.adoc # solr/solr-ref-guide/modules/indexing-guide/pages/transforming-and-indexing-custom-json.adoc # solr/webapp/web/js/angular/controllers/core-overview.js # solr/webapp/web/js/angular/controllers/cores.js # solr/webapp/web/js/angular/services.js

…ateTool The generated SolrJ request gained a contentType argument on this branch, and bin/solr create started using it on main (SOLR-18320).

…entions {collectionName} placeholders, quoted URLs, --data-binary for update payloads, and one command per line, as standardized on main by SOLR-18460, SOLR-18495 and SOLR-18500.

… pass the content type

janhoy · 2026-11-10T21:08:20Z

🤖 Claude Fable 5 merged upstream/main into this branch (220 commits behind) and resolved the conflicts. Follow-up commits on top of the merge:

Conflicts resolved in the merge commit

indexing-with-update-handlers.adoc, partial-document-updates.adoc, transforming-and-indexing-custom-json.adoc (20 hunks): this PR rebuilt the examples into V1/V2 tab blocks while main standardized every curl example (SOLR-18460, SOLR-18495, SOLR-18500). Took this PR's structure in every hunk; the text main had edited in place (posting XML from a file, response format) already lives elsewhere in the PR's restructured pages.
core-overview.js, cores.js: took main's updated injection lists ($timeout, ReplicationV2, NodeTarget, ApiErrorHandler) minus the Update service this PR removes.
services.js: dropped the Update $resource factory as this PR does, kept main's new ParamSetsV2 factory next to it.

Semantic conflicts, 8afa6bd and the commit after it: the api.mustache change on this branch gives every generated multi-content-type request a contentType argument. Callers added on main since this PR branched (bin/solr create from SOLR-18320, TestDistribFileStore, ConfigsetPinnedPackageVersionTest) now pass application/zip or application/octet-stream, matching this PR's own adaptations. UpdateAPIMetricsDoubleCountingTest imports ContentStreamBase from its new home in solr-core (SOLR-18351).

Curl conventions, b98bc2f: swept the three pages above to main's current style: {collectionName} placeholders, quoted URLs, --data-binary for update payloads, one command per line. Mechanical, no wording changes.

Validated locally with compileTestJava, the Admin UI browser suite (-Ptests.selenium=true) and the Ref Guide build with link check.

github-actions · 2026-11-10T21:28:14Z

Automated PR readiness judgements (LLM-generated; may be wrong; not updated on later pushes).

✅ Jira issue in title: already present
✅ Changelog: type: changed
✅ Reference Guide update: included
⚠️ Upgrade notes — missing (confidence: medium): The V2 javabin endpoint path changes from /update/bin to /update/javabin, and the Admin UI no longer supports targeting custom update handlers, which are backwards-incompatible changes for users relying on the old paths or UI behavior.
✅ Tests: included

janhoy

Quick browse through all files...

janhoy · 2026-11-10T21:10:50Z

      throws SolrServerException, IOException {
    try (final var stream = new ByteBufferInputStream(buffer)) {
-      final var uploadReq = new FileStoreApi.UploadFile(name, stream);
+      final var uploadReq = new FileStoreApi.UploadFile(name, stream, "application/octet-stream");


Use a constant in place of string?

janhoy · 2026-11-10T21:11:09Z

      var uploadReq =
          new ConfigsetsApi.UploadConfigSet(
-              confName, new ByteArrayInputStream(Utils.zipDirectory(confPath)));
+              confName, new ByteArrayInputStream(Utils.zipDirectory(confPath)), "application/zip");


Constant instead of string? Either from Jetty or from JDK

janhoy · 2026-11-10T21:29:45Z

        // Create and execute the upload request using generated client
        final var uploadRequest =
-            new ConfigsetsApi.UploadConfigSet(configSetName + suffix, fileBytes);
+            new ConfigsetsApi.UploadConfigSet(configSetName + suffix, fileBytes, "application/zip");


Constant? Also line 1283

janhoy · 2026-11-10T21:30:02Z

-    final var uploadReq = new FileStoreApi.UploadFile(name, new ByteBufferInputStream(buffer));
+    final var uploadReq =
+        new FileStoreApi.UploadFile(
+            name, new ByteBufferInputStream(buffer), "application/octet-stream");


janhoy · 2026-11-10T21:37:33Z

 Increments or decrements a numeric field's value by a specific amount, specified as a single integer or float.
 Positive amounts increment the field's value, and negative decrement.

+NOTE: Atomic update syntax is supported exclusively on the `/update` endpoint. 


I don't remember why this was. And is it precise to say "exclusively on the /update endpoint", when the /update endpoint now is a dispatcher to various sub handlers, which behave quite differently. It is only the json one with Content-Type: application/json that handles atomic update???

janhoy · 2026-11-10T21:41:27Z

-[source,bash]
----
-curl 'http://localhost:8983/api/collections/techproducts/update/json/docs'\
+curl 'http://localhost:8983/api/collections/techproducts/update/json'\


It's a pity that v2 paths changelike this between v1 and v2 but I guess we considered all alternatives...

Copilot AI and others added 14 commits March 1, 2026 13:45

Initial plan

7b437c7

code warning cleanups

3cc0acc

Lint

e40d417

We didn't need this.

92d70a8

Properly route reqeusts by returning an empty getApis

74bdf15

current v2 doesnt support multiple commands

6a675bf

Relocate test to be in the same package as what we are testing.

4d2cbc0

Add indexing-with-v2-apis.adoc documenting V2 update API endpoints

7337062

Co-authored-by: epugh <22395+epugh@users.noreply.github.com>

Tweak formatting

b932cb2

github-actions Bot added documentation Improvements or additions to documentation tests cat:api labels Mar 2, 2026

epugh requested a review from Copilot March 2, 2026 18:15

epugh changed the title ~~docs: Add indexing-with-v2-apis.adoc for V2 /update API endpoints~~ Migrate V2 /update endpoints to using JAX-RS Mar 2, 2026

Copilot started reviewing on behalf of epugh March 2, 2026 18:16

View session

Copilot AI reviewed Mar 2, 2026

View reviewed changes

epugh and others added 2 commits March 3, 2026 08:18

Update solr/solr-ref-guide/modules/indexing-guide/pages/indexing-with…

055f200

…-v2-apis.adoc Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

relocate via intellij went werid

296f33a

epugh commented Mar 3, 2026

View reviewed changes

Comment thread solr/solr-ref-guide/modules/indexing-guide/pages/indexing-with-v2-apis.adoc Resolved

janhoy mentioned this pull request Sep 14, 2026

SOLR-18445 Let /update handler support JSONL / NDJSON #4910

Open

Deal with atomic updates that we don't support with an error.

ac83841

github-actions Bot added the cat:index label Sep 25, 2026

epugh added 5 commits September 25, 2026 09:42

Deal with atomic updates that we don't support with an error.

ba5bb51

Merge branch 'copilot/migrate-update-api-to-jax-rs' of github.com:epu…

ddffebb

…gh/solr into copilot/migrate-update-api-to-jax-rs

Deal with v2 api "re-entering twice" the V1 handleRequest by pulling …

675e724

…out everything but metrics Committing as single chunk for furthur reveiw and feedback.

Revamp openapi generation to not assume content-type

8a51c10

github-actions Bot added cat:cloud cat:packagemanager labels Sep 25, 2026

epugh added 4 commits September 25, 2026 13:36

I hacked up not realizing the tail part!

3198bb2

Deal with wt behavior internally.

4fe0bb2

I suppose...

576a0c7

Much better changelog!

ea3d27b

lint

d681a31

janhoy marked this pull request as draft November 10, 2026 19:38

janhoy added 4 commits November 10, 2026 20:40

Pass the content type to the generated UploadConfigSet request in Cre…

8afa6bd

…ateTool The generated SolrJ request gained a contentType argument on this branch, and bin/solr create started using it on main (SOLR-18320).

Ref Guide: align the V1/V2 update examples with the current curl conv…

b98bc2f

…entions {collectionName} placeholders, quoted URLs, --data-binary for update payloads, and one command per line, as standardized on main by SOLR-18460, SOLR-18495 and SOLR-18500.

Follow main: ContentStreamBase moved to solr-core, UploadFile callers…

107a5a9

… pass the content type

janhoy marked this pull request as ready for review November 10, 2026 21:08

github-actions Bot added the cat:cli label Nov 10, 2026

janhoy reviewed Nov 10, 2026

View reviewed changes

Conversation

epugh commented Mar 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

API changes

Documentation and Admin UI

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

epugh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

janhoy commented Sep 11, 2026

Uh oh!

epugh commented Sep 12, 2026

Uh oh!

janhoy commented Sep 12, 2026

Uh oh!

janhoy commented Sep 13, 2026

Uh oh!

epugh commented Sep 25, 2026

Uh oh!

epugh commented Sep 25, 2026

Uh oh!

epugh commented Sep 25, 2026

Uh oh!

janhoy commented Sep 26, 2026

Uh oh!

epugh commented Sep 27, 2026

Uh oh!

janhoy commented Nov 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

janhoy commented Nov 10, 2026

Uh oh!

github-actions Bot commented Nov 10, 2026

Uh oh!

janhoy left a comment

Choose a reason for hiding this comment

Uh oh!

janhoy Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

janhoy Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

janhoy Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

janhoy Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

janhoy Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

janhoy Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

epugh commented Mar 2, 2026 •

edited

Loading

janhoy commented Nov 10, 2026 •

edited

Loading