[syahra712]

Description

raw-request reports every request as successful, whatever Solr answers.

SolrClient.raw_request returns the requests.Response without checking the status. SolrSearch calls resp.raise_for_status() itself for this reason, but RawRequest doesn't. It also isn't a SolrRunner, so it has no error translation. A 400, 404 or 500 comes back as {"weight": 1, "http-status": 500, ...} with no success key, and execute_single then records it as a success. The error rate stays at 0 and the failing request's latency is reported as normal.

On solr:9, GET /api/collections/nope and POST /api/collections with an invalid body both return 500, and both were reported as successful operations.

This also affects converted workloads. convert-workload keeps raw-request operations as they are, so the converted elb_logs workload still sends POST /elb_logs/_flush and POST /_plugins/_ppl (the t1-* PPL tasks). Solr answers those with 404, and today they are timed as successful requests.

The fix:

• RawRequest now extends SolrRunner and calls resp.raise_for_status().
• The existing decorator turns the HTTPError into a BenchmarkTransportError, or BenchmarkNotFoundError for a 404.
• The sample is then recorded as success: False with http-status and an error description, the same as the other runners. With the default --on-error continue the run carries on.

Successful requests are unchanged. One behaviour change to note: workloads that send OpenSearch-only paths through raw-request will now show those tasks as errors instead of hiding them.

Issues Resolved

None filed. I found it while reading the runner.

Testing

• New functionality includes testing

New TestRawRequest in tests/unit/solr/test_runner.py:

• A 200 still returns http-status and weight.
• A 500 raises BenchmarkTransportError with status_code 500.
• A 404 raises BenchmarkNotFoundError.

The 500 and 404 tests fail on main.

I also ran it against solr:9 through execute_single with on_error="continue":

• GET /api/collections/nope now gives {'success': False, 'error-type': 'transport', 'http-status': 500, ...} with 0 ops, where it used to be a success.
• GET /api/collections is still success: True.

The full tests/ suite passes locally on Python 3.13 (1178 passed, 5 skipped), and so does ruff check ..

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[Copilot]

🟢 Approval recommended

No active review issues are recorded for the change aligning raw-request HTTP error reporting with other Solr runners.

No open issues 🎉

What changed in this PR

This PR makes raw-request report HTTP errors as failed benchmark requests, aligning its behavior with other Solr runners instead of understating error rates.

Changes:

• Makes RawRequest inherit from SolrRunner and check response status, enabling existing error translation for 4xx and 5xx responses.
• Adds tests for successful 200 responses and exception translation for 500 and 404 responses.

File	Description
tests/​unit/​solr/​test_runner.py	Adds regression tests for raw-request success and HTTP error handling.
solrorbit/​worker_coordinator/​runner.py	Routes raw-request HTTP failures through standard Solr error handling and documents the behavior.

_{Review effort: Balanced}

[janhoy]

Thanks!