Skip to content

*: split ExtractedErrors into MultipleKeyErrors and MultipleRegionErrors - #557

Merged
ti-chi-bot[bot] merged 1 commit into
tikv:masterfrom
getwyrd:error-variants
Oct 19, 2026
Merged

ti-chi-bot[bot] merged 1 commit into
tikv:masterfrom
getwyrd:error-variants

Conversation

@eduralph

@eduralph eduralph commented Jul 25, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #544 — please review the last commit only. Until #544 merges, the diff below also
contains its two-line fix.

Follows @pingyu's suggestion on #544. Thanks for it — the problem turned out to be a bit worse than
redundancy.

Why. ExtractedErrors carried either key errors or region errors, so a caller had to pop an
element and type-test it to learn which it was holding. Worse, which variant reached the caller was
decided by plan shape
, not by what the server reported:

  • no collapsing merge → ExtractError sees the per-shard Vec<Result<_>> and normalizes to
    ExtractedErrors;
  • with merge(CollectSingle) → the merge pops the single Result out first, so MultipleKeyErrors
    becomes the plan's own error and ExtractError re-raises it without ever calling key_errors().

That is exactly the bug #544 fixes, and it is already visible in the public raw API for the same
class of failure:

method plan shape error the user sees today
delete, put_with_ttl retry_multi_region → merge(CollectSingle) → extract_error MultipleKeyErrors
batch_delete, batch_put_with_ttl, delete_range retry_multi_region → extract_error ExtractedErrors

What. ExtractedErrors is removed. ExtractError now reports key errors as MultipleKeyErrors
and region errors as a new MultipleRegionErrors, so the variant names what the server said and is
independent of how the plan was composed. #544's two-variant match arm collapses back to one.

A latent bug this removes. CleanupLocks::execute popped the last error to type-test it and, on
the key-error path, assigned the remaining vec to key_error — so a lone key error was dropped and
key_errors() went on to report Some(vec![]), an error carrying no errors. With the kind known
from the variant, neither arm pops to decide.

One wrinkle the split exposes rather than creates. HasKeyErrors for Result<T, Error> reports
any per-shard Err as a key error — that is how a plan keeping its Vec<Result<_>> carries a hard
failure through. Naming the variant MultipleKeyErrors would then be a lie for a gRPC or
exhausted-retry error, so ExtractError wraps only what the response reported and propagates
anything else unchanged. "Reported by the response" means the two error types the HasKeyErrors
response impls construct: KeyError from a kvrpcpb::KeyError field, and KvError from a raw
endpoint's string error field — both count, since testing for KeyError alone would leave raw
operations plan-shape-dependent, which is the very thing being removed. No current plan composes
retry_multi_region_preserve_results with extract_error, so this changes no existing behaviour; it
keeps the new variant honest for the compositions that could.

Breaking. Error::ExtractedErrors is removed. Callers matching it should match
Error::MultipleKeyErrors (what the server said about the request) or Error::MultipleRegionErrors
(region failures); callers that inspected the payload's type to tell them apart can now match the
variant instead. Per @pingyu's note on #544, the next release looks like a semver-major anyway.

Verification. 4 new tests. One pins the invariant this buys — a response key error surfaces as
MultipleKeyErrors both with and without a collapsing merge, so recomposing a plan can no longer flip
the variant under a distant match arm. One pins the region side on the shape that actually reaches
ExtractError (a plan with no retry layer above it, as in resolve_lock_with_retry, since
RetryableMultiRegion consumes region errors itself). One pins that a hard per-shard error keeps its
own class. One pins that a raw endpoint's string error is classified the same way under either plan
shape. 72 lib tests; make check green; txn/raw/failpoint integration suites green against a local
api-v2 cluster.

Summary by CodeRabbit

  • Bug Fixes

    • Fixed an issue where key errors could be lost during cleanup, so they are reported correctly.
    • Improved recovery from lock and transaction-status failures, including cases where the primary record was never written.
    • Preserved non-key and transport errors instead of reporting them as key errors.
  • Improvements

    • Distinguished key-related errors from region-related errors in reporting.
    • Clarified error documentation and expanded coverage for transactional, raw endpoint, retry, and region failure scenarios.

@ti-chi-bot ti-chi-bot Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the dco. labels Jul 25, 2026
@coderabbitai

coderabbitai Bot commented Jul 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 44c88806-9b9b-4d02-86c2-9c32d48d29ca
📥 Commits

Reviewing files that changed from the base of the PR and between 6908ad3 and ea84837.

📒 Files selected for processing: 3
  • src/common/errors.rs
  • src/request/plan.rs
  • src/transaction/lock.rs

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

📝 Walkthrough

The change replaces Error::ExtractedErrors with separate key and region error variants. Request planning classifies response key errors and propagates non-response failures. Cleanup, lock handling, transaction status checks, and commit retries use the new variants.

Changes: Error variant split

Layer / File(s) Summary
Error variants and extraction classification
src/common/errors.rs, src/request/plan.rs
The error enum adds MultipleRegionErrors and removes ExtractedErrors. ExtractError wraps key errors in MultipleKeyErrors only when all reported errors originate in responses. It propagates the first non-response error unchanged.
Cleanup handling and classification tests
src/request/plan.rs
CleanupLocks handles key and region errors separately and preserves the key-error vector. Tests cover merged and unmerged plans, raw response errors, region errors, and non-response failures.
Transaction error consumers
src/transaction/lock.rs, src/transaction/transaction.rs
Lock resolution and transaction status checks handle the new key and region variants. Commit retry handling matches MultipleKeyErrors.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to ea848

No confirmed issue remains that would prevent merging after normal checks.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 42.86% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 4 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly identifies the main change: replacing ExtractedErrors with separate MultipleKeyErrors and MultipleRegionErrors variants. The leading '*:' is minor noise but does not reduce clarity.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI

Warning

Some tools did not complete. Review the errors below.

🔧 Clippy (1.98.1)

Clippy execution failed

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot ti-chi-bot Bot added contribution This PR is from a community contributor. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 25, 2026
@eduralph
eduralph marked this pull request as ready for review July 25, 2026 22:16
@ti-chi-bot ti-chi-bot Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 25, 2026
@eduralph eduralph closed this Sep 8, 2026
@eduralph eduralph reopened this Sep 8, 2026
`ExtractedErrors` carried either key errors or region errors, so a caller had
to pop an element and type-test it to learn which it was holding. Worse, WHICH
variant reached the caller was decided by plan shape rather than by what the
server reported: without a collapsing merge, `ExtractError` normalized the
per-shard `Vec<Result<_>>` into `ExtractedErrors`; with `CollectSingle`, the
merge popped the single `Result` out first, so `MultipleKeyErrors` became the
plan's own error and `ExtractError` re-raised it without ever calling
`key_errors()`.

That is not a theoretical hazard. It is the bug fixed in the parent commit,
where `check_txn_status` matched only one variant and an orphaned lock whose
primary was never written stayed unresolved forever. It is also visible in the
public raw API today: `delete` and `put_with_ttl` report key errors as
`MultipleKeyErrors` while `batch_delete`, `batch_put_with_ttl` and
`delete_range` report them as `ExtractedErrors`, for the same class of failure
and with nothing in either signature to suggest it.

Remove `ExtractedErrors`. `ExtractError` now reports key errors as
`MultipleKeyErrors` and region errors as a new `MultipleRegionErrors`, so the
variant names what the server said and is independent of how the plan was
composed. The parent commit's two-variant match arm collapses back to one.

This also fixes a leak in `CleanupLocks::execute`, which popped the last error
to type-test it and, on the key-error path, assigned the REMAINING vec to
`key_error` — dropping a lone key error and leaving `key_errors()` to report
`Some(vec![])`, an error carrying no errors. With the kind known from the
variant, neither arm pops to decide.

Breaking: `Error::ExtractedErrors` is removed. Callers matching it should match
`Error::MultipleKeyErrors` (per-key failures) or `Error::MultipleRegionErrors`
(region failures); callers that inspected the payload's type to tell them apart
can now match the variant instead.

One wrinkle the split exposes rather than creates: `HasKeyErrors for
Result<T, Error>` reports ANY per-shard `Err` as a key error, which is how a
plan keeping its `Vec<Result<_>>` carries a hard failure through. Naming the
variant `MultipleKeyErrors` would then be a lie for a gRPC or exhausted-retry
error, so `ExtractError` wraps only what the RESPONSE reported and
propagates anything else unchanged. "Reported by the response" means the two
error types the `HasKeyErrors` response impls construct: `KeyError` from a
`kvrpcpb::KeyError` field, and `KvError` from a raw endpoint's string `error`
field. Both must count — testing for `KeyError` alone would leave raw
operations plan-shape-dependent, which is the very thing being removed. No
current plan composes `retry_multi_region_preserve_results` with
`extract_error`, so this changes no existing behaviour; it keeps the new
variant honest for the compositions that could.

Tests: 4 new. One pins the invariant this buys — a response key error surfaces
as `MultipleKeyErrors` both with and without a collapsing merge, so recomposing
a plan can no longer flip the variant under a distant match arm. The other pins
the region-error side on the shape that actually reaches `ExtractError`: a plan
with no retry layer above it, as in `resolve_lock_with_retry`, since
`RetryableMultiRegion` consumes region errors itself. The third pins that a hard
per-shard error keeps its own class instead of being relabelled, and the fourth
that a raw endpoint's string error is classified the same way under either plan
shape. 72 lib tests green; txn/raw/failpoint integration suites green against a local api-v2
cluster.

Signed-off-by: Eduard R. <eduard@ralphovi.net>

@pingyu pingyu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM~

@ti-chi-bot ti-chi-bot Bot added needs-1-more-lgtm Indicates a PR needs 1 more LGTM. approved labels Oct 18, 2026
@pingyu

pingyu commented Oct 18, 2026

Copy link
Copy Markdown
Collaborator

/cc @ekexium

@ti-chi-bot
ti-chi-bot Bot requested a review from ekexium October 18, 2026 07:24
@ti-chi-bot ti-chi-bot Bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Oct 19, 2026
@ti-chi-bot

ti-chi-bot Bot commented Oct 19, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ekexium, pingyu

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot

ti-chi-bot Bot commented Oct 19, 2026

Copy link
Copy Markdown

[LGTM Timeline notifier]

Timeline:

  • 2026-10-18 07:23:04.902723978 +0000 UTC m=+1147794.649954847: ☑️ agreed by pingyu.
  • 2026-10-19 01:54:21.008971221 +0000 UTC m=+1214470.756202140: ☑️ agreed by ekexium.

@ti-chi-bot
ti-chi-bot Bot merged commit d98fb52 into tikv:master Oct 19, 2026
8 checks passed
@eduralph
eduralph deleted the error-variants branch October 20, 2026 11:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved contribution This PR is from a community contributor. dco-signoff: yes Indicates the PR's author has signed the dco. lgtm size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants