Skip to content

transaction: add a test-utils probe that drives 2PC one phase at a time - #577

Draft
eduralph wants to merge 1 commit into
tikv:masterfrom
getwyrd:test-probes
Draft

eduralph wants to merge 1 commit into
tikv:masterfrom
getwyrd:test-probes

Conversation

@eduralph

Copy link
Copy Markdown
Contributor

Draft: opened early for testing and review. This is new public API, behind a feature flag.

What

A test-utils Cargo feature that adds Transaction::probe(), returning a TransactionProbe that drives a transaction's two-phase commit one phase at a time:

  • set_primary_key(key): use this primary, which need not be a key the transaction writes;
  • set_lock_ttl(ms): prewrite with this lock TTL instead of the computed one;
  • prewrite(keys): prewrite only these keys (each must already be written in the transaction), and stop;
  • commit_primary(): commit the primary alone, leaving the secondaries locked.

Nothing the probe does starts the heartbeat, commits secondaries or rolls back, so dropping the transaction afterwards leaves exactly the locks a crashed client would.

Safety rules the probe keeps:

  • It drives two-phase commit only: a transaction with 1PC or async commit is refused, since there the prewrite is the commit point.
  • commit_primary goes through commit_primary_with_retry, so a commit ts TiKV rejects as too small is retried as in commit().
  • Once commit_primary has sent the commit, rollback() is refused unless TiKV cleanly rejected it: the primary may be durable, and rolling back its secondaries would tear the transaction. A lost answer comes back as UndeterminedError, as from commit().

Without the feature nothing changes. With it, the only change outside the probe is that Committer takes an optional lock TTL override; None keeps the computed TTL as before (unit test without_the_probe_the_lock_ttl_is_still_computed).

Why

client-go exports the same capability from a non-test file (CommitterProbe in txnkv/transaction/test_probe.go), and its own tests construct lock states with it directly. client-rust has no equivalent: its mocks are #[cfg(test)], so a caller (including this repo's integration tests) cannot build those states without races or compile-time failpoints. For example, an orphaned secondary lock (primary never written) currently needs a region split and a racing transaction to manufacture; with the probe it is:

txn.put(secondary, value).await?;
let mut probe = txn.probe();
probe.set_primary_key(primary).set_lock_ttl(1000);
probe.prewrite([secondary]).await?;
drop(txn); // the crash

That makes lock-resolution behaviour (#543, fixed in #561) testable deterministically, and the same holds for roll-forward of a committed primary and, later, async-commit locks.

Tests

  • Unit tests against the mock PD: the probe sends exactly one prewrite with the named primary, keys and TTL and nothing else; it refuses a key the transaction did not write, and 1PC or async-commit transactions, without sending anything; commit_primary commits only the primary and closes rollback; a lost commit answer is UndeterminedError and keeps rollback closed; a clean rejection leaves rollback open; the normal commit path still computes its TTL.
  • Integration tests (integration-tests + test-utils): txn_probe_leaves_an_orphaned_secondary_lock (scan finds the one orphan lock; after its TTL a reader resolves it as rolled back) and txn_probe_commit_primary_leaves_the_secondary_to_roll_forward (only the secondary stays locked; a reader rolls it forward).
  • make check, make unit-test and make integration-test-txn enable test-utils.

Open questions for review

  • Shape. A probe borrowed from the transaction keeps the API small and mirrors CommitterProbe. Alternatives: methods directly on Transaction behind the feature, or a probe on the client. Happy to change.
  • Size. This can be split into two PRs if preferred: first the Committer TTL override (no new API), then the feature-gated probe.
  • Scope. Only what lock-resolution tests need today. Other client-go probes (commit ts control, async-commit fallback, region split helpers) can follow as tests need them.
  • Noticed, not changed here. Transaction::commit() leaves the status at StartedCommit when the primary commit is undetermined, so a following rollback() is allowed there too. The probe closes that for itself; the same change in commit() seems worth its own PR.

Behind a new test-utils feature, Transaction::probe() returns a
TransactionProbe that can set the primary key (one the transaction need not
write), set an exact lock TTL, prewrite only chosen keys, and commit the
primary alone. Nothing it does starts the heartbeat, commits secondaries or
rolls back, so dropping the transaction afterwards leaves the locks a crashed
client would. It mirrors client-go's CommitterProbe.

Without the feature nothing changes. With it, Committer takes an optional lock
TTL override; None keeps the computed TTL.

Signed-off-by: Eduard Ralph <eduard@ralphovi.net>
@ti-chi-bot ti-chi-bot Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the dco. labels Oct 20, 2026
@ti-chi-bot

ti-chi-bot Bot commented Oct 20, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign marsishandsome for approval. For more information see the Code Review Process.
Please ensure that each of them provides their approval before proceeding.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@coderabbitai

coderabbitai Bot commented Oct 20, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ti-chi-bot ti-chi-bot Bot added contribution This PR is from a community contributor. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contribution This PR is from a community contributor. dco-signoff: yes Indicates the PR's author has signed the dco. do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant