Skip to content

feat(cache): RemoveManyAsync and SetManyAsync over many keys, and hash cache fixes - #267

Merged
cosmin-staicu merged 1 commit into
mainfrom
perf/multi-key
Oct 23, 2026
Merged

cosmin-staicu merged 1 commit into
mainfrom
perf/multi-key

Conversation

@cosmin-staicu

@cosmin-staicu cosmin-staicu commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

RemoveManyAsync and SetManyAsync

On ICache, ICache<T>, IHashCache and IHashCache<T>, served by ISpanKeyCache* and ISpanKeyHashCache*, which gain these members and RemoveFieldsAsync with default bodies, so an implementation built against 3.0.0 still loads. Any other implementation, a mock included, falls back to chunked single-key calls. They take an IEnumerable<CacheKey> and return the keys that failed; an empty list means all succeeded.

  • Any slots. One command per key, so a cluster never answers CROSSSLOT.
  • Chunks. InMemoryRedisCacheOptions.BatchChunkSize (500) keys per chunk, BatchConcurrentChunks (2) in flight. A chunk's writes or removes start together, then that chunk's events, so a chunk costs about two round trips and memory stays with the chunk. A failure never stops the rest. The caller's cancellation throws: no command is sent after it, in the chunk or a later one, and what ran stays done.
  • Order per key. Remove drops the local entry, unlinks, then announces, even if the unlink failed (as RemoveAsync does). Set writes, then announces; a failed write gets no event. Both take their place in the key's write order from perf(cache): write a set's value before its event, and an opt-in background L2 write #266, so a background write of the key is never overtaken, and every local write is tracked so an earlier read cannot put an old value back.
  • A key named twice. Within a chunk the key ends as its writes in order would leave it: the last write wins, and hash merges combine in order. Across chunks, which run two at a time, the order is not set.
  • Topic-wide event. Above the opt-in BatchTopicInvalidationThreshold, one TopicInvalidated event replaces the per-key events. It drops the local tier of every cache sharing the topic, since events name no cache. A node on an older version ignores it, so enable it only once every node runs this version. The size is known through Cache<T> and its key strategy too.
  • Background writes. The calls stay awaited under BackgroundDistributedWrite and wait only for keys with a pending background write.

The notify doorbell now keeps one PUBLISH in flight and coalesces the rings after it into one more. One PUBLISH per event overflowed a subscriber's client-output-buffer-limit pubsub at 100k events, and Redis closed the subscription. A PUBLISH stalled on Redis holds the rings behind it for at most one second.

RemoveAsync(CacheKey[]) and the batch SetAsync are unchanged: the batch set is one atomic transaction, which the new path would give up.

Hash cache

  • HashReplace keeps the local hash whole. A HashReplace write left only the written fields in the local tier, so the writing node lost the others until the entry expired. It now merges into the held local hash, as one locked step so two concurrent merges keep both sets of fields; with none held, it stores nothing and the next read fetches the whole hash.
  • GetCacheEntryAsync in one round trip. An EXISTS before the read is gone, as the docs already said.
  • RemoveFieldsAsync removes single fields (HDEL). Peers drop the whole local entry through an ordinary remove event, so older nodes need no new format. The event goes out whenever the delete was sent, since some chunks may have landed even if it failed.
  • Large hashes. A write of more than RedisCacheOptions.HashFieldBatchSize (1000) fields goes in bounded HSET chunks. A KeyReplace write stays atomic and keeps its TTL through a same-slot staging key and RENAME; a hash with no expiry is renamed and persisted in one transaction, so it never lands with the staging key's lifetime; a HashReplace merge is not atomic across chunks (documented). Field reads are split too, and a whole-hash read pages by HSCAN above ten batches on Redis 7+.

Measured (local Docker)

N=1 N=10 N=225
RemoveManyAsync, single node 0.9 ms 1.1 ms 10.2 ms
RemoveManyAsync, cluster 0.9 ms 1.35 ms 10.8 ms
SetManyAsync, cluster 0.9 ms 1.3 ms 11.3 ms
RemoveAsync in a loop 0.9 ms 8.9 ms 217 ms
  • 100k keys: about 23–30k keys/s for both calls on single node and cluster, no failures.
  • A 1M-field hash on a single node: writes at 122k fields/s; the longest server command in SLOWLOG was about 3.5 ms.

Tests

  • Unit: chunk boundaries, a failing middle chunk, cancellation between and within chunks, the in-flight bound, pending background writes, the topic-wide switch and its failure, the size through Cache<T>'s key strategy, a key named twice, an older node's decode, the fallback without the capability; the HashReplace merge, two merges at once; RemoveFieldsAsync; large hashes; the doorbell coalescing and its one-second bound.
  • Redis integration on single node and cluster: 130 keys across slots, the topic-wide drop on a peer, hash set and remove, about two round trips for 300 keys under delayed writes, 20k keys in chunks, the merge, RemoveFieldsAsync, 5000-field hashes, a chunked replace of a hash with no expiry.
  • Three cluster-only integration failures also happen on main against a cluster; they are standalone-only tests.

Follow-up

  • A chunked hash write starts every chunk at once, each a copy of its fields: a 1M-field hash at HashFieldBatchSize 1000 puts about 1000 commands on the wire before any is awaited and doubles its payload in memory. A window of chunks in flight, as BatchConcurrentChunks gives the batch calls, would bound both; it needs its own benchmark, so it is left for a separate change.

@github-actions

Copy link
Copy Markdown

🔎 Maintainer heads-up: automated triage flagged this PR as potentially material, so it may need a signed CLA in addition to the DCO sign-off.

Strong signals

  • adds public API surface (PublicAPI.Unshipped.txt in src/UiPath.Caching.Abstractions, src/UiPath.Caching)

Other signals

  • large production change (+1522 lines under src/)

This is advisory only — the bot does not decide. Please judge against the CLA criteria (material, product-critical, patent-sensitive, corporate contributor, broad commercial use). Note that thresholds can be gamed by splitting PRs, so use your judgement.

  • If a CLA is needed → add the cla-required label (a contributor comment with signing steps is posted automatically).
  • If it is not needed → replace needs-cla-review with cla-not-required so later pushes don't re-flag it.

@github-actions github-actions Bot added the needs-cla-review A maintainer should assess whether a signed CLA is required (see CONTRIBUTING.md) label Oct 21, 2026
@cosmin-staicu
cosmin-staicu force-pushed the perf/background-write branch 6 times, most recently from ec1666d to eb72d20 Compare October 22, 2026 06:14
Base automatically changed from perf/background-write to main October 22, 2026 08:05
@cosmin-staicu
cosmin-staicu force-pushed the perf/multi-key branch 7 times, most recently from 781a66a to a4cb23c Compare October 22, 2026 15:47
@cosmin-staicu cosmin-staicu added cla-not-required Maintainer reviewed: no CLA required for this contribution and removed needs-cla-review A maintainer should assess whether a signed CLA is required (see CONTRIBUTING.md) labels Oct 22, 2026
@cosmin-staicu cosmin-staicu self-assigned this Oct 22, 2026
@cosmin-staicu
cosmin-staicu force-pushed the perf/multi-key branch 2 times, most recently from 4902eb7 to 760637c Compare October 22, 2026 17:04
…h cache fixes

RemoveManyAsync and SetManyAsync, on the single-key and the hash caches, take
any number of keys across any slots and return the ones that failed. Keys go in
chunks of BatchChunkSize (500), BatchConcurrentChunks (2) in flight; a chunk's
commands start together, then its events, so a call costs about two round trips
per chunk and memory stays with the chunk. A failure never stops the rest.
Above the opt-in BatchTopicInvalidationThreshold, one TopicInvalidated event
replaces the per-key events, and goes out whatever ends the batch once a chunk
has started. The calls stay awaited under
BackgroundDistributedWrite. The notify doorbell keeps one PUBLISH in flight and
coalesces the rest, so a large batch cannot overflow a subscriber's buffer.

Hash cache:
- a HashReplace write merges into the local hash instead of replacing it with
  the written fields;
- GetCacheEntryAsync reads in one round trip;
- RemoveFieldsAsync removes single fields; peers drop the whole entry;
- a hash of more than HashFieldBatchSize (1000) fields is written and read in
  bounded commands; a KeyReplace write stays atomic through a same-slot staging
  key and RENAME.

Signed-off-by: Cosmin Staicu <cosmin.staicu@uipath.com>
@sonarqubecloud

Copy link
Copy Markdown

@cosmin-staicu
cosmin-staicu merged commit 2a81c15 into main Oct 23, 2026
10 checks passed
@cosmin-staicu
cosmin-staicu deleted the perf/multi-key branch October 23, 2026 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-not-required Maintainer reviewed: no CLA required for this contribution

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants