Repository navigation
feat(cache): RemoveManyAsync and SetManyAsync over many keys, and hash cache fixes - #267
Merged
Merged
Conversation
cosmin-staicu
requested review from
alinahornet,
cosminvlad,
litheon,
lucianaparaschivei and
razvalex
as code owners
October 21, 2026 12:31
|
🔎 Maintainer heads-up: automated triage flagged this PR as potentially material, so it may need a signed CLA in addition to the DCO sign-off. Strong signals
Other signals
This is advisory only — the bot does not decide. Please judge against the CLA criteria (material, product-critical, patent-sensitive, corporate contributor, broad commercial use). Note that thresholds can be gamed by splitting PRs, so use your judgement.
|
cosmin-staicu
force-pushed
the
perf/background-write
branch
6 times, most recently
from
October 22, 2026 06:14
ec1666d to
eb72d20
Compare
cosmin-staicu
force-pushed
the
perf/multi-key
branch
7 times, most recently
from
October 22, 2026 15:47
781a66a to
a4cb23c
Compare
cosmin-staicu
force-pushed
the
perf/multi-key
branch
2 times, most recently
from
October 22, 2026 17:04
4902eb7 to
760637c
Compare
…h cache fixes RemoveManyAsync and SetManyAsync, on the single-key and the hash caches, take any number of keys across any slots and return the ones that failed. Keys go in chunks of BatchChunkSize (500), BatchConcurrentChunks (2) in flight; a chunk's commands start together, then its events, so a call costs about two round trips per chunk and memory stays with the chunk. A failure never stops the rest. Above the opt-in BatchTopicInvalidationThreshold, one TopicInvalidated event replaces the per-key events, and goes out whatever ends the batch once a chunk has started. The calls stay awaited under BackgroundDistributedWrite. The notify doorbell keeps one PUBLISH in flight and coalesces the rest, so a large batch cannot overflow a subscriber's buffer. Hash cache: - a HashReplace write merges into the local hash instead of replacing it with the written fields; - GetCacheEntryAsync reads in one round trip; - RemoveFieldsAsync removes single fields; peers drop the whole entry; - a hash of more than HashFieldBatchSize (1000) fields is written and read in bounded commands; a KeyReplace write stays atomic through a same-slot staging key and RENAME. Signed-off-by: Cosmin Staicu <cosmin.staicu@uipath.com>
cosmin-staicu
force-pushed
the
perf/multi-key
branch
from
October 22, 2026 17:23
760637c to
da0ada1
Compare
|
razvalex
approved these changes
Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



RemoveManyAsync and SetManyAsync
On
ICache,ICache<T>,IHashCacheandIHashCache<T>, served byISpanKeyCache*andISpanKeyHashCache*, which gain these members andRemoveFieldsAsyncwith default bodies, so an implementation built against 3.0.0 still loads. Any other implementation, a mock included, falls back to chunked single-key calls. They take anIEnumerable<CacheKey>and return the keys that failed; an empty list means all succeeded.CROSSSLOT.InMemoryRedisCacheOptions.BatchChunkSize(500) keys per chunk,BatchConcurrentChunks(2) in flight. A chunk's writes or removes start together, then that chunk's events, so a chunk costs about two round trips and memory stays with the chunk. A failure never stops the rest. The caller's cancellation throws: no command is sent after it, in the chunk or a later one, and what ran stays done.RemoveAsyncdoes). Set writes, then announces; a failed write gets no event. Both take their place in the key's write order from perf(cache): write a set's value before its event, and an opt-in background L2 write #266, so a background write of the key is never overtaken, and every local write is tracked so an earlier read cannot put an old value back.BatchTopicInvalidationThreshold, oneTopicInvalidatedevent replaces the per-key events. It drops the local tier of every cache sharing the topic, since events name no cache. A node on an older version ignores it, so enable it only once every node runs this version. The size is known throughCache<T>and its key strategy too.BackgroundDistributedWriteand wait only for keys with a pending background write.The notify doorbell now keeps one
PUBLISHin flight and coalesces the rings after it into one more. OnePUBLISHper event overflowed a subscriber'sclient-output-buffer-limit pubsubat 100k events, and Redis closed the subscription. APUBLISHstalled on Redis holds the rings behind it for at most one second.RemoveAsync(CacheKey[])and the batchSetAsyncare unchanged: the batch set is one atomic transaction, which the new path would give up.Hash cache
HashReplacekeeps the local hash whole. AHashReplacewrite left only the written fields in the local tier, so the writing node lost the others until the entry expired. It now merges into the held local hash, as one locked step so two concurrent merges keep both sets of fields; with none held, it stores nothing and the next read fetches the whole hash.GetCacheEntryAsyncin one round trip. AnEXISTSbefore the read is gone, as the docs already said.RemoveFieldsAsyncremoves single fields (HDEL). Peers drop the whole local entry through an ordinary remove event, so older nodes need no new format. The event goes out whenever the delete was sent, since some chunks may have landed even if it failed.RedisCacheOptions.HashFieldBatchSize(1000) fields goes in boundedHSETchunks. AKeyReplacewrite stays atomic and keeps its TTL through a same-slot staging key andRENAME; a hash with no expiry is renamed and persisted in one transaction, so it never lands with the staging key's lifetime; aHashReplacemerge is not atomic across chunks (documented). Field reads are split too, and a whole-hash read pages byHSCANabove ten batches on Redis 7+.Measured (local Docker)
RemoveManyAsync, single nodeRemoveManyAsync, clusterSetManyAsync, clusterRemoveAsyncin a loopSLOWLOGwas about 3.5 ms.Tests
Cache<T>'s key strategy, a key named twice, an older node's decode, the fallback without the capability; theHashReplacemerge, two merges at once;RemoveFieldsAsync; large hashes; the doorbell coalescing and its one-second bound.RemoveFieldsAsync, 5000-field hashes, a chunked replace of a hash with no expiry.mainagainst a cluster; they are standalone-only tests.Follow-up
HashFieldBatchSize1000 puts about 1000 commands on the wire before any is awaited and doubles its payload in memory. A window of chunks in flight, asBatchConcurrentChunksgives the batch calls, would bound both; it needs its own benchmark, so it is left for a separate change.