diff --git a/design/client-side-cache.md b/design/client-side-cache.md index 6a917fa44..53c7c0ae9 100644 --- a/design/client-side-cache.md +++ b/design/client-side-cache.md @@ -122,15 +122,52 @@ invalidation, validating against a key that changed); the cross-index. we would survive it (generations captured at send), but there is no reason to pay for a protocol that needs it. Without RESP3, caching is **refused loudly** at connect (`RespHandshake`), never silently degraded into a cache nothing invalidates. -- **`Tracking` (plain `CLIENT TRACKING ON`) by default**; `Broadcast` (`BCAST`) when `Prefixes` are given, since - `PREFIX` is only accepted with `BCAST`. Originally `BCAST` with an empty prefix was the default: the server keeps - no per-client table, so it cannot run out of room, and we paid in hearing about keys we never read. Reversed - (2026-10-22) because that cost is unbounded on a busy server and lands on every caching client, whereas the - tracking table's cost is bounded by `tracking-table-max-keys` and an eviction there is a refill, not a stale read. - `OPTIN`/`OPTOUT` are not offered, because `CLIENT CACHING YES|NO` applies to "the next command on that - connection", which a multiplexer cannot promise; measured against Redis 8.9, `ON OPTOUT` is otherwise identical to - `ON`, so it is available later at no behavioural cost. The enum's zero is `Default` ("no opinion"), resolved in - one place, so the library can change its mind without overriding anybody who chose. +- **`Tracking` (`CLIENT TRACKING ON OPTOUT`) by default**; `TrackingByPrefix` when `Prefixes` are given (below). + Originally `BCAST` with an empty prefix was the default: the server keeps no per-client table, so it cannot run + out of room, and we paid in hearing about keys we never read. Reversed (2026-10-22) because that cost is unbounded + on a busy server and lands on every caching client, whereas the tracking table's cost is bounded by + `tracking-table-max-keys` and an eviction there is a refill, not a stale read. Prefixes kept selecting `Broadcast` + for one more day, because `PREFIX` is only accepted with `BCAST` and no per-key mode read the list; once one did + (2026-10-23) the same argument applied to that case too, and `Default` with prefixes moved off broadcast. The + enum's zero is `Default` ("no opinion"), resolved in one place, so the library can change its mind without + overriding anybody who chose. +- **`TrackingByPrefix` (2026-10-23): `OPTIN` on the wire, `Prefixes` applied here.** Making `Prefixes` a filter in + the per-key modes was first proposed as a hard boundary - "the keys I am interested in", with anything outside + forced to `CacheOptOut` - and rejected because it demotes the flags: a `CacheOptIn` that cannot reach outside the + list is a weaker statement than the one it makes everywhere else. So it is a mode instead, and the list is the + policy for reads that passed no flag; the flags win. An unflagged read whose keys all match is treated as + `CacheOptIn` (sent behind `YES`, tracked, stored on first miss, admission skipped - a `YES` that tracks a key we + then refuse to store would be the worst of both); an unflagged read outside, or straddling, is sent plain and + refused as `RefusedNotTracked`; `CacheOptIn` outside is cached (the server can track any key it is told to, unlike + under `BCAST`, where the list went to the server and is final); `CacheOptOut` inside is uncached with nothing on + the wire, `OPTIN` having no `NO` to say. `OPTIN` rather than `OPTOUT` because of where the extra frame lands: on + the misses of the set chosen for caching, which the cache exists to shrink, rather than on every read outside it + (all of which reach the server). The inverse shape - "everything but these" - is `Tracking` with `CacheOptOut` on + the few, not a switch on this mode. The unflagged outside read costs a lookup that is bound to miss before the + refusal; checking the list first would cost about the same, and keeps the hit path untouched. `Prefixes` is + rejected with `Tracking`/`TrackingOptIn` named (nothing reads it) and required by `TrackingByPrefix` (nothing else + to decide with). +- **`OPTIN`/`OPTOUT` are offered (2026-10-23), as `TrackingOptIn` and `Tracking`, driven by a two-bit per-command + field: `CacheOptIn`, `CacheOptOut`, both-set reserved.** They were previously off the table because + `CLIENT CACHING YES|NO` applies to "the next command on that connection", which a multiplexer could not promise. + It can now: a request carries a *prefix* (`RespRequest.Prefix` → `IRespMessage.Prefix`), one frame the connection + writes immediately ahead of the request under the same write lock, enqueueing a discard sink for its reply - so + the server's "next command" is exactly ours, whatever `SELECT`, `ASKING` or `SCRIPT LOAD` heads went in front, and + per element inside a batch. Measured against Redis 8.9: `ON OPTOUT` is identical to `ON` except that it permits + `CLIENT CACHING NO`, so `Tracking` negotiates `OPTOUT` for free; `CACHING YES|NO` is consumed by whatever command + follows (a `PING` or `SET` included); `OPTIN`/`OPTOUT` are refused with `BCAST`; `CACHING` before `MULTI` governs + the whole transaction; pipelining the pair in one write scopes correctly; and a key is dropped from the table + once its invalidation has been sent. Where the frame goes: under `OPTIN`, `YES` ahead of the read that fills (and + of a refresh); under `OPTOUT`, `NO` ahead of a read the caller flagged `CacheOptOut`, and ahead of a transaction's + `MULTI` when the transaction contains a read (reads inside `MULTI` are never cached, so tracking them is pure + server cost). Never ahead of a write (never tracked), never inside `MULTI` (it would be queued and desync `EXEC`), + never under `BCAST`, and not for the command surface's `NeverCached` marker (random members, TTLs), which stays a + separate internal bit precisely so that a `CLIENT CACHING NO` per `HPTTL` does not cost more than the entry it + saves. An error reply to a prefix is swallowed and counted on the connection (`RespConnection.PrefixErrors`), so a + refused modifier degrades a read to uncached rather than failing it. `CacheOptIn` also bypasses admission: the + caller has decided, and under `OPTIN` the first miss is the one that paid for `YES`. The fourth encoding is + reserved: the candidates - "skip the lookup but store" (a refresh, already `WithMaxCacheAge(0)`) and "look up + but never store" - were not worth spending it on today. - **Prefixes are a connection-level knob, never derived from context key-prefixes**: server prefixes are global, must not overlap, and cannot be removed one at a time, while context prefixes nest. A prefix list is also a statement about what may be cached: a key outside it is never announced, so it is refused (`RefusedNotTracked`). @@ -163,11 +200,11 @@ The gates, in the order a fill meets them (`TryBeginFill`, `TryCompleteCore`): | Gate | Refusal counter | Why | |---|---|---| | Retry category **declared** and ≤ `CommandRetryReadOnly` | `RefusedByFlags` | Undeclared is zero, which a naive `<=` reads as safest; undeclared must mean "no". Read-only is necessary, not sufficient: non-deterministic and cursor commands are excluded in command metadata, beside the category. | -| Not `NoClientCache`, not `FireAndForget` | `RefusedByFlags` | Opt-out, not opt-in (§6.10: opt-in touches every method and fails silently by omission). Fire-and-forget promises `default`; a hit would return something else. | +| Not `CacheOptOut`, not `NeverCached`, not `FireAndForget`; under `TrackingOptIn`, `CacheOptIn` present | `RefusedByFlags` | Opt-out by default (§6.10: opt-in touches every method and fails silently by omission); `TrackingOptIn` is the explicit opt-in mode, where the server tracks nothing we did not ask for, so nothing else may be kept. `TrackingByPrefix` passes unflagged reads here and lets the prefix gate below decide. Fire-and-forget promises `default`; a hit would return something else. | | Not inside a transaction | - | A cached answer inside `MULTI` returns a value from before the transaction's own writes. Batches are fine: pipelining, not atomicity. | | At least one key, and all keys listable (≤ 62 arguments) | `RefusedNoKeys` | Invalidation reports keys; a keyless entry is valid forever - not even a flush reaches it. A partial key list is worse than none. | -| Every key inside `Prefixes` | `RefusedNotTracked` | Every key, not any: one unannounced key makes the whole reply uninvalidatable. | -| Admission (§9) | `RefusedNotAdmitted` | Last of the refusals before anything is allocated. | +| Every key inside `Prefixes`, unless `TrackingByPrefix` and `CacheOptIn` | `RefusedNotTracked` | Every key, not any: one unannounced key makes the whole reply uninvalidatable. Under `TrackingByPrefix` the list is the default policy, not a boundary: a flagged opt-in outside it is sent behind `YES` and the server tracks it. | +| Admission (§9), unless `CacheOptIn` or `TrackingByPrefix` | `RefusedNotAdmitted` | Last of the refusals before anything is allocated. `CacheOptIn` skips it: the caller decided, and under `OPTIN` this miss already paid for `CLIENT CACHING YES`. Under `TrackingByPrefix` every read that reaches this gate is an opt-in, by flag or by prefix, and the same reasoning applies. | | Reply is not an error | `RefusedError` | An error need not be a function of the keys (config, ACLs, topology, OOM); cached, a transient failure becomes permanent. Classified by the first **content** element - RESP3 attributes may precede it. Nulls *are* cached: Redis tracks keys that do not exist. | | Keys unchanged since send | `RefusedRaced` | Invariant 3. | | Reply ≤ `MaxPayloadBytes` | `RefusedTooLarge` | Cheapest bound, needs no bookkeeping. | diff --git a/design/command-flow.md b/design/command-flow.md index a0e4eb953..9a4bbec0a 100644 --- a/design/command-flow.md +++ b/design/command-flow.md @@ -81,7 +81,7 @@ caller *takes* the result - `RespPayloadOperation.GetResult` - not on the rea frame (`WithDefaultCategory`) - the one place it happens, and before the cache reads the flags. A write then stamps its keys in the cache *before* it is sent (read-your-own-writes). A read probes the context's `RespClientCache` if there is one, unless the executor is transactional or the flags forbid - it (`NoClientCache` suppresses the probe as well as the store). A hit returns without routing or + it (`CacheOptOut` suppresses the probe as well as the store). A hit returns without routing or sending; an identical request already in flight is awaited rather than duplicated (asynchronous callers only); otherwise a fill is opened - if admission lets it (by default only on a repeated miss) - so the reply populates the cache on the way back. See `client-side-cache.md`. diff --git a/design/interpolated-resp-writer.md b/design/interpolated-resp-writer.md index 91343b7a9..81a236f7a 100644 --- a/design/interpolated-resp-writer.md +++ b/design/interpolated-resp-writer.md @@ -1986,6 +1986,13 @@ a server-side opt-out buys is invalidation-table memory — which under `BCAST` `CLIENT CACHING yes` pipelined immediately ahead of each command, with two traps — it applies to **all** commands in a following `MULTI`, and to **all** commands executed by a following Lua script. +> **Revisited 2026-10-23.** We went per-key by default (see `client-side-cache.md` §6) and then did exactly this: +> `CacheTrackingMode.Tracking` negotiates `OPTOUT`, `TrackingOptIn` negotiates `OPTIN`, and the per-command field +> is two bits (`CacheOptIn`, `CacheOptOut`, both reserved). The "pipelined immediately ahead" requirement is met +> by a per-request prefix frame written under the connection's write lock with a discard sink for its reply, not +> by a separate send. The `MULTI` trap became a feature: `NO` ahead of a transaction that reads. The Lua trap is +> moot for `YES` (scripts are not opted in) and accepted for `NO` (a `CacheOptOut` `EVAL_RO` is opted out whole). + **Invalidate locally on every write. Do not enable `NOLOOP`.** Two separate decisions that look like one. Local invalidation of the keys a write touches is a strict improvement, independent of `NOLOOP`: diff --git a/docs/ClientSideCaching.md b/docs/ClientSideCaching.md index 74467bd7a..54df5941d 100644 --- a/docs/ClientSideCaching.md +++ b/docs/ClientSideCaching.md @@ -23,12 +23,12 @@ options.ClientCache = new CacheOptions using var conn = await ConnectionMultiplexer.ConnectAsync(options); var db = conn.GetDatabase(); -await db.Strings.GetAsync("app:user:42"); // a round trip; remembered, not yet cached -await db.Strings.GetAsync("app:user:42"); // a round trip; read again, so now the reply is cached +await db.Strings.GetAsync("app:user:42"); // a round trip; the key is inside the prefixes, so the reply is cached await db.Strings.GetAsync("app:user:42"); // answered from memory +await db.Strings.GetAsync("other"); // a round trip, every time: outside the prefixes, so never cached ``` -A reply is cached the second time it is read, not the first: a request read only once never costs the cache anything. That is the default admission rule, and it can be changed - see [Admission](#admission-caching-what-is-read-again). +With `Prefixes` set, the prefixes decide what is cached: a read inside them is cached on its first miss, a read outside them is not cached at all (unless the call says `CommandFlags.CacheOptIn`). Without `Prefixes`, every read is cacheable, and a reply is cached the second time it is read rather than the first, so a request read only once never costs the cache anything - see [Tracking modes](#tracking-modes-and-prefixes) and [Admission](#admission-caching-what-is-read-again). Requirements: @@ -67,11 +67,12 @@ A multi-key read such as `MGET a b c` is one entry; a change to any of its keys - **reads that report time or access** - `TTL`, `PTTL`, the hash-field TTL reads, `TOUCH`, `OBJECT FREQ` / `IDLETIME` / `REFCOUNT`, `XINFO CONSUMERS`, and the per-entry form of `XPENDING` - whose answers change without any write; - **`SORT` with a `BY` or `GET` pattern** (`BY weight_*`, `GET object_*->name`) - it reads keys it never names, so their changes could not invalidate it. A plain `SORT`, `BY nosort` and `GET #` are cached; - **error replies**; -- **reads inside a transaction** (`MULTI`/`EXEC`): those are answered by `EXEC`, atomically with the transaction. Reads inside a *batch* do use the cache - a batch is pipelining, not atomicity; +- **reads inside a transaction** (`MULTI`/`EXEC`): those are answered by `EXEC`, atomically with the transaction. Reads inside a *batch* do use the cache - a batch is pipelining, not atomicity. In `Tracking` mode a transaction that contains a read is sent behind `CLIENT CACHING NO`, so the server does not track keys nobody here will keep; - **fire-and-forget** commands (`CommandFlags.FireAndForget`) - there is no reply to keep; - **replies larger than `MaxPayloadBytes`** (1 MiB by default); -- **keys outside `Prefixes`**, when prefixes are configured - the server would never announce their changes; -- anything sent with `CommandFlags.NoClientCache`, or through a context made with `WithoutCache()`. +- **keys outside `Prefixes`**, when prefixes are configured - in `Broadcast` mode the server would never announce their changes; in `TrackingByPrefix` mode (what prefixes select by default) a call can still opt such a read in with `CommandFlags.CacheOptIn`; +- anything sent with `CommandFlags.CacheOptOut`, or through a context made with `WithoutCache()`; +- in `TrackingOptIn` mode, anything sent **without** `CommandFlags.CacheOptIn` - see [Tracking modes](#tracking-modes-and-prefixes). Commands the client does not know - a module's, say - are not cached unless you declare them read-only with `flags.WithRetryCategory(CommandFlags.CommandRetryReadOnly)` (see [Extending the client](Extending)). Only do that if the server announces changes to the keys the command reads. For module commands it usually does: Redis signals a key as modified - which is what drives invalidation - automatically when a module closes a key it opened for writing, unless the module has opted out (`REDISMODULE_OPTION_NO_IMPLICIT_SIGNAL_MODIFIED`) and signals by hand. To confirm for a particular module, cache a read (by default, read it twice), change the key with the module's own write command from a second connection, and check that the next read sees the change. @@ -87,7 +88,7 @@ What you can rely on: What it does **not** cover: -- **Key expiry is not announced.** When a key reaches its TTL, Redis sends no invalidation, so a cached copy outlives it until `TimeToLive` retires the entry. If you rely on keys disappearing on time, keep `TimeToLive` short or pass `CommandFlags.NoClientCache` for those reads. +- **Key expiry is not announced.** When a key reaches its TTL, Redis sends no invalidation, so a cached copy outlives it until `TimeToLive` retires the entry. If you rely on keys disappearing on time, keep `TimeToLive` short or pass `CommandFlags.CacheOptOut` for those reads. - **A dead connection nobody has noticed** is not detected by the cache; it relies on the multiplexer's own failure detection, with `TimeToLive` as the backstop. That is why `TimeToLive` is always finite. - **Undeclared keys in scripts** - see [Scripts](#scripts). @@ -104,8 +105,8 @@ Controlling it | option | default | what it does | |---|---|---| | `Enabled` | `true` | Whether the cache exists. Setting `ClientCache` at all is what turns caching on; this lets you keep the options and switch it off. | -| `TrackingMode` | `Default` (= `Tracking` today, or `Broadcast` when `Prefixes` are set) | How the server decides which changes to announce - see below. | -| `Prefixes` | none (all keys) | Which keys to track and cache, in `Broadcast` mode; setting any selects that mode. | +| `TrackingMode` | `Default` (= `Tracking` today, or `TrackingByPrefix` when `Prefixes` are set) | How the server decides which changes to announce, and whether reads are cached by default (`Tracking`, `Broadcast`), only inside `Prefixes` (`TrackingByPrefix`), or only when a call asks (`TrackingOptIn`) - see below. | +| `Prefixes` | none (all keys) | Which keys to cache. Setting any selects `TrackingByPrefix`, where the list decides for calls that pass no flag; under `Broadcast` it is also sent to the server to filter what is announced. | | `MaxBytes` | unbounded | A memory budget for cached replies. Entries that have not been read since they were stored are evicted, oldest first, to get back under it - see [Eviction](#eviction). | | `MaxEntries` | unbounded | A limit on the number of entries - use it alongside `MaxBytes` when replies are small and numerous. | | `MaxPayloadBytes` | 1 MiB | The largest single reply that may be cached. | @@ -165,6 +166,8 @@ options.ClientCache = new CacheOptions A reply that misses is not stored straight away. The first miss for a request is **remembered** - in a small, fixed-size filter - and only a request that misses again soon afterwards is cached. So a request read once never touches the cache, and a request read often is cached after its second read rather than its first. This is TinyLFU's "doorkeeper", a well-established technique. +It applies to reads that are cacheable by default - every read in `Tracking` and `Broadcast` mode. A read that **opted in** is stored on its first miss regardless: one flagged `CommandFlags.CacheOptIn`, or any read inside `Prefixes` in `TrackingByPrefix` mode (the default when prefixes are set). Those reads have already paid for `CLIENT CACHING YES`, and a tracked key whose reply is then not stored would be the worst of both. + Why it is the default: storing a reply costs something - it is copied, and an entry is kept until it is evicted - and one-off reads that are stored push out the entries that are read again. Measured on a skewed workload (a few hot keys and a long tail, Zipf s = 0.99), caching on the second miss held a **higher** hit rate than caching on the first (76.6% against 72.4%) at about 50% more throughput; on a workload that never repeats, it removed the cache's overhead entirely. What it costs: one extra round trip for each key before it is cached - and again whenever it re-enters the cache after being evicted or invalidated. That only loses on a workload where every request is read exactly twice and never again. @@ -189,11 +192,23 @@ Under `MaxBytes`, replies are stored in large slabs in the order they arrive and ### Tracking modes and prefixes -- **`Tracking`** (the default): the server remembers which keys this client read and announces only those. Precise - invalidation traffic scales with what this client holds, not with how busy the server is - but the server holds a table per client, and evicts from it (announcing what it drops) under pressure, which costs a refill here. `Prefixes` cannot be combined with it. -- **`Broadcast`** (the default when `Prefixes` are set): the server announces every change to every key matching your `Prefixes`, whoever made it, and keeps no record of what you read. Cheap for the server; the cost is on the client, which hears about keys it never cached. **Set `Prefixes` to the parts of the keyspace you actually want cached**: with none, this connection hears about every write to every key in the instance, which on a busy server is a lot of traffic. Keys outside the prefixes are simply not cached. Prefixes must not overlap one another (`"app:"` and `"app:user:"` together are rejected), and they are compared with the key as sent - so if you use `WithKeyPrefix`, include that prefix. +- **`Tracking`** (the default without `Prefixes`): the server remembers which keys this client read and announces only those. Precise - invalidation traffic scales with what this client holds, not with how busy the server is - but the server holds a table per client, and evicts from it (announcing what it drops) under pressure, which costs a refill here. `Prefixes` cannot be combined with it. Negotiated as `CLIENT TRACKING ON OPTOUT`: every read is tracked and cacheable unless a call says otherwise with `CommandFlags.CacheOptOut`, which also tells the server not to track that read (a `CLIENT CACHING NO` goes out as one unit with it). +- **`TrackingByPrefix`** (the default when `Prefixes` are set): `Prefixes` decide for the calls that pass no flag. A read whose keys all fall under a prefix is treated as if it had said `CommandFlags.CacheOptIn`: it goes out behind `CLIENT CACHING YES`, the server tracks it, and it is stored on its first miss. Any other read goes to the server as it would without a cache. The flags still win: `CacheOptIn` on a read outside the prefixes caches it, `CacheOptOut` on a read inside them does not. Negotiated as `CLIENT TRACKING ON OPTIN`; the list is **not** sent to the server (the server only accepts `PREFIX` with `BCAST`), so the server tracks exactly the keys this client holds. The price is an extra frame and reply per miss inside the prefixes, which is the traffic the cache exists to shrink. Prefixes must not overlap one another (`"app:"` and `"app:user:"` together are rejected), and they are compared with the key as sent - so if you use `WithKeyPrefix`, include that prefix. +- **`TrackingOptIn`**: opt-in by flag alone. Negotiated as `CLIENT TRACKING ON OPTIN`: the server tracks nothing, and nothing is cached, unless a call says so with `CommandFlags.CacheOptIn` - that read goes out behind `CLIENT CACHING YES`, is tracked, and is stored on its first miss. Every other read goes to the server as it would without a cache. For "cache these few hot keys and leave everything else alone" when the keys are not a prefix; the price is an extra frame and an extra reply per opted-in miss. `Prefixes` cannot be combined with it - that is `TrackingByPrefix`. +- **`Broadcast`**: the server announces every change to every key matching your `Prefixes`, whoever made it, and keeps no record of what you read. Cheap for the server; the cost is on the client, which hears about keys it never cached, and that cost is set by how busy the server is rather than by anything this client does - which is why prefixes no longer select this mode by default. **Set `Prefixes`** if you choose it: with none, this connection hears about every write to every key in the instance. Keys outside the prefixes are simply not cached, and here the list is a hard boundary: the server was never asked about those keys, so no flag can opt one in. The per-call flags keep only their local meaning: the server has no per-client table to opt in to or out of. `CacheTrackingMode.Default` (the value you get by not setting it) means "the library's choice", resolved as above today; name a mode explicitly if you depend on it. +What the two flags do, by mode: + +| | `Tracking` | `TrackingByPrefix` | `TrackingOptIn` | `Broadcast` | +|---|---|---|---|---| +| no flag | cached; tracked | inside `Prefixes`: cached, stored on first miss, `CLIENT CACHING YES` sent; outside: not cached, not tracked | not cached; not tracked | cached if inside `Prefixes` | +| `CacheOptIn` | cached, stored on first miss | cached, stored on first miss; `CLIENT CACHING YES` sent - inside or outside `Prefixes` | cached, stored on first miss; `CLIENT CACHING YES` sent | cached if inside `Prefixes`, stored on first miss | +| `CacheOptOut` | not cached; `CLIENT CACHING NO` sent | not cached | not cached | not cached | + +Passing both flags is rejected. Neither flag can make an uncacheable command cacheable (a write, a keyless read, a random or time-based read). In `Broadcast` mode `CacheOptIn` does not bypass `Prefixes`; in `TrackingByPrefix` mode it does, because there the list is only the policy for calls that passed no flag. + ### Invalidating by hand The server announces every change it sees, so you normally never touch the cache. For the change it cannot announce - a key whose expiry you know has passed (expiry is not announced), a module command that opted out of signalling, or a value you simply know is wrong - the cache can be told directly: @@ -210,10 +225,16 @@ The same object carries the cache's counters - `Count`, `Bytes`, `Stored`, `Evic ### Per call, and per context -- **`CommandFlags.NoClientCache`** - this call neither reads from nor writes to the cache: +- **`CommandFlags.CacheOptOut`** - this call neither reads from nor writes to the cache, and in `Tracking` mode the server is told not to track it: + + ```csharp + var exact = await db.Strings.GetAsync(key, CommandFlags.CacheOptOut); + ``` + +- **`CommandFlags.CacheOptIn`** - this call's reply is stored on its first miss, skipping [admission](#admission-caching-what-is-read-again); in `TrackingOptIn` mode it is also the only way a reply is cached at all, and in `TrackingByPrefix` mode it caches a read outside the prefixes: ```csharp - var exact = await db.Strings.GetAsync(key, CommandFlags.NoClientCache); + var hot = await db.Strings.GetAsync(key, CommandFlags.CacheOptIn); ``` - **`WithMaxCacheAge(TimeSpan)`** - a context that accepts cached answers only up to a given age. It can only tighten `TimeToLive`, never extend it: @@ -234,7 +255,7 @@ Contexts are cheap to make and immutable; create them where you need them, or ke Scripts --- -`EVAL_RO` / `EVALSHA_RO` (the read-only script commands, via `ScriptEvaluateReadOnly` or `Scripts.EvaluateReadOnlyAsync`) are cached, and invalidated by changes to the keys in `KEYS[]`. **A script that reads a key it did not declare** - building a key name inside Lua, say - will not be invalidated when that key changes, so its cached result can go stale until `TimeToLive`. Declare every key a script reads, or pass `CommandFlags.NoClientCache` for that script. Ordinary `EVAL` / `EVALSHA` are treated as writes and never cached. +`EVAL_RO` / `EVALSHA_RO` (the read-only script commands, via `ScriptEvaluateReadOnly` or `Scripts.EvaluateReadOnlyAsync`) are cached, and invalidated by changes to the keys in `KEYS[]`. **A script that reads a key it did not declare** - building a key name inside Lua, say - will not be invalidated when that key changes, so its cached result can go stale until `TimeToLive`. Declare every key a script reads, or pass `CommandFlags.CacheOptOut` for that script. Ordinary `EVAL` / `EVALSHA` are treated as writes and never cached. Results from the cache --- diff --git a/docs/Extending.md b/docs/Extending.md index e592541d9..3ae109149 100644 --- a/docs/Extending.md +++ b/docs/Extending.md @@ -243,7 +243,7 @@ This one is easy to miss and has no visible symptom. The client keeps a retry ca **Declaring a command read-only also makes it [client-side cacheable](ClientSideCaching)**, for callers who have enabled the cache: its reply may be served from memory until a change to one of its keys invalidates it. That is right for an ordinary read - and for a module's, since Redis signals a key as modified when a module closes a key it opened for writing. It is wrong for a read whose answer changes **without a write** to the keys it names: one that reports idle or elapsed time, returns something random, or reads keys it does not name as `RedisKey`. For those, declare the category and opt out of the cache too: ```csharp -flags.WithRetryCategory(CommandFlags.CommandRetryReadOnly) | CommandFlags.NoClientCache +flags.WithRetryCategory(CommandFlags.CommandRetryReadOnly) | CommandFlags.CacheOptOut ``` ### Synchronous forms diff --git a/src/RESPite/Operations/IRespMessage.cs b/src/RESPite/Operations/IRespMessage.cs index 6fbaecb67..e7aa17a04 100644 --- a/src/RESPite/Operations/IRespMessage.cs +++ b/src/RESPite/Operations/IRespMessage.cs @@ -67,6 +67,31 @@ internal interface IRespMessage : IValueTaskSource /// void OnEnqueued(object connection, long bytesSent, long bytesReceived); + /// + /// One complete RESP frame to write immediately before the request, on the same connection and + /// under the same write lock, or for none. + /// + /// + /// + /// For a server-side modifier that applies to "the next command on this connection" - + /// CLIENT CACHING YES|NO is the motivating case - which is only honoured if nothing else can + /// slip in between. A separate preamble send cannot promise that under a multiplexer; a prefix carried + /// by the message can, because the connection writes the two as one unit. + /// + /// + /// The connection writes it, expects one reply for it, and discards that reply - the message + /// never sees it. That is what distinguishes a prefix from a preamble: a preamble is an operation of its + /// own, with its own completion; a prefix is bytes that belong to this request and a reply nobody is + /// waiting for. An error reply is swallowed too, and counted on the connection, so a prefix the server + /// did not accept degrades the request rather than failing it. + /// + /// + /// Exactly one frame, by contract: the connection enqueues exactly one discard for it. A fixed buffer, + /// never returned to a pool, so it can be shared by every message that needs the same words. + /// + /// + byte[]? Prefix { get; } + /// Deliver a reply held in a single span. /// The version this caller holds. /// The complete reply frame. diff --git a/src/RESPite/Operations/RespConnection.cs b/src/RESPite/Operations/RespConnection.cs index 997069777..33f4fd387 100644 --- a/src/RESPite/Operations/RespConnection.cs +++ b/src/RESPite/Operations/RespConnection.cs @@ -85,6 +85,44 @@ private readonly struct PendingEntry(IRespMessage message, short token) /// private readonly object _writeLock = new(); + /// Where the reply to every prefix goes; see . + private readonly RespPrefixReplySink _prefixReplies = new(); + + /// How many prefix replies were errors; see . + public long PrefixErrors => _prefixReplies.Errors; + + /// The most recent prefix error reply, as text, or null. + public string? LastPrefixError => _prefixReplies.LastError; + + /// Stamp, enqueue and stage one reserved request - and its prefix in front of it, if it has one. + /// The message being written. + /// The life of that was reserved. + /// Its request bytes. + /// The receive counter, snapshotted by the caller. + /// + /// The one place a request's bytes meet the pending queue, so that a prefix is handled identically + /// on every write path - single, pair, run, inbox drain. A prefix is one frame and earns one entry naming + /// the sink, enqueued ahead of the message's own: the server answers in order, so the sink takes the + /// prefix's reply and the message still gets the next one. The counters are snapshotted BEFORE this + /// request's own bytes, prefix included: what a timeout wants to know is whether the connection moved at + /// all while this operation waited, and including our own write in the baseline would make every + /// command look like progress. + /// + private void Stage(IRespMessage message, short token, ReadOnlySpan payload, long received) + { + message.OnEnqueued(this, _bytesSent, received); + if (message.Prefix is { } prefix) + { + _pending.Enqueue(new(_prefixReplies, 0)); + Write(prefix); + Volatile.Write(ref _bytesSent, _bytesSent + prefix.Length); + } + + _pending.Enqueue(new(message, token)); + Write(payload); + Volatile.Write(ref _bytesSent, _bytesSent + payload.Length); + } + /// /// Sends that found the write lock busy, waiting for the holder to write them; only used when /// is on. @@ -163,10 +201,7 @@ private void DrainInboxLocked() try { - message.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - _pending.Enqueue(entry); - Write(payload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + payload.Length); + Stage(message, entry.Token, payload.Span, Volatile.Read(ref _bytesReceived)); } finally { @@ -406,15 +441,8 @@ public bool Send(IRespMessage message) try { - // the counters are snapshotted BEFORE this request's own bytes: what a timeout wants to - // know is whether the connection moved at all while this operation waited, and including - // our own write in the baseline would make every command look like progress - message.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - alone = _pending.IsEmpty; // nothing else in flight: the sequential shape; see FlushAlone - _pending.Enqueue(new(message, message.Token)); - Write(payload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + payload.Length); + Stage(message, message.Token, payload.Span, Volatile.Read(ref _bytesReceived)); } finally { @@ -491,10 +519,7 @@ public bool Send( if (!message.TryReserveRequest(message.Token, out var only)) return false; try { - message.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - _pending.Enqueue(new(message, message.Token)); - Write(only.Span); - Volatile.Write(ref _bytesSent, _bytesSent + only.Length); + Stage(message, message.Token, only.Span, Volatile.Read(ref _bytesReceived)); } finally { @@ -509,15 +534,9 @@ public bool Send( if (!message.TryReserveRequest(message.Token, out var bodyPayload)) return false; try { - var bytes = headPayload.Length + bodyPayload.Length; - head.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - message.OnEnqueued(this, _bytesSent + headPayload.Length, Volatile.Read(ref _bytesReceived)); - - _pending.Enqueue(new(head, head.Token)); - _pending.Enqueue(new(message, message.Token)); - Write(headPayload.Span); - Write(bodyPayload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + bytes); + var received = Volatile.Read(ref _bytesReceived); + Stage(head, head.Token, headPayload.Span, received); + Stage(message, message.Token, bodyPayload.Span, received); } finally { @@ -596,10 +615,7 @@ public bool Send( if (!second.TryReserveRequest(second.Token, out var only)) return false; try { - second.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - _pending.Enqueue(new(second, second.Token)); - Write(only.Span); - Volatile.Write(ref _bytesSent, _bytesSent + only.Length); + Stage(second, second.Token, only.Span, Volatile.Read(ref _bytesReceived)); } finally { @@ -614,15 +630,9 @@ public bool Send( if (!second.TryReserveRequest(second.Token, out var secondPayload)) return false; try { - var bytes = firstPayload.Length + secondPayload.Length; - first.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - second.OnEnqueued(this, _bytesSent + firstPayload.Length, Volatile.Read(ref _bytesReceived)); - - _pending.Enqueue(new(first, first.Token)); - _pending.Enqueue(new(second, second.Token)); - Write(firstPayload.Span); - Write(secondPayload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + bytes); + var received = Volatile.Read(ref _bytesReceived); + Stage(first, first.Token, firstPayload.Span, received); + Stage(second, second.Token, secondPayload.Span, received); wroteFirst = true; } finally @@ -695,10 +705,7 @@ public bool Send( { try { - head.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - _pending.Enqueue(new(head, head.Token)); - Write(headPayload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + headPayload.Length); + Stage(head, head.Token, headPayload.Span, Volatile.Read(ref _bytesReceived)); } finally { @@ -713,10 +720,7 @@ public bool Send( if (!first.TryReserveRequest(first.Token, out var firstPayload)) return false; try { - first.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - _pending.Enqueue(new(first, first.Token)); - Write(firstPayload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + firstPayload.Length); + Stage(first, first.Token, firstPayload.Span, Volatile.Read(ref _bytesReceived)); wroteFirst = true; } finally @@ -728,10 +732,7 @@ public bool Send( if (!second.TryReserveRequest(second.Token, out var secondPayload)) return false; try { - second.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - _pending.Enqueue(new(second, second.Token)); - Write(secondPayload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + secondPayload.Length); + Stage(second, second.Token, secondPayload.Span, Volatile.Read(ref _bytesReceived)); } finally { @@ -774,15 +775,9 @@ public bool Send(IRespMessage first, IRespMessage second) if (!second.TryReserveRequest(second.Token, out var secondPayload)) return false; try { - var bytes = firstPayload.Length + secondPayload.Length; - first.OnEnqueued(this, _bytesSent, Volatile.Read(ref _bytesReceived)); - second.OnEnqueued(this, _bytesSent + firstPayload.Length, Volatile.Read(ref _bytesReceived)); - - _pending.Enqueue(new(first, first.Token)); - _pending.Enqueue(new(second, second.Token)); - Write(firstPayload.Span); - Write(secondPayload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + bytes); + var received = Volatile.Read(ref _bytesReceived); + Stage(first, first.Token, firstPayload.Span, received); + Stage(second, second.Token, secondPayload.Span, received); } finally { @@ -848,10 +843,7 @@ public bool Send( { try { - head.OnEnqueued(this, _bytesSent, received); - _pending.Enqueue(new(head, head.Token)); - Write(headPayload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + headPayload.Length); + Stage(head, head.Token, headPayload.Span, received); } finally { @@ -867,10 +859,7 @@ public bool Send( try { - message.OnEnqueued(this, _bytesSent, received); - _pending.Enqueue(new(message, token)); - Write(payload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + payload.Length); + Stage(message, token, payload.Span, received); } finally { @@ -932,10 +921,7 @@ public bool Send(RespRunEntry[] operations, int count) // enqueued before IT is written, which is all the ordering this needs: a reply to an // earlier element can arrive while a later one is still being copied, but no reply can // arrive for something not yet written - message.OnEnqueued(this, _bytesSent, received); - _pending.Enqueue(new(message, token)); - Write(payload.Span); - Volatile.Write(ref _bytesSent, _bytesSent + payload.Length); + Stage(message, token, payload.Span, received); } finally { diff --git a/src/RESPite/Operations/RespMessageBase.cs b/src/RESPite/Operations/RespMessageBase.cs index b7bfa200b..191a8725a 100644 --- a/src/RESPite/Operations/RespMessageBase.cs +++ b/src/RESPite/Operations/RespMessageBase.cs @@ -364,6 +364,15 @@ private void UnregisterCancellation() protected ReadOnlyMemory RequestForDiagnostics => Volatile.Read(ref _requestRefCount) == 0 ? default : _request; + /// + /// + /// Per life, like the request: set after by whoever attaches the request, and + /// cleared by the reset that ends the life, so a recycled instance cannot carry a previous command's + /// modifier in front of the next one. Plain field semantics - it is written by the single owner before + /// the message is handed to a connection, and read by the connection under its write lock. + /// + public byte[]? Prefix { get; protected set; } + /// public bool TryReserveRequest(short token, out ReadOnlyMemory payload, bool recordSent = true) { @@ -740,6 +749,7 @@ private void Reset() UnregisterCancellation(); TryReleaseRequest(); _diagnostics = default; // the whole point of keeping it in one struct + Prefix = null; // a modifier belongs to the command it was attached for, not to the instance OnReset(); _asyncCore.Reset(); diff --git a/src/RESPite/Operations/RespPrefixReplySink.cs b/src/RESPite/Operations/RespPrefixReplySink.cs new file mode 100644 index 000000000..8e81ccc5b --- /dev/null +++ b/src/RESPite/Operations/RespPrefixReplySink.cs @@ -0,0 +1,123 @@ +using System; +using System.Buffers; +using System.Text; +using System.Threading; +using System.Threading.Tasks.Sources; +using RESPite.Buffers; + +namespace RESPite.Operations; + +/// +/// The pending-queue entry that stands in for a prefix's reply (see ): +/// it takes the frame, counts it if it was an error, and completes nothing. +/// +/// +/// +/// One per connection, enqueued once per prefix written. It never changes state, so one instance +/// can sit in the queue any number of times at once; its token is a constant, so every entry naming it is +/// "current" and takes exactly one frame. That is the whole mechanism by which a prefix costs the caller +/// nothing: the frame that answers it has somewhere to go that is not the next operation. +/// +/// +/// An error reply is the case worth recording. CLIENT CACHING YES on a connection whose tracking +/// mode does not allow it answers -ERR; the command behind it still runs and still replies, so +/// nothing fails - but the modifier was not applied, and "why is this not being tracked?" should have an +/// answer. and are that answer; the request itself is not +/// faulted, because degrading to "uncached" is strictly better than failing a read that succeeded. +/// +/// +internal sealed class RespPrefixReplySink : IRespMessage +{ + private long _errors; + private string? _lastError; + + /// How many prefix replies were errors. + public long Errors => Volatile.Read(ref _errors); + + /// The most recent error reply, as text, or null. + public string? LastError => Volatile.Read(ref _lastError); + + short IRespMessage.Token => 0; + + bool IRespMessage.AllowInlineParsing => true; + + bool IRespMessage.IsFinished => true; // nothing is ever owed to anybody for it + + byte[]? IRespMessage.Prefix => null; + + void IRespMessage.Wait(short token, TimeSpan timeout) => throw NotWaitable(); + + bool IRespMessage.TryReserveRequest(short token, out ReadOnlyMemory payload, bool recordSent) + { + payload = default; + return false; // it is never written; it only receives + } + + void IRespMessage.ReleaseRequest() + { + } + + void IRespMessage.OnEnqueued(object connection, long bytesSent, long bytesReceived) + { + } + + bool IRespMessage.TrySetResult(short token, scoped ReadOnlySpan response, IPayloadReservationProvider? source) + { + Note(response); + return true; + } + + bool IRespMessage.TrySetResult(short token, in ReadOnlySequence response) + { + if (response.IsSingleSegment) + { + Note(response.First.Span); + } + else if (!response.IsEmpty) + { + // an error reply is short; a multi-segment one is almost certainly not an error, but check the + // first byte rather than assume + Span first = stackalloc byte[1]; + response.Slice(0, 1).CopyTo(first); + if (first[0] == (byte)'-') Note(response.ToArray()); + } + + return true; + } + + private void Note(ReadOnlySpan response) + { + if (response.IsEmpty || response[0] != (byte)'-') return; + + // strip the leading '-' and the trailing CRLF; the text is for a human + var text = response.Slice(1); + if (text.Length >= 2 && text[text.Length - 2] == (byte)'\r') text = text.Slice(0, text.Length - 2); + Volatile.Write(ref _lastError, Encoding.UTF8.GetString(text.ToArray())); + Interlocked.Increment(ref _errors); + } + + // the connection is closing, or an operation is being timed out: a prefix has nobody to tell + bool IRespMessage.TrySetException(short token, Exception exception, bool definite) => false; + + bool IRespMessage.TrySetCanceled(short token, CancellationToken cancellationToken) => false; + + void IRespMessage.TrySetCanceled() + { + } + + void IRespMessage.TrySetTimedOut() + { + } + + bool IRespMessage.TryTimeoutIfOlderThan(short token, TimeSpan age) => false; + + void IValueTaskSource.GetResult(short token) => throw NotWaitable(); + + ValueTaskSourceStatus IValueTaskSource.GetStatus(short token) => ValueTaskSourceStatus.Succeeded; + + void IValueTaskSource.OnCompleted(Action continuation, object? state, short token, ValueTaskSourceOnCompletedFlags flags) + => throw NotWaitable(); + + private static InvalidOperationException NotWaitable() + => new("A prefix reply is discarded by the connection; nothing can wait on it."); +} diff --git a/src/StackExchange.Redis/Caching/CacheOptions.cs b/src/StackExchange.Redis/Caching/CacheOptions.cs index cefcd2f76..ce3b1df5f 100644 --- a/src/StackExchange.Redis/Caching/CacheOptions.cs +++ b/src/StackExchange.Redis/Caching/CacheOptions.cs @@ -32,7 +32,7 @@ public sealed class CacheOptions /// Whether a cache is created at all. /// /// The global switch. Suppressing the cache for one command is - /// , which suppresses the probe as well as the store; this + /// , which suppresses the probe as well as the store; this /// is the deployment-level decision that there is no cache to probe. /// public bool Enabled { get; init; } = true; @@ -42,8 +42,8 @@ public sealed class CacheOptions /// unless you say otherwise, which resolves to /// today: the server announces only what this client /// read, so the invalidation traffic scales with what we hold rather than with what the server - /// does. Giving resolves it to - /// instead, since prefixes only mean anything there. Read + /// does. Giving resolves it to + /// instead: the prefixes say what to cache, and the server still hears only about that. Read /// rather than this one when you need the mode that will actually be negotiated. /// public CacheTrackingMode TrackingMode { get; init; } @@ -56,11 +56,13 @@ public sealed class CacheOptions /// The one place the choice is made. Every consumer reads this rather than comparing against /// a mode itself, so changing what "no opinion" means is an edit here and nowhere else - and so that /// no caller can accidentally treat an unset mode as a third behaviour. Today: - /// mean broadcast, because PREFIX is only accepted with BCAST; otherwise tracking. + /// mean (broadcast until 2026-10-23, when a per-key + /// mode that reads the list arrived; its cost is bounded by our own misses, broadcast's by the server's + /// write rate); otherwise tracking. /// internal CacheTrackingMode ResolvedTrackingMode => TrackingMode switch { - CacheTrackingMode.Default when HasPrefixes => CacheTrackingMode.Broadcast, + CacheTrackingMode.Default when HasPrefixes => CacheTrackingMode.TrackingByPrefix, CacheTrackingMode.Default => CacheTrackingMode.Tracking, var explicitMode => explicitMode, }; @@ -251,12 +253,17 @@ public int EvictionSampleSize /// total one. /// /// - /// Broadcast only. PREFIX is meaningless under - /// - the server announces what we read, so there is nothing - /// to filter - and CLIENT TRACKING rejects the combination outright. So giving prefixes with - /// left at selects broadcast; giving - /// them with named is an error, raised when the cache is built - /// rather than at the handshake. + /// Two modes read it. Under it goes to the server as + /// PREFIX arguments and filters what is announced, as well as what is cached. Under + /// it stays here and decides which unflagged reads + /// are opted in (sent behind CLIENT CACHING YES, tracked, stored); the per-call flags override + /// it there. It is rejected with or + /// named - the server announces what we read, so there + /// is nothing to filter, and CLIENT TRACKING refuses PREFIX without BCAST anyway - + /// and required by , which has nothing to decide with + /// otherwise. Giving prefixes with left at + /// selects . + /// Either mistake is raised when the cache is built rather than at the handshake. /// /// public IReadOnlyList Prefixes @@ -274,7 +281,8 @@ public IReadOnlyList Prefixes /// Whether restricts what may be cached. /// - /// Only ever true in : under + /// Only ever true in and + /// , the two modes that accept a list: under /// the server announces exactly what we read, so there is no /// such thing as an untracked key and nothing for the gate to refuse. /// @@ -291,13 +299,24 @@ public IReadOnlyList Prefixes /// internal void Validate() { - if (HasPrefixes && ResolvedTrackingMode != CacheTrackingMode.Broadcast) + var mode = ResolvedTrackingMode; + if (HasPrefixes && mode is not (CacheTrackingMode.Broadcast or CacheTrackingMode.TrackingByPrefix)) { throw new ArgumentException( - $"Cache prefixes require {nameof(CacheTrackingMode)}.{nameof(CacheTrackingMode.Broadcast)};" + $"Cache prefixes require {nameof(CacheTrackingMode)}.{nameof(CacheTrackingMode.Broadcast)}" + + $" or {nameof(CacheTrackingMode)}.{nameof(CacheTrackingMode.TrackingByPrefix)};" + $" {TrackingMode} announces the keys that were read, so there is nothing to filter.", nameof(Prefixes)); } + + if (mode == CacheTrackingMode.TrackingByPrefix && !HasPrefixes) + { + throw new ArgumentException( + $"{nameof(CacheTrackingMode)}.{nameof(CacheTrackingMode.TrackingByPrefix)} needs {nameof(Prefixes)}:" + + $" without a list there is nothing to opt in by. Use {nameof(CacheTrackingMode.TrackingOptIn)}" + + $" to opt in by flag alone, or {nameof(CacheTrackingMode.Tracking)} to cache every read.", + nameof(Prefixes)); + } } /// diff --git a/src/StackExchange.Redis/Caching/CacheTrackingMode.cs b/src/StackExchange.Redis/Caching/CacheTrackingMode.cs index a34388c66..d8e34afca 100644 --- a/src/StackExchange.Redis/Caching/CacheTrackingMode.cs +++ b/src/StackExchange.Redis/Caching/CacheTrackingMode.cs @@ -8,30 +8,31 @@ namespace StackExchange.Redis.Caching /// /// /// - /// The two modes of CLIENT TRACKING, and they are a trade rather than a ranking: broadcasting + /// The modes of CLIENT TRACKING, and they are a trade rather than a ranking: broadcasting /// costs the client noise, and per-key tracking costs the server memory. Which is cheaper depends on /// whose resource is scarce, which is not something a library can know. /// /// - /// An enum rather than a bool Broadcast because the server has two further modes - - /// OPTIN and OPTOUT - and a boolean cannot grow to hold them. Whether they will ever be - /// offered here is genuinely open; see . + /// Per-key tracking comes in two refinements, OPTOUT () and OPTIN + /// (), which differ in what a read gets by default and which way the + /// per-command flags - and - + /// push. Both are honoured by writing CLIENT CACHING YES|NO as one unit with the command it + /// modifies, so the "next command on this connection" the server speaks of really is that command. /// /// public enum CacheTrackingMode { /// - /// Let the library choose; today that is , or when - /// are given, since prefixes only mean anything there. + /// Let the library choose; today that is , or + /// when are given. /// /// /// - /// Zero means "no opinion", not a mode. Which of the two modes is cheaper depends on - /// whose resource is scarce - the remarks on this type say the library cannot know that - and - /// wiring a real mode to zero says the opposite: that an unset value is a considered choice. It - /// also makes the two indistinguishable, so the library can never tell a caller who asked for - /// a mode from one who never thought about it, and can never change its mind without silently - /// overriding the first. + /// Zero means "no opinion", not a mode. Which mode is cheaper depends on whose resource is + /// scarce - the remarks on this type say the library cannot know that - and wiring a real mode to + /// zero says the opposite: that an unset value is a considered choice. It also makes the two + /// indistinguishable, so the library can never tell a caller who asked for a mode from one who + /// never thought about it, and can never change its mind without silently overriding the first. /// /// /// So the answer this resolves to may move - if server or client heuristics make one clearly @@ -47,27 +48,34 @@ public enum CacheTrackingMode /// /// /// - /// What resolves to when are given, and - /// otherwise a choice to make by name. The server keeps no per-client key table at all, so it + /// A choice to make by name: does not resolve to it even when + /// are given (that is ), because + /// what broadcasting costs is set by the server's write rate, which this client cannot see or bound. + /// The server keeps no per-client key table at all, so it /// cannot run out of room for one; what we pay instead is being told about every key anyone /// changes, whether or not we hold it, which exists to bound. /// Without prefixes on a busy server that is a lot of pushes, nearly all of them misses here. /// /// /// The consequence that reaches the cache: a key outside the prefixes is never announced, so it is - /// never cached either - see . + /// never cached either - see . The per-command + /// flags have only their local half here: bypasses the cache + /// for a call, stores on the first miss; neither can say + /// anything to a server that tracks nothing per client, and neither can reach a key outside the + /// prefixes. /// /// Broadcast = 1, /// - /// CLIENT TRACKING without BCAST: the server remembers which keys we read, - /// and announces only those. + /// CLIENT TRACKING ON OPTOUT: the server remembers which keys we read, and announces + /// only those; every read is tracked and cacheable unless a call says otherwise. /// /// /// /// What resolves to today, unless are - /// given. Precise - no key we did not ask about is ever mentioned, so the invalidation traffic is + /// given (then ). Precise - no key we did not ask about is ever + /// mentioned, so the invalidation traffic is /// proportional to what this client holds rather than to what the server does - and /// is meaningless here, because there is nothing to filter. /// The cost moves to the server, which must hold a table of keys per client and evicts from it @@ -75,16 +83,77 @@ public enum CacheTrackingMode /// costs us a refill, never a stale read. /// /// - /// Two hazards specific to this mode. The server stops tracking a key once it has told us - /// about it, so anything that reads without re-registering goes quietly stale - which is also why - /// NOLOOP is not simply switched on (design notes 6.13). And OPTIN/OPTOUT, the - /// two refinements that only exist here, are driven by CLIENT CACHING YES|NO applying to the - /// next command on that connection - which a multiplexer does not give a caller any way to - /// control. Offering them would need the pipeline to write the pair atomically, the way it already - /// does for a transaction; until that exists they are not on the table, which is the honest reason - /// they are absent from this enum rather than present and throwing. + /// OPTOUT rather than plain ON because, measured against Redis 8.9, the two are + /// identical except that OPTOUT permits CLIENT CACHING NO. That is the wire half of + /// : a read the cache will not keep is sent behind it, and + /// the server does not add the key to its table or push an invalidation for it. The library does + /// the same for a transaction that contains a read, since reads inside MULTI are never + /// cached. + /// + /// + /// One hazard specific to per-key tracking. The server stops tracking a key once it has told + /// us about it, so anything that reads without re-registering goes quietly stale - which is also + /// why NOLOOP is not simply switched on (design notes 6.13). /// /// Tracking = 2, + + /// + /// CLIENT TRACKING ON OPTIN: the server remembers only the keys we explicitly ask it to, and + /// nothing is cached unless a call says so with . + /// + /// + /// + /// The inverse of , for a workload where a few hot reads deserve caching and + /// everything else should cost the server nothing: no table entry, no invalidation push. A read + /// without goes to the server as it would without a cache, + /// and is not stored - it could not be, because nothing would ever invalidate it. A read with it is + /// sent behind CLIENT CACHING YES, as one unit, tracked, and stored. + /// + /// + /// The price is one extra frame and one extra reply per opted-in miss, so this is a mode for + /// "cache these particular keys", not for general use; for "cache everything except a few", + /// with on the few is the cheaper + /// shape. are rejected here as they are for + /// - opting in by prefix rather than by flag is ; + /// has only its local meaning, there being nothing on the + /// server to opt out of. + /// + /// + TrackingOptIn = 3, + + /// + /// where decide for the reads that do + /// not say: a read whose keys all fall under a prefix is treated as , + /// any other read as it would be without a cache. + /// + /// + /// + /// What resolves to when are given. The + /// same wire negotiation as - CLIENT TRACKING ON OPTIN, no + /// PREFIX, which the server would refuse without BCAST - so the server tracks only + /// what is sent behind CLIENT CACHING YES, and the prefixes are applied here, not there: + /// an unflagged read inside them goes out behind YES, is tracked, and is stored on its first + /// miss, exactly as if the call had said ; an unflagged read + /// outside them, or one whose keys straddle the boundary, is sent plain and kept nowhere. + /// + /// + /// The flags still win. Prefixes are the policy for reads that did not state one; + /// on a read outside the prefixes caches it (the server can + /// track any key here, unlike under ), and + /// on a read inside them leaves it uncached and untracked. That is what distinguishes this from + /// with the same list, where the prefixes are a hard boundary nothing can + /// cross. + /// + /// + /// OPTIN rather than OPTOUT because of where the extra frame lands: on the misses of + /// the set chosen for caching, which is the traffic the cache exists to shrink, rather than on every + /// read outside it. A prefix list is by nature "these few things"; "everything but" is + /// with on the few. Preferred over + /// for the default because its cost is bounded by this client's own misses, + /// where broadcasting is billed by the server's write rate. + /// + /// + TrackingByPrefix = 4, } } diff --git a/src/StackExchange.Redis/Caching/RespClientCache.cs b/src/StackExchange.Redis/Caching/RespClientCache.cs index 89aa09a7b..51606217e 100644 --- a/src/StackExchange.Redis/Caching/RespClientCache.cs +++ b/src/StackExchange.Redis/Caching/RespClientCache.cs @@ -92,16 +92,30 @@ internal sealed partial class RespClientCache : IDisposable /// Create a cache. /// How the cache is built; when null. /// Initial size hint for the tracked-key table. + /// + /// How commands are spelled on this connection, so the CLIENT CACHING frames honour a renamed + /// CLIENT exactly as the handshake's CLIENT TRACKING does; the default map when null. + /// /// /// One constructor rather than an overload pair: two constructors both carrying optional parameters /// is ambiguous for callers, and the analyzers say so (RS0026). Named arguments cover the cases an /// overload would have. /// - public RespClientCache(CacheOptions? options = null, int keyCapacity = 256) + public RespClientCache(CacheOptions? options = null, int keyCapacity = 256, CommandMap? commandMap = null) { Options = options ?? CacheOptions.Default; Options.Validate(); // settings that constrain one another; see CacheOptions.Validate _keys = new RespKeyTable(keyCapacity); + + // the two CLIENT CACHING frames, rendered once through the command map (CLIENT can be renamed like + // anything else, and the handshake's CLIENT TRACKING honours the same map) and shared by every + // request that needs one: a prefix is a fixed buffer, never pooled, so sharing is free + var mode = Options.ResolvedTrackingMode; + _optIn = mode == CacheTrackingMode.TrackingOptIn; + _byPrefix = mode == CacheTrackingMode.TrackingByPrefix; + _optOut = mode == CacheTrackingMode.Tracking; + if (_optIn || _byPrefix) _cachingYes = RenderClientCaching(commandMap ?? CommandMap.Default, "YES"u8); // both negotiate OPTIN + if (_optOut) _cachingNo = RenderClientCaching(commandMap ?? CommandMap.Default, "NO"u8); _slabSize = ChooseSlabSize(Options.MaxBytes); if (Options.ResolvedAdmission == CacheAdmission.OnRepeatedMiss) { @@ -120,6 +134,91 @@ public RespClientCache(CacheOptions? options = null, int keyCapacity = 256) /// This cache as callers see it; one instance, created with the cache. See . internal ClientCache View { get; } + // _optIn: TrackingOptIn, where only a CacheOptIn read may be kept. _byPrefix: TrackingByPrefix, the same + // wire mode with Prefixes deciding for unflagged reads. _optOut: Tracking, where CacheOptOut has a wire half. + private readonly bool _optIn, _byPrefix, _optOut; + private readonly byte[]? _cachingYes, _cachingNo; + + /// CLIENT CACHING {verb} as one RESP frame, with the command spelled as the map says. + private static byte[] RenderClientCaching(CommandMap map, ReadOnlySpan verb) + { + var client = map.GetResp(RedisCommand.CLIENT); // "$6\r\nCLIENT\r\n", or the renamed equivalent + if (client.IsEmpty) throw ExceptionFactory.CommandDisabled(RedisCommand.CLIENT); + ReadOnlySpan head = "*3\r\n"u8, caching = "$7\r\nCACHING\r\n$"u8, crlf = "\r\n"u8; + var frame = new byte[head.Length + client.Length + caching.Length + 1 + crlf.Length + verb.Length + crlf.Length]; + var span = frame.AsSpan(); + head.CopyTo(span); + span = span.Slice(head.Length); + client.CopyTo(span); + span = span.Slice(client.Length); + caching.CopyTo(span); + span = span.Slice(caching.Length); + span[0] = (byte)('0' + verb.Length); // 2 or 3 + span = span.Slice(1); + crlf.CopyTo(span); + span = span.Slice(crlf.Length); + verb.CopyTo(span); + span = span.Slice(verb.Length); + crlf.CopyTo(span); + Debug.Assert(span.Length == crlf.Length, "frame sized wrongly"); + return frame; + } + + /// + /// The frame to write ahead of a request whose reply this cache is about to store, or null. + /// + /// + /// CLIENT CACHING YES under and + /// : the server tracks nothing there unless told, and + /// a fill only begins under those modes for a read that opted in - by flag, or by falling inside the + /// prefixes - so "about to store" and "needs the prefix" coincide. Elsewhere the server already tracks + /// (or broadcasts) everything, and nothing is said. + /// + internal byte[]? FillPrefix => _cachingYes; + + /// + /// The frame to write ahead of a request this cache will not store, or null. + /// + /// + /// CLIENT CACHING NO under (which the handshake + /// negotiates as OPTOUT precisely so this is legal), for a read the caller opted out with + /// : the server would otherwise add the key to its table and + /// push an invalidation nobody here holds anything for. Only for reads - the server tracks the keys + /// of read-only commands, so on a write the frame would buy nothing - and only for the caller's + /// flag, not the command surface's NeverCached marker (see CommandFlagsInternal.NeverCachedFlag + /// for why that one stays local). + /// + internal byte[]? OptOutPrefix(CommandFlags flags) + => _optOut && (flags & CommandFlags.CacheOptOut) != 0 && IsReadOnly(flags) ? _cachingNo : null; + + /// + /// The frame to write ahead of a transaction's MULTI, or null. + /// + /// + /// Reads inside MULTI/EXEC are never cached (their answers come from EXEC, + /// atomically with the transaction's own writes), so under + /// every key they read is a server-side tracking entry and a later invalidation push for nothing. + /// CLIENT CACHING NO before MULTI applies to the whole transaction - measured against + /// Redis 8.9, as the documentation says of YES. Only when the transaction contains a read: a + /// write-only transaction tracks nothing, and the frame would be pure cost. + /// + internal byte[]? TransactionPrefix(List operations) + { + if (!_optOut) return null; + foreach (var queued in operations) + { + if (IsReadOnly(queued.Operation.Flags)) return _cachingNo; + } + + return null; + } + + private static bool IsReadOnly(CommandFlags flags) + { + var category = flags & CommandFlagsInternal.MaskRetryCategory; + return category != 0 && category <= CommandFlags.CommandRetryReadOnly; + } + /// How entries in this cache behave, unless a caller overrides it. public CachePolicy Policy => Options.DefaultPolicy; @@ -754,7 +853,10 @@ public bool TryBeginFill(ref RespRequestFrame frame, int database, CommandFlags // EVERY key, not any: the entry depends on all of them, so one key the server was never asked // to watch is enough to make the whole reply uninvalidatable. MGET tracked untracked is not // "mostly fine". - if (Options.HasPrefixes) + // ...except under TrackingByPrefix with CacheOptIn said outright: there the list is the policy for + // reads that did not state one, the flag is a statement, and the server can track any key we + // send behind YES. Under Broadcast the list went to the server, and nothing can cross it. + if (Options.HasPrefixes && !(_byPrefix && (flags & CommandFlags.CacheOptIn) != 0)) { for (var i = 0; i < count; i++) { @@ -769,7 +871,14 @@ public bool TryBeginFill(ref RespRequestFrame frame, int database, CommandFlags // last of the refusals and before anything is allocated: a request seen for the first time is remembered, // not stored - see CacheAdmission.OnRepeatedMiss. The database is folded in, as it is in EntryKey. - if (_doorkeeper is not null && !_doorkeeper.SeenBefore((frame.AsLookupKey().GetHashCode() * 397) ^ database)) + // ...unless the caller said "cache this" (CacheOptIn): they have already made the admission + // decision, and under OPTIN mode the first miss is the one that pays for CLIENT CACHING YES. The + // same under TrackingByPrefix for every read that gets this far: inside the prefixes it IS an + // opt-in, and a YES that tracks a key we then decline to store would be the worst of both. + if (_doorkeeper is not null + && (flags & CommandFlags.CacheOptIn) == 0 + && !_byPrefix + && !_doorkeeper.SeenBefore((frame.AsLookupKey().GetHashCode() * 397) ^ database)) { Interlocked.Increment(ref _refusedNotAdmitted); fill = default; @@ -783,7 +892,9 @@ public bool TryBeginFill(ref RespRequestFrame frame, int database, CommandFlags deps[i] = new Dependency(node, generation); } - var key = frame.Detach(flags); + // the prefix travels with the request that fills: under OPTIN mode this is the read that must + // say CLIENT CACHING YES, or the server will never announce its keys + var key = frame.Detach(flags, FillPrefix); // Register as the leader for this request, so concurrent misses can wait on us instead of each // sending their own copy. Losing the race is not a failure: we simply lead without a slot, which @@ -1323,7 +1434,7 @@ private static bool IsCacheableBehindAttributes(ReadOnlySpan response) } /// - /// As , but observed: a refusal is counted. + /// As , but observed: a refusal is counted. /// /// /// The orchestration skips the cache entirely for a command whose flags forbid it - it does not @@ -1363,12 +1474,24 @@ internal bool PermitsCaching(CommandFlags flags) /// it cannot work for exactly that reason. /// /// - internal static bool IsCacheable(CommandFlags flags) + internal bool IsCacheable(CommandFlags flags) => IsCacheable(flags, _optIn); + + /// Whether permit a reply to be served from or stored in a cache. + /// The command's flags, with its retry category applied. + /// + /// Whether the cache is in , where only a read flagged + /// is tracked by the server and so only that read may be kept. + /// + internal static bool IsCacheable(CommandFlags flags, bool optIn = false) { - if ((flags & (CommandFlags.NoClientCache | CommandFlags.FireAndForget)) != 0) return false; + // CacheOptOut and the command surface's NeverCached marker both mean "not this one"; fire-and-forget + // promises default, which a hit would contradict + if ((flags & (CommandFlags.CacheOptOut | CommandFlagsInternal.NeverCachedFlag | CommandFlags.FireAndForget)) != 0) return false; - var category = flags & CommandFlagsInternal.MaskRetryCategory; - return category != 0 && category <= CommandFlags.CommandRetryReadOnly; + // under OPTIN mode only an opted-in read is tracked by the server, so only that read may be kept + if (optIn && (flags & CommandFlags.CacheOptIn) == 0) return false; + + return IsReadOnly(flags); } /// One key a cached entry depends on, and the generation it had when the request was sent. diff --git a/src/StackExchange.Redis/ConnectionMultiplexer.cs b/src/StackExchange.Redis/ConnectionMultiplexer.cs index 1ff3948c7..7266ce48b 100644 --- a/src/StackExchange.Redis/ConnectionMultiplexer.cs +++ b/src/StackExchange.Redis/ConnectionMultiplexer.cs @@ -227,7 +227,7 @@ private ConnectionMultiplexer(ConfigurationOptions configuration, ServerType? se // would be handed the pushes of a connection it does not own. if (RawConfig.ClientCache is { Enabled: true } cacheOptions) { - ClientCache = new RespClientCache(cacheOptions); + ClientCache = new RespClientCache(cacheOptions, commandMap: map); } var configChannel = configuration.ConfigurationChannel; diff --git a/src/StackExchange.Redis/Contexts/RespConnectionExecutor.cs b/src/StackExchange.Redis/Contexts/RespConnectionExecutor.cs index 09158103a..57033f2d7 100644 --- a/src/StackExchange.Redis/Contexts/RespConnectionExecutor.cs +++ b/src/StackExchange.Redis/Contexts/RespConnectionExecutor.cs @@ -99,7 +99,7 @@ private RespPayloadOperation Dispatch(in RespRequest request, CancellationToken // the request's bytes are the caller's until this call completes, so the operation takes its // own copy rather than a reference: the connection may still be writing after we return, and // fire-and-forget returns before the write at all - operation.Attach(request.Span, request.Flags, cancellationToken); + operation.Attach(request.Span, request.Flags, cancellationToken, request.Prefix); operation.Database = Database; if (!_connection.Send(operation)) diff --git a/src/StackExchange.Redis/Contexts/RespEndpointExecutor.cs b/src/StackExchange.Redis/Contexts/RespEndpointExecutor.cs index c6d3d4357..a1dd90659 100644 --- a/src/StackExchange.Redis/Contexts/RespEndpointExecutor.cs +++ b/src/StackExchange.Redis/Contexts/RespEndpointExecutor.cs @@ -893,7 +893,7 @@ internal override RespPayload Send(RespRequest preamble, RespRequest request, IR pre.Database = Database; var body = RespPayloadOperation.Rent(); - body.Attach(request.Span, request.Flags, cancellationToken); + body.Attach(request.Span, request.Flags, cancellationToken, request.Prefix); body.Slot = request.Slot; body.Command = request.Command; body.Observer = this; diff --git a/src/StackExchange.Redis/Contexts/RespExecutor.cs b/src/StackExchange.Redis/Contexts/RespExecutor.cs index 3355d911b..032407b61 100644 --- a/src/StackExchange.Redis/Contexts/RespExecutor.cs +++ b/src/StackExchange.Redis/Contexts/RespExecutor.cs @@ -741,8 +741,9 @@ internal static ValueTask SendWithPreambleAsync( // still hold the buffer, and disposing it would return an array that is still being written - // which shows up as somebody else's reply arriving for your command. Taking them by ref means // the caller's frames really are emptied, and their Dispose is the no-op it looks like. + DemandCacheFlags(flags, ref request); var head = preamble.Detach(CommandFlags.CommandRetryAlways); - var body = request.Detach(flags); + var body = request.Detach(flags, UncachedPrefix(executor, context.Cache, flags)); return Blocks(context) ? SendPairBlocking(executor, head, body, gate, handler) : AwaitPair(executor, head, body, gate, handler, cancellationToken); @@ -777,7 +778,8 @@ internal static ValueTask SendWithPreambleAsync( flags = flags.WithDefaultCategory(request.Command); // see the note in SendAsync var executor = context.Executor ?? throw new InvalidOperationException("No executor is configured for this context."); - var body = request.Detach(flags); + DemandCacheFlags(flags, ref request); + var body = request.Detach(flags, UncachedPrefix(executor, context.Cache, flags)); return Blocks(context) ? SendPairBlocking(executor, preamble, body, gate, handler) : AwaitPair(executor, preamble, body, gate, handler, cancellationToken); @@ -1106,12 +1108,13 @@ public static TResult Send( // the category has to be settled before PermitsCaching sees it. At the call site that was // convention; here it is the order of the statements. flags = flags.WithDefaultCategory(request.Command); + DemandCacheFlags(flags, ref request); var executor = context.Executor ?? ThrowNoExecutor(ref request); var cache = context.Cache; NoteLocalWrite(cache, in request, flags); - // NoClientCache suppresses the PROBE as well as the store: opting out must mean the caller does + // CacheOptOut suppresses the PROBE as well as the store: opting out must mean the caller does // not get a cached answer either, not merely that this reply is not kept if (cache is not null && UsesCache(executor, cache, flags)) { @@ -1160,8 +1163,9 @@ public static TResult Send( // not cacheable, which is precisely the uncached case - fall through to it } - // the executor may need the bytes past this call, so hand it something it can retain - var owned = request.Detach(flags); + // the executor may need the bytes past this call, so hand it something it can retain - with + // CLIENT CACHING NO ahead of it where the caller opted this read out of a tracking server + var owned = request.Detach(flags, UncachedPrefix(executor, cache, flags)); try { var response = executor.Send(owned); @@ -1218,6 +1222,7 @@ public static ValueTask SendAsync( // the category has to be settled before PermitsCaching sees it. At the call site that was // convention; here it is the order of the statements. flags = flags.WithDefaultCategory(request.Command); + DemandCacheFlags(flags, ref request); var executor = context.Executor ?? ThrowNoExecutor(ref request); var cache = context.Cache; @@ -1246,11 +1251,12 @@ public static ValueTask SendAsync( // An executor that copies the bytes during the call is handed a view of the frame, and the frame's // array goes back to the pool here, on the thread that rented it: no lease, and nothing of the // caller's kept for the round trip. Everything else gets the lease, as before. + var prefix = UncachedPrefix(executor, cache, flags); if (executor.CopiesRequestOnSend && typeof(TResult) != typeof(RespPayload)) { try { - return executor.SendTypedAsync(request.AsLookupKey(flags), handler, cancellationToken); + return executor.SendTypedAsync(request.AsLookupKey(flags, prefix), handler, cancellationToken); } finally { @@ -1258,7 +1264,43 @@ public static ValueTask SendAsync( } } - return executor.SendTypedAsync(request.Detach(flags), handler, cancellationToken); + return executor.SendTypedAsync(request.Detach(flags, prefix), handler, cancellationToken); + } + + /// + /// The frame to write ahead of a request that is going to the server uncached, or null. + /// + /// + /// CLIENT CACHING NO, when the caller opted a read out under a tracking server - see + /// . Never inside a transaction: the frame would be queued + /// behind MULTI and add a slot to EXEC's array, which is the same reason + /// SELECT is not injected there; the transaction's own MULTI carries the opt-out for + /// the whole run instead (). + /// + private static byte[]? UncachedPrefix(RespExecutorBase executor, RespClientCache? cache, CommandFlags flags) + => cache is not null && !executor.Transactional ? cache.OptOutPrefix(flags) : null; + + /// + /// Refuse the reserved encoding of the client-cache field: both bits set. + /// + /// + /// Two bits, three meanings - default, , + /// - and the fourth is kept free rather than given a meaning + /// chosen for symmetry. Checked here, at the one place every command's flags pass, whether or not a + /// cache is configured: a reserved value should fail the same way everywhere. One mask and compare. + /// + private static void DemandCacheFlags(CommandFlags flags, ref RespRequestFrame request) + { + if ((flags & CommandFlagsInternal.MaskClientCache) == CommandFlagsInternal.MaskClientCache) + { + request.Dispose(); + ThrowBothCacheFlags(); + } + + [DoesNotReturn] + static void ThrowBothCacheFlags() => throw new ArgumentException( + $"{nameof(CommandFlags.CacheOptIn)} and {nameof(CommandFlags.CacheOptOut)} cannot be combined; the pair is reserved.", + "flags"); } /// @@ -1503,7 +1545,9 @@ private static bool TryServeFromCache( { // this caller claimed the refresh, so the request cannot simply be dropped: it IS the thing // that needs re-sending. Detach hands ownership to the background send, which disposes it. - StartRefresh(executor, request.Detach(flags), cache); + // The refreshing read re-registers the keys, so under OPTIN mode it carries CLIENT CACHING + // YES exactly as the original fill did. + StartRefresh(executor, request.Detach(flags, cache.FillPrefix), cache); } else { diff --git a/src/StackExchange.Redis/Contexts/RespHandshake.cs b/src/StackExchange.Redis/Contexts/RespHandshake.cs index 5a1502c48..7c11a5049 100644 --- a/src/StackExchange.Redis/Contexts/RespHandshake.cs +++ b/src/StackExchange.Redis/Contexts/RespHandshake.cs @@ -1347,7 +1347,16 @@ await context.SendAsync($"{command}{RedisLiterals.TRACKING}{RedisLiterals.ON}{Re } else { - await context.SendAsync($"{command}{RedisLiterals.TRACKING}{RedisLiterals.ON}").ConfigureAwait(false); + // OPTOUT for Tracking: measured identical to plain ON against Redis 8.9 (design notes 6.13), + // and it is what makes CLIENT CACHING NO - the wire half of CommandFlags.CacheOptOut - legal on + // this connection. OPTIN for TrackingOptIn and TrackingByPrefix: nothing is tracked unless a + // read is written behind CLIENT CACHING YES, which is what CommandFlags.CacheOptIn does - and + // what TrackingByPrefix does for an unflagged read inside its prefixes. The prefixes themselves + // stay client-side there: PREFIX is refused without BCAST. + var refinement = options.ResolvedTrackingMode is Caching.CacheTrackingMode.TrackingOptIn or Caching.CacheTrackingMode.TrackingByPrefix + ? RedisLiterals.OPTIN + : RedisLiterals.OPTOUT; + await context.SendAsync($"{command}{RedisLiterals.TRACKING}{RedisLiterals.ON}{refinement}").ConfigureAwait(false); } } diff --git a/src/StackExchange.Redis/Contexts/RespOperationBatch.cs b/src/StackExchange.Redis/Contexts/RespOperationBatch.cs index f9012a347..6b6afcd97 100644 --- a/src/StackExchange.Redis/Contexts/RespOperationBatch.cs +++ b/src/StackExchange.Redis/Contexts/RespOperationBatch.cs @@ -215,7 +215,7 @@ internal override ValueTask SendVoidAsync(RespRequest request, IRespHandlerFalse if the command is fire-and-forget, and so already answered. private bool Enqueue(RespPayloadOperation operation, in RespRequest request, CancellationToken cancellationToken) { - operation.Attach(request.Span, request.Flags, cancellationToken); + operation.Attach(request.Span, request.Flags, cancellationToken, request.Prefix); operation.Diagnostics.Status = RespCommandStatus.WaitingInBacklog; operation.Slot = request.Slot; operation.Database = Database; // read if it is ever re-sent on its own, after a redirect @@ -279,7 +279,7 @@ public override ValueTask SendAsync( head.Database = Database; var body = RespPayloadOperation.Rent(); - body.Attach(request.Span, request.Flags, cancellationToken); + body.Attach(request.Span, request.Flags, cancellationToken, request.Prefix); body.Diagnostics.Status = RespCommandStatus.WaitingInBacklog; body.Slot = request.Slot; body.Database = Database; diff --git a/src/StackExchange.Redis/Contexts/RespPayloadOperation.cs b/src/StackExchange.Redis/Contexts/RespPayloadOperation.cs index f8bb5fd90..9dc880783 100644 --- a/src/StackExchange.Redis/Contexts/RespPayloadOperation.cs +++ b/src/StackExchange.Redis/Contexts/RespPayloadOperation.cs @@ -590,6 +590,7 @@ internal void Attach(in RespRequest request, CancellationToken cancellationToken span.CopyTo(buffer); _flags = request.Flags; SetRequest(new ReadOnlyMemory(buffer, 0, span.Length), owner: null, cancellationToken); + Prefix = request.Prefix; return; } @@ -597,19 +598,26 @@ internal void Attach(in RespRequest request, CancellationToken cancellationToken { _flags = request.Flags; SetRequest(memory, owner, cancellationToken); + Prefix = request.Prefix; return; } - Attach(request.Span, request.Flags, cancellationToken); + Attach(request.Span, request.Flags, cancellationToken, request.Prefix); } - internal void Attach(ReadOnlySpan request, CommandFlags flags, CancellationToken cancellationToken) + /// Begin a life for a request given as raw bytes, which are copied. + /// The request bytes, copied. + /// The request's flags. + /// Cancels the request. + /// A frame the connection writes ahead of the request, or null; see . + internal void Attach(ReadOnlySpan request, CommandFlags flags, CancellationToken cancellationToken, byte[]? prefix = null) { _flags = flags; var pool = ArrayPool.Shared; var buffer = pool.Rent(request.Length); request.CopyTo(buffer); SetRequest(new ReadOnlyMemory(buffer, 0, request.Length), pool, cancellationToken); + Prefix = prefix; } /// Fault an operation the connection refused, naming how far it actually got. diff --git a/src/StackExchange.Redis/Contexts/RespTransaction.cs b/src/StackExchange.Redis/Contexts/RespTransaction.cs index 669ee1b86..245b37022 100644 --- a/src/StackExchange.Redis/Contexts/RespTransaction.cs +++ b/src/StackExchange.Redis/Contexts/RespTransaction.cs @@ -352,7 +352,8 @@ internal async Task ExecuteAsync(CommandFlags flags = CommandFlags.None) out var exec, conditions is null ? null : OnAborted, (run, count) => target.TryWriteRun(connection, run, count), - flags)) + flags, + _context.Cache?.TransactionPrefix(queue ?? EmptyBody))) { Fail(queue, "The connection was lost before the transaction could be sent."); return false; @@ -607,19 +608,26 @@ private static void Fail(List? queue, string message) /// carrying its retry category; without them a refused EXEC is classified as never /// retryable and a caller's retry policy does nothing. /// + /// + /// A frame to write ahead of MULTI, or null: the cache's CLIENT CACHING NO for a + /// transaction that contains reads (see ). + /// internal static bool TrySendOver( RespConnection? connection, List operations, out ValueTask exec, Action? onAborted = null, Func? send = null, - CommandFlags flags = CommandFlags.None) + CommandFlags flags = CommandFlags.None, + byte[]? multiPrefix = null) { exec = default; if (connection is null || connection.IsClosed) return false; + // the MULTI carries the transaction's CLIENT CACHING NO, if the cache wants one: inside the + // transaction it would be queued and counted in EXEC's array; ahead of MULTI it governs all of it var multi = RespPayloadOperation.Rent(); - multi.Attach(MultiFrame, CommandFlags.None, default); + multi.Attach(MultiFrame, CommandFlags.None, default, multiPrefix); var execOperation = new RespExecOperation(); execOperation.Attach(ExecFrame, operations, onAborted, flags); diff --git a/src/StackExchange.Redis/Enums/CommandFlags.Category.cs b/src/StackExchange.Redis/Enums/CommandFlags.Category.cs index 273170276..fb6052f0f 100644 --- a/src/StackExchange.Redis/Enums/CommandFlags.Category.cs +++ b/src/StackExchange.Redis/Enums/CommandFlags.Category.cs @@ -77,11 +77,14 @@ internal static CommandFlags WithScanCursorCategory(this CommandFlags flags, in /// (design notes 6.13). /// /// - /// It uses the same bit as the caller-facing deliberately: the - /// effect is identical, and a caller cannot unset what the command surface has already or-ed in. + /// Locally it is - the cache is neither consulted nor filled, and a + /// caller cannot unset what the command surface has already or-ed in. It is a separate, internal bit + /// because also has a wire half (CLIENT CACHING NO ahead of + /// the command, under ), and these commands do not want + /// it: an extra frame in front of every HPTTL costs more than the tracking entry it would save. /// /// - internal static CommandFlags NeverCached(this CommandFlags flags) => flags | CommandFlags.NoClientCache; + internal static CommandFlags NeverCached(this CommandFlags flags) => flags | CommandFlagsInternal.NeverCachedFlag; internal static CommandFlags WithDefaultCategory(this CommandFlags flags, RedisCommand command) { diff --git a/src/StackExchange.Redis/Enums/CommandFlags.Internal.cs b/src/StackExchange.Redis/Enums/CommandFlags.Internal.cs index ac72747b7..7274d8eed 100644 --- a/src/StackExchange.Redis/Enums/CommandFlags.Internal.cs +++ b/src/StackExchange.Redis/Enums/CommandFlags.Internal.cs @@ -17,7 +17,16 @@ internal const CommandFlags CommandServerSpecific = (CommandFlags)(1 << 18), // "probe" (bit 19): health-check traffic. Deliberately *not* InternalCallFlag, which also decides // queuing - see Message.IsCallerFacing. - ProbeFlag = (CommandFlags)(1 << 19); + ProbeFlag = (CommandFlags)(1 << 19), + // "never cached" (bit 22): set by the command surface for reads whose reply is not a function of + // their keys - random members, TTLs. Locally it is CacheOptOut: the cache is neither consulted nor + // filled. On the wire it is NOTHING, which is the reason it is a separate bit: CacheOptOut also + // tells a tracking server not to track the key, and a CLIENT CACHING NO in front of every HPTTL + // would cost more than the tracking entry it saves. See CommandFlagsExtensions.NeverCached. + NeverCachedFlag = (CommandFlags)(1 << 22); + + /// The caller-facing client-cache field: two bits, three meanings, both set reserved. + internal const CommandFlags MaskClientCache = CommandFlags.CacheOptIn | CommandFlags.CacheOptOut; internal const CommandFlags MaskPrimaryServerPreference = CommandFlags.DemandMaster | CommandFlags.DemandReplica @@ -38,7 +47,7 @@ internal const CommandFlags | CommandFlags.FireAndForget | CommandFlags.NoRedirect | CommandFlags.NoScriptCache - | CommandFlags.NoClientCache + | MaskClientCache // CacheOptIn and CacheOptOut | MaskRetryCategory // caller may override the retry category... | CommandServerSpecific // ...and the server-specific flag | NoFlushFlag // we'll allow this one even though not advertised diff --git a/src/StackExchange.Redis/Enums/CommandFlags.cs b/src/StackExchange.Redis/Enums/CommandFlags.cs index 3a6e0bb17..76f2ae14a 100644 --- a/src/StackExchange.Redis/Enums/CommandFlags.cs +++ b/src/StackExchange.Redis/Enums/CommandFlags.cs @@ -135,14 +135,28 @@ public enum CommandFlags /// NoScriptCache = 512, + // 524288 (bit 19): "probe" - health-check traffic; never user-specified, so not visible on the public API + // (see CommandFlagsInternal.ProbeFlag) + /// - /// Indicates that this command must not be served from, or stored in, the client-side cache. + /// This command must not be served from, or stored in, the client-side cache - and, where the + /// server is tracking what this client reads, the server is told not to track it either. /// /// /// - /// Client-side caching is opt-out rather than opt-in: a command that declares a retry category no - /// more severe than and names at least one key is cacheable by - /// default. This suppresses that. + /// Client-side caching is opt-out by default: a command that declares a retry category no more + /// severe than and names at least one key is cacheable. This + /// suppresses that for one call: the cache is neither consulted nor filled. + /// + /// + /// The wire half. Under the server keeps a + /// table of the keys this client has read, and a read it will never cache is a table entry and an + /// invalidation push for nothing; so there the command goes out behind CLIENT CACHING NO, + /// written as one unit with it, and the key is not tracked. Under + /// nothing is sent, because the server tracks + /// nothing per client; under nothing is sent, + /// because nothing is tracked unless asked. Writes are never tracked, so the flag on a write does + /// nothing at all. /// /// /// Two reasons to reach for it. A value that changes so often that invalidation traffic costs more @@ -160,8 +174,41 @@ public enum CommandFlags /// than introducing it, and cannot fix it - declaring every key touched is the fix, and was the fix /// before any of this existed. /// + /// + /// Combining this with is rejected: the pair is reserved. + /// /// - NoClientCache = 1 << 19, + CacheOptOut = 1 << 20, + + /// + /// This command's reply should be cached, and the server told to track its keys, even where the + /// default would be not to. + /// + /// + /// + /// Under this is the only way anything is + /// cached. The server tracks nothing unless a read is preceded by CLIENT CACHING YES; + /// this flag writes that ahead of the command, as one unit with it, and stores the reply. A read + /// without it goes to the server and is not kept, because nothing would ever invalidate it. + /// + /// + /// Under the other modes the server already tracks (or broadcasts) everything, so nothing is sent; + /// what remains is the local effect: the reply is stored on its first miss, bypassing + /// . The caller has said "cache this", and making them + /// say it twice would be odd. + /// + /// + /// It cannot make a command cacheable that is not: a write, a keyless read, a read whose answer is + /// not a function of its keys (random members, TTLs), or - under + /// - a key outside + /// , is still refused, with the same counters saying why. + /// Combining this with is rejected: the pair is reserved. + /// + /// + CacheOptIn = 1 << 21, + + // 4194304 (bit 22): "never cached" - set by the command surface for reads whose reply is not a function + // of their keys; never user-specified, so not visible on the public API (see CommandFlagsInternal.NeverCachedFlag) // 1024: used for "no flush"; never user-specified, so not visible on the public API diff --git a/src/StackExchange.Redis/Groups/Keys.Methods.cs b/src/StackExchange.Redis/Groups/Keys.Methods.cs index ef7aa526d..1c02668ef 100644 --- a/src/StackExchange.Redis/Groups/Keys.Methods.cs +++ b/src/StackExchange.Redis/Groups/Keys.Methods.cs @@ -361,7 +361,7 @@ public static ValueTask MoveAsync(this RespKeys keys, RedisKey key, int da /// other read. It is just rarely worth it: DUMP is a migration and backup primitive, so the /// read is usually one-shot and the payload is the whole value. A bulk migration will therefore /// spend the cache's budget on entries nothing will ever read again. Pass - /// for that; genuinely large payloads are refused anyway, + /// for that; genuinely large payloads are refused anyway, /// by . /// /// diff --git a/src/StackExchange.Redis/Protocol/RespRequest.cs b/src/StackExchange.Redis/Protocol/RespRequest.cs index a0d7aa32d..3dddf872b 100644 --- a/src/StackExchange.Redis/Protocol/RespRequest.cs +++ b/src/StackExchange.Redis/Protocol/RespRequest.cs @@ -21,7 +21,7 @@ namespace StackExchange.Redis.Protocol /// past the call. /// /// - /// Lifetime. Whoever retains, releases. hands back a key holding + /// Lifetime. Whoever retains, releases. hands back a key holding /// one reference; takes another. Dispose each one exactly once. The rule for the /// dictionary is that the STORED key holds its own reference for as long as it is in the dictionary - /// see the remarks on - which is what section 6.4 of the design doc means by "the @@ -56,9 +56,11 @@ internal RespRequest( int argCount = 0, CommandFlags flags = CommandFlags.None, RedisCommand command = RedisCommand.UNKNOWN, - int database = -1) + int database = -1, + byte[]? prefix = null) { _array = array; + Prefix = prefix; _lease = lease; _offset = offset; _length = length; @@ -88,6 +90,20 @@ internal RespRequest( /// public CommandFlags Flags { get; } + /// + /// One complete frame to write immediately ahead of this request, as one unit with it, or + /// ; its reply is discarded by the connection. + /// + /// + /// CLIENT CACHING YES|NO, decided by the cache from the tracking mode and the command's + /// flags. Not part of the key: equality, hashing and see only the + /// command, so a prefixed request and its unprefixed twin look up the same entry - the prefix + /// says how the server should treat the read, not what was read. Dropped by + /// for the same reason. A fixed, shared buffer: never pooled, never + /// released. See RESPite.Operations.IRespMessage.Prefix. + /// + internal byte[]? Prefix { get; } + /// /// How many arguments were keys, or -1 when the request cannot report them. /// @@ -115,7 +131,7 @@ public ReadOnlySpan GetKey(in KeyRange range) /// Whether this key owns a reference to its buffer, and so may be stored. /// /// - /// False for a key from , which borrows the frame's buffer and is + /// False for a key from , which borrows the frame's buffer and is /// valid only until the frame is disposed. Storing a borrowed key would put a pooled array into a /// cache and then hand it back to the pool - design doc section 6.4, whose failure mode is wrong data /// served from cache rather than a crash. refuses, so the documented @@ -134,7 +150,7 @@ internal RespRequest ToOwned() var array = ArrayPool.Shared.Rent(Math.Max(span.Length, 1)); span.CopyTo(array); return new RespRequest( - array, RefCountedBuffer.Adopt(array, array.Length), 0, span.Length, _keyMarks, Slot, ArgCount, Flags, Command, Database); + array, RefCountedBuffer.Adopt(array, array.Length), 0, span.Length, _keyMarks, Slot, ArgCount, Flags, Command, Database, Prefix); } /// Take a reference of our own on these bytes, rather than copying them. @@ -265,7 +281,7 @@ public bool Equals(RespRequest other) /// /// /// Tests the buffer rather than the lease, because owning a lease is not the same question as - /// having content: a BORROWED request - , or a shared + /// having content: a BORROWED request - , or a shared /// fixed rendering - has no lease and is perfectly readable. Asking the lease reported every one of /// those as "(empty)" in diagnostics, which is exactly when you are reading them. /// diff --git a/src/StackExchange.Redis/Protocol/RespRequestFrame.cs b/src/StackExchange.Redis/Protocol/RespRequestFrame.cs index 7e71b8084..d9b77fbf1 100644 --- a/src/StackExchange.Redis/Protocol/RespRequestFrame.cs +++ b/src/StackExchange.Redis/Protocol/RespRequestFrame.cs @@ -151,7 +151,7 @@ public readonly int TryGetKeys(scoped Span target) /// /// As , for a caller holding the buffer and marks rather than a frame - - /// which is how answers the same question after . + /// which is how answers the same question after . /// internal static int ResolveKeys( byte[] buffer, @@ -309,7 +309,12 @@ private static KeyRange PayloadOf(byte[] buffer, int offset) /// the frame, so a copy taken earlier still holds the array reference and must not be disposed. /// /// - public RespRequest Detach(CommandFlags flags = CommandFlags.None) + public RespRequest Detach(CommandFlags flags = CommandFlags.None) => Detach(flags, prefix: null); + + /// + /// The request's flags. + /// A frame to write ahead of the request, or null; see . + internal RespRequest Detach(CommandFlags flags, byte[]? prefix) { var buffer = _buffer ?? throw new ObjectDisposedException(nameof(RespRequestFrame)); _buffer = null; // ownership moves to the lease @@ -323,7 +328,8 @@ public RespRequest Detach(CommandFlags flags = CommandFlags.None) ArgCount, flags, Command, - Database); + Database, + prefix); } /// @@ -333,17 +339,22 @@ public RespRequest Detach(CommandFlags flags = CommandFlags.None) /// /// /// This is the zero-allocation path, and it is the common one: on a cache HIT the caller never wanted - /// the buffer, so paying for a lease to find that out is pure waste. allocates a + /// the buffer, so paying for a lease to find that out is pure waste. allocates a /// - small, but one per lookup, which is exactly the kind of per-call - /// cost this whole design exists to remove. Measured: 48 bytes a lookup with , + /// cost this whole design exists to remove. Measured: 48 bytes a lookup with , /// zero with this. /// /// - /// The returned key cannot be retained and so cannot be stored; call on a miss, + /// The returned key cannot be retained and so cannot be stored; call on a miss, /// when ownership is actually wanted. /// /// - public RespRequest AsLookupKey(CommandFlags flags = CommandFlags.None) + public RespRequest AsLookupKey(CommandFlags flags = CommandFlags.None) => AsLookupKey(flags, prefix: null); + + /// + /// The request's flags. + /// A frame to write ahead of the request, or null; see . + internal RespRequest AsLookupKey(CommandFlags flags, byte[]? prefix) { var buffer = _buffer ?? throw new ObjectDisposedException(nameof(RespRequestFrame)); return new RespRequest( @@ -356,7 +367,8 @@ public RespRequest AsLookupKey(CommandFlags flags = CommandFlags.None) ArgCount, flags, Command, - Database); + Database, + prefix); } /// Return the underlying buffer to the pool; safe to call more than once. diff --git a/src/StackExchange.Redis/PublicAPI/PublicAPI.Unshipped.txt b/src/StackExchange.Redis/PublicAPI/PublicAPI.Unshipped.txt index 323c2484c..972745ffa 100644 --- a/src/StackExchange.Redis/PublicAPI/PublicAPI.Unshipped.txt +++ b/src/StackExchange.Redis/PublicAPI/PublicAPI.Unshipped.txt @@ -73,6 +73,8 @@ StackExchange.Redis.Caching.CacheTrackingMode StackExchange.Redis.Caching.CacheTrackingMode.Broadcast = 1 -> StackExchange.Redis.Caching.CacheTrackingMode StackExchange.Redis.Caching.CacheTrackingMode.Default = 0 -> StackExchange.Redis.Caching.CacheTrackingMode StackExchange.Redis.Caching.CacheTrackingMode.Tracking = 2 -> StackExchange.Redis.Caching.CacheTrackingMode +StackExchange.Redis.Caching.CacheTrackingMode.TrackingOptIn = 3 -> StackExchange.Redis.Caching.CacheTrackingMode +StackExchange.Redis.Caching.CacheTrackingMode.TrackingByPrefix = 4 -> StackExchange.Redis.Caching.CacheTrackingMode StackExchange.Redis.Caching.ClientCache StackExchange.Redis.Caching.ClientCache.Bytes.get -> long StackExchange.Redis.Caching.ClientCache.Count.get -> int @@ -87,7 +89,8 @@ StackExchange.Redis.Caching.ClientCache.Refreshes.get -> long StackExchange.Redis.Caching.ClientCache.ServedStale.get -> long StackExchange.Redis.Caching.ClientCache.Stored.get -> long StackExchange.Redis.Caching.ClientCache.TrackedKeys.get -> int -StackExchange.Redis.CommandFlags.NoClientCache = 524288 -> StackExchange.Redis.CommandFlags +StackExchange.Redis.CommandFlags.CacheOptIn = 2097152 -> StackExchange.Redis.CommandFlags +StackExchange.Redis.CommandFlags.CacheOptOut = 1048576 -> StackExchange.Redis.CommandFlags StackExchange.Redis.CommandFlagsExtensions static StackExchange.Redis.CommandFlagsExtensions.WithRetryCategory(this StackExchange.Redis.CommandFlags flags, StackExchange.Redis.CommandFlags category) -> StackExchange.Redis.CommandFlags StackExchange.Redis.Config diff --git a/src/StackExchange.Redis/RedisLiterals.cs b/src/StackExchange.Redis/RedisLiterals.cs index 2e8e7a125..6a2bdc7e5 100644 --- a/src/StackExchange.Redis/RedisLiterals.cs +++ b/src/StackExchange.Redis/RedisLiterals.cs @@ -124,6 +124,8 @@ public static readonly RedisValue TRACKING = RedisValue.FromRaw("TRACKING"u8), // ON is declared once, up with OFF; CLIENT TRACKING and CLIENT MAINT_NOTIFICATIONS share it BCAST = RedisValue.FromRaw("BCAST"u8), + OPTIN = RedisValue.FromRaw("OPTIN"u8), + OPTOUT = RedisValue.FromRaw("OPTOUT"u8), PREFIX = RedisValue.FromRaw("PREFIX"u8), SETINFO = RedisValue.FromRaw("SETINFO"u8), SETNAME = RedisValue.FromRaw("SETNAME"u8), diff --git a/tests/RESPite.Tests/RespConnectionTests.cs b/tests/RESPite.Tests/RespConnectionTests.cs index 252262acf..d90dcc518 100644 --- a/tests/RESPite.Tests/RespConnectionTests.cs +++ b/tests/RESPite.Tests/RespConnectionTests.cs @@ -694,4 +694,127 @@ public async Task ConcurrentSendersDoNotInterleaveTheirBytes() Assert.Equal(messages.Length, connection.PendingCount); } + /// A message that carries a , as a cache-modified read does. + private sealed class PrefixedMessage : RespMessageBase + { + protected override string? Parse(ref RespReader reader) + => reader.TryGetSpan(out var span) ? Encoding.UTF8.GetString(span.ToArray()) : null; + + internal static PrefixedMessage For(string prefix, string command) + { + var message = new PrefixedMessage(); + message.SetRequest(Encoding.UTF8.GetBytes(command), null, default); + message.Prefix = Encoding.UTF8.GetBytes(prefix); + return message; + } + } + + private const string CachingYes = "*3\r\n$6\r\nCLIENT\r\n$7\r\nCACHING\r\n$3\r\nYES\r\n"; + + [Fact] + public void APrefixIsWrittenImmediatelyBeforeItsRequestAndItsReplyIsSwallowed() + { + // the whole point: CLIENT CACHING applies to the next command on the connection, so the two must be + // one write; and the +OK it earns is nobody's result + var (connection, transport) = Connect(); + var message = PrefixedMessage.For(CachingYes, "*1\r\n$4\r\nPING\r\n"); + + Assert.True(connection.Send(message)); + Assert.Equal("*3|$6|CLIENT|$7|CACHING|$3|YES|*1|$4|PING|", transport.Written); + Assert.Equal(2, connection.PendingCount); // one reply owed for the prefix, one for the request + + transport.Receive("+OK\r\n+PONG\r\n"); + Assert.Equal("PONG", message.GetResult(message.Token)); + Assert.Equal(0, connection.PendingCount); + Assert.Equal(0, connection.PrefixErrors); + } + + [Fact] + public void APrefixErrorIsCountedNotSurfaced() + { + // the modifier was refused; the command behind it still ran, so failing the read would be worse + // than the degradation - but the refusal must be visible to whoever asks why nothing is tracked + var (connection, transport) = Connect(); + var message = PrefixedMessage.For(CachingYes, "*1\r\n$4\r\nPING\r\n"); + Assert.True(connection.Send(message)); + + transport.Receive("-ERR CLIENT CACHING YES is only valid when tracking is enabled in OPTIN mode.\r\n+PONG\r\n"); + Assert.Equal("PONG", message.GetResult(message.Token)); + Assert.Equal(1, connection.PrefixErrors); + Assert.Equal("ERR CLIENT CACHING YES is only valid when tracking is enabled in OPTIN mode.", connection.LastPrefixError); + } + + [Fact] + public void APrefixInARunStaysNextToItsOwnRequest() + { + // a batch is a run; the second element's modifier must land between the first and the second, not + // at the front of the run, and the replies must still find their owners + var (connection, transport) = Connect(); + var first = TextMessage.For("*1\r\n$1\r\nA\r\n"); + var second = PrefixedMessage.For(CachingYes, "*1\r\n$1\r\nB\r\n"); + var third = TextMessage.For("*1\r\n$1\r\nC\r\n"); + + Assert.True(connection.Send([RespRunEntry.Of(first), RespRunEntry.Of(second), RespRunEntry.Of(third)], 3)); + Assert.Equal("*1|$1|A|*3|$6|CLIENT|$7|CACHING|$3|YES|*1|$1|B|*1|$1|C|", transport.Written); + + transport.Receive("+a\r\n+OK\r\n+b\r\n+c\r\n"); + Assert.Equal("a", first.GetResult(first.Token)); + Assert.Equal("b", second.GetResult(second.Token)); + Assert.Equal("c", third.GetResult(third.Token)); + Assert.Equal(0, connection.PendingCount); + } + + [Fact] + public void APrefixIsWrittenWhenTheHolderDrainsTheInbox() + { + // the combining path hands a busy sender's message to whoever holds the lock; the prefix has to + // travel with it, or a CLIENT CACHING is silently dropped exactly when the connection is busiest + var transport = new InterceptingTransport(); + var connection = new RespConnection(transport) { CombineWrites = true }; + var blocker = TextMessage.For("*1\r\n$1\r\nX\r\n"); + var combined = PrefixedMessage.For(CachingYes, "*1\r\n$1\r\nY\r\n"); + transport.OnWrite = () => + { + transport.OnWrite = null; + // inside the holder's write: the lock is taken, so this lands in the inbox and is drained by the holder + Assert.True(connection.Send(combined)); + }; + + Assert.True(connection.Send(blocker)); + Assert.Equal(3, connection.PendingCount); // X, the prefix's discard, Y + + transport.Receive("+x\r\n+OK\r\n+y\r\n"); + Assert.Equal("x", blocker.GetResult(blocker.Token)); + Assert.Equal("y", combined.GetResult(combined.Token)); + } + + [Fact] + public void AnOutOfBandPushBetweenThePrefixReplyAndTheRequestReplyConsumesNothing() + { + // RESP3 invalidations arrive whenever the server likes, including between a prefix's +OK and the + // reply it precedes. A push is classified before anything is dequeued, so it must not take the + // sink's entry or the request's: +OK to the sink, the push out of band, the reply to the request. + var (connection, transport) = Connect(); + var message = PrefixedMessage.For(CachingYes, "*2\r\n$3\r\nGET\r\n$1\r\nk\r\n"); + Assert.True(connection.Send(message)); + + transport.Receive("+OK\r\n>2\r\n$10\r\ninvalidate\r\n*1\r\n$1\r\nk\r\n$1\r\nv\r\n"); + Assert.Equal("v", message.GetResult(message.Token)); + Assert.Equal(0, connection.PendingCount); + Assert.Equal(0, connection.PrefixErrors); + } + + [Fact] + public void ThePrefixDoesNotOutliveTheLifeItWasAttachedFor() + { + // a pooled operation is rented again for an unrelated command; the previous life's modifier must + // not go out in front of it + var (connection, transport) = Connect(); + var message = PrefixedMessage.For(CachingYes, "*1\r\n$4\r\nPING\r\n"); + Assert.True(connection.Send(message)); + transport.Receive("+OK\r\n+PONG\r\n"); + Assert.Equal("PONG", message.GetResult(message.Token)); // the life ends here + + Assert.Null(message.Prefix); + } } diff --git a/tests/StackExchange.Redis.Tests/FormatTests.cs b/tests/StackExchange.Redis.Tests/FormatTests.cs index 1e41dcecf..c36e42f34 100644 --- a/tests/StackExchange.Redis.Tests/FormatTests.cs +++ b/tests/StackExchange.Redis.Tests/FormatTests.cs @@ -93,6 +93,16 @@ public void ParseAbstractUnixDomainSocketEndPoint() #else [InlineData(CommandFlags.PreferReplica | CommandFlags.FireAndForget, "PreferMaster, FireAndForget, PreferReplica")] // 2-bit flag is hit-and-miss [InlineData(CommandFlags.DemandReplica | CommandFlags.FireAndForget, "PreferMaster, FireAndForget, DemandReplica")] // 2-bit flag is hit-and-miss +#endif +#if NETFRAMEWORK + // netfx appends the second zero-valued name (PreferMaster) to every non-zero value; see the note on CommandFlags + [InlineData(CommandFlags.CacheOptOut, "PreferMaster, CacheOptOut")] + [InlineData(CommandFlags.CacheOptIn, "PreferMaster, CacheOptIn")] + [InlineData(CommandFlags.FireAndForget | CommandFlags.CacheOptIn, "PreferMaster, FireAndForget, CacheOptIn")] +#else + [InlineData(CommandFlags.CacheOptOut, "CacheOptOut")] + [InlineData(CommandFlags.CacheOptIn, "CacheOptIn")] + [InlineData(CommandFlags.FireAndForget | CommandFlags.CacheOptIn, "FireAndForget, CacheOptIn")] #endif public void CommandFlagsFormatting(CommandFlags value, string expected) { diff --git a/tests/StackExchange.Redis.Tests/RespClientCacheTests.cs b/tests/StackExchange.Redis.Tests/RespClientCacheTests.cs index f908fdd0b..9b2ab6b9a 100644 --- a/tests/StackExchange.Redis.Tests/RespClientCacheTests.cs +++ b/tests/StackExchange.Redis.Tests/RespClientCacheTests.cs @@ -566,13 +566,13 @@ public void UnsetCategoryIsRefusedEvenThoughItComparesBelowReadOnly() } [Fact] - public void NoClientCacheSuppressesStoring() + public void CacheOptOutSuppressesStoring() { using var cache = new RespClientCache(new CacheOptions { Admission = CacheAdmission.OnFirstMiss }); var frame = Get("abc"); Assert.False(cache.TryBeginFill( - ref frame, 0, CommandFlags.CommandRetryReadOnly | CommandFlags.NoClientCache, out _)); + ref frame, 0, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut, out _)); frame.Dispose(); Assert.Equal(0, cache.Count); @@ -580,7 +580,7 @@ public void NoClientCacheSuppressesStoring() } [Fact] - public void NoClientCacheAlsoSuppressesServingFromCache() + public void CacheOptOutAlsoSuppressesServingFromCache() { using var cache = new RespClientCache(new CacheOptions { Admission = CacheAdmission.OnFirstMiss }); var executor = new FakeExecutor("$5\r\nhello\r\n"); @@ -592,7 +592,7 @@ public void NoClientCacheAlsoSuppressesServingFromCache() // opting out must mean the caller does not RECEIVE a cached answer either - not merely that this // reply is not kept. Otherwise "don't cache this" silently still serves stale data. var opted = Get("abc"); - Via(executor, cache).Raw.Send(ref opted, CommandFlags.CommandRetryReadOnly | CommandFlags.NoClientCache, TextHandler.Instance, default); + Via(executor, cache).Raw.Send(ref opted, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut, TextHandler.Instance, default); Assert.Equal(2, executor.Sent); // ... and the entry is untouched for callers who did not opt out @@ -632,13 +632,13 @@ public void RefusalCountersSayWhyNothingWasCached() } [Fact] - public void NoClientCacheIsUserSelectable() + public void CacheOptOutIsUserSelectable() { // an external surface has to be able to pass it through Execute, or the opt-out is unreachable // for exactly the callers who need it Assert.Equal( - CommandFlags.NoClientCache, - CommandFlagsInternal.UserSelectableFlags & CommandFlags.NoClientCache); + CommandFlags.CacheOptOut, + CommandFlagsInternal.UserSelectableFlags & CommandFlags.CacheOptOut); } [Fact] @@ -752,10 +752,10 @@ public void ADetachedRequestCanStillAnswerForItself() // the point of the widening: an executor decorator sees a RespRequest and nothing else, so the // request has to carry what routing, retry and caching each need to ask using var frame = Ctx.Render($"{RedisCommand.MGET}{(RedisKey)"a"}{(RedisKey)"b"}{(RedisKey)"c"}"); - using var request = frame.Detach(CommandFlags.CommandRetryReadOnly | CommandFlags.NoClientCache); + using var request = frame.Detach(CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut); Assert.Equal(4, request.ArgCount); - Assert.Equal(CommandFlags.CommandRetryReadOnly | CommandFlags.NoClientCache, request.Flags); + Assert.Equal(CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut, request.Flags); Assert.Equal(3, request.KeyCount); var ranges = new KeyRange[request.KeyCount]; @@ -781,7 +781,7 @@ public void RoutingNeedsOnlyTheSlotAndTheRequestCarriesIt() public void MetadataDoesNotAffectRequestIdentity() { using var plain = Get("abc").Detach(CommandFlags.CommandRetryReadOnly); - using var different = Get("abc").Detach(CommandFlags.CommandRetryNever | CommandFlags.NoClientCache); + using var different = Get("abc").Detach(CommandFlags.CommandRetryNever | CommandFlags.CacheOptOut); // identity is the rendered bytes and only the rendered bytes: two callers issuing the same command // with different flags are asking the same question, so they must share a cache entry @@ -1222,7 +1222,7 @@ public async Task ConcurrentSweepsReclaimEachEntryExactlyOnce() /// pass or fail on line ordering. The message names both halves, since either one could be the mistake. /// [Fact] - public void PrefixesRequireBroadcastTracking() + public void PrefixesRequireAModeThatReadsThem() { var options = new CacheOptions { @@ -1233,25 +1233,33 @@ public void PrefixesRequireBroadcastTracking() var ex = Assert.Throws(() => new RespClientCache(options)); Assert.Contains("Broadcast", ex.Message); + Assert.Contains("TrackingByPrefix", ex.Message); - // ...and the same list is fine the other way round, whichever order it was written in + // ...and the same list is fine with either mode that reads it, whichever order it was written in using var ok = new RespClientCache(new CacheOptions { Admission = CacheAdmission.OnFirstMiss, Prefixes = ["app:"], TrackingMode = CacheTrackingMode.Broadcast }); using var alsoOk = new RespClientCache(new CacheOptions { Admission = CacheAdmission.OnFirstMiss, TrackingMode = CacheTrackingMode.Broadcast, Prefixes = ["app:"] }); + using var byPrefix = new RespClientCache(new CacheOptions { TrackingMode = CacheTrackingMode.TrackingByPrefix, Prefixes = ["app:"] }); + + // the mode that is nothing but the list cannot do without it + var missing = Assert.Throws(() => new RespClientCache(new CacheOptions { TrackingMode = CacheTrackingMode.TrackingByPrefix })); + Assert.Contains("Prefixes", missing.Message); } /// - /// means tracking, unless prefixes are given - which only mean - /// anything under broadcast, so giving them selects it rather than failing. + /// means tracking, unless prefixes are given - then the mode that + /// reads them locally, , rather than broadcast: its cost + /// is bounded by this client's own misses, where broadcast is billed by the server's write rate. /// [Fact] public void DefaultModeIsTrackingUnlessPrefixesAreGiven() { Assert.Equal(CacheTrackingMode.Tracking, new CacheOptions().ResolvedTrackingMode); - Assert.Equal(CacheTrackingMode.Broadcast, new CacheOptions { Prefixes = ["app:"] }.ResolvedTrackingMode); + Assert.Equal(CacheTrackingMode.TrackingByPrefix, new CacheOptions { Prefixes = ["app:"] }.ResolvedTrackingMode); // a named mode is never second-guessed Assert.Equal(CacheTrackingMode.Broadcast, new CacheOptions { TrackingMode = CacheTrackingMode.Broadcast }.ResolvedTrackingMode); Assert.Equal(CacheTrackingMode.Tracking, new CacheOptions { TrackingMode = CacheTrackingMode.Tracking }.ResolvedTrackingMode); + Assert.Equal(CacheTrackingMode.Broadcast, new CacheOptions { TrackingMode = CacheTrackingMode.Broadcast, Prefixes = ["app:"] }.ResolvedTrackingMode); // and prefixes with Default build a cache without complaint using var cache = new RespClientCache(new CacheOptions { Admission = CacheAdmission.OnFirstMiss, Prefixes = ["app:"] }); @@ -1823,12 +1831,12 @@ public async Task OurOwnWriteEvictsWhatWeCached() /// A read of ours does not invalidate anything. /// /// The other half: if the mutation test were simply "anything that is not cacheable invalidates", a - /// fire-and-forget read or a NoClientCache read would evict the very entry it declined to use. + /// fire-and-forget read or a CacheOptOut read would evict the very entry it declined to use. /// [Theory] [InlineData(CommandFlags.CommandRetryReadOnly)] [InlineData(CommandFlags.CommandRetryReadOnly | CommandFlags.FireAndForget)] - [InlineData(CommandFlags.CommandRetryReadOnly | CommandFlags.NoClientCache)] + [InlineData(CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut)] public async Task ReadsDoNotInvalidate(CommandFlags readFlags) { using var cache = new RespClientCache(new CacheOptions { Admission = CacheAdmission.OnFirstMiss }); @@ -1950,4 +1958,291 @@ public async Task AWriteWithUnknowableKeysStillInvalidatesTheKeysItNamed() Assert.False(cache.TryGet(probe.AsLookupKey(), 0, out _), "a key past the bitmap must still be invalidated"); } + // ---- opt-in / opt-out: the three-valued cache field and its wire half ---- + + private const string CachingYes = "*3|$6|CLIENT|$7|CACHING|$3|YES|"; + private const string CachingNo = "*3|$6|CLIENT|$7|CACHING|$2|NO|"; + + private static CacheOptions Mode(CacheTrackingMode mode, CacheAdmission admission = CacheAdmission.OnFirstMiss) + => new() { Admission = admission, TrackingMode = mode }; + + private static string? PrefixOf(in RespRequest request) => request.Prefix is { } prefix ? Text(prefix) : null; + + [Fact] + public void OptInModeStoresOnlyAReadThatOptedIn() + { + // under OPTIN the server tracks nothing unless asked, so a default read must not be kept - nothing + // would ever invalidate it - and must not even probe; only CacheOptIn reads take part + using var cache = new RespClientCache(Mode(CacheTrackingMode.TrackingOptIn)); + var executor = new FakeExecutor("$5\r\nhello\r\n") { ParkRequests = true }; + + var plain = Get("abc"); + Via(executor, cache).Raw.Send(ref plain, CommandFlags.CommandRetryReadOnly, TextHandler.Instance, default); + Assert.Equal(1, executor.Sent); + Assert.Equal(0, cache.Count); + Assert.Equal(1, cache.RefusedByFlags); + Assert.Null(PrefixOf(executor.Parked[0])); // and nothing was said to the server + + var opted = Get("abc"); + Via(executor, cache).Raw.Send(ref opted, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, TextHandler.Instance, default); + Assert.Equal(2, executor.Sent); + Assert.Equal(1, cache.Count); + Assert.Equal(CachingYes, PrefixOf(executor.Parked[1])); // the read that fills says so + + var again = Get("abc"); + Via(executor, cache).Raw.Send(ref again, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, TextHandler.Instance, default); + Assert.Equal(2, executor.Sent); // a hit + + // a default read does not get the opted-in entry either: it did not ask to take part + var plainAgain = Get("abc"); + Via(executor, cache).Raw.Send(ref plainAgain, CommandFlags.CommandRetryReadOnly, TextHandler.Instance, default); + Assert.Equal(3, executor.Sent); + Assert.Null(PrefixOf(executor.Parked[2])); + } + + [Fact] + public void TrackingModeSaysNoForAnOptedOutReadAndNothingOtherwise() + { + using var cache = new RespClientCache(Mode(CacheTrackingMode.Tracking)); + var executor = new FakeExecutor("$5\r\nhello\r\n") { ParkRequests = true }; + + var plain = Get("abc"); + Via(executor, cache).Raw.Send(ref plain, CommandFlags.CommandRetryReadOnly, TextHandler.Instance, default); + Assert.Null(PrefixOf(executor.Parked[0])); // OPTOUT tracks by default; nothing to add + + var opted = Get("def"); + Via(executor, cache).Raw.Send(ref opted, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut, TextHandler.Instance, default); + Assert.Equal(CachingNo, PrefixOf(executor.Parked[1])); // not kept here, so not tracked there + Assert.Equal(1, cache.Count); + + // a write is never tracked, so the flag on a write buys nothing and says nothing + var write = Ctx.Render($"{RedisCommand.SET}{(RedisKey)"abc"}{(RedisValue)"x"}"); + Via(executor, cache).Raw.Send(ref write, CommandFlags.CommandRetryNever | CommandFlags.CacheOptOut, TextHandler.Instance, default); + Assert.Null(PrefixOf(executor.Parked[2])); + + // and CacheOptIn under OPTOUT has only its local meaning: stored, nothing said + var optedIn = Get("ghi"); + Via(executor, cache).Raw.Send(ref optedIn, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, TextHandler.Instance, default); + Assert.Null(PrefixOf(executor.Parked[3])); + Assert.Equal(2, cache.Count); + } + + [Fact] + public void BroadcastModeNeverSpeaksToTheServer() + { + // BCAST and OPTIN/OPTOUT are mutually exclusive on the server, so neither flag may put a CLIENT + // CACHING on the wire; both keep their local meaning + using var cache = new RespClientCache(Mode(CacheTrackingMode.Broadcast)); + var executor = new FakeExecutor("$5\r\nhello\r\n") { ParkRequests = true }; + + var optedOut = Get("abc"); + Via(executor, cache).Raw.Send(ref optedOut, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut, TextHandler.Instance, default); + Assert.Null(PrefixOf(executor.Parked[0])); + Assert.Equal(0, cache.Count); + + var optedIn = Get("abc"); + Via(executor, cache).Raw.Send(ref optedIn, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, TextHandler.Instance, default); + Assert.Null(PrefixOf(executor.Parked[1])); + Assert.Equal(1, cache.Count); + } + + [Fact] + public void CacheOptInBypassesAdmission() + { + // the caller has made the admission decision; under OPTIN mode the first miss is also the one that + // pays for CLIENT CACHING YES, so refusing it would track a key and then not keep it + using var cache = new RespClientCache(Mode(CacheTrackingMode.Tracking, CacheAdmission.OnRepeatedMiss)); + + var first = Get("abc"); + Assert.False(cache.TryBeginFill(ref first, 0, CommandFlags.CommandRetryReadOnly, out _)); + first.Dispose(); + Assert.Equal(1, cache.RefusedNotAdmitted); + + var opted = Get("def"); + Assert.True(cache.TryBeginFill(ref opted, 0, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, out var fill)); + Assert.True(Complete(cache, fill, "$1\r\nx\r\n")); + Assert.Equal(1, cache.RefusedNotAdmitted); + Assert.Equal(1, cache.Count); + } + + [Fact] + public void BothCacheBitsAreRejectedWithOrWithoutACache() + { + // two bits, three meanings; the fourth encoding is reserved, and a reserved value must fail the same + // way everywhere rather than quietly meaning something on a connection that happens to cache + var executor = new FakeExecutor("$5\r\nhello\r\n"); + const CommandFlags both = CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn | CommandFlags.CacheOptOut; + + var withCache = Get("abc"); + using var cache = new RespClientCache(Mode(CacheTrackingMode.Tracking)); + Assert.Throws(() => Via(executor, cache).Raw.Send(ref withCache, both, TextHandler.Instance, default)); + + var withoutCache = Get("abc"); + Assert.Throws(() => Via(executor).Raw.Send(ref withoutCache, both, TextHandler.Instance, default)); + Assert.Equal(0, executor.Sent); + + // the gate itself errs towards a miss if the pair ever reaches it + Assert.False(RespClientCache.IsCacheable(both)); + Assert.False(RespClientCache.IsCacheable(both, optIn: true)); + } + + [Fact] + public void IsCacheableHonoursTheModeAndTheMarkers() + { + Assert.True(RespClientCache.IsCacheable(CommandFlags.CommandRetryReadOnly)); + Assert.False(RespClientCache.IsCacheable(CommandFlags.CommandRetryReadOnly, optIn: true)); + Assert.True(RespClientCache.IsCacheable(CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, optIn: true)); + Assert.True(RespClientCache.IsCacheable(CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn)); + Assert.False(RespClientCache.IsCacheable(CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut)); + + // the command surface's marker is a CacheOptOut locally - and only locally; see NeverCached + Assert.False(RespClientCache.IsCacheable(CommandFlags.CommandRetryReadOnly.NeverCached())); + Assert.Equal(CommandFlags.None, CommandFlagsInternal.UserSelectableFlags & CommandFlagsInternal.NeverCachedFlag); + using var cache = new RespClientCache(Mode(CacheTrackingMode.Tracking)); + Assert.Null(cache.OptOutPrefix(CommandFlags.CommandRetryReadOnly.NeverCached())); + Assert.NotNull(cache.OptOutPrefix(CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut)); + } + + [Fact] + public void CacheOptInIsUserSelectable() + { + Assert.Equal( + CommandFlags.CacheOptIn, + CommandFlagsInternal.UserSelectableFlags & CommandFlags.CacheOptIn); + } + + [Fact] + public void ATransactionOptsOutOnlyWhenItReads() + { + // reads inside MULTI are never cached, so under OPTOUT the whole transaction is told not to track; + // a write-only transaction tracks nothing anyway, and the frame would be pure cost + using var tracking = new RespClientCache(Mode(CacheTrackingMode.Tracking)); + using var optIn = new RespClientCache(Mode(CacheTrackingMode.TrackingOptIn)); + + var write = RespPayloadOperation.Rent(); + write.Attach(Ctx.Render($"{RedisCommand.SET}{(RedisKey)"a"}{(RedisValue)"x"}").AsLookupKey().Span, CommandFlags.CommandRetryNever, default); + var read = RespPayloadOperation.Rent(); + read.Attach(Get("a").AsLookupKey().Span, CommandFlags.CommandRetryReadOnly, default); + + var writeOnly = new List { QueuedOperation.Of(write) }; + var mixed = new List { QueuedOperation.Of(write), QueuedOperation.Of(read) }; + + Assert.Null(tracking.TransactionPrefix(writeOnly)); + Assert.Equal(CachingNo, Text(tracking.TransactionPrefix(mixed)!)); + Assert.Null(optIn.TransactionPrefix(mixed)); // nothing is tracked there unless asked + + using var byPrefix = new RespClientCache(new CacheOptions { TrackingMode = CacheTrackingMode.TrackingByPrefix, Prefixes = ["a"] }); + Assert.Null(byPrefix.TransactionPrefix(mixed)); // same wire mode, same answer, prefix match or not + } + + [Fact] + public void ByPrefixModeOptsInTheReadsInsideItsPrefixes() + { + // the default admission on purpose: a read inside the prefixes is an opt-in, and an opt-in is stored + // on its FIRST miss, since that is the read that paid for CLIENT CACHING YES + using var cache = new RespClientCache(new CacheOptions { TrackingMode = CacheTrackingMode.TrackingByPrefix, Prefixes = ["app:"] }); + var executor = new FakeExecutor("$5\r\nhello\r\n") { ParkRequests = true }; + + // inside, unflagged: YES, stored, then a hit + var inside = Get("app:1"); + Via(executor, cache).Raw.Send(ref inside, CommandFlags.CommandRetryReadOnly, TextHandler.Instance, default); + Assert.Equal(1, executor.Sent); + Assert.Equal(1, cache.Count); + Assert.Equal(CachingYes, PrefixOf(executor.Parked[0])); + + var again = Get("app:1"); + Via(executor, cache).Raw.Send(ref again, CommandFlags.CommandRetryReadOnly, TextHandler.Instance, default); + Assert.Equal(1, executor.Sent); + + // outside, unflagged: sent plain, kept nowhere, and counted as the prefix refusal it is + var outside = Get("other"); + Via(executor, cache).Raw.Send(ref outside, CommandFlags.CommandRetryReadOnly, TextHandler.Instance, default); + Assert.Equal(2, executor.Sent); + Assert.Equal(1, cache.Count); + Assert.Null(PrefixOf(executor.Parked[1])); + Assert.Equal(1, cache.RefusedNotTracked); + + // straddling the boundary, unflagged: the entry would depend on a key outside, so it is out + var straddle = Ctx.Render($"{RedisCommand.MGET}{(RedisKey)"app:1"}{(RedisKey)"other"}"); + Via(executor, cache).Raw.Send(ref straddle, CommandFlags.CommandRetryReadOnly, TextHandler.Instance, default); + Assert.Equal(3, executor.Sent); + Assert.Equal(1, cache.Count); + Assert.Null(PrefixOf(executor.Parked[2])); + Assert.Equal(2, cache.RefusedNotTracked); + } + + [Fact] + public void ByPrefixModeLetsTheFlagsOverrideTheList() + { + // the list is the policy for reads that did not state one; a flag is a statement. Unlike Broadcast, + // where the list went to the server and nothing can cross it, here the server tracks whatever we + // send behind YES + using var cache = new RespClientCache(new CacheOptions { TrackingMode = CacheTrackingMode.TrackingByPrefix, Prefixes = ["app:"] }); + var executor = new FakeExecutor("$5\r\nhello\r\n") { ParkRequests = true }; + + // outside, CacheOptIn: YES, stored, then a hit (for the unflagged read too - the entry exists) + var optedIn = Get("other"); + Via(executor, cache).Raw.Send(ref optedIn, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, TextHandler.Instance, default); + Assert.Equal(1, executor.Sent); + Assert.Equal(1, cache.Count); + Assert.Equal(CachingYes, PrefixOf(executor.Parked[0])); + Assert.Equal(0, cache.RefusedNotTracked); + + var plain = Get("other"); + Via(executor, cache).Raw.Send(ref plain, CommandFlags.CommandRetryReadOnly, TextHandler.Instance, default); + Assert.Equal(1, executor.Sent); + + // inside, CacheOptOut: sent plain - OPTIN tracks nothing without a YES, so there is no NO to say - + // and kept nowhere + var optedOut = Get("app:1"); + Via(executor, cache).Raw.Send(ref optedOut, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut, TextHandler.Instance, default); + Assert.Equal(2, executor.Sent); + Assert.Equal(1, cache.Count); + Assert.Null(PrefixOf(executor.Parked[1])); + + // and a write is a write: no prefix in either direction + var write = Ctx.Render($"{RedisCommand.SET}{(RedisKey)"app:1"}{(RedisValue)"x"}"); + Via(executor, cache).Raw.Send(ref write, CommandFlags.CommandRetryNever, TextHandler.Instance, default); + Assert.Null(PrefixOf(executor.Parked[2])); + } + + [Fact] + public void ThePrefixHonoursARenamedClientCommand() + { + // the handshake's CLIENT TRACKING goes through the command map; a prefix that did not would send a + // command the server has never heard of, and under OPTIN nothing would ever be tracked + var map = CommandMap.Create(new Dictionary { ["client"] = "cli" }); + using var cache = new RespClientCache(Mode(CacheTrackingMode.TrackingOptIn), commandMap: map); + + var frame = Get("abc"); + Assert.True(cache.TryBeginFill(ref frame, 0, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, out var fill)); + Assert.Equal("*3|$3|CLI|$7|CACHING|$3|YES|", PrefixOf(fill.Key)); // the map upper-cases, as it does for every command + fill.Abandon(); + } + + [Fact] + public void OptInModeRejectsPrefixes() + { + // PREFIX needs BCAST, and OPTIN cannot be combined with BCAST - the same refusal Tracking gets + var ex = Assert.Throws(() => new RespClientCache( + new CacheOptions { TrackingMode = CacheTrackingMode.TrackingOptIn, Prefixes = ["app:"] })); + Assert.Contains("Broadcast", ex.Message); + } + + [Fact] + public void ThePrefixIsNotPartOfTheKey() + { + // a prefixed request and its unprefixed twin name the same entry: the prefix says how the server + // should treat the read, not what was read + using var cache = new RespClientCache(Mode(CacheTrackingMode.TrackingOptIn)); + var frame = Get("abc"); + Assert.True(cache.TryBeginFill(ref frame, 0, CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptIn, out var fill)); + Assert.NotNull(fill.Key.Prefix); + Assert.True(Complete(cache, fill, "$1\r\nx\r\n")); + + using var plain = Get("abc").Detach(CommandFlags.CommandRetryReadOnly); + Assert.Null(plain.Prefix); + Assert.True(cache.TryGet(plain, 0, out var hit)); + hit.Release(); + } } diff --git a/tests/StackExchange.Redis.Tests/RespInProcTrackingTests.cs b/tests/StackExchange.Redis.Tests/RespInProcTrackingTests.cs index d9016a6a9..9d5c4948b 100644 --- a/tests/StackExchange.Redis.Tests/RespInProcTrackingTests.cs +++ b/tests/StackExchange.Redis.Tests/RespInProcTrackingTests.cs @@ -48,13 +48,24 @@ private static async Task WaitFor(Func> condition, int millis = /// the first read is not guaranteed to stick; reading until one does costs nothing when it sticks first /// time, and is the difference between a test that passes alone and one that passes under load. /// - private static async Task PrimeAsync(IDatabase db, RespClientCache cache, string key, string expected, int expectedCount = 1) + private static async Task PrimeAsync(IDatabase db, RespClientCache cache, string key, string expected, int expectedCount = 1, CommandFlags flags = CommandFlags.None) { Assert.True( - await WaitFor(async () => (string?)await db.Strings.GetAsync(key) == expected && cache.Count == expectedCount), + await WaitFor(async () => (string?)await db.Strings.GetAsync(key, flags) == expected && cache.Count == expectedCount), $"'{key}' never stayed in the cache (stored: {cache.Stored}, raced: {cache.RefusedRaced}, count: {cache.Count})"); } + /// The one client on that negotiated tracking. + private static RedisClient TrackedClient(InProcessTestServer server) + { + var tracked = new List(); + server.ForAllClients(client => + { + if (client.TrackingEnabled) tracked.Add(client); + }); + return Assert.Single(tracked); + } + private async Task<(InProcessTestServer Server, ConnectionMultiplexer Muxer, RespClientCache Cache)> ConnectAsync( CacheOptions? options = null) { @@ -105,7 +116,9 @@ await WaitFor(async () => (string?)await db.Strings.GetAsync("k") == "v2"), [Fact] public async Task PrefixesAreSentAndHonoured() { - var (server, muxer, cache) = await ConnectAsync(new CacheOptions { Admission = CacheAdmission.OnFirstMiss, Prefixes = ["app:"] }); + // Broadcast by name: it is the mode that sends the list to the server, and no longer what prefixes + // alone select (that is TrackingByPrefix, which keeps them here) + var (server, muxer, cache) = await ConnectAsync(new CacheOptions { Admission = CacheAdmission.OnFirstMiss, TrackingMode = CacheTrackingMode.Broadcast, Prefixes = ["app:"] }); using var _ = server; await using var __ = muxer; @@ -248,4 +261,233 @@ public async Task Resp2WithACacheRefusesLoudly() log.WriteLine(reported); Assert.Contains("RESP3", reported); } + + // ---- opt-in / opt-out, against the server's own record of what it was told ---- + + [Fact] + public async Task TrackingModeNegotiatesOptOut() + { + // OPTOUT is identical to plain ON except that it makes CLIENT CACHING NO legal, which is what the + // CacheOptOut flag's wire half needs; so that is what the handshake asks for + var (server, muxer, cache) = await ConnectAsync(); + using var _ = server; + await using var __ = muxer; + + var db = muxer.GetDatabase(); + await db.StringSetAsync("k", "v1"); + await PrimeAsync(db, cache, "k", "v1"); + + var client = TrackedClient(server); + Assert.True(client.TrackingOptOut, "Tracking must negotiate OPTOUT"); + Assert.False(client.TrackingOptIn); + Assert.False(client.TrackingBroadcast); + Assert.True(client.IsTrackingKey(0, "k")); + } + + [Fact] + public async Task OptInModeCachesAndTracksOnlyWhatOptedIn() + { + // default admission on purpose: CacheOptIn has to be stored on its FIRST miss, since that is the + // read that paid for CLIENT CACHING YES + var (server, muxer, cache) = await ConnectAsync(new CacheOptions { TrackingMode = CacheTrackingMode.TrackingOptIn }); + using var _ = server; + await using var __ = muxer; + + var db = muxer.GetDatabase(); + await db.StringSetAsync("k", "v1"); + + // a default read: answered by the server, kept nowhere, tracked nowhere + Assert.Equal("v1", await db.Strings.GetAsync("k")); + Assert.Equal("v1", await db.Strings.GetAsync("k")); + Assert.Equal(0, cache.Count); + var client = TrackedClient(server); + Assert.True(client.TrackingOptIn, "TrackingOptIn must negotiate OPTIN"); + Assert.Equal(0, client.CachingYesCount); + Assert.False(client.IsTrackingKey(0, "k")); + + // an opted-in read: CLIENT CACHING YES went out ahead of it, the server tracks it, the cache keeps it + await PrimeAsync(db, cache, "k", "v1", flags: CommandFlags.CacheOptIn); + Assert.True(client.CachingYesCount >= 1, "the server never saw CLIENT CACHING YES"); + Assert.True(client.IsTrackingKey(0, "k"), "the server is not tracking the opted-in key"); + + var yes = client.CachingYesCount; + var stored = cache.Stored; + Assert.Equal("v1", await db.Strings.GetAsync("k", CommandFlags.CacheOptIn)); // a hit: nothing sent + Assert.Equal(yes, client.CachingYesCount); + Assert.Equal(stored, cache.Stored); + + // and the invalidation loop closes as it does in any tracking mode + await using var other = await ConnectionMultiplexer.ConnectAsync(server.GetClientConfig()); + await other.GetDatabase().StringSetAsync("k", "v2"); + Assert.True( + await WaitFor(async () => (string?)await db.Strings.GetAsync("k", CommandFlags.CacheOptIn) == "v2"), + "the invalidation for an opted-in key never arrived"); + } + + [Fact] + public async Task ByPrefixModeNegotiatesOptInAndTheListStaysHere() + { + // prefixes alone select this mode; default admission on purpose, since a read inside the prefixes is + // an opt-in and must be stored on the miss that paid for CLIENT CACHING YES + var (server, muxer, cache) = await ConnectAsync(new CacheOptions { Prefixes = ["app:"] }); + using var _ = server; + await using var __ = muxer; + + var db = muxer.GetDatabase(); + await db.StringSetAsync("app:k", "v1"); + await db.StringSetAsync("other", "o"); + + // outside the prefixes, unflagged: answered by the server, kept nowhere, tracked nowhere + Assert.Equal("o", await db.Strings.GetAsync("other")); + Assert.Equal("o", await db.Strings.GetAsync("other")); + Assert.Equal(0, cache.Count); + var client = TrackedClient(server); + Assert.True(client.TrackingOptIn, "TrackingByPrefix must negotiate OPTIN"); + Assert.False(client.TrackingBroadcast, "PREFIX would be refused without BCAST; the list must stay client-side"); + Assert.Empty(client.TrackingPrefixes); + Assert.Equal(0, client.CachingYesCount); + Assert.False(client.IsTrackingKey(0, "other")); + + // inside, unflagged: YES went out ahead of it, the server tracks it, the cache keeps it on first miss + await PrimeAsync(db, cache, "app:k", "v1"); + Assert.True(client.CachingYesCount >= 1, "the server never saw CLIENT CACHING YES"); + Assert.True(client.IsTrackingKey(0, "app:k"), "the server is not tracking the key inside the prefixes"); + + var yes = client.CachingYesCount; + var stored = cache.Stored; + Assert.Equal("v1", await db.Strings.GetAsync("app:k")); // a hit: nothing sent + Assert.Equal(yes, client.CachingYesCount); + Assert.Equal(stored, cache.Stored); + + // the flags win over the list: outside with CacheOptIn is tracked and kept... + await PrimeAsync(db, cache, "other", "o", expectedCount: 2, flags: CommandFlags.CacheOptIn); + Assert.True(client.IsTrackingKey(0, "other"), "an opted-in key outside the prefixes must still be tracked"); + + // ...and inside with CacheOptOut is neither, with nothing extra on the wire (OPTIN has no NO to say) + await db.StringSetAsync("app:j", "j"); + Assert.Equal("j", await db.Strings.GetAsync("app:j", CommandFlags.CacheOptOut)); + Assert.Equal(2, cache.Count); + Assert.Equal(0, client.CachingNoCount); + Assert.False(client.IsTrackingKey(0, "app:j")); + + // and the invalidation loop closes for a key opted in by prefix + await using var other = await ConnectionMultiplexer.ConnectAsync(server.GetClientConfig()); + await other.GetDatabase().StringSetAsync("app:k", "v2"); + Assert.True( + await WaitFor(async () => (string?)await db.Strings.GetAsync("app:k") == "v2"), + "the invalidation for a key opted in by prefix never arrived"); + } + + [Fact] + public async Task OptOutFlagTellsATrackingServerNotToTrack() + { + var (server, muxer, cache) = await ConnectAsync(); + using var _ = server; + await using var __ = muxer; + + var db = muxer.GetDatabase(); + await db.StringSetAsync("k", "v1"); + + Assert.Equal("v1", await db.Strings.GetAsync("k", CommandFlags.CacheOptOut)); + Assert.Equal(0, cache.Count); + var client = TrackedClient(server); + Assert.Equal(1, client.CachingNoCount); + Assert.False(client.IsTrackingKey(0, "k"), "an opted-out read must not leave a tracking entry behind"); + + // the same key read normally is tracked and kept, with nothing extra on the wire + await PrimeAsync(db, cache, "k", "v1"); + Assert.True(client.IsTrackingKey(0, "k")); + Assert.Equal(1, client.CachingNoCount); + } + + [Theory] + [InlineData(CacheTrackingMode.Broadcast)] + [InlineData(CacheTrackingMode.TrackingOptIn)] + public async Task OptOutFlagSaysNothingWhereTheServerWouldRefuseIt(CacheTrackingMode mode) + { + // BCAST has no per-client table to opt out of, and OPTIN tracks nothing unless asked; CLIENT CACHING + // NO would be an error on both, so the flag keeps only its local meaning + var (server, muxer, cache) = await ConnectAsync(new CacheOptions { TrackingMode = mode }); + using var _ = server; + await using var __ = muxer; + + var db = muxer.GetDatabase(); + await db.StringSetAsync("k", "v1"); + Assert.Equal("v1", await db.Strings.GetAsync("k", CommandFlags.CacheOptOut)); + Assert.Equal(0, cache.Count); + Assert.Equal(0, TrackedClient(server).CachingNoCount); + } + + [Fact] + public async Task ATransactionThatReadsOptsTheWholeTransactionOut() + { + // reads inside MULTI are never cached, so under OPTOUT the MULTI goes out behind CLIENT CACHING NO + // and the server tracks none of the transaction's keys; a write-only transaction gets nothing + var (server, muxer, cache) = await ConnectAsync(); + using var _ = server; + await using var __ = muxer; + + var db = muxer.GetDatabase(); + await db.StringSetAsync("k", "v1"); + var client = TrackedClient(server); + + var tran = db.CreateTransaction(); + var read = tran.StringGetAsync("k"); + var write = tran.StringSetAsync("w", "x"); + Assert.True(await tran.ExecuteAsync()); + Assert.Equal("v1", await read); + await write; + Assert.Equal(1, client.CachingNoCount); + Assert.False(client.IsTrackingKey(0, "k"), "a read inside MULTI must not leave a tracking entry behind"); + Assert.Equal(0, cache.Count); + + var writeOnly = db.CreateTransaction(); + var only = writeOnly.StringSetAsync("w", "y"); + Assert.True(await writeOnly.ExecuteAsync()); + await only; + Assert.Equal(1, client.CachingNoCount); // nothing to opt out of + + // the key is still perfectly cacheable outside a transaction + await PrimeAsync(db, cache, "k", "v1"); + Assert.True(client.IsTrackingKey(0, "k")); + } + + [Fact] + public async Task AnInvalidationBetweenThePrefixReplyAndTheReadReplyIsRoutedNotMatched() + { + // the server may push an invalidation at any point, including between the +OK for CLIENT CACHING + // YES and the reply to the GET behind it. A push is routed before anything is dequeued, so neither + // the prefix's discard nor the read's own entry may be consumed by it - and every reply after it + // must still land on its own command. Forced here, because a real server cannot be timed. + var (server, muxer, cache) = await ConnectAsync(new CacheOptions { TrackingMode = CacheTrackingMode.TrackingOptIn }); + using var _ = server; + await using var __ = muxer; + + var db = muxer.GetDatabase(); + await db.StringSetAsync("k", "v1"); + await db.StringSetAsync("other", "o"); + + var pushes = 0; + server.AfterReply = (client, command, subcommand) => + { + if (command == RedisCommand.CLIENT && string.Equals(subcommand, "CACHING", StringComparison.OrdinalIgnoreCase)) + { + pushes++; + // the key being read, so the fill is refused as raced - the correct answer - and the result + // still has to be the GET's reply, not the push and not the +OK + server.SendInvalidation(client, [(RedisKey)"k"]); + } + }; + + Assert.Equal("v1", await db.Strings.GetAsync("k", CommandFlags.CacheOptIn)); + Assert.True(pushes >= 1, "the hook never fired; the prefix was not sent"); + Assert.Equal(0, cache.Count); + Assert.True(cache.RefusedRaced >= 1, "the invalidation should have refused the fill"); + + // the connection is still in step: an unrelated read, then the opted-in read again with no push + Assert.Equal("o", await db.Strings.GetAsync("other")); + server.AfterReply = null; + await PrimeAsync(db, cache, "k", "v1", flags: CommandFlags.CacheOptIn); + Assert.Equal("v1", await db.Strings.GetAsync("k", CommandFlags.CacheOptIn)); + } } diff --git a/tests/StackExchange.Redis.Tests/RespSurfaceTests.cs b/tests/StackExchange.Redis.Tests/RespSurfaceTests.cs index 14a09950a..38df6a9b9 100644 --- a/tests/StackExchange.Redis.Tests/RespSurfaceTests.cs +++ b/tests/StackExchange.Redis.Tests/RespSurfaceTests.cs @@ -330,14 +330,14 @@ public async Task WritesAreNotCached() } [Fact] - public async Task NoClientCacheOptsASingleCallOut() + public async Task CacheOptOutOptsASingleCallOut() { using var cache = new RespClientCache(new CacheOptions { Admission = CacheAdmission.OnFirstMiss }); var executor = new FakeExecutor("$5\r\nhello\r\n"); var target = Target(executor, cache); await target.Strings.GetAsync("mykey"); - await target.Strings.GetAsync("mykey", CommandFlags.CommandRetryReadOnly | CommandFlags.NoClientCache); + await target.Strings.GetAsync("mykey", CommandFlags.CommandRetryReadOnly | CommandFlags.CacheOptOut); Assert.Equal(2, executor.Sent.Count); // the opted-out call did not read the cached entry } diff --git a/toys/StackExchange.Redis.Server/RedisClient.cs b/toys/StackExchange.Redis.Server/RedisClient.cs index 067ec9adb..2c75e4f1b 100644 --- a/toys/StackExchange.Redis.Server/RedisClient.cs +++ b/toys/StackExchange.Redis.Server/RedisClient.cs @@ -190,6 +190,48 @@ public virtual void OnKey(in RedisKey key, KeyFlags flags) /// Whether tracking is in BCAST mode; the alternative is per-key. public bool TrackingBroadcast { get; private set; } + /// CLIENT TRACKING ON OPTIN: a read is tracked only if CLIENT CACHING YES preceded it. + public bool TrackingOptIn { get; private set; } + + /// CLIENT TRACKING ON OPTOUT: every read is tracked unless CLIENT CACHING NO preceded it. + public bool TrackingOptOut { get; private set; } + + /// + /// What the last CLIENT CACHING said, until the next command consumes it: true for YES, false + /// for NO, null for nothing pending. + /// + /// + /// As the real server does it: the flag applies to the next command on the connection, + /// whatever that is - a PING consumes it as surely as a GET - and a MULTI carries it across the + /// whole transaction (measured against Redis 8.9, and documented for YES). + /// + public bool? CachingOverride { get; set; } + + /// How many CLIENT CACHING YES this connection has received. + public int CachingYesCount { get; private set; } + + /// How many CLIENT CACHING NO this connection has received. + public int CachingNoCount { get; private set; } + + /// Record a CLIENT CACHING, for the next command. + public void SetCachingOverride(bool yes) + { + CachingOverride = yes; + if (yes) CachingYesCount++; + else CachingNoCount++; + } + + /// Whether the server is executing this client's EXEC right now; the override survives it. + internal bool InExec { get; set; } + + /// Whether this client is inside MULTI, queueing; the override survives that too. + internal bool InMulti => _transactionState is TransactionState.MultiHopeful + or TransactionState.MultiDoomedByTouch or TransactionState.MultiAbortByError; + + /// Whether a per-key tracking server currently holds for this client. + public bool IsTrackingKey(int database, string key) + => _trackedReads is not null && _trackedReads.Contains((database, key)); + private List _trackingPrefixes; private HashSet<(int Database, string Key)> _trackedReads; @@ -205,18 +247,26 @@ public virtual void OnKey(in RedisKey key, KeyFlags flags) /// Prefixes are broadcast-only, as on a real server: there is nothing to filter when the server is /// announcing what this client read. /// - public void SetTracking(bool enabled, bool broadcast, List prefixes) + public void SetTracking(bool enabled, bool broadcast, List prefixes, bool optIn = false, bool optOut = false) { TrackingEnabled = enabled; TrackingBroadcast = enabled && broadcast; + TrackingOptIn = enabled && optIn; + TrackingOptOut = enabled && optOut; _trackingPrefixes = enabled ? prefixes : null; _trackedReads = enabled && !broadcast ? new() : null; + CachingOverride = null; } /// In per-key mode, note that this client has read a key, so a change to it is announced. public void NoteTrackedRead(int database, in RedisKey key) { if (!TrackingEnabled || TrackingBroadcast) return; + + // OPTIN tracks only what was asked for; OPTOUT tracks everything that was not declined; the + // default mode has no escape hatch at all (CLIENT CACHING is an error there, see ClientCaching) + if (TrackingOptIn && CachingOverride != true) return; + if (TrackingOptOut && CachingOverride == false) return; _trackedReads?.Add((database, (string)key)); } diff --git a/toys/StackExchange.Redis.Server/RedisServer.cs b/toys/StackExchange.Redis.Server/RedisServer.cs index de4c5dae8..cbdce97f4 100644 --- a/toys/StackExchange.Redis.Server/RedisServer.cs +++ b/toys/StackExchange.Redis.Server/RedisServer.cs @@ -534,7 +534,19 @@ public override TypedRedisValue Execute(RedisClient client, in RedisRequest requ return TypedRedisValue.Error( $"ERR only [P|S][UN]SUBSCRIBE / PING / QUIT allowed in this context (got: '{request.Command}')"); } - return base.Execute(client, request); + + var result = base.Execute(client, request); + + // CLIENT CACHING applies to the NEXT command and is consumed by it, whatever it is - except that a + // MULTI carries it across the transaction: the queued commands keep it while they are queued and + // while EXEC runs them, and EXEC (or DISCARD) is what consumes it + var command = request.KnownCommand; + if (command is not (RedisCommand.CLIENT or RedisCommand.MULTI) && !client.InMulti && !client.InExec) + { + client.CachingOverride = null; + } + + return result; static bool IsAuthCommand(RedisCommand cmd) => cmd is RedisCommand.AUTH or RedisCommand.HELLO; static bool IsPubSubCommand(RedisCommand cmd) @@ -788,6 +800,7 @@ protected virtual TypedRedisValue Exec(RedisClient client, in RedisRequest reque var results = TypedRedisValue.Rent(commands.Length, out var span, RespPrefix.Array); int index = 0; var lease = RedisRequest.GetLease(); + client.InExec = true; try { foreach (var cmd in commands) @@ -799,6 +812,7 @@ protected virtual TypedRedisValue Exec(RedisClient client, in RedisRequest reque } finally { + client.InExec = false; RedisRequest.ReleaseLease(ref lease); } return results; @@ -827,7 +841,7 @@ protected virtual TypedRedisValue ClientTracking(RedisClient client, in RedisReq else if (request.IsString(2, "off"u8) || request.IsString(2, "OFF"u8)) on = false; else return TypedRedisValue.Error("ERR syntax error"); - bool broadcast = false; + bool broadcast = false, optIn = false, optOut = false; List prefixes = null; for (int i = 3; i < request.Count; i++) { @@ -835,6 +849,14 @@ protected virtual TypedRedisValue ClientTracking(RedisClient client, in RedisReq { broadcast = true; } + else if (request.IsString(i, "optin"u8) || request.IsString(i, "OPTIN"u8)) + { + optIn = true; + } + else if (request.IsString(i, "optout"u8) || request.IsString(i, "OPTOUT"u8)) + { + optOut = true; + } else if (request.IsString(i, "prefix"u8) || request.IsString(i, "PREFIX"u8)) { if (++i >= request.Count) return TypedRedisValue.Error("ERR syntax error"); @@ -842,7 +864,7 @@ protected virtual TypedRedisValue ClientTracking(RedisClient client, in RedisReq } else { - // OPTIN/OPTOUT/NOLOOP/REDIRECT: not modelled, and saying so is the point + // NOLOOP/REDIRECT: not modelled, and saying so is the point return TypedRedisValue.Error("ERR " + request.GetString(i) + " is not supported by this server"); } } @@ -852,16 +874,53 @@ protected virtual TypedRedisValue ClientTracking(RedisClient client, in RedisReq return TypedRedisValue.Error("ERR PREFIX option requires BCAST mode to be enabled"); } + // the real server's spellings, so a client that reads the message learns the same thing + if (optIn && optOut) return TypedRedisValue.Error("ERR You can't specify both OPTIN mode and OPTOUT mode"); + if ((optIn || optOut) && broadcast) return TypedRedisValue.Error("ERR OPTIN and OPTOUT are not compatible with BCAST"); + if (on && client.Protocol < RedisProtocol.Resp3) { // RESP2 clients must redirect invalidation to a subscriber connection; we do not model that return TypedRedisValue.Error("ERR Client tracking in RESP2 requires REDIRECT, which is not supported by this server"); } - client.SetTracking(on, broadcast, prefixes); + client.SetTracking(on, broadcast, prefixes, optIn, optOut); return TypedRedisValue.OK; } + /// CLIENT CACHING YES|NO - track, or do not track, the keys of the next command. + [RedisCommand(3, nameof(RedisCommand.CLIENT), "caching", LockFree = true)] + protected virtual TypedRedisValue ClientCaching(RedisClient client, in RedisRequest request) + { + bool yes; + if (request.IsString(2, "yes"u8) || request.IsString(2, "YES"u8)) yes = true; + else if (request.IsString(2, "no"u8) || request.IsString(2, "NO"u8)) yes = false; + else return TypedRedisValue.Error("ERR syntax error"); + + // the real server's messages, verbatim: YES needs OPTIN, NO needs OPTOUT, and plain ON has neither + if (yes && !client.TrackingOptIn) + { + return TypedRedisValue.Error("ERR CLIENT CACHING YES is only valid when tracking is enabled in OPTIN mode."); + } + + if (!yes && !client.TrackingOptOut) + { + return TypedRedisValue.Error("ERR CLIENT CACHING NO is only valid when tracking is enabled in OPTOUT mode."); + } + + client.SetCachingOverride(yes); + return TypedRedisValue.OK; + } + + /// + /// Called after each reply is queued; see . The command is the + /// one just answered; for CLIENT the second argument is the subcommand, else null. + /// + internal Action AfterReply { get; set; } + + private protected override void OnReplyQueued(RedisClient client, RedisCommand command, string subcommand) + => AfterReply?.Invoke(client, command, subcommand); + [RedisCommand(2, nameof(RedisCommand.CLIENT), "getname", LockFree = true)] protected virtual TypedRedisValue ClientGetname(RedisClient client, in RedisRequest request) => TypedRedisValue.BulkString(client.Name); diff --git a/toys/StackExchange.Redis.Server/RespServer.cs b/toys/StackExchange.Redis.Server/RespServer.cs index 68680adde..6b2122820 100644 --- a/toys/StackExchange.Redis.Server/RespServer.cs +++ b/toys/StackExchange.Redis.Server/RespServer.cs @@ -341,7 +341,22 @@ internal void FlushPendingInvalidations() } } - private void SendInvalidation(RedisClient client, List keys) + /// + /// A reply has just been queued for (with + /// for CLIENT, else null); anything queued here lands immediately behind it and ahead of the + /// next command's reply. + /// + /// + /// A test seam: it is how a test forces an out-of-band push to arrive between a prefix's +OK + /// and the reply of the command the prefix modified, which a real server does whenever it likes + /// and a test cannot otherwise time. + /// + private protected virtual void OnReplyQueued(RedisClient client, RedisCommand command, string subcommand) + { + } + + /// Push an invalidate to one client for the given keys, regardless of what it is tracking. + public void SendInvalidation(RedisClient client, List keys) { var msg = TypedRedisValue.Rent(2, out var span, RespPrefix.Push); span[0] = TypedRedisValue.BulkString("invalidate"); @@ -427,8 +442,12 @@ public async Task RunClientAsync(IDuplexPipe pipe, RedisServer.Node node = null, } else { + // captured before the awaits: the request is a ref struct and cannot cross them + var repliedCommand = request.KnownCommand; + var repliedSubcommand = repliedCommand == RedisCommand.CLIENT && request.Count > 1 ? request.GetString(1) : null; await ClientPauseAsync(client, request); await client.AddOutboundAsync(response); + OnReplyQueued(client, repliedCommand, repliedSubcommand); } client.ResetAfterRequest();