Skip to content

Rebuild the client around connections, transports and commands - #90

Merged
scothis merged 37 commits into
componentized:mainfrom
scothis:resp-socket
Oct 16, 2026
Merged

scothis merged 37 commits into
componentized:mainfrom
scothis:resp-socket

Conversation

@scothis

@scothis scothis commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Replace the store interface and the ops component with a
componentized:valkey@0.3.0-dev package that separates the mechanics of
a connection from the transport carrying it, and from the commands sent
over it. Connections always speak RESP3.

  • connection: pairs commands with replies, keeps push messages apart,
    re-establishes state after RESET, and advertises the client's name and
    version. Pub/sub, transactions and WATCH are part of the connection,
    with a subscription resource for channels, patterns and shard
    channels. connection-minimal is the same interface without pub/sub,
    for a smaller component.
  • tcp and tcp-connector: connections over wasi:sockets to the server.
    The connector defines a no-arg method to establish a pre-configured (via
    wasi:config) connection.
  • commands: typed functions for Valkey's command set, following
    Valkey's command definitions, with the shared types in a types
    interface. Values are string, bytes or string-or-bytes, so
    binary data round trips.
  • componentized-valkey crate: the types, the RESP codec and the
    conversions between commands, replies and the WIT types, shared by
    the components.

Network access can be limited with componentized:sockets gates.
tcp-connector-latch allows only the server in the connector's config
(its hostname, by name or IP address, on its port, over TCP) and denies
every other socket. The cli is always behind a gate with this latch,
composed so that components behind the gate cannot import wasi:sockets
around it.

Tests run components in wasmtime with a shared harness. Guest-side
suites cover the commands and the latch, the latch over stub sockets so
that denials are certain. make test-valkey runs the integration tests
against Valkey in a Docker container.

scothis and others added 30 commits October 9, 2026 23:36
Replace the `store` interface, a connection resource carrying every
command, with interfaces that separate the mechanics of talking to a
server from the transport carrying it and from the commands sent over
it. Connections always speak RESP3.

WIT, componentized:valkey:
- `connection`: request/reply over a pair of value streams, for any
  transport. `new` returns the requests stream and a future resolving
  to the connection once `HELLO 3` succeeds, which pins RESP3 and
  confirms the server can be reached. `call` pairs replies with
  commands in order with several in flight, `send` is fire and forget,
  `pushes` broadcasts push messages to every reader, closing a reader
  that falls behind rather than holding up replies, `set-keep-alive`
  pings an idle connection
- `tcp`: connections over wasi:sockets, a `transport` resource and
  `connect`
- `connector`: connections whose transport and settings are chosen by
  the provider
- command interfaces for each of Valkey's ACL categories, e.g.
  `hash-commands`, functions borrowing a connection so each category
  can be implemented independently. The commands not yet implemented
  are listed in each interface
- `store` is removed

Components:
- `resp`: exports the RESP codec, the types every component shares
- `connection`, `tcp` and `tcp-connector`, which connects to the
  server configured with wasi:config, with keys `hostname`, `port`,
  `username`, `password`, `client-name` and `keep-alive-interval-ms`
- `commands`, every command interface in one component, and a
  component for each category, e.g. `hash-commands`, compiled from the
  shared sources in crates/commands
- `as-keyvalue` gets connections from a `connector`, rather than
  connecting over TCP itself
- `cli` and `client` are composed from the new components
- `ops` is removed

RESP:
- the codec moves to crates/resp-codec, a regular crate the components
  share through wit-bindgen's `with`
- messages that span several socket reads are framed before they are
  decoded, bytes past a message are kept for the next
- nested values are copied as received, rather than decoded and
  encoded again

Tests:
- crates/test-harness instantiates components with wasmtime, and plays
  the server over a stub transport, or over TCP
- the tests against a Valkey server are ignored unless run with
  `make test-valkey`, which CI runs after `make test`
- CI and the dev container use Valkey 9, which as-keyvalue's ttls need
  for HEXPIRE

Signed-off-by: Scott Andrews <scott@andrews.me>
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Run tests against a real ephemeral valkey-severs running in docker.

Signed-off-by: Scott Andrews <scott@andrews.me>
Skip non-essential features, like pub/sub.

Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
BulkStrings cary double duty as actual utf-8 encoded strings and binary
blobs. we now handle both cases. Any value that can be decodded as a
string is returned as a string, otherwise the bytes are provided. When
sending commands, either form is acceptable.

Command values that are aways strings remain strings. Values that are
always binary are list<u8> and values that may alternate depending on
the content or context are the string-or-bytes variants. Users will
need to detect which form was returned.

Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
The `tcp-connector-latch` limits network access to only the essential
capabilities necessary to establish a connection with the
`tcp-connector`. In order, it will only lookup IP address for the
configured hostname. TCP sockets can only connect to the IP address
returned by the hostname lookup, on the port specified by the config.
No inbound connections are allow, nor any UDP activity.

The gate and latch are installed by default into the `cli` component as
that uses the `tcp-connector`. Other users of the `tcp` transport will
need to configure a latch appropriate for their environment.

Signed-off-by: Scott Andrews <scott@andrews.me>
@scothis scothis changed the title Resp socket Rebuild the client around connections, transports and commands Oct 15, 2026
@scothis
scothis marked this pull request as ready for review October 15, 2026 23:45
The tests that need a server spin up an ephemeral container.

Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
Signed-off-by: Scott Andrews <scott@andrews.me>
@scothis
scothis enabled auto-merge (squash) October 16, 2026 02:05
@scothis
scothis merged commit 414d3d3 into componentized:main Oct 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant