Skip to content

gorums: authenticate peers using node TLS identities - #359

Merged
meling merged 6 commits into
masterfrom
feat/325/peer-identity
Oct 21, 2026
Merged

meling merged 6 commits into
masterfrom
feat/325/peer-identity

Conversation

@meling

@meling meling commented Oct 11, 2026 •

Copy link
Copy Markdown
Member

Closes #325.

Builds on the local-server options from #361, which is merged.

Summary

Servers configured with WithPeers trusted the node ID claimed in stream metadata and whoever answered at a peer's address.
Any client that could reach a server could pose as a configured peer.
Under WithStreamDedup, a forged claim for a lower ID could also redirect the server's own calls to the attacker.

This PR authenticates peers using TLS identities stored with their addresses in one node map.
A node value that implements TLSIdentity() string automatically enables verification when its source is passed to WithPeers.
There is no separate verifier option or identity map to keep in sync.

API and Configuration

type replica struct {
    address  string
    identity string
}

func (r replica) Addr() string        { return r.address }
func (r replica) TLSIdentity() string { return r.identity }

nodes := map[gorums.ID]replica{
    1: {"node1:9000", "spiffe://relab/node/1"},
    2: {"node2:9000", "spiffe://relab/node/2"},
    3: {"node3:9000", "spiffe://relab/node/3"},
}

srv := gorums.NewServer(
    gorums.WithPeers(myID, gorums.WithNodes(nodes), dialOpts...),
)
  • NodeAddress stays unchanged; TLSIdentity is an optional companion interface.
  • WithNodes captures membership, addresses, and identities together.
    Later changes to the map or its node values do not affect the source.
    Embedding a node source preserves its authentication requirement.
  • If any node declares a TLS identity, every node in the peer set, including the local node, must supply a unique identity that a certificate SAN can match.
    NewServer panics at startup for a missing, empty, or duplicate identity, and for an identity that no SAN can match: a wildcard DNS name, an IP or host:port address, or text that is not printable ASCII or is neither a URI nor a DNS name.
    An empty identity never disables authentication.
  • WithLocalNodes(func(id ID, addr string) NodeAddress) supplies node values for local clusters after all listeners are allocated.
    The factory runs once per node in ID order, and each node must use its supplied listener address.
    NewLocalServers returns configuration errors and closes allocated listeners on failure.
  • Verification is internal; there is no public verifier API.
  • Address-only sources retain their existing unauthenticated behavior.
    Standalone NewConfig does not activate peer identity checks; conn: should NewConfig check the TLS identities of the servers it dials? #370 asks whether it should.

Verification Rules

Every peer stream is checked, with and without stream deduplication:

  • Inbound: a stream that claims a configured peer's ID is checked before attachment to the peer's node and before any connect callback or handler runs.
    A failed check or an unknown claimed peer ID ends the stream with codes.Unauthenticated.
  • Outbound: a stream opened to a peer is checked against the dialed node's ID before any request is sent.
    A failed check closes the stream and fails the waiting request with codes.Unauthenticated.

The TLS handshake must have verified the peer's certificate against a trusted CA.
An identity with a URI scheme, such as spiffe://relab/node/2, matches only a URI SAN in the same canonical form.
Other identities match only DNS SANs, without regard to case.
Wildcard DNS SANs, IP SANs, and the Common Name do not establish peer identity.
Duplicate identities are checked using the same matching rules.

Applications must still configure TLS credentials on the server and dialer, present each replica's certificate, and verify certificates against the cluster's CA.
Each replica's certificate must support both client and server authentication.
A stream without a verified TLS certificate is refused; verification never falls back to trusting the claimed ID.
The refusal names the TLS settings to check: the dialer's certificate, the server's ClientAuth, and InsecureSkipVerify.

Sender IDs and Clients

ServerContext.SenderID is authenticated for configured peers when TLS identities are declared.
It identifies only the last hop, not the original signer of a forwarded message.

Unknown claimed peer IDs have two distinct outcomes:

  • With TLS peer identities: the stream is refused before any handler runs.
  • Without TLS peer identities: the stream is served as an untracked client, and handlers see SenderID 0.

Clients that claim no peer ID are not subject to peer identity checks.
TLS settings and server interceptors remain responsible for client authentication and authorization.
Back-channel client IDs are assigned by the server and are not authenticated peer identities.

Commits

  1. doc: align the Makefile targets table in the developer guide (whitespace only)
  2. stream: check the dialed peer with an optional verifier
  3. conn: verify peer claims and the peers a server dials
  4. gorums: authenticate peers whose nodes declare TLSIdentity
  5. gorums: add WithLocalNodes for local peer records
  6. internal/tests/tls: test per-replica peer certificates

Documentation and Tests

  • Update the user guide with automatic activation, one-map configuration, the identity rules, certificate requirements, local node factories, and separate authenticated and unauthenticated ID behavior.
  • Clarify the inbound and outbound checks in the developer guide and the SenderID doc comment.
  • Cover automatic activation, empty, mixed, and unmatchable identities, URI and DNS duplicate rules, source snapshots and embedding, local factory validation, and listener cleanup.
  • Retain per-layer stream and connection checks.
  • Test TLS end to end with WithLocalNodes and per-replica certificates: successful calls and authenticated sender IDs, wrong certificates in both directions, lower-ID impersonation under deduplication, and ordinary clients without certificates.

CI lint and tests pass. Validation run locally:

go test ./...
go test -tags=integration ./...
go test -race ./internal/stream ./internal/conn .
go test -race -tags=integration ./internal/tests/tls
golangci-lint run ./... ./examples/... ./benchkit/...
make modernize
make goplscheck
git diff --check

Each commit builds, passes go vet ./..., and passes the tests of internal/stream, internal/conn, gorumstest, internal/tests/tls, and the root package.
The examples and benchkit tests pass on the tip of #360, which contains this branch.

Known Limitation

A peer refused by the remote server resends its pending two-way calls on new streams until their contexts end.
Those calls fail with the context's error rather than codes.Unauthenticated, and the refusal reason is available in Node.LastErr.
This retry policy predates this PR and is unchanged.
Calls between peers need deadlines.
#360 fixes this: calls on a refused stream fail with the refusal, and the dialer backs off before it opens a new stream.

Follow-ups: #369 (double gorums: prefix in configuration errors) and #370 (identity checks for NewConfig).

Related discussion: the design comment on #325.

Copilot AI balanced review requested due to automatic review settings October 11, 2026 18:02

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@meling

meling commented Oct 11, 2026

Copy link
Copy Markdown
Member Author

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

I have only used 78% of my AI credits!! Please learn to compute!

@meling
meling force-pushed the feat/325/peer-identity branch from f8677d9 to 4cc9887 Compare October 11, 2026 20:40
@meling
meling changed the base branch from master to feat/325/local-server-options October 11, 2026 20:40
@meling
meling added this pull request to stack #362 October 11, 2026 20:40
@meling
meling force-pushed the feat/325/peer-identity branch from 4cc9887 to 38665fe Compare October 12, 2026 20:10
Base automatically changed from feat/325/local-server-options to master October 12, 2026 20:21
@meling
meling force-pushed the feat/325/peer-identity branch from 38665fe to 6e77169 Compare October 12, 2026 20:22
@meling
meling force-pushed the feat/325/peer-identity branch 2 times, most recently from d759c2b to 022fdef Compare October 17, 2026 19:29
@meling meling changed the title gorums: authenticate peer identities with WithPeerVerifier gorums: authenticate peers using node TLS identities Oct 20, 2026
Pad the last four rows of the Makefile targets table to the column
widths of the rows above them. The table content is unchanged.
An outbound channel trusted whichever server answered at the dialed
address. Add PeerVerifier and OutboundOptions.Verify: when set, the
channel checks the remote end of each new stream against its node ID,
using the stream's context, before it sends requests on the stream. A
failed check closes the stream and fails the waiting request with
codes.Unauthenticated; the channel then retries as for any failed
stream.

Reading the stream's context commits the stream to its connection.
gRPC then cannot retry the stream on a connection that nobody checked,
so every request sent on the stream reaches the checked peer.
A server attached an inbound stream to peer X whenever the stream's
metadata claimed X, so any client could pose as a configured peer.
NewInboundManager now takes an optional PeerVerifier. With it,
AcceptPeer attaches a stream that claims a peer's node ID only after
the verifier accepts the claim, and refuses claims for unknown node
IDs, both with codes.Unauthenticated. The verifier runs without the
manager's lock. The outbound manager hands the same verifier to the
channels of the configured peers that the server dials.

WithNodes now captures each node's address and, for a node that
implements TLSIdentity, its expected certificate identity, so later
changes to the caller's map do not affect the source. NewPeerVerifier
builds the TLS verifier from these identities. It accepts a stream for
a node only if the verified certificate has a URI SAN equal to the
node's URI identity, or a DNS SAN equal to its DNS identity without
regard to case; wildcard DNS SANs, IP SANs, and the Common Name never
match. It rejects mixed and duplicate identity sets, and identities
that no SAN can match: empty or wildcard names, network addresses, and
text that is not printable ASCII or is neither a URI nor a DNS name.
Servers configured with WithPeers trusted the node ID that a peer
claimed when it connected, and whoever answered at a peer's address.
So ServerContext.SenderID was not authenticated, and under
WithStreamDedup a forged claim for a lower ID could redirect the
server's own calls to the impostor.

A node type passed to WithNodes may now implement TLSIdentity next to
NodeAddress. If any node in the WithPeers source does, the server
checks every peer on each stream, in both directions, before callbacks
or handlers run, and NewServer panics if the identity set is invalid.
Keeping each identity with its node's address means that membership
and authentication cannot drift apart. Address-only sources and
NewConfig keep their unauthenticated behavior.

Document the activation, the certificate and TLS settings that
replicas need, and which SenderID values are authenticated.
NewLocalServers built its peer source from listener addresses only, so
a local cluster could not declare TLS identities. WithLocalNodes takes
a factory that NewLocalServers calls once per node, in increasing ID
order, after it has allocated every listener. Each returned node must
use the supplied listener address. NewLocalServers returns an error,
and closes the allocated listeners, if a node is invalid or the peer
configuration is rejected.

Test the factory with address-only and identity-bearing nodes,
including the refusals of peers that connect without TLS.
Run peer groups whose servers declare TLS identities and present
certificates from a test CA, with and without stream deduplication.
The tests cover peers that connect and see each other's authenticated
sender IDs, a server that presents another node's certificate (refused
by the dial-side and the inbound check), an impostor that claims a
lower peer's ID and must not take over its stream, and a client
without a certificate that is still served.
@meling
meling force-pushed the feat/325/peer-identity branch from 8e8d531 to b469186 Compare October 20, 2026 23:05
Comment thread internal/stream/outbound.go Outdated
Comment thread internal/conn/peer_verifier.go
@meling
meling merged commit 1e53f97 into master Oct 21, 2026
3 checks passed
@meling
meling deleted the feat/325/peer-identity branch October 21, 2026 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: stronger peer identity guarantees

2 participants