Repository navigation
Conversation
1978abc to
66dc711
Compare
66dc711 to
6b82aa3
Compare
6b82aa3 to
0acb890
Compare
0acb890 to
fec7394
Compare
meling
left a comment
There was a problem hiding this comment.
Reviewed against the stacked base feat/325/peer-identity, not master. Four findings are attached, with reproductions. The existing LastErr-preservation discussion is not duplicated here.
Simplifications
- In
sendFailed, removing thes.ended()escape from the ownership decision both simplifies the state machine and fixes the handoff interleaving described below. Cancellation should not bypass the CAS that decides who disposes of pending calls. - Extract one received-frame-or-age predicate and use one time origin for health eligibility, readiness promotion, and backoff reset. These currently use
opened, a separate probation timer, and the run loop'sstart. Keep the timer as a wakeup mechanism. Do not reuseviable()unchanged for backoff reset: its!suspectshortcut has different semantics. paceandbackoffduplicate timer creation and capped doubling.pacecan retain its non-eager early return and otherwise delegate tobackoff, discarding the deadline.- Give readiness probation its own named duration instead of coupling it to the maximum reconnect delay. More broadly, consider separating the heuristic readiness change from the refusal/backoff fix until its GOAWAY and slow-refusal contracts are settled.
Validation
Existing tests pass for ./internal/stream ./internal/conn ./internal/tests/tls . in default, -race, and -tags=integration modes. Scratch Go-overlay tests reproduce the four findings without changing tracked files. The handoff failure uses a deliberately forced legal interleaving; an additional ungated 300,000-iteration race stress run passed.
The wider go test ./... ./examples/... ./benchkit/... run failed in the unchanged benchkit/cmd/sweep test TestWriteReplayScript: replay_test.go:57: mode = -rwx------, want 0755. This was not investigated or changed as part of the PR review.
54bc386 to
be90084
Compare
be90084 to
e7e337c
Compare
A server that refuses a stream, for example because the dialer fails its identity check, ends the stream with a status before it sends any frame. The outbound channel treated that as a lost stream: it requeued the pending calls and opened a new stream at once, because a request was waiting. The calls failed only when their contexts ended, never with the refusal's code, and both servers spun at thousands of streams per second. A stream now counts as refused when it ends with Unauthenticated, PermissionDenied, InvalidArgument, or Unimplemented before it delivered a frame. Only the receive loop learns the stream's status. So a send that fails with io.EOF before the first frame only cancels the stream and leaves the pending calls to the receive loop, and an atomic handoff decides which goroutine disposes of them. A one-way send that fails this way waits for the receive loop's verdict, so it fails with the refusal too. If a frame arrives after the failed send, the server accepted the stream: the buffered frames are delivered, and the calls still pending are retried. Calls on a refused stream fail with the refusal, streaming calls included. The channel then backs off (50 ms, doubling to 2 s) before it opens the next stream, with or without eager reconnect, and fails each request at once with the refusal until the backoff ends. A failed dial-side verification backs off the same way. Lost streams are still requeued and reconnected. Connection state stays optimistic: a new stream counts as up when it opens, before the server's verdict. The user guide gains a Refused Streams section, since the behavior applies to every client, not only to peers.
A send completes locally before the server's verdict on a new stream arrives. So after a refusal, each new stream's first send cleared LastErr, and LastErr flipped between the refusal and nil for about one round trip per attempt. Callers that sort nodes with ByLastError then saw a refusing node as healthy. A session opened after a refusal is now suspect until a stream proves viable: the stream delivers a frame, or it lives 2 s. A send on a suspect session clears LastErr only once its stream has proven viable. A received frame clears LastErr as before, and a newer error may replace the refusal. The run loop uses the same test to start its backoff over.
e7e337c to
23264be
Compare

Stacked on #359. Merge #359 first; this PR's base will then switch to
master.Peer identity setup is defined by #359.
This PR changes how the dialer handles refused streams, not the peer identity API.
Problem
When a server refused a dialed stream at accept time (for example, a peer whose TLS identity fails verification), the dialer requeued the pending call and opened a new stream at once.
One pending call caused about 15,000 refused streams per second; the call failed only at its deadline, with the context's error instead of the refusal's code.
Each attempt also counted as up until the refusal arrived, so the refused side's
ConnectedPeersandWithPeerChangeflapped, and underWithStreamDedupitsWaitForAllcould return although every stream was refused.Change
A dialed stream is refused when it ends with
Unauthenticated,PermissionDenied,InvalidArgument, orUnimplementedbefore the server sent anything.Node.LastErruntil a stream proves viable: it delivers a message, or a send succeeds on a stream that has lived 2 s. A newer error, such asUnavailable, may replace the refusal.Transient failures (a broken connection, a server restart, GOAWAY,
Unknown, or any status after the server has sent a frame) retain the existing retry and reconnect behavior.Readiness and health limits
The timer-based readiness change was removed during review.
Connection reporting remains optimistic: each new stream counts as up when opened and verified locally, before the server accepts or refuses it.
ConnectedPeersandWithPeerChangecan therefore report a refused peer once per backoff period, andWaitForAllis not confirmation of remote acceptance.Idle accepted streams become ready without a two-second probation, including under repeated GOAWAY.
An explicit acceptance signal is deferred to a separate issue.
With one-way-only traffic and GOAWAY rotation that keeps every stream shorter than 2 s,
LastErrcan remain stale after recovery even while messages reach the server.This limit was reproduced and documented; health behavior was not changed.
Earlier measurements
These measurements predate the rebase and were not rerun on the rebased branch.
The runtime refusal and retry implementation is unchanged by the rebase.
Testing
LastErr, recovery, optimistic stream reporting, and transient retries.Validation passed on the rebased branch:
make testincludes benchmark validation.Docs:
WithPeers, the user guide's Authenticating Peers section, and the developer guide's stream layer describe the behavior and its limits.