Skip to content

fix(qwp): close browser WebSockets that refuse the client's close code - #80

Open
glasstiger wants to merge 5 commits into
mainfrom
ia_browser_close_codes
Open

glasstiger wants to merge 5 commits into
mainfrom
ia_browser_close_codes

Conversation

@glasstiger

Copy link
Copy Markdown
Collaborator

Summary

Fixes #77. In a browser, the QWP client closed some WebSockets with codes that the browser's WebSocket.close() refuses: 1001 in QwpClient.close(), and 1002 or 1011 on protocol errors, failed sends, failed reconnects and a cancelled query that does not drain. The close threw and the connection settled as closed, but the native socket stayed open. Pooled query sockets outlived QwpClient.close(), and a reconnect: false session whose cancel did not drain kept receiving the cancelled result.

Changes

  • When a socket without terminate() refuses a close, closeSocket() retries it, first with 1000 and the same reason, then with no code. Every caller of the connection's close() gets the retry. A WHATWG socket validates its arguments before it changes state, so retrying is safe.
  • Node's ws accepts these codes, so on Node they are sent unchanged. A close that ws refuses is still terminated immediately, because ws enters CLOSING before it validates and ignores any later close().
  • QWP.md: a browser can only start the closing handshake. Without flow control, QuestDB reads the close only once the query has ended. In a browser the query therefore continues until then, or until the browser abandons the handshake, after 60 s in Chromium. The generated API reference is regenerated for the QWP.md change.

Not marked breaking. For a browser user, the socket now closes and a 1000 close frame goes on the wire. closed reports the server's close instead of a synthetic 1006 "QWP WebSocket close failed". Node with ws is unchanged.

Validation

  • Reproduced against QuestDB 10.1.0 in Chromium. Before the fix, the pooled socket was still OPEN after client.close(), and the cancelled session's socket stayed OPEN, receiving 300 frames in 3 s. With the fix, the pooled socket is CLOSED once client.close() resolves. The cancelled session's socket is CLOSING at once and receives no further frames, and QuestDB logs the client's close with code 1000. Without flow control, QuestDB answered that close only after its 1-minute query.timeout aborted the query. A server that never reads makes Chromium drop the connection after 60 s.
  • Unit tests use a fake socket that validates close() as a browser does. They cover the transport, QwpClient.close(), and the undrained cancel. A ws-like fake checks that a refused close is still terminated immediately. Two Chromium e2e tests check the native socket's readyState after QwpClient.close() and after a reconnect: false cancel times out. The new browser tests fail without the fix.
  • Lint, formatting, all typechecks, pnpm test, pnpm test:dist, pnpm typecheck:dist, pnpm test:qwp-browser and the package checks pass.

A browser's WebSocket.close() accepts only 1000 and 3000-4999 and throws
InvalidAccessError for anything else. The client closes with 1001 in
QwpClient.close(), and with 1002 or 1011 on protocol errors, failed sends,
failed reconnects and a cancelled query that does not drain. In a browser
that close threw, the fallback found no terminate() to call, and the
connection settled as closed with a synthetic 1006 while the native socket
stayed open. Pooled query sockets outlived QwpClient.close(), and a
reconnect: false session whose cancel did not drain kept receiving the
cancelled result until the page unloaded.

closeSocket() now retries a close that a socket without terminate() refuses,
first with 1000 and the same reason, then with no code at all. A WHATWG
socket validates its arguments before it changes state, so the retry is
safe. ws accepts these codes and still sends them unchanged. A close that ws
refuses is still terminated at once, because ws enters CLOSING before it
validates and ignores any later close().

A browser still cannot drop a connection itself. Without flow control
QuestDB reads the close only once the query has ended, so in a browser the
query runs until then or until the browser abandons the closing handshake,
after 60 seconds in Chromium. QWP.md now says so.

Fixes #77
The comment said the close codes "still reach a Node server", but the
server is QuestDB; `ws` is the Node client. It now says that from Node
the codes still reach the server. Comment-only change.
@glasstiger

Copy link
Copy Markdown
Collaborator Author

Review

Reviewed head 8508928a against base 0ce79e1d. af65bde1, pushed after the review, changes only the comment listed under Minor.

Verdict: approve. No Critical or Moderate findings.

Minor

  • Fixed in af65bde1: the startClosingHandshake() JSDoc in packages/client-core/src/_qwp/_internal/websocket-connection.ts said "ws accepts them, so they still reach a Node server". The server is QuestDB and ws is the Node client. It now reads "so from Node they still reach the server".

Evidence

  • Regression tests: with only the websocket-connection.ts change reverted, 5 of the 6 new unit tests fail. The sixth, the ws "terminates instead of retrying" test, passes on both revisions, which is expected because Node behaviour is unchanged. Both new Chromium e2e tests fail with the change reverted and pass at head.

  • Partial fix: a variant that retries the close but still settles the connection as closed at once passes all six unit tests, because their fake sockets fire close synchronously. The pooled-socket Chromium test catches it ([2] instead of [3]), and that test runs in the qwp-browser CI job.

  • Issue QWP browser: closing with code 1001 or 1011 leaves the WebSocket open #77 reproduction, QuestDB 10.1.0 in Chromium:

    Base Head
    Pooled query socket after client.close() OPEN after 2.5 s CLOSED
    Socket of a cancelled reconnect: false query OPEN, about 360 frames in 3 s CLOSING, no frames
    QuestDB log for the pooled close nothing close code 1000
  • QWP.md: against a peer that never answers the close, Chromium abandoned the closing handshake after 60,005 ms, as the new text says.

  • ws: 8.21.3 enters CLOSING before it validates the code, so terminating a refused close at once is still correct for Node.

  • Checks at head: vitest run test/qwp 1082/1082, browser e2e 12/12, eslint, prettier, tsc for the root, browser and test configs, and the docs-reference test all pass.

  • Callers: traced every caller of close() and every reader of closed (reconnect, failover, endpoint deprioritization, ingress poison-frame classification, browser durable-ACK negotiation). None makes a worse decision because of the change.

Trade-offs

  • In a browser, QwpClient.close() while a leased query is still streaming without flow control now waits up to cancelDrainTimeoutMs (5 s by default). Measured against QuestDB: 5,001 ms at head and 1 ms at base, where the socket was left open. Node takes the same ~5 s on both revisions, and QWP.md already documents both bounds.
  • In a browser, closed now reports the server's close (for example code 1000) instead of the synthetic 1006 "QWP WebSocket close failed". CONTRIBUTING.md counts a change to behaviour or wire bytes as breaking even when it is a fix. The PR explains why it isn't marked breaking, as fix(qwp): end cancelled queries promptly and keep their session usable #75 wasn't, but a maintainer may want to confirm the release classification.

Submodules: none. Findings: 0 Critical, 0 Moderate, 1 Minor (fixed).

The test that keeps a `ws` socket from being retried closed with 1000. That
skips the retry with 1000 which startClosingHandshake() gives a refused code
other than 1000, so moving the terminate() check after that retry still
passed all of test/qwp. With `ws` that change postpones terminate() by the
whole close deadline: `ws` enters CLOSING before it validates, and the
retried close() sends nothing. The test now also closes with 1011, and fails
on that variant. Test-only change.
…close

In a browser, a failed ingress negotiation waited for its close(1002) before
rethrowing. Since the refused 1002 is retried with 1000 and waits for the
server's reply, the connect deadline could fire first and turn the terminal
QwpDurableAckUnavailableError into a retryable upgrade timeout. Start the
close without waiting for it, as Node's upgrade-validation failures do.
@glasstiger

Copy link
Copy Markdown
Collaborator Author

Review

Reviewed head c5fe7585 against base 0ce79e1d. 11f34dd7, pushed after the review, changes only the QWP.md sentence listed under Minor and regenerates docs/.

Verdict: approve. No Critical or Moderate findings.

Minor

  • Fixed in 11f34dd7: in the QWP.md paragraph on cancellation draining, the two new browser sentences separated "A query ended with query.cancel() then fails with QwpEgressQueryCancelTimeoutError" from the sentence its "then" referred to. It read as if the error arrived only after Chromium abandons the closing handshake (60 s), but the drain timer raises it when cancelDrainTimeoutMs expires (egress-session.ts:2174), as the new Chromium test also shows. It now reads "When the bound expires, a query ended with query.cancel() fails with …".

Evidence

  • Regression tests: with websocket-connection.ts and browser qwp.ts reverted, 7 of the 8 new unit test cases fail. The Node ws case passes on both revisions, as expected. Both new Chromium tests fail with readyStates [1] and pass at head. Reverting only c5fe7585 makes both durable-ACK cases fail after about 1,000 ms, when the connect deadline wins.
  • Partial fix: a variant that retries the close but settles the connection at once passes all 147 session.test.ts tests, because the fake sockets emit close synchronously. The pooled-socket Chromium test catches it ([2] instead of [3]), and the qwp-browser job runs on every pull request and before publishing.
  • Checks at c5fe7585: vitest run test/qwp 1084/1084, test:qwp-browser 12/12, the docs-reference test, eslint, prettier, and typecheck, typecheck:qwp-browser and typecheck:test all pass; the GitHub checks are green.
  • Callers: traced every caller of close() and every reader of closed: ingress poison-frame classification, egress reconnect, failover demotion and deprioritization, the browser durable-ACK verdict, the query pool and both sessions. The poison-frame classifier never sees the echo of a close the client started, because a send failure calls requestReconnect() first, and no other deadline races an awaited close whose code the browser refuses. Node is unchanged, since ws sockets have terminate().

Trade-offs

Measured with fake sockets. In each case the browser now behaves as Node already did on both revisions; base was faster only because it left the old socket open and still streaming.

  • Returning a query lease that cannot be reused, while the server ignores the close (a result streamed without flow control), now holds its pool slot for up to one more cancelDrainTimeoutMs, as QwpClient.close() waits (noted in the earlier review). A concurrent borrowQuery() on a full pool can then get QwpPoolAcquireTimeoutError: at 303 ms at head, where base handed out a new socket at 205 ms (queryPoolMax: 1, acquireTimeoutMs: 300, cancelDrainTimeoutMs: 200).
  • When a replay send times out on a new connection, the reconnect loop now waits for that connection's close, up to closeTimeoutMs (15 s by default), before its next attempt: 1,105 ms at head versus 104 ms at base, with sendTimeoutMs: 100 and closeTimeoutMs: 1000. For egress, fewer attempts fit within failoverMaxDurationMs.
  • Release classification: the PR lists the browser-visible changes and calls them non-breaking. CONTRIBUTING.md's rule reads more broadly, so a maintainer may want to confirm.

Submodules: none. Findings: 0 Critical, 0 Moderate, 1 Minor (fixed).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

QWP browser: closing with code 1001 or 1011 leaves the WebSocket open

1 participant