Repository navigation
Unread request data after an early response can deadlock HTTP/2 streams and shutdown #393
Description
Hypercorn 0.18.0, h2 4.4.1, h11 0.16.0, Python 3.14.7 on Linux and 3.14.3 on macOS, asyncio worker, cleartext HTTP/2 prior knowledge.
A completed early response can leave incoming request data in the ten-message application queue. The connection reader then waits while putting more body data into that queue, and stream cleanup waits while putting http.disconnect into the same full queue. A second request on the HTTP/2 connection receives no response, and supervisor SIGTERM does not finish within the configured graceful timeout plus two seconds.
This reproduces with a plain ASGI application, without a framework, TLS, a proxy, or a streaming response. It appears distinct from #389 (late data for a stream already removed), #390 (outgoing response backpressure), and #314 (HTTP/1 TLS close-notify failure). The shutdown half overlaps #308: the stuck connection never closes, so Server.wait_closed never returns. With #388 applied, the supervisor exits in about 4.1 seconds, at the graceful timeout, but stream 3 still receives no response. With #391 applied, the result does not change. An idle HTTP/2 connection left open by the client does not delay shutdown (about 0.1 seconds), so the full queue is what holds this connection.
Save the application below as early_app.py and the client as client.py. Install hypercorn==0.18.0 h2==4.4.1, then run:
hypercorn --bind 127.0.0.1:8080 --workers 1 --graceful-timeout 4 early_app:appApplication:
async def app(scope, receive, send):
if scope["type"] == "lifespan":
while True:
message = await receive()
if message["type"] == "lifespan.startup":
await send({"type": "lifespan.startup.complete"})
elif message["type"] == "lifespan.shutdown":
await send({"type": "lifespan.shutdown.complete"})
return
if scope["type"] == "http":
# Refuse one request without reading its body; ordinary requests succeed.
status = 413 if scope["path"] == "/early" else 200
await send({"type": "http.response.start", "status": status,
"headers": [(b"content-length", b"2")]})
await send({"type": "http.response.body", "body": b"ok"})Client:
import json
import socket
import sys
import time
import h2.config
import h2.connection
import h2.events
def probe(port, case, before_close=None):
connection = h2.connection.H2Connection(
config=h2.config.H2Configuration(client_side=True, header_encoding="utf-8")
)
connection.initiate_connection()
def request(stream, method, path, end):
connection.send_headers(stream, [(":method", method), (":scheme", "http"),
(":authority", "localhost"), (":path", path)], end_stream=end)
with socket.create_connection(("127.0.0.1", port), timeout=2) as sock:
request(1, "POST", "/early", False)
if case == "unread":
for _ in range(40):
connection.send_data(1, b"x" * 1024)
request(3, "GET", "/next", True)
else:
connection.send_data(1, b"x" * 1024)
sock.sendall(connection.data_to_send())
statuses, ended, late_sent = {}, set(), False
started = time.monotonic()
deadline = started + 8
error = None
while 3 not in ended and time.monotonic() < deadline:
if case == "removed" and 1 in ended and not late_sent:
time.sleep(0.2)
connection.send_data(1, b"x" * 4096, end_stream=True)
request(3, "GET", "/next", True)
sock.sendall(connection.data_to_send())
late_sent = True
sock.settimeout(max(0.001, deadline - time.monotonic()))
try:
data = sock.recv(65536)
if not data:
error = "EOF before the following response"
break
for event in connection.receive_data(data):
if isinstance(event, h2.events.ResponseReceived):
statuses[event.stream_id] = dict(event.headers)[":status"]
elif isinstance(event, h2.events.DataReceived):
connection.acknowledge_received_data(
event.flow_controlled_length, event.stream_id)
elif isinstance(event, h2.events.StreamEnded):
ended.add(event.stream_id)
sock.sendall(connection.data_to_send())
except TimeoutError:
error = "No following response within 8 seconds"
break
except ConnectionError as exc:
error = type(exc).__name__
break
result = {"case": case, "statuses": statuses, "ended": sorted(ended),
"next_status": statuses.get(3), "error": error,
"elapsed_seconds": round(time.monotonic() - started, 4)}
if before_close is not None:
before_close(result)
return result
if __name__ == "__main__":
print(json.dumps(probe(int(sys.argv[1]), sys.argv[2])))In another terminal:
python client.py 8080 unreadThe client sends headers, forty valid 1 KiB DATA frames on stream 1, and the following GET on stream 3 together. The body is below the initial connection/stream flow-control windows. Observed: stream 1 receives a complete 413, but stream 3 receives no response within eight seconds. Keeping that client socket open while signaling only the Hypercorn supervisor also leaves it running after six seconds (--graceful-timeout 4). Local verification used an equivalent parent process that kept the client open during that signal; its bounded teardown then terminated the process group.
The source path is H2Protocol._handle_events -> HTTPStream.handle(Body) -> app_queue.put. Response completion separately goes through HTTPStream._send_closed and stream closure -> HTTPStream.handle(StreamClosed) -> another app_queue.put. The application never consumes those messages after its response. Increasing max_app_queue_size moves the point where this waits but does not release a queue whose consumer has finished.
Expected: a completed response/finished application should release unread input so that other HTTP/2 streams and stream cleanup can progress, and should not leave supervisor shutdown waiting on that input queue. This is separate from requiring HTTP/1 connection reuse after an early answer. A local workaround releasing that queue before final response cleanup, then dropping subsequent input to the finished application, restores progress in this reproduction; #389 still needs its separate removed-stream handling.
The same client supports removed to isolate #389: wait for stream 1 to finish, then send late DATA and a following GET.
Activity