Skip to content

fix(http): correct chunked request body size counting - #59

Open
bladehan1 wants to merge 5 commits into
release_v4.8.3from
fix/http-chunked-body-size-limit
Open

bladehan1 wants to merge 5 commits into
release_v4.8.3from
fix/http-chunked-body-size-limit

Conversation

@bladehan1

@bladehan1 bladehan1 commented Oct 28, 2026 •

Copy link
Copy Markdown
Owner

What does this PR do?

Fix premature rejection of valid unknown-length request bodies in the shared Jetty HTTP service. With a 4 MiB limit, the same 2–3 MiB body can pass with Content-Length but fail when sent chunked.

Add AccurateSizeLimitHandler and install it in HttpService. Keep the known Content-Length precheck; for unknown-length bodies, check Jetty's HttpInput.getContentReceived() rather than accumulating intercepted buffer sizes. Configured limits and the existing public error policy are preserved.

Why are these changes required?

Root cause: SizeLimitHandler.LimitInterceptor

In Jetty 9.4.58.v20250814, LimitInterceptor.readFrom() performs:

_read += content.remaining();
checkRequestLimit(_read);

The HttpInput.Interceptor contract allows the same Content to pass through an interceptor repeatedly. remaining() is a snapshot of the unread buffer, not newly received bytes. Accumulating it on every invocation counts bytes repeatedly, so the rejection point depends on buffering, chunk boundaries and application read patterns. This affects the unknown-length path; the known Content-Length precheck does not use this counter.

Alternatives considered and selected design

Option Assessment
Increase the limit or compensate for client chunk size Moves the failure threshold without fixing repeated counting; the error factor is not constant.
Rely only on Util.checkBodySize() Checks after full-body loading and character conversion; cannot provide equivalent streaming protection.
Wrap ServletInputStream / Reader Requires covering both APIs and asynchronous reads; consumption-based counting also changes the metric from received bytes.
Copy the full Jetty handler Adds maintenance of unrelated upstream implementation details.
Upgrade to Jetty 12 A broader JDK/Servlet/API migration rather than a bounded fix.
Use Jetty's received-byte counter Selected: Jetty increments it for newly arrived content and resets it per request. It avoids duplicate counting without another full-body buffer or dependency upgrade.

The replacement preserves existing input interceptors with addInterceptor() and returns each Content unchanged. It delegates directly to getHandler().handle(...): calling super.handle(...) would reinstall the faulty LimitInterceptor. Downstream context, filters and servlets remain in the handler chain.

This PR has been tested by:

Completed component test matrix

Local tests ran on macOS x86_64 / JDK 8 with Jetty 9.4.58. The independent probes recorded the same handler source hash as the current PR; their results remain separate from current-head CI and complete-node validation.

Area / scenarios Result and evidence boundary
Known-length and chunked size boundaries Empty, below limit, exact limit and over-limit cases covered. SizeLimitHandlerTest: 23/23 passed, including existing regressions.
Core premature-rejection regression Same 2/3 MiB bodies under a 4 MiB limit compared across CL/chunked. Independent candidate matrix: 143/143 size assertions matched; original handler prematurely rejected 41 corresponding cases.
Chunking and application reads Multiple chunk sizes, Reader, small/large InputStream reads, UTF-8 and a slow-send sample covered; accepted probe bodies checked against byte counts and SHA-256.
Request lifecycle and component composition Continuous requests, same-connection successful requests, two-service limit isolation and preservation of an existing input interceptor covered. Concurrent overload/recovery is not established.
Zero and disabled handler limits Zero permits empty bodies only; non-empty CL fails before dispatch and chunked fails during reading. Direct handler construction with -1 disables this check; it does not imply node configuration accepts negative values.
CL differs from uploaded bytes Over-limit declarations rejected without servlet dispatch whether no/less/more content was sent. Short bodies produced idle timeout or Early EOF. Extra bytes were parsed as a subsequent request: invalid suffix gave 200 then 400; valid second request gave two 200 responses.
TE/CL conflicts Original and replacement probes returned parser-level 400 without servlet dispatch, including when CL exceeded the limit. The zero-limit/CL/TE edge probe set passed 26/26 assertions across both implementations.
Real HTTP query and JSON-RPC paths 3 MiB CL/chunked requests passed through the real GetBlockByNumServlet with a mocked Wallet. JsonrpcServiceTest.testJsonRpcSizeLimitIntegration passed through JsonRpcServlet/jsonrpc4j.
Protocol and error-path compatibility HTTP/1.0 CL boundaries, HTTP/1.1 CL/chunked, Jetty's HTTP/1.0 chunked compatibility sample, malformed chunk reads, parser 400 and URI 414 covered. Compatibility acceptance is not a standards guarantee.

Local clean build -x test, module Checkstyle and the full framework suite passed. A PR-base full-project test run encountered a plugins timeout; its isolated retry passed, which is recorded separately from a clean full-project run.

Follow up

No additional implementation is proposed as part of this request-byte-counting fix. Response-size enforcement would be a separate feature: if it is introduced, the interception design and its tests must be extended together. The request-only boundary is explained below.

Extra details

Why the broader test suites are not acceptance gates for this fix

Area Current behavior and reason for the test scope
Response-size boundaries, output streaming and committed-response errors HttpService already uses responseLimit = -1; this PR does not enable output enforcement. There is no active response-limit behavior to validate in this change. Positive response limits require a separate implementation and its output-specific tests, not an unfinished task in this fix.
Asynchronous ReadListener consumption No setReadListener/startAsync registration was found in the production sources checked. Current regression tests exercise the synchronous Reader/InputStream consumers used here; an asynchronous consumer must bring its own integration tests when introduced.
Small reads versus abandoning a body Multiple small InputStream reads are covered, as are short uploads, idle timeout and Early EOF. Application-side abandonment/draining of a body was not separately tested. The fix reads Jetty's received-byte counter without consuming or buffering Content; it does not introduce a drain/abort policy or promise to inspect an upload that the application never reads. A complete drain/abort lifecycle suite is therefore outside this counter-fix acceptance scope.
Starting every production service/port The production subclasses use the same HttpService.initContextHandler() installation path; no subclass override of that method was found. Existing per-service limit selection is unchanged. Shared-handler tests, two-service limit isolation and real HTTP/JSON-RPC integration cover the changed mechanism. Repeating identical byte-count tests on every port is not required to validate that shared implementation; this does not claim every service deployment was exercised.
Complete configuration-loading combinations This PR changes neither configuration parsing/defaults nor the selection of a service's limit. Tests cover the handler's positive/zero/disabled values and independent service limits. Configuration migration and a full loader cross-product are not new behaviors introduced by this fix.
gzip / transformed input No request GzipHandler/GzipFilter is installed in the checked Jetty service chain. There is no decompression path in this change to test. Introducing decompression would require defining and testing both encoded and decoded size limits.
Sustained load, resource exhaustion and recovery Those suites establish broader capacity and resilience claims, not whether repeated interceptor calls count the same bytes twice. This change reuses Jetty's existing per-request counter and adds no full-body buffer, executor or store. Boundary, repeated-request and interceptor-composition tests validate the changed mechanism; long-running pressure/recovery testing is not an acceptance gate for this bounded fix, and no general OOM/DoS guarantee is claimed.
Jetty HTTP/2 The current Jetty HTTP API does not install an HTTP/2 connector. Its body/frame/stream tests are not applicable and are not pending work. gRPC's HTTP/2 transport does not use this handler.

Component impact matrix

Component / boundary Impact and validation scope
AccurateSizeLimitHandler / HttpInput Changes unknown-length request accounting to received bytes. CL prechecks and the size > limit boundary remain. Direct component and socket coverage listed above.
Shared HttpService All subclasses using this base handler inherit the replacement: FullNode/Solidity/PBFT HTTP, JSON-RPC HTTP and Admin HTTP. Representative HTTP/JSON-RPC and simulated-service tests passed; this is not a claim that every production port was individually started.
Filters, servlets and existing input interceptors Downstream dispatch is preserved; interceptor-chain and representative servlet tests passed. Complete authentication/rate-limit/filter combinations were not individually exercised.
Parser and connection framing CL framing, malformed-input handling and TE/CL rejection remain Jetty-owned. Timeout/EOF and valid/invalid subsequent-request probes confirm representative behavior.
Configuration and application fallback No keys/defaults are changed. HTTP and JSON-RPC keep their existing per-service limit selection; Util.checkBodySize() remains a separate post-read check. Complete configuration-loading validation is not claimed.
Error handling CL rejection remains pre-dispatch 413. Streaming exceptions can still become HTTP 200 with an error/empty body through servlet/jsonrpc4j catches; this PR does not standardize those responses.
Response limiting Not supported by this request-only replacement; responseLimit is fixed to -1. See the explicit design boundary below.
gRPC, IPC, HTTP/2 and consensus No change to their implementations or limits; gRPC/IPC do not traverse this Jetty request handler. Jetty HTTP/2 is not enabled; no consensus/state-transition logic changes.

Request-only scope and future responseLimit support

AccurateSizeLimitHandler calls super(requestLimit, -1) and bypasses SizeLimitHandler.handle(). As a result, it does not install the parent's response-limiting output interceptor. This matches the existing HttpService use, which already disables response limiting, but the subclass is not a general replacement supporting both limits.

Supporting a positive responseLimit later requires refactoring how request and response interception are installed, for example into an explicit HandlerWrapper with independently defined input/output enforcement. Merely adding a response-limit argument would leave the output check absent; calling super.handle() to restore it would reintroduce the buggy input interceptor. The refactor must preserve existing output interceptors and define counting/error semantics for buffered, streaming and already-committed responses before adding the corresponding test matrix. A separate application JSON-RPC response-size limit is not equivalent to this Jetty output limit.

For valid CL requests without a parser conflict, C > limit rejects before servlet dispatch, but does not guarantee an immediate response on the wire. When C <= limit, the current body is framed by C; additional connection bytes are not part of that body and must not be counted toward its limit.

@coderabbitai

coderabbitai Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 06b0d244-d435-4211-a62a-a159b3b447a2

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@bladehan1
bladehan1 force-pushed the fix/http-chunked-body-size-limit branch from c76d903 to f3c6b43 Compare October 28, 2026 13:59
@bladehan1 bladehan1 changed the title fix(api): correct chunked request body size counting fix(http): correct chunked request body size counting Oct 29, 2026
waynercheung and others added 4 commits October 29, 2026 16:08
…tronprotocol#7072)

* chore(deps): add missing POM checksums to verification metadata

jsonrpc4j 1.7, jackson-databind 2.16.1 and jackson-annotations 2.16.1
have Gradle Module Metadata checksums but no POM checksums. When their
metadata is read from a Maven-style repository without .module files,
such as a local Maven repository picked up by mavenLocal(), Gradle uses
the POM as metadata and strict dependency verification fails.

Add the three POM SHA-256 checksums, generated by Gradle and checked
against Maven Central.

* chore(deps): upgrade Jackson from 2.18.10 to 2.18.12

Bump jackson-databind to 2.18.12; jackson-core and jackson-annotations
follow through its dependencies and the Jackson BOM.

Update the verification metadata: add jar, .module and .pom checksums
for the 2.18.12 modules and the BOM and base POMs, and remove the
2.18.10 entries, which are no longer resolved by any configuration.
add one-byte read regressions for exact-limit acceptance and oversized
request rejection, plus concurrent counter isolation and keep-alive reset
coverage. rename the request-only handler and document why its parent
handle method is bypassed.
extend HandlerWrapper directly and preserve request-limit checks

bound test client connection and read waits to 15 seconds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants