[xxo1shine]

What does this PR do?

Add pending-write accounting and timeout handling to improve TCP connection resource management.

• Set a 200 MiB pending-write threshold per connection.
• Trigger eviction of the connection with the most pending data when the shared total reaches 500 MiB.
• Close connections when a write remains incomplete for 10 seconds.
• Reserve framed payload bytes before event-loop submission and release them when writes complete or fail.

The shared threshold triggers asynchronous eviction rather than enforcing a hard global admission limit.

Why are these changes required?

This PR has been tested by:

• Unit Tests
• Manual Testing

Follow up

Extra details

[SeriousCoding789]

Since this change moved here from tronprotocol/java-tron#7071 with the same code (only the logger name differs), here is a consolidated version of the points raised there:

1. Victim selection can hit honest peers. selectVictim picks the connection with the most pending bytes. During full-block sync a single batch is up to 100 blocks of about 2 MB each, so an honest peer that is syncing from this node can easily be the largest one. It is then disconnected the same way as a peer that exceeded its own 200 MiB limit, so it also gets the 60 s reconnect cooldown that every disconnect applies through banNodeOnDisconnect, and trusted or configured active peers are not treated differently. Both cases log the same pending outbound data limit exceeded line, without saying which limit fired or how many bytes were pending. What separates a stalled peer from an honest one working through a large batch is progress rather than size: the honest peer keeps completing writes, while the stalled one completes none. Choosing the victim by the oldest last-completed write, exempting peers evicted only because of global pressure from the cooldown, deprioritizing trusted/active peers, and logging the limit type with the per-peer and total pending bytes would make eviction less likely to hit the wrong peer and easier to diagnose.

2. The limits are hardcoded in the library but depend on the embedding node. The 200 MiB per-connection limit appears to be sized for one java-tron sync batch: MAX_BLOCK_FETCH_PER_PEER (100) × BLOCK_SIZE (2,000,000 bytes) ≈ 191 MiB, which leaves only about 9 MiB of headroom, and nothing ties the two together, so raising either value in java-tron would make an honest full-block batch exceed the limit right away. The 500 MiB global limit is meant to protect direct memory, so it depends on the node's -XX:MaxDirectMemorySize (1g in java-tron's JDK 17 vmoptions). The budget also counts payload bytes, while ProtobufVarint32LengthFieldPrepender grows its output buffer to the next power of two below 4 MiB, so the direct memory actually held can be close to 2x the counted bytes. Making the per-connection limit, the global limit and the write timeout configurable through P2pConfig, with java-tron deriving them from its sync batch size and direct memory limit, would keep them correct when those values change.

3. Writability as a possible follow-up. Exposing Channel.isWritable() together with a configurable write buffer water mark (above one block, e.g. a few MiB) would let java-tron pause serving bulk replies to a peer while its channel is not writable and resume afterwards. That keeps each connection's backlog near the high water mark, so pending outbound memory is roughly bounded by the number of connections × (high water mark + one max-size message), while a peer that stops reading is still closed by the 10 s write timeout and the global eviction mostly becomes a safety net. Simply stopping when the channel turns unwritable would not work on the java-tron side, because the channel also becomes unwritable briefly on fast links once the socket buffer fills, so the remaining requests would need to be served later.

[xxo1shine]

The current policy prioritizes the connection with the most pending data so that, under global resource pressure, we can release more of the budget with fewer disconnections. It does not classify that peer as malicious. An honest syncing peer can indeed become the victim; this is a trade-off in overload protection. Selecting solely by the time of the last completed write could instead choose a connection with little pending data that has not completed a write for a long time, releasing too little of the budget. Conversely, a connection that keeps completing small writes can still accumulate an increasing backlog. Write progress can therefore be considered in a future evaluation, but I would retain the current backlog-based selection policy for now. Being trusted or configured as an active peer does not, by itself, prevent a connection from creating resource pressure.

Cooldown and reconnection behavior currently follow the existing connection-management logic. Outbound and trusted connections already receive different admission handling, so the 60-second reconnection restriction does not apply uniformly to every connection. Distinguishing the trigger and logging the pending data at eviction would help diagnosis and can be considered as a follow-up observability improvement.

Regarding the thresholds, 100 maximum-size blocks is an extreme batch estimate; it does not mean normal syncing will have that much data pending simultaneously. Based on our production observations, blocks are typically around 100 KB, or roughly 10 MB per batch, and completed writes continuously release the budget. This does not guarantee that every large-block or congested-network scenario will avoid triggering the protection. The current budget accounts for pending write bytes including the frame prefix. The 500 MiB threshold triggers soft eviction; it is neither an exact measure of direct-memory usage nor a hard cap on it. Allocator capacity rounding and other memory overhead must be considered separately. Making the parameters configurable has value, but the defaults still require workload measurements and cannot be derived directly from only the sync batch size or the JVM direct-memory limit.

I agree that writability and application-level backpressure are suitable for a separate follow-up. They require java-tron to implement pause/resume behavior and bounded pending-request management. Merely exposing isWritable() or setting watermarks does not automatically limit the rate at which writes are produced. For this PR, I would retain the current combination of pending-write accounting, soft eviction, and write timeouts.