Repository navigation
kestrel: drain bulk-IN during bring-up on USB; redo a bring-up the 8852C fw halted - #496
Conversation
Nothing read the USB bulk-IN between intf_init arming the RX path and the end of bring-up. On the 8852C the RX DMA then flags itself stuck during the H2C-heavy BB/RF table load (HAXI_IDCT B_AX_RXMDA_STUCK_IDCT -> DMAC_ERR_ISR HAXIDMA_ERR -> L1 pre-error 0x999 -> fw assert 0x30000024). Every step still reports success; the radio works for a while and then its TX queues stop draining for good (bulk-OUT timeouts). BringUpMonitorOnce() now starts the WP/bulk-IN drain right after PowerOnFwAndTrx instead of at the end of the TX bring-up. As a safety net, BringUpCheckingFwHalt() reads HALT_C2H_CTRL after a monitor or TX bring-up and, if the fw halted, logs the DMAC error registers (plus the HAXI block on the 8852C, as the vendor's dump_err_status_haxidma_ax does). On USB it then redoes the whole bring-up from power_on, up to 3 times, after reset_for_rebringup() drops the host state one bring-up leaves behind (halbb/halrf contexts, fast-retune caches, H2C sequence). On PCIe the redo is unverified, so a halt fails the bring-up there. A failed or throwing pass stops the drain; a posted halt whose error code reads 0 counts as a halt. tests/kestrel_bringup_halt.sh counts halted bring-ups; the entry in docs/8852c-quirks.md has the mechanism and the numbers. Measured on the TX50UH (8852C), back-to-back txdemo bring-ups (200 frames, ch149, log level warn), same halt check in both arms, the undrained arm with only the early start_wp_drain() removed: - 2026-10-18 16:46-17:00: undrained 17 of 30 runs halted (3 failed all 3 attempts); drained 0 of 30. - 2026-10-20 07:56-08:07: undrained 7 of 66; drained 0 of 50. Every halt read HAXI_IDCT 0x2. TX20U Nano (8852B): 0 of 20 both ways, same bring-up time.
PR Summary by QodoPrevent 8852C firmware halts during USB bring-up
AI Description
Diagram
High-Level Assessment
Files changed (9)
|
Code Review by Qodo
1.
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
Review follow-ups for the bring-up halt check: - fw_err_state(): once HALT_C2H_CTRL says the fw halted, the error code and the DMAC/HAXI diagnostic reads are best-effort. A failed USB read there (the transport throws) is now caught and logged, and the halt is still returned, so the USB redo still happens. The probe itself now runs inside BringUpCheckingFwHalt's cleanup scope, so a failing HALT_C2H_CTRL read stops the bulk-IN drain before it propagates. - tests/kestrel_bringup_halt.sh: a run where txdemo exits non-zero without the "halted on all" line (adapter absent, open or bring-up error, timeout) tested nothing: it is reported as an error, adds no clean attempt, and makes the script exit 1. - src/kestrel/CLAUDE.md: the note is reduced to the rule and a pointer; mechanism, numbers and reproducer stay in docs/8852c-quirks.md only. Tested 2026-10-20 on the TX50UH (8852C) with a test build that fakes a halt on the first check and throws on every diagnostic read: 3 of 3 runs logged the halt as "error registers unreadable", redid the bring-up and exited 0. Stock build: 20 of 20 (8852C) and 10 of 10 (8852B) clean bring-ups through the script, RX on both after the handoff; the script exits 1 with the adapter absent.
josephnef
left a comment
josephnef
left a comment
There was a problem hiding this comment.
Reviewed at 87eab7f. Code read against the drain/RX-loop handoff, the redo's host-state reset and every fw_err_state caller; built with the default options and ctest (83/83); then reproduced on a Linux host, which the PR lists as not covered.
Linux reproduction (Arch, kernel 6.18, TX50UH 35bc:0101 on a USB 2 hub port, no rtw89 loaded). tests/kestrel_bringup_halt.sh, N=30 per arm, ch149, 200 frames, back-to-back, same build with only the early start_wp_drain() removed for the undrained arm:
| arm | runs halted | halted attempts | runs failed (3/3) |
|---|---|---|---|
| undrained | 30 of 30 | 90 of 90 | 30 |
| this PR | 0 of 30 | 0 of 30 | 0 |
Every halt read err=0x30000024, dmac_isr=0x4000, haxi_idct=0x2, the same chain as on your bench. Two things differ from the Windows numbers and belong in docs/8852c-quirks.md as the adversarial counterpart to the 17-of-30 row: here the halt is deterministic, and the redo recovered none of the 30 halted runs (the chip side does clear: the drained arm's first run came up clean straight after the undrained arm left the fw asserted). So on Linux fix 1 is the whole fix and the retry has no measured value yet; fine to keep as a net, but please don't let the doc read as though it recovers most halts in general.
Also checked on this host: rxdemo (Init path, where no drain thread existed before) hands the bulk-IN from the drain to the RX loop and receives frames on ch6; a 200-frame txdemo completes with the drain started before the table load.
Two non-blocking nits:
tests/kestrel_bringup_halt.sh: theenvline does not passDEVOURER_USB_BUS/DEVOURER_USB_PORT, andsudostrips them, so two identical adapters (this rig has two TX50UH) cannot be told apart withoutSUDO="sudo --preserve-env=...". Forwarding both in theenvline when set would match the other kestrel scripts' knob style.- The doc's "Not covered: ... Linux hosts" can be replaced by the table above.
Approving; the nits can ride a follow-up.
josephnef
added
the
skip-qodo-gate
In short
On the 8852C (TP-Link TX50UH), a bring-up could leave the chip's firmware crashed while every step reported success. The adapter then worked for a while, and minutes later it stopped sending anything (every USB send timed out). In a long AirDrop transfer this looked like a random mid-transfer hang.
The cause: while the chip is being set up, it already starts handing received data to the host over USB, but devourer only started reading that data at the very end of the bring-up. On the 8852C the chip's receive path counts that as "stuck", raises an internal error, and the firmware crashes a moment later. The 8852B tolerates it.
The fix has two parts:
Tested on both chips, many bring-ups in a row, in two sessions on different days:
How often it crashes without the fix varies a lot from one session to the next, but with the fix it never did. A new script,
tests/kestrel_bringup_halt.sh, counts the crashes so anyone can repeat this.Start-up takes the same time with and without the fix on both chips (a full test run: 4.7 s on the 8852C, 7.9 s on the 8852B).
Technical detail
Failure chain
intf_initarms the USB RX path, but nothing reads the bulk-IN untilInitWritestarts the WP-release drain (orInitstarts the RX loop) at the end. During the H2C-heavy BB/RF table load the 8852C then raises:HAXI_IDCTbit 1 (B_AX_RXMDA_STUCK_IDCT) ->DMAC_ERR_ISRB_AX_HAXIDMA_ERR_FLAG(0x4000) -> L1 pre-error0x999(L1_PREERR_DMAC) posted inHALT_C2H-> with no host L1 handshake, fw assert0x30000024.Nothing in the bring-up reads
HALT_C2Hafter the first steps, so the bring-up returns success. TX keeps working until the TX queues stop draining (bulk-OUT timeouts on every send).Changes
BringUpMonitorOnce()starts the WP/bulk-IN drain right afterPowerOnFwAndTrx()on USB.StartRxLoop()already stops the drain before taking the endpoint, andInitWrite's laterstart_wp_drain()is a no-op when it is already running, so the handoffs are unchanged. No bring-up step reads the bulk-IN synchronously, so the drain cannot steal a response. PCIe gets no drain thread (its release reports arrive on the RPQ ring).BringUpCheckingFwHalt()wraps the monitor bring-up (Init) and the whole TX bring-up (InitWrite, so a halt in its TX half also redoes everything). After each pass it callsfw_err_state(); on a halt it logs a warning, stops the drain, callsHalKestrel::reset_for_rebringup()(destroys the halbb/halrf contexts and clears their one-shot flags, the fast-retune caches and the H2C sequence number) and runs the bring-up again frompower_on, which forces the MAC off. Up to 3 attempts on USB;InitWritethrows after the third, as it does for any refused bring-up. On PCIe the redo (and the ring re-init under it) is unverified, so there a halt fails the bring-up at once instead of being retried. A pass that fails or throws stops the drain before returning.fw_err_state()now also logsDMAC_ERR_ISRandSER_DBG_INFO(both dies), plus the HAXI registers on the 8852C only, matching the vendor'sdump_err_status_haxidma_ax, which is built for the AXI dies only. OnceHALT_C2H_CTRLsays the firmware halted, the rest is best-effort: if the error code reads 0 or the diagnostic reads fail (the USB transport throws), the failure is logged andHALT_C2H_CTRLis returned, so the caller still sees the halt and the USB redo still happens (a zero code was never observed: every halt in our logs carried 0x999 or 0x30000024). The register names are added toMacRegAx.h(mac_reg_ax.hvalues).docs/8852c-quirks.md: a new entry in the file's usual shape (what the chip does, what devourer does, residual cost, reproducer).src/kestrel/CLAUDE.md: under "Dispatch and cold boot", only the rule and a pointer to that entry.tests/kestrel_bringup_halt.sh: N back-to-back txdemo bring-ups at log level warn, counts the halted attempts from the halt check's log line, exits 1 on any halt, and also on any run where txdemo failed without one (adapter absent, open error, timeout), since such a run tested nothing. For the A/B, run it against a build with the earlystart_wp_drain()removed.Measurements
Windows 11, MSVC Release build, rear USB 3 port. txdemo, 200 canonical beacons on ch149,
DEVOURER_LOG_LEVEL=warn(the halt is timing-sensitive: debug logging slows the H2C stream and hides it). Back-to-back processes, no power cycle between them. "Without fix 1" is this branch with only thestart_wp_drain()call inBringUpMonitorOnce()removed, so the same halt check counts the halts (master has no check at the end of bring-up, so it cannot count them).Session 1, 2026-10-18 16:46-17:00 local time:
All 24 halts read
HAXI_IDCT=0x00000002(RX DMA stuck) andDMAC_ERR_ISR=0x00004000. The retry recovered 14 of the 17 halted runs, which is why it stays only as a safety net. RX: 3 rxdemo bring-ups per die with the fix received frames after the drain handed the bulk-IN over to the RX loop.Session 2, 2026-10-20 07:56-08:07 local time, TX50UH only, on 35fe35e (partly through
tests/kestrel_bringup_halt.sh): without fix 1, 7 of 66 runs halted (8 of 74 attempts, none failed all 3; again every oneHAXI_IDCT=0x2,DMAC_ERR_ISR=0x4000); with it, 0 of 50. The rate without the fix was much lower than in session 1. The halt is timing-sensitive, and how much traffic the radio hears during the bring-up may also matter (the bulk-IN only fills if frames arrive); I have not isolated that. The script reproduced the halt (and exited 1) in the undrained arm.Earlier, in the AirDrop receiver on the same TX50UH (2026-10-16): 8 of 48 bring-ups halted without fix 1, 0 of 80 with it. The rate differs from txdemo's because the timing differs; both point the same way.
Not covered: the 8852CE on PCIe (no drain thread there; the halt check runs but does not retry), and Linux hosts (the halt depends on timing, and I only have this Windows bench). The two
Kestrel coex: LTE interface not readyerrors the 8852C logs at every bring-up are unrelated and also appear on master.