Repository navigation
Move radio TX to a dedicated task so the HA WebSocket stays connected - #35
Merged
Merged
Conversation
Web command handlers ran on the async_tcp task and blocked up to 3s waiting for loop() to bit-bang radio frames. async_tcp also services the WebSocket on port 8080, so pongs stalled and Home Assistant (ping_timeout=15) disconnected when several covers were commanded. - Add a radio_tx FreeRTOS task with a job queue; sendFrame/repeatFrame and repeater frames enqueue instead of transmitting inline. - Serialize CC1101 access with a recursive mutex (TX task, apply, end, frequency scan, RSSI read). - Make Web::queueCommand fire-and-forget; drop the done semaphore. - Route /setMyPosition and /unpairShade through the command queue. - Clamp repeats to MAX_TX_REPEATS (40). - Pin AsyncTCP to core 0 on dual-core targets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Abort in-flight transmissions on flush so config save, frequency scan and OTA never wait on the radio lock for a whole job - Suspend TX during frequency scan and OTA - Refuse commands the radio queue cannot take before consuming a rolling code or moving simulated shade state; shift movement start when a job goes on air late - Apply radio config changes under the radio lock - Re-check receiver idle before repeater transmissions - Return 503 from web handlers when the command queue is full - Restore 200 ms settle delay before the set-My command - Report RSSI 0 instead of a stale value when the radio is busy - Encode each frame once and centralise the repeat clamp Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- Serialize queueTx and flushTx so a queued job can't be lost to a flush. - Settings changes preempt only the job in flight; the queue survives. - Flush drains the queue and reports each job as dropped. - Track suspend reasons (scan, OTA) separately. - Report started/dropped for every job on a single event queue. - Add priority jobs that go to the front and may use reserved slots. - Remove the MAX_TX_REPEATS clamp. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Send functions return whether the radio took the command. - Shade, group and tilt state only change when the command was queued. - Rolling code is committed only after the frame is queued. - repeatFrame advances the repeat counter only on success. - Remove canTransmit/canQueueTx; queueTx is the single authority. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Hold a shade's simulated movement until its command is on air. - Send stops at the target as priority jobs. - If a stop can't be sent or is dropped, simulate the run to the limit. - Roll back targets when a movement command is dropped. - Only store a favorite position when SETMY was queued. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- endFrequencyScan always resumes TX; applying radio settings ends a scan. - Frequency scans time out after 5 minutes. - Reboot when an OTA upload is abandoned mid-transfer. - /sendRemoteCommand returns 503 when the radio can't take the frame. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
markTxPending ran inside sendCommandAfter before callers set the new target, so isAtTarget() kept idle shades from ever being held, while shades already moving were frozen although their motor kept running. Hold a shade only when it was idle before the command and release the hold when the command didn't set it moving. The single mark in sendCommandAfter now covers group members too, so the second pass in SomfyGroup::sendCommand is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A priority stop always went to the front of the radio queue, ahead of an earlier frame of the same remote that was still waiting. The older rolling code then went out last and the motor rejected it. Split "may use the reserved slots" (priority) from "goes to the front" (front) and only send a stop to the front when its remote has nothing queued at the back. The radio task and flushes track which back jobs have been taken so Transceiver::isQueued() can tell. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The radio task bit-bangs a whole job on loop()'s core at a higher priority. A job with 127+ repeats ran for 18-36s without loop() ever running and the 15s task watchdog rebooted the device. Limit a job to 50 repeats (about 7s) in encodeTxJob, which every send path goes through, and advance the 80-bit repeat counter by the number of frames actually queued. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The favorite was persisted as soon as the long My press was queued. A flush or radio reconfiguration could drop or cut it short and the app kept a favorite the motor never stored. The radio task now reports finished/aborted for every job. Setting a favorite goes through sendSetMy, which commits it on finished and puts the previous one back otherwise. The new favorite is set before the My is sent so the simulation no longer heads for the old one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Restoring a backup wrote the transceiver config in place from the async task while the radio task could be transmitting with it. Read the record into a copy and swap it in through the new Transceiver::setConfig, which Transceiver::fromJSON now uses as well. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Web commands are answered once queued, but the main loop handed all of them to the radio at once and anything past its free slots was dropped silently. Unpair also cleared paired even when the Prog wasn't sent. - processQueue leaves commands queued until the radio has room. - Commands that transmit get a 503 straight away during a frequency scan instead of being dropped later. - Unpair restores paired when the Prog can't be sent. - Failed web commands are logged. - Note that /setPositions and /setSensor answer with the state before the command; the socket events carry the result. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds an [env:native] for host-side unit tests (pio test -e native) with a minimal Arduino.h stub, and covers the Utils helpers: _trim, toBoolean, SETCHARPROP, sortArray and the Timestamp conversions. The firmware envs now extend a shared [esp32] section instead of [env] so the native env doesn't inherit the Arduino framework, the ESP32 libraries or the build scripts. The half-hour calcTZOffset cases fail: +05:30 comes back as 2300. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a "Unit tests" job running pio test -e native on ubuntu-latest. The firmware build matrix waits for it, so a failing test also stops release uploads. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
calcTZOffset added the leftover seconds instead of minutes to the hhmm offset, so +05:30 came out as 2300 and ISO timestamps showed +2300. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The command and protocol enums, somfy_frame_t and its encode/decode methods, and both translateSomfyCommand overloads move verbatim out of Somfy.cpp/Somfy.h, which now include SomfyFrame.h. The codec doesn't depend on the radio, storage or network, so on its own it can be unit tested on the PC. decodeFrame(somfy_rx_t*) stays in Somfy.cpp since it reads the RSSI from the CC1101. No behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Builds SomfyFrame.cpp in the native env, with a minimal String and no-op ESP_LOGx in the stubs, and tests: - a fixed 56-bit frame, byte for byte, so what goes on air is pinned - encode/decode round trips for 56-bit, 80-bit (repeats, Favorite, Stop, steps, Toggle), RTW and RTV frames - rejection of bad checksums, addresses, rolling codes, keys and 80-bit parity - repeat/synonym tracking in copy() - parsing and naming of commands The command-name round trip fails: names the firmware reports, such as "Step Up" and "My+Up", parse back to the wrong command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shadeCommand events (and the Home Assistant events built from them) name commands "Step Up", "Sun Flag", "My+Up" and so on, but parsing those names fell through to the prefix shortcuts: "Step Up" became SunFlag, "Sun Flag" became StepUp and "My+Up" became My. Match the reported names exactly before the shortcuts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
JsonFormatter and BufferedJsonFormatter move verbatim out of WResp.cpp/WResp.h, which now include JsonFormatter.h. They don't need the web server, so on their own they can be unit tested on the PC; the native env builds the new file. AsyncJsonResp and JsonSockEvent stay in WResp. No behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tests BufferedJsonFormatter output: nesting and commas, number and bool values, skipped null strings and escaping. A canary after the buffer catches writes past its end. Three fail: control characters other than \b\f\n\r\t aren't escaped, calcEscapedLength skips the first character, and so a value starting with an escaped character can be written one byte past the buffer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
calcEscapedLength looped from strlen(raw) down to 1, counting the terminator instead of the first character. A value starting with a character that needs escaping was counted one byte short, so the fit check could let it be written one byte past the buffer. Control characters other than \b\f\n\r\t were copied raw, which isn't valid JSON. Both now go through one escapeChar helper, which also escapes the remaining control characters as \u00XX. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When something didn't fit, the formatters skipped it silently, closing brackets included, and clients got malformed JSON. Two writes could also go past the buffer: - AsyncJsonResp flushed when full and then wrote the value without checking again, so a value bigger than the whole buffer overflowed. It is now streamed straight out instead. - JsonSockEvent::closeEvent added the closing ] when only one byte was left, putting the terminator one byte past the buffer. The formatters now set an overflowed() flag and log an error, and a socket event that overflowed is logged and not sent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
checkMovement worked out the position four times, nearly copy-pasted, for lift down, lift up, tilt down and tilt up. All four now call travelPosition(), which keeps the same arithmetic, including how the elapsed time wraps with millis(). Tilt up with no tilt time still stops the tilt direction in the caller. transformPosition and transformCommand keep their behaviour through the new displayPosition() and flipCommand() helpers. The native env builds ShadeMotion.cpp so all of this can be unit tested on the PC. No behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tests travelPosition (part-way positions both ways, separate up and down times, stopping at the ends, zero travel time, steady progress and staying within 0-100), displayPosition with and without flipping, and flipCommand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At -70 dBm the RSSI moves a dBm or two constantly, so the >1 dBm check every 1.5s sent wifiStrength to every client about 270 times an hour. That is extra traffic on an already lossy link and a state write in Home Assistant for each one. Broadcast on a channel change, or for a change of at least 3 dBm no more than every 10s. A newly connected client still gets the current value straight away. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add Timing.h and use it to log what holds up the main loop, how long a new socket client's setup takes, slow socket sends, and how long each web request took and from which client. Socket connects and disconnects are logged with how long the client stayed, and late pings as warnings. Routine sends and pings are not logged, as each line costs several ms on a 115200 baud console. Report a stored crash once with the crashed and running build IDs, then erase it, since it was reported on every boot and its backtrace only decodes against the build that crashed. Mount the file system once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Arduino core's log_printf() has its UART lock commented out and writes with ets_printf(), so lines logged at the same time from async_tcp and loop() came out interleaved character by character. It also formats short lines into one static buffer shared by every task. Wrap log_printf at link time and hold a mutex around the core's log_printfv(). It never blocks before the scheduler runs, in an ISR or in a critical section, and waits at most 50ms before printing anyway. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
serializeShadeRef sent flipCommands as flipPosition, so group linked shades and the /shadeCommand reply showed the wrong setting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The roaming scan found two APs for the SSID every 10 minutes but the log never showed their signal, so there was no way to tell whether the device at -70 dBm had a better AP to move to. Log each AP on the roaming scans, show roams at info level, and switch for a 6 dB gain instead of 10. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It went out whenever free heap moved by 1.5 KB and 7s had passed, which with normal traffic was about 240 times an hour. Send it every 30s instead. It is also the one event always sent, so 30s keeps Home Assistant well inside its 60s receive timeout. Per-client sends and the web UI's room are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The /discovery handler takes about 200ms on the async_tcp task on every call with no obvious cause. When it takes 50ms or more, log how long the header, memory stats, rooms, shades, groups and sending took. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Arduino core already links with --wrap=log_printf for the ESP Insights hook in libesp_diagnostics, so defining __wrap_log_printf failed to link with a duplicate definition. That hook prints through log_printfv, so wrap that instead and take the lock there. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Home Assistant's aiohttp client only pings after 20s without a message from the device, so with memStatus every 30s pings arrive every 30-60s and every one was logged as late. The gap depends on our own traffic, not the link, so log pings at debug level and keep the time of the last one for the disconnect line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every append found the end of the buffer with strlen(), up to four times per append and four appends per field, so building a response cost the square of its size. /discovery fills a 4 KB buffer and spent about 195 ms of its ~210 ms there (shades 126 ms, groups 69 ms), all on the async_tcp task that serves every socket and web request. Keep the length in the formatter and write every append at the end through one helper. The web response and socket event subclasses and ControllerChunker use the tracked length too. Output is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The log lock was a mutex, so when async_tcp waited on it loop() inherited async_tcp's priority and could preempt the radio task mid-frame. Use a binary semaphore, which has no priority inheritance, track the holder so a nested log doesn't wait on itself, and cap the wait at 20ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every HTTP request logged two timing lines and every socket connect five, all from the async_tcp task, at several ms each on a 115200 baud console. Raise the default TIMING_LOG_MIN_MS to 100 and drop the unused ScopeTimer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Erasing the dump on the first boot after a crash lost it on headless devices where nobody saw the serial summary. Leave it until the next crash replaces it and use the reset reason to tell a fresh crash (WARN) from one reported on an earlier boot (INFO). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The radio task took a delayed job (the My after a shade reaches its target) off the queue and slept until it was due, so a stop queued at the front waited for the sleep plus the whole delayed job. Wait on a task notification instead and send front jobs while the delayed one is held. A held job still counts as queued for isQueued(), so a stop for the same remote goes behind it and the rolling codes stay in order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e JSON The formatter drops a value that doesn't fit but keeps appending the commas and braces after it, so an oversized shade or group streamed invalid JSON. Check overflowed() after each unit, log the item and skip it, and keep producing until a unit or the end so a skip never ends the stream early. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
processQueue stopped draining web commands whenever the radio queue was near full, including set_positions, which never transmits. Look at the command at the head and only hold it back if it needs the radio, keeping the commands in order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Splitting getNextRollingCode into peek and commit opened the ShadeCodes namespace twice per command. Once a code has been written for an address lastRollingCode matches NVS, so peek uses it and only commit opens NVS. Changing the address drops back to reading NVS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
emitSockets() sent memStatus and then emitSockets(255) sent it again, so room 0 got two identical events in one pass after every /discovery. Only the per-client path sends it from emitSockets(num). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
markTxPending copied the shade and linked-remote address matching from SomfyShade::processFrame. Both now use findRemote() so they can't drift. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
app_elf_sha256 has no terminator (IDF 4.4 fills all 17 bytes), so strcmp never matched the running build and %s printed whatever followed it. Copy it into a terminated buffer first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The frame was popped before queueTx(), so whenever the radio queue was busy it was lost. Peek it instead, only remove it once it is queued (or TX is suspended), and don't try while the queue has no room. The repeater buffer moves to TxBuffer.h/.cpp so it can be unit tested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
beginFrequencyScan, endFrequencyScan and apply() run on async_tcp from the web handlers and emitted into the socket event buffer loop() also writes, so an event could be mixed up or dropped. They now only flag the change and Transceiver::loop() sends it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review flagged the outer My/Favorite losing to the nested Up/Down it queues. Every such nested send is from a simMy shade, where the Up/Down is what starts the motor, so the behaviour stays. The hold moves into TxHold in ShadeMotion with a comment explaining the order and unit tests pinning it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setConfig() swapped the config in and released the radio lock, and apply() only ran later from save(), so the radio task could send queued jobs with the new TX pin and settings on a CC1101 that wasn't set up for them. setConfig() now swaps, saves and applies under one lock and save() goes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The emit to a newly connected client also stored the RSSI and time, which restarted the 10s hold-off and hid a real change from the clients already connected. The baseline moves into RssiBroadcast, which ignores per-client emits, with unit tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The log lock is a binary semaphore, so when the radio task preempted loop() mid-log every log from async_tcp waited the full 20ms for a holder that couldn't let go until the radio job ended. Waiting now stops as soon as the holder is preempted on another core. The rule lives in LogLockPolicy.h with unit tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
timeRequest built both 112-byte descriptions (with a String copy of the URL) on async_tcp for every request, and logElapsed threw them away for anything under 100ms. Check worthLogging() first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…web command hold Move the pieces of the radio TX path that don't need FreeRTOS or NVS out of Somfy.cpp and Web.cpp so they build in the native test env: the TX job and event types and job encoding (TxBuffer), the queue rules (TxQueuePolicy.h), a shade's pending radio jobs (ShadeTxState), the next rolling code (RollingCode.h) and the web command queue types (WebCommand.h). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The order radio jobs go out in was spread across callers, file statics and the radio task: each remote tracked its last back job so a stop could decide whether it may jump the queue, and the task kept a separate count of the back jobs it had taken. RadioScheduler now owns the waiting jobs, their tickets, the stop-goes-first rule, delayed jobs, flush and suspend, and the outcome of every job. It is plain C++, with the time passed in and the caller locking, so all of it is covered by native tests. Transceiver keeps the FreeRTOS task, the locks and the CC1101. sendCommandAfter returns the job's ticket instead of leaving it in lastTxSeq. Two stops for the same remote now go out in the order they were sent; queued to the front they used to come out reversed, putting the rolling codes out of order. Adds GLOSSARY.md for the radio terms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Home Assistant's WebSocket (port 8080,
ping_timeout=15) disconnects during radio operations, especially when several covers are moved at once.Causes:
Web::queueCommandruns on theasync_tcptask and waited up to 3 s forloop()to finish transmitting.async_tcpalso serves the WebSocket, so pongs stalled. Back-to-back HA commands added up past 15 s./setMyPosition(35 repeats, about 5 s) and/unpairShadetransmitted directly onasync_tcp. They also racedloop()on the CC1101.Transceiver::sendFramebusy-waits about 140 ms per frame, with all repeats sent in one call.repeatwas unbounded (up to 127, about 18 s).Changes
radio_txtask (priority 5) fed by a 32-slot job queue.SomfyShadeController::sendFrame,SomfyRemote::repeatFrameand repeater frames now queue a copy of the frame instead of transmitting inline.delay()spacing incheckMovement.config.apply(),Transceiver::end(), frequency scan, and the RSSI read (try-lock only).apply()/end()flush pending jobs.queueCommandno longer waits for the command to run (semaphore removed, queue 32 deep)./setMyPositionand/unpairShadego through the command queue.MAX_TX_REPEATS(40).CONFIG_ASYNC_TCP_RUNNING_CORE=0on dual-core envs (esp32dev, esp32devdbg, esp32s3).Behaviour changes
loop()is still paused while a job transmits on its core (at most about 6 s per job with the cap). The 100 ms gap between jobs keeps the watchdog fed.Testing
Draft: not yet built or tested on hardware.
repeat=127is capped; WebSocket stays up🤖 Generated with Claude Code