Skip to content

Fix NVS, radio and shade bugs; split Somfy.cpp into tested modules - #36

Merged
cjkas merged 40 commits into
mainfrom
fix/live-bugs
Oct 14, 2026
Merged

cjkas merged 40 commits into
mainfrom
fix/live-bugs

Conversation

@cjkas

@cjkas cjkas commented Oct 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

1. Live bugs

NVS: one Preferences handle per function instead of one global. loop() and async_tcp shared a single handle:

 loop()                              async_tcp (/saveRadio)
   pref.begin("ShadeCodes")
                                       pref.begin("CC1101")   // fails: handle already open
                                       pref.clear()           // clears ShadeCodes
   pref.putUShort(remote, code)
-  → every rolling code erased, paired motors ignore the next frames
+  each function opens its own Preferences; NVS serializes access itself

No lock or wrapper is added (see docs/adr/0001-local-nvs-handles-no-wrapper.md).

Where Bug Fix
SomfyGroup::emitState Read a missing linked shade's flags through a null pointer Only read flags for a shade that exists
/repeatCommand (shade) Shade fields written inside an array: invalid JSON Return an object, as the group branch and the web UI expect
MQTT groups/+/sunFlag/set 1 turned the sun flag off, 0 on Shades and groups share sunFlagCommand()
Received StepDown, tilt-only shade Tilt step set as the lift target Set the tilt target
Received StepUp Guarded downTime == 0 but divided by upTime Guard the travel time of the direction stepped
Set-My (favorite) 200 ms settle delay counted from queueing, so it ran out while the stop ahead was still on air A delayed job waits after the job ahead goes off air
Radio event ring A full ring dropped a job outcome: favorite never committed, held shade waited 30 s Jobs are refused unless both outcomes fit
Tilt and set-My commands A command the radio refused was counted as sent They return false; the web command queue logs it
/unpairShade Cleared paired on the web server task; if TX was suspended the Prog was dropped after the client was told paired:false Only loop() clears it, and the unpair waits for TX to resume
Sun and wind, tilt-only shade Three callers set three different targets (lift, lift and tilt, tilt) sensorTargets(): a tilt-only shade moves its tilt, others their lift
Sun and wind timers 64-bit timestamps of a 32-bit millis() fired at once after the 49.7-day rollover 32-bit, wrap-safe

2. Somfy.cpp split into files by concern

Somfy.cpp was 4,766 lines with ~690 that never compiled (#ifdef USE_NVS, commented-out code). The dead code is gone, and the rest moved unchanged:

src/
├── SomfyShade.cpp        # what a shade does (renamed Somfy.cpp, history follows)
├── SomfyRemote.cpp       # sending commands, rolling codes
├── SomfyGroup.cpp
├── ShadeController.cpp   # the shade list, rooms
├── ShadeGpio.cpp         # relay outputs of dry-contact shades
├── SomfyMqtt.cpp         # publish, Home Assistant discovery
├── SomfyEvents.cpp       # socket events
├── SomfyJson.cpp         # all config parsing
├── Transceiver.h/.cpp    # CC1101, radio task, frequency scan
├── PulseDecoder.h/.cpp   # received edges → frame bits (IRAM)   ← native tests
├── ShadeSensors.h/.cpp   # sun and wind timers and targets       ← native tests
├── ShadeMotion.cpp       # + stepTargets() for remote and group steps ← native tests
└── ShadeTypes.h          # shade, tilt and group kinds

The three tested modules replace logic that existed in two or three drifting copies. The plan and what changed are in docs/SOMFY_SPLIT_PLAN.md.

3. Debug build: --param=large-unit-insns=1000

At -Og, GCC only caps inlining growth in units over 10,000 instructions. Splitting Somfy.cpp let the smaller files inline ArduinoJson into every caller, and the image grew past the 1.5 MB partition. The flag treats every file as large; it is set for [env:esp32devdbg] only.

Evidence

  • Before: sun timer on a tilt-only shade with favorite tilt 30 → lift target set to the lift favorite; tilt unchanged.
    After: test_sensors → test_tilt_only_shade_only_moves_its_tilt:

    sensorTargets(extend,  tiltonly, myPos 60, myTiltPos 30) → tilt 30, lift untouched
    sensorTargets(retract, tiltonly, ...)                    → tilt 0,  lift untouched
    
  • Before: the receive decoder had no tests.
    After: test_pulse feeds it the edges sendFrame() produces:

    56-bit first frame (2 syncs) and repeat (7)  → payload decoded, bit_length 56
    80-bit first frame (12 syncs) and repeat (6) → payload decoded, bit_length 80
    glitch < bitMin → skipped; broken pulse mid-frame → next frame still decoded
    
  • pio test -e native: 165 → 222 test cases, all passing.

  • Flash, esp32devdbg: 1,561,977 → 1,545,717 bytes of 1,572,864 (99.3% → 98.3%). RAM 98,732 → 98,100.

  • Builds locally: esp32devdbg, esp32dev, esp32c3, esp32s3. esp32c6 is left to CI.

  • The split keeps all 253 top-level definitions of Somfy.cpp; git blame -C -C follows moved lines to their original commits.

  • Hardware (not yet done):

    • Remote presses received from a 56-bit and an 80-bit remote
    • Shade commands go on air; frequency scan starts and stops
    • Save radio settings while sending commands; every shade still responds
    • MQTT groups/<id>/sunFlag/set 1 → flag on, 0 → off
    • Remote and group step on a lift shade and a tilt-only shade move one step
    • Sun flag on/off moves only a tilt-only shade's tilt; wind blocks a remote Down for 30 s
    • Unpair a shade; set a favorite

Merge Danger

Door: two-way

No stored formats change: NVS namespaces and keys, the shades file and the API routes are as before.

Blast Radius: radio

  • Receiving: the decoder moved into its own IRAM function. A mistake there means no remote or sensor frames are received.
  • Debug build: the inlining flag changes compiled code across the whole esp32devdbg image.
  • Settings: every NVS read and write opens its own handle; a missed conversion would show as settings or rolling codes not loading or saving.
  • Tilt-only shades with a sun sensor now move their tilt, not their lift, on sun and wind.
  • MQTT: automations that inverted a group's sunFlag value to work around the old bug will now do the opposite.
  • /repeatCommand for a shade now returns an object instead of invalid JSON.
  • Roaming: a 6 dB stronger AP is now enough to roam; the old check needed 7 dB despite saying 6.
  • Group steps on a tilt-only shade at its end, and steps on a shade with no travel time, now behave like remote steps.

🤖 Generated with Claude Code

cjkas and others added 30 commits October 12, 2026 20:51
loop() and the async web server task shared one global Preferences.  A
begin() from one task failed while the other had a namespace open, and the
clear() or remove() after it then hit the other task's namespace: saving the
radio settings while a command committed its rolling code could erase every
rolling code.

Each function now opens NVS with a local Preferences.  NVS serializes access
internally and the destructor closes the handle, so no lock is added (see
docs/adr/0001).  No new tests: nothing here builds natively, and deleting the
global makes any missed use a compile error.

The read-then-commit races on rolling codes across tasks remain and are
tracked in docs/KNOWN_ISSUES.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SomfyGroup::emitState skipped a linked shade id that no longer resolves when
listing the shades, but still read its flags through the null pointer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A StepDown frame for a tilt-only shade computed the new tilt position but set
it as the lift target, so the simulated tilt never moved.  StepUp and the
internal StepDown already set the tilt target.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shade branch opened an array and wrote the shade's named fields straight
into it, so the response was invalid JSON.  The web UI reads it as one shade,
as the group branch already returns it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
groups/+/sunFlag/set sent Flag for 1 and SunFlag for 0, the opposite of the
shade topic.  SunFlag turns the flag on.  Both topics now take the command
from sunFlagCommand() so they can't drift apart again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lift StepUp checked downTime for 0 but divided by upTime, so a shade with
an up time of 0 had its target sent to 0% instead of the step being ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The delay was counted from when the job was queued.  The stop at the
target is queued just before the favorite and is on air for about 245ms,
so the 200ms settle delay had run out before the favorite's turn and only
the 100ms beat between jobs was left.  A delayed job now waits its delay
after whichever came later: being queued, or the job ahead of it going off
air.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ring

A full event ring threw an outcome away, breaking the rule that every job
we submit ends with exactly one of finished, aborted or dropped.  A lost
outcome left a favorite uncommitted and a held shade waiting out its 30s
timeout.  submit() and hasRoom() now keep room for two outcomes of every
job not yet ended, so an event can't be lost; if loop() stops collecting
them, new jobs are refused instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
moveToTiltTarget, sendTiltCommand and setMyPosition returned nothing, so
the web command queue counted tilt_command and set_my_position as sent
and never logged a refusal.  They now return false when the radio refused
a command they sent.

No native test: these live in Somfy.cpp, which the native env can't build.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
queueUnpairShade cleared shade->paired on the async_tcp task while loop()
owns the shade.  A second unpair queued before the first ran recorded
wasPaired=false and, if its Prog was refused, put back the wrong value.
The shade is now only changed by loop() once the Prog is queued, and the
response reports paired=false through a serializeShade overload instead.

No native test: Web.cpp can't be built by the native env.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
processFrame worked out a StepUp/StepDown target with the same formula in
eight places, each with its own divide-by-zero guard.  4dd4a98 fixed one
that guarded on downTime but divided by upTime.  stepPosition() takes the
travel time once, for both the guard and the maths, so they can't drift
apart again, and it is now covered by native tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- setMyPosition stored a new favorite the same way in each of its three
  tilt branches; storeMyPosition() now does it.  The plain-lift branch
  now emits its state even when the radio refuses the favorite, which is
  harmless since sendSetMy() has already put the old one back.
- tiltRepeats() replaces the repeated tiltmotor ? TILT_REPEATS : repeats.
- timeRequest's two timing log blocks share logRequestTime().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Transceiver::flushTx() had no caller; suspendTx() is the only flush.
RadioScheduler::hasRoom() was only ever asked about a normal job, so it no
longer takes a priority flag.  The tests check the reserved stop slots
through submit() instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The repeater-frame flag was called raw and the ticket was seq, jobSeq,
stopSeq or mySeq depending on the file.  They are now repeater and ticket
(stopTicket, myTicket, lastTicket) throughout, matching the glossary.
Renames only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
clientMap[], connectedAt[] and lastPing[] were parallel arrays indexed by
the same slot.  They are now one ws_slot_t per slot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A websocket event too large for its buffer has been logged and not
  sent since 8881289, not sent truncated.  Retitle the entry and point at
  MAX_SOCK_RESPONSE by name rather than a stale line number.
- List the web server task handlers that still change the shade model
  loop() owns: /setPaired, /setRollingCode and restore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/unpairShade answers paired:false as soon as the unpair is queued. If TX
was suspended before loop() reached it, the Prog was dropped and the shade
stayed paired, with nothing to tell the client. The unpair now waits in the
queue until TX resumes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…riters

List every web handler that still changes the shade model, add the
oversized /controller items that are left out without telling the client,
the /linkRemote rolling-code race, and the /setMyPosition tilt default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two test names still said seq after the rename to ticket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move the roam check into worthRoaming() and the memStatus routing into
heapEmitTarget() in Utils so the native tests reach them.

worthRoaming() roams for a gain of at least ROAM_MIN_GAIN_DB, as the
comment and the 6 dB commit said. The old check needed one dB more.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
loop(), socket client init and /discovery each read a step's time and
restarted the timer by hand. lap() does both. Also put the request timing
comment back on timeRequest() and say why /discovery logs its steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
USE_NVS is never defined, so the NVS-era shade storage (loadLegacy,
SomfyShade::load and the NVS branches) never built. Also drop the
commented-out toJSON methods, snprintf emitters and unused CC1101 options,
useNVS() and sort_asc(), which nothing calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At -Og GCC only caps inlining growth in units over large-unit-insns
(10000 instructions). Smaller files inline ArduinoJson's helpers into
every caller: moving the JSON parsing out of Somfy.cpp grew the image by
19 KB, past the 1.5 MB app partition. Treating every file as large keeps
inlining at the level the big files already had and frees 13 KB.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The CC1101 setup, receive decoding, radio task and frequency scan move out
of Somfy.cpp, and their types out of Somfy.h into Transceiver.h, which
Somfy.h includes. The code is unchanged apart from the bit length the
radio was set up for: bit_length becomes radioBitLength, shared with the
remotes that send without one.

The radio's fromJSON stays in Somfy.cpp for now so the JSON parsing can
move into one file: ArduinoJson costs flash in every file that uses it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The publish, unpublish and Home Assistant discovery code for rooms, shades
and groups moves out of Somfy.cpp unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The emitState and emitCommand events for rooms, shades, groups and the
controller move out of Somfy.cpp unchanged. Also drop the log tag from
SomfyMqtt.cpp, which never logs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
validateJSON and fromJSON for shades, rooms, groups and the radio move out
of Somfy.cpp unchanged. Keeping every ArduinoJson parser in one file
shares its inlined helpers, which saves 2.8 KB of flash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
setGPIOs, triggerGPIOs and usesPin, which drive the relay outputs of dry
contact and GPIO shades, move out of Somfy.cpp unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sending commands, repeats and sensor frames, rolling codes and the remote
flags move out of Somfy.cpp unchanged, with startsMovement(), which only
sendCommandAfter uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Linking shades, the group flags and direction, sending group commands and
saving move out of Somfy.cpp unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cjkas and others added 10 commits October 13, 2026 20:45
Loading and committing the shade list, adding, finding and deleting rooms,
shades and groups, repeaters, and handing frames and radio events to the
shades move out of Somfy.cpp unchanged, with SomfyRoom's clear and save.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only SomfyShade's behaviour is left: receiving frames, commands, moving to
targets and the favorite. Drop the includes and externs it no longer uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the Somfy.cpp split plan with phases 0 and 1 marked done, what the
split taught about flash size in the debug build, and the phase 2 work
still open. KNOWN_ISSUES.md now names the new files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop files the Arduino PlatformIO build never reads: the ESP32 SVD,
the ESP-IDF CMakeLists/idf_component.yml, the PlatformIO template
READMEs, and the already-ignored debug.cfg. Fix the esp32.vsd typo in
.gitignore so the SVD stays ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
group_types, shade_types and tilt_types move out of Somfy.h unchanged into
a header without the shade model, so the shade maths in ShadeMotion and
the sun and wind rules can use tilt_types in the native tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sun and wind timers were spread over processFrame, checkMovement and
processInternalCommand. ShadeSensors now holds them and decides when the
weather extends or retracts a shade, when turning the sun flag on moves
it, and when wind blocks a remote Down. Each caller still sets the targets
itself, so no shade moves differently yet.

The timers are now 32-bit like millis(). As 64-bit values holding a 32-bit
millis() they fired at once after the rollover every 49.7 days. The debug
logs of each timer go: they were compiled out at CORE_DEBUG_LEVEL 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The three places that act on the sun and wind each set different targets
for a tilt-only shade: the sun timer moved its lift, the no-sun and wind
timers moved tilt and lift, a group's sun flag moved its lift and only a
remote's sun flag moved its tilt. They now share sensorTargets(): a
tilt-only shade moves its tilt to the favorite tilt (or fully) and back to
0, and every other shade its lift, as Up and Down already do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A received step and a group step each worked out where a step takes the
shade, the group copy with its own inline maths. stepTargets() in
ShadeMotion now does it for both: a tilt-only shade steps its tilt, an
integrated tilt turns its slats to the end first and then steps the lift,
and every other shade steps its lift.

Only edge cases change. A group step on a tilt-only shade already at the
end now goes through like a remote step, and a step that can't be sized
(no travel time) leaves the targets alone in both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The decoding of the time between received edges into frame bits moves out
of Transceiver::handleReceive unchanged into decodePulse(), with the RTS
timings and the receive buffer type. It stays in IRAM on the ESP32 and is
now covered by native tests that feed it the edges sendFrame() produces
for 56 and 80-bit first frames and repeats, a glitch and a broken pulse.

handleReceive keeps the noise guard, the edge timing and the RX queue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GLOSSARY.md gains the shade terms the new modules use (favorite, step,
sensor frame, sun flag, extend and retract) and the pulse decoder. The
split plan marks phase 2 done, with what changed in behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cjkas cjkas changed the title Fix the shared NVS handle and five shade/MQTT bugs Fix NVS, radio and shade bugs; split Somfy.cpp into tested modules Oct 14, 2026
@cjkas
cjkas merged commit 377a121 into main Oct 14, 2026
5 checks passed
@cjkas
cjkas deleted the fix/live-bugs branch October 18, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant