Add generic per-device check-in action mechanism#1850
Draft
TheJulianJES wants to merge 7 commits into
Draft
Conversation
When a Zigbee device boots after an OTA, it sends a `QueryNextImageCommand` with its running `current_file_version`. If that version differs from the previously cached value for this cluster, schedule `device.reinterview()` to rebuild endpoints, clusters, and quirks against the new firmware. This makes post-OTA recovery stateless: it works for slow-rebooting devices that miss the post-flash polling window in `update_firmware()`, devices flashed outside HA, and HA restarts during the reboot window.
Replace the fixed post-OTA read_attributes / image_notify / reinterview sequence with a confirmation step that resolves on whichever signal arrives first: - A successful active read of `current_file_version` (kept from the old flow as a probe for quietly-rebooting mains devices, but now best-effort: exhausted retries no longer abort the post-OTA steps). - A new `QueryNextImageCommand` whose `current_file_version` differs from the value the OTA cluster cached pre-flash. The OTA cluster's listener also schedules a re-interview when this happens. - A `Device_annce` from this device after the post-flash reboot. If none arrive within `POST_OTA_CONFIRMATION_TIMEOUT` (5 min), update_firmware() still returns SUCCESS — recovery is deferred to the version-change listener if the device wakes up later. After confirmation, a best-effort `image_notify` is sent so the device refreshes its OTA query cache, and a re-interview is driven directly unless the version-change listener already replaced this device with a re-interviewed one. This fixes the slow-reboot race where the read_attributes retry budget was exhausted before the device finished rebooting (e.g. Hue LLC020 ~102s), causing update_firmware() to throw and skip image_notify and reinterview. Claude-Session: https://claude.ai/code/session_01VnDASUDhbLfANUM9UbnAeo
Switch the post-OTA wait's announce signal from a `device.zdo` listener for `device_announce` to an `application.add_listener` for `device_joined`. `device_joined` is the existing application-level event for new joins / NWK changes (including the Device_annce path), so we don't add a new ZDO-level listener. The version-change signal still covers same-NWK rejoins. Also adds a test verifying that a `device_joined` event for a different device does NOT resolve this device's wait.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #1850 +/- ##
==========================================
- Coverage 99.47% 99.47% -0.01%
==========================================
Files 57 57
Lines 12026 12121 +95
==========================================
+ Hits 11963 12057 +94
- Misses 63 64 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds infrastructure to defer device work until a sleepy end device is likely awake (“check-in actions”), and wires additional wake / OTA-related triggers so queued work (and post-OTA flows) can run more reliably in zigpy’s device lifecycle.
Changes:
- Introduces a per-device check-in action registry with cooldown + no-double-start semantics, triggered on inbound traffic and on stack-reported joins/announces.
- Updates Poll Control check-in responses to request fast polling while check-in actions are pending.
- Adds OTA-related logic to detect firmware version changes via QueryNextImage and updates the post-OTA flow to wait for confirmation signals before proceeding.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| zigpy/zcl/clusters/general.py | Schedules a re-interview when QueryNextImage reports a firmware version change. |
| zigpy/device.py | Implements check-in action registry/runner; adjusts Poll Control behavior; updates post-OTA confirmation + re-interview behavior. |
| zigpy/application.py | Triggers check-in actions on stack-reported joins/announces. |
| tests/test_zcl_clusters.py | Adds coverage for re-interview scheduling on QueryNextImage version changes. |
| tests/test_device.py | Adds coverage for check-in actions, Poll Control behavior, and the updated post-OTA confirmation flow. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
176
to
178
| self._on_remove_callbacks.clear() | ||
| self._checkin_actions.clear() | ||
|
|
Comment on lines
+483
to
+485
| def remove_checkin_action(self, name: str) -> None: | ||
| """Remove a previously registered check-in action, if present.""" | ||
| self._checkin_actions.pop(name, None) |
Comment on lines
+527
to
+528
| if done and self._checkin_actions.get(action.name) is action: | ||
| del self._checkin_actions[action.name] |
|
|
||
| confirmed.set() | ||
|
|
||
| probe_task = asyncio.ensure_future(probe_version()) |
Comment on lines
+1149
to
+1157
| # Wait for the device to confirm it booted into the new firmware, | ||
| # whichever signal arrives first: | ||
| # - a successful active read of `current_file_version` (a quietly | ||
| # rebooted mains device that sends no traffic on its own), | ||
| # - QueryNextImage with a `current_file_version` that differs from | ||
| # the value we cached pre-flash (the OTA cluster's listener will | ||
| # also schedule a re-interview when this happens), or | ||
| # - Device_annce after the post-flash reboot. | ||
| try: |
- Rename `POST_OTA_PROBE_ATTEMPTS` to `POST_OTA_PROBE_RETRIES`: `read_attributes(retries=N)` performs N+1 total attempts, so the old name misrepresented the behavior (which matches dev's existing `retries=10`). - Document that the QueryNextImage confirmation deliberately accepts any query when no pre-flash baseline was cached: the wait only establishes that the device is alive after the flash. - Use `asyncio.create_task()` instead of `asyncio.ensure_future()` for the probe task. Claude-Session: https://claude.ai/code/session_01VnDASUDhbLfANUM9UbnAeo
Sleepy end devices only receive requests for a short window after they send something themselves. Add a small named-action registry on `Device`: `register_checkin_action()` queues a coroutine factory that is run whenever the device shows signs of being awake (any inbound packet, or a stack-reported join/announce). The coroutine returns `True` to unregister the action or `False`/raises to retry on a later check-in, gated by a per-action cooldown (default 5 minutes) so a frequently reporting device does not hammer a failing action. Actions run as application tasks, not device tasks: device-owned tasks are cancelled when a re-interview swaps the device object, which would cancel an action that is itself driving the re-interview. Poll Control check-ins now also request fast polling while check-in actions are pending, keeping the device awake for the queued work. Claude-Session: https://claude.ai/code/session_01VnDASUDhbLfANUM9UbnAeo
- Re-registering an existing action now mutates the stored `_CheckinAction` in place instead of replacing it: a running attempt unregisters itself by object identity on success, so a replacement object would have left completed actions permanently registered. - Document why neither `remove_checkin_action()` nor `on_remove()` cancels a running attempt: `_device_reinterviewed()` calls the old device's `on_remove()` mid-swap, and an action driving that very re-interview must not cancel its own call stack. Claude-Session: https://claude.ai/code/session_01VnDASUDhbLfANUM9UbnAeo
TheJulianJES
force-pushed
the
tjj/checkin-actions
branch
from
6a2237f to
b986d60
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DRAFT / EXPERIMENTAL.
Note
Part of a four-PR series making post-OTA re-interviews reliable for sleepy end devices. Merge order (first to last):
Infrastructure for queueing work until a sleepy device is awake. Branch:
tjj/checkin-actions, stacked ontjj/reinterview_ota_announcement. Pure mechanism — no behavior change until something registers an action (the re-interview consumer lands intjj/reinterview-on-checkin).Problem
Sleepy end devices only receive requests for a second or two after they send something themselves. Anything zigpy wants from such a device (a re-interview after OTA, an update start, …) fails if sent while the device sleeps — the reliable moment to act is right after any inbound traffic from it.
Changes
Device.register_checkin_action(name, coro_factory, *, cooldown)queues a coroutine factory that runs whenever the device shows signs of being awake. The coroutine returnsTrueto unregister the action,Falseto retry on a later check-in; raising is logged and retries likeFalse. Re-registering an existing name swaps the factory but keeps the running-attempt and cooldown state. Plusremove_checkin_action(name)andhas_pending_checkin_actions.Device.packet_received, before duplicate filtering — a duplicate is still a wake signal) and stack-reported joins/announces (handle_join, covering radios that don't deliver the ZDO frame).CHECKIN_ACTION_RETRY_COOLDOWN= 5 min default) is measured from attempt start, so a failing action isn't hammered by a device that reports every 30 s, and the trailing packets of an attempt can't retrigger it. An action whose task is still running is never started twice — this also makes actions immune to their own traffic._device_reinterviewed()cancels the old device's tasks during the swap, which would cancel an action that is itself driving the re-interview.on_remove()clears the registry.Testing
New tests: run-and-unregister, cooldown gating, exception retention + retry, no double-start while running, duplicate-packet wake,
handle_jointrigger,on_removecleanup, idempotent removal, and the Poll Control fast-poll response with pending actions. Full suite passes.