Fix mobile_app local push failing permanently after a missed confirmation

[markfrancisonly]

Proposed change

Local push is what lets a notification reach a device without ever touching a cloud — this change makes that local-first path resilient, so devices stay on it instead of being silently forced onto cloud delivery, or losing notifications entirely, because of a single late acknowledgment.

What's broken

Local push notifications are expected to be acknowledged by the app within 10 seconds. When a phone confirms late — asleep, slow to wake, behind a buffering proxy — Home Assistant doesn't just give up on that one message. It silently kills the entire push channel. The app is never told: it keeps its socket open, believes it's still subscribed, and never re-subscribes.

For a device registered with local push only, the consequence is severe: no notifications at all until the app is force-restarted. Cloud-capable devices fare better but silently lose local delivery until an app restart. Both companion apps use confirmations, so both are exposed — iOS registrations virtually always include a cloud path (silent degradation), while Android builds without Google services are the typical local-push-only case (hard failure).

The constraint: iOS misses confirmations on purpose

While the app is suspended, iOS deliberately leaves Live Activity updates unconfirmed (home-assistant/iOS#4857) so the confirm timeout forwards them through APNs — the only path that can start a Live Activity in the background. Prompt APNs routing for unconfirmed Live Activity updates is therefore a hard requirement, not an accident of the old behavior.

The fix: a missed confirmation affects routing, never registration

Event	New behavior
1 missed confirm	The message was sent locally, but delivery is unknown — that is what the confirmation was for. The timeout triggers a safety copy via cloud for devices that have one. The channel stays registered
2 misses in a row	Channel degraded: cloud-capable devices route straight to cloud, no 10 s wait (keeps iOS Live Activities fast); local-push-only devices keep local delivery
While degraded: one probe per 5 min	One message is sent locally as a test (all others stay on cloud). Confirmed in time → the channel returns to normal local delivery. Not confirmed → it stays degraded and the next probe comes ~5 min later
Any timely confirm	Immediately restores normal local delivery and zeroes the miss counter — so two misses only degrade the channel if no message got confirmed between them

Everything else is untouched: when a device unsubscribes or disconnects, messages still awaiting confirmation are sent via cloud once, exactly as before; re-subscribing replaces the channel with a fresh one; and the confirmation messages the apps already send work unchanged. Genuinely dead sockets are still reaped by the server's 55 s websocket heartbeat, typically within a couple of minutes; a complementary client-side ping is proposed in home-assistant/android#7164. Degrade / probe / restore transitions are debug-logged.

What changes in practice

Scenario	Before	After
Local-push-only device, one late confirm	Notifications dead until force-restart	Cloud fallback cannot deliver (there is no cloud path), but the channel remains registered and later notifications continue trying local delivery
iOS Live Activity burst, app suspended	First update +10 s, rest instant via APNs	In-flight updates +10 s each, later ones instant via APNs; one 10 s probe per 5 min
Device delivers but confirms slowly (>10 s)	1 duplicate, then cloud-only	Sequential sends: up to 2 duplicates, then cloud-only until a probe confirms. Messages already in flight when the channel degrades each keep their own timer and can each duplicate

Inherited limitations (unchanged by this PR)

These compromises predate the change:

• Delivery is inferred, never reported. The server guesses delivery from a 10 s timer; a client can't say "got it, don't resend" or "can't present this, use cloud now". Every duplicate and every 10 s wait — before and after this PR — stems from that inference.
• Duplicates on a late confirm already existed (exactly one, after which the channel was removed and everything went cloud-only); this PR bounds them at two per degrade cycle instead. Client-side tag matching usually collapses a duplicate into a re-alert rather than a second entry.
• A local-push-only device with a silently dead socket loses the messages sent into the dead window — they have no second path, and the cloud fallback can only log. Previously that window was permanent (the channel was gone until an app restart); it's now bounded by the 55 s heartbeat reap and the client's reconnect.

Potential follow-up (not an alternative to this change): an explicit client acknowledgement protocol — "delivered, don't resend" / "can't present this, route via cloud" — would remove the inference for clients that speak it. It would layer on top of this PR rather than replace it: the server must keep timeout-based inference for older apps indefinitely, and this change is that layer. Cross-repo (core + both companions), so it belongs in an architecture discussion if there's interest.

Tests

New tests cover: a channel surviving a single missed confirmation · the full degrade → probe → recover cycle, for both ways Home Assistant sends notifications (the notify service and notify entities) · only one probe going local while the rest of a burst stays on cloud · a timely confirmation between two misses preventing degradation · local-push-only devices never being routed to cloud · unsubscribing still delivering unconfirmed messages via cloud once.

Type of change

• Dependency upgrade
• Bugfix (non-breaking change which fixes an issue)
• New integration (thank you!)
• New feature (which adds functionality to an existing integration)
• Deprecation (breaking change to happen in the future)
• Breaking change (fix/feature causing existing functionality to break)
• Code quality improvements to existing code or addition of tests

Additional information

• This PR fixes or closes issue:
  • fixes #176371
  • fixes #174164
• This PR is related to issue:
  • #172669
  • android#7164
  • iOS#4857
• Link to documentation pull request:
• Link to developer documentation pull request:
• Link to frontend pull request:

Checklist

• I understand the code I am submitting and can explain how it works.
• The code change is tested and works locally.
• Local tests pass. Your PR cannot be merged unless tests pass
• There is no commented out code in this PR.
• I have followed the development checklist
• I have followed the perfect PR recommendations
• The code has been formatted using Ruff (ruff format homeassistant tests)
• Tests have been added to verify that the new code works.
• Any generated code has been carefully reviewed for correctness and compliance with project standards.

If user exposed functionality or configuration variables are added/changed:

• Documentation added/updated for www.home-assistant.io

If the code communicates with devices, web services, or third-party tools:

• The manifest file has all fields filled out correctly.
  Updated and included derived files by running: python3 -m script.hassfest.
• New or updated dependencies have been added to requirements_all.txt.
  Updated by running python3 -m script.gen_requirements_all.
• For the updated dependencies a diff between library versions and ideally a link to the changelog/release notes is added to the PR description.

To help with the load of incoming pull requests:

• I have reviewed two other open pull requests in this repository.

[home-assistant]

Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration (mobile_app) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of mobile_app can trigger bot actions by commenting:

• @home-assistant close Closes the pull request.
• @home-assistant mark-draft Mark the pull request as draft.
• @home-assistant ready-for-review Remove the draft status from the pull request.
• @home-assistant rename Awesome new title Renames the pull request.
• @home-assistant reopen Reopen the pull request.
• @home-assistant unassign mobile_app Removes the current integration label and assignees on the pull request, add the integration domain after the command.
• @home-assistant update-branch Update the pull request branch with the base branch.
• @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component, problem in config, problem in device, feature-request) to the pull request.
• @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component, problem in config, problem in device, feature-request) on the pull request.

[Copilot]

Pull request overview

Keeps mobile app local-push channels registered when a notification confirmation times out.

Changes:

• Falls back only the unconfirmed notification to cloud delivery.
• Updates tests for channel retention, confirmation timers, and teardown flushing.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
homeassistant/components/mobile_app/push_notification.py	Removes channel teardown on confirmation timeout.
tests/components/mobile_app/test_notify.py	Verifies revised confirmation and fallback behavior.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

homeassistant/components/mobile_app/push_notification.py:20

• Restore the omitted Pull Request template items before approval. The description deletes the unchecked requirements/changelog dependency items and the final “reviewed two other open pull requests” item, although repository policy requires every template section and checkbox to remain present.

# How long a degraded channel routes via cloud before the next send probes
# local delivery again.
PUSH_DEGRADED_PROBE_INTERVAL = 300  # seconds

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (1)

homeassistant/components/mobile_app/push_notification.py:16

• Restore the omitted PR-template checklist entries before merging. The description is missing both dependency-related checklist items and the final “reviewed two other open pull requests” item; unchecked items must remain present even when they do not apply.

PUSH_DEGRADED_AFTER_TIMEOUTS = 2

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[markfrancisonly]

# PR #176833 use-case readout (Sol 5.6)

Use case	Old behavior (iOS)	Old behavior (Android)	New behavior	Assessment
Confirmed within 10 seconds	Local delivery continues.	Same.	Same.	Unchanged.
One late or missing confirmation	Message falls back to APNs after 10 seconds. Core silently removes the local channel, so later messages use APNs.	Same with FCM. Local-only builds lose their only route.	Only that message falls back. The channel stays registered and the next message tries local delivery.	Major improvement.
A timeout followed by a successful confirmation	The channel is already gone, so it cannot recover.	Same.	The success resets the failure count and preserves local delivery.	Improvement.
Repeated failures with cloud available	The first timeout switches the device to cloud until it subscribes again.	Same.	Two consecutive timeouts open the breaker. Later messages go directly to cloud; one local probe is allowed every five minutes.	Good failover with automatic recovery.
Android local-only registration	Not normally applicable.	One timeout removes the only route; later sends fail until the app subscribes again.	Core always keeps trying local delivery. A later confirmation restores healthy state.	Critical improvement.
Slow iOS attachment or notification presentation	One operation taking over 10 seconds permanently switches delivery to APNs.	Android confirms before rendering, so this is normally not an issue.	One slow message is tolerated. Two consecutive timeouts temporarily switch delivery to APNs.	Improvement.
iOS background Live Activity	The first update waits 10 seconds for APNs fallback; removing the channel sends later updates directly through APNs.	Not applicable.	The breaker opens after two timeouts, then later updates use APNs directly. One update may be used as a probe every five minutes.	Acceptable trade-off. One extra update and periodic probes may be delayed.
Several messages already in flight	The first timeout removes the channel and immediately flushes all pending messages to cloud.	Same.	Messages already sent locally retain their own 10-second timers. Only messages sent after the breaker opens bypass local delivery.	Small burst-latency regression.
Silently stale socket with cloud available	The first missed confirmation switches delivery to cloud, but the client may still think it is subscribed.	Same.	Two misses switch delivery to cloud without deleting the subscription. A later probe can restore local delivery.	Improvement. Avoids server/client split-brain.
Proper socket close, unsubscribe, or channel replacement	Core removes the channel and flushes pending messages.	Same.	Same; the probe timer is also cancelled.	Correctly unchanged.
Home Assistant Core restart	The client must reconnect and restore its subscription.	Released Android versions may fail to keep retrying until the app restarts.	Same. The circuit breaker cannot survive or repair a Core restart.	Out of scope. Android still needs its reconnect fix.
Cloud unavailable while the breaker is open	After one timeout, later cloud sends can fail indefinitely.	Same.	Cloud sends can still fail, but Core retries local delivery every five minutes.	Improvement, not a complete fix.
Client does not support confirmations	Local delivery has no confirmation timer.	Same.	Same; the breaker is never used.	Backward compatible.

Cloud means APNs on iOS and FCM on cloud-capable Android builds. Messages already in flight when the breaker opens are not limited to two; each keeps its own timeout.

Overall, the PR is a net improvement. It keeps channel lifetime tied to the WebSocket, falls back individual messages, and uses a reversible circuit breaker for repeated failures. A future explicit negative acknowledgement would handle iOS Live Activity fallback more precisely.

References: Core PR, live-socket issue, Core-restart issue, Android reconnect PR, iOS Live Activity PR.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

homeassistant/components/mobile_app/notify.py:115

• Avoid scheduling a cloud fallback for local-push-only registrations. When cloud_capable is false, this still retains _send_message and invokes it on every missed confirmation, but _send_message indexes the absent ATTR_PUSH_URL at notify.py:290, so each timeout raises an unhandled KeyError; the new test masks this by patching the helper. Pass an absent/no-op fallback for local-only channels and exercise the real timeout path.

            not cloud_capable or push_channel.async_should_send_local()
        ):
            push_channel.async_send_notification(
                data,
                partial(_send_message, self._session, self._config_entry),

homeassistant/components/mobile_app/notify.py:234

• Do not attach a remote fallback for local-push-only targets. This branch deliberately sends locally when cloud_capable is false, yet a missed confirmation invokes _async_send_remote_message_target, whose _send_message call indexes the absent ATTR_PUSH_URL at notify.py:290; because only HomeAssistantError is caught, every timeout produces an unhandled KeyError. Pass an absent/no-op fallback and cover the unmocked timeout path.

                not cloud_capable or push_channel.async_should_send_local()
            ):
                push_channel.async_send_notification(
                    data,
                    partial(self._async_send_remote_message_target, entry),

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

[bgoncal]

Was this tested on real devices or just on core side?

[TimoPtr]

Client-side tag matching usually collapses a duplicate into a re-alert rather than a second entry.

This should be address, the client should be able to determine if it's a retry notification and check if it was already displayed or not. Otherwise if I understand well after this PR users will start to see duplicate notifications (even if it improve the reliability).