Identify the joined row by primary key when detecting orphaned rows - #178764
Open
soldier2008 wants to merge 1 commit into
Open
Identify the joined row by primary key when detecting orphaned rows#178764soldier2008 wants to merge 1 commit into
soldier2008 wants to merge 1 commit into
Conversation
The correlated subquery that picks one row per data_id and attributes_id selected a timestamp column, and both are nullable. When the picked row had no timestamp the equality never held, the join matched nothing, and a row still referenced by events or states was reported as unused. Purge then tried to delete it, which a database enforcing the foreign key rejects. Selecting the primary key keeps the query shape and cannot be null.
home-assistant
Bot
added
bugfix
cla-signed
core
has-tests
integration: recorder
small-pr
Contributor
|
Hey there @home-assistant/core, mind taking a look at this pull request as it has been labeled with an integration ( Code owner commandsCode owners of
|
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes recorder purge queries so referenced shared data is not misidentified as orphaned when timestamps are null.
Changes:
- Uses non-null primary keys in correlated subqueries.
- Adds regression tests for nullable timestamps.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
homeassistant/components/recorder/queries.py |
Identifies joined rows by primary key. |
tests/components/recorder/test_queries.py |
Tests referenced IDs with null timestamps. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| @@ -0,0 +1,94 @@ | |||
| """Tests for the recorder queries.""" | |||
| data_id = data.data_id | ||
| for time_fired_ts in timestamps: | ||
| session.add( | ||
| Events(event_type_id=1, data_id=data_id, time_fired_ts=time_fired_ts) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed change
Before deleting from
event_dataandstate_attributes, purge subtracts the ids that are still referenced. Where a range scan is slow, currently PostgreSQL and older MariaDB, that question is answered by joining against one arbitrary row per id, picked by a correlated subquery onEvents.time_fired_tsandStates.last_updated_ts.Both columns are nullable. When the picked row has no timestamp the equality never holds, the join matches nothing, and an id that events or states still point at is reported as unused. Purge then deletes it and a database enforcing the foreign key rejects the statement. One such row is enough to hide the id even when other rows with a timestamp reference it, since nothing pins down which row the subquery picks.
Selecting the primary key instead keeps the query shape, the correlation and the indexes it uses, and cannot be null. Rows without a timestamp exist on databases predating the timestamp migration, which is what the
WHERE time_fired_ts is NULLbackfill inmigration.pyis for.Type of change
Additional information
Checklist
ruff format homeassistant tests)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest.requirements_all.txt.Updated by running
python3 -m script.gen_requirements_all.To help with the load of incoming pull requests: