Skip to content

Prevent emulator.wtf reruns from failing on expired artifacts - #7298

Open
loganrosen wants to merge 1 commit into
home-assistant:mainfrom
loganrosen:loganrosen-prevent-emulator-wtf-expired-artifacts
Open

Prevent emulator.wtf reruns from failing on expired artifacts#7298
loganrosen wants to merge 1 commit into
home-assistant:mainfrom
loganrosen:loganrosen-prevent-emulator-wtf-expired-artifacts

Conversation

@loganrosen

@loganrosen loganrosen commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Summary

PR Emulator.wtf runs on workflow_run and downloads the emulator-wtf-inputs artifact from the triggering Pull Request run. That artifact had retention-days: 1, but GitHub allows reruns for 30 days, and a partial rerun re-triggers the downstream workflow without rebuilding the APKs. Past the first day the download fails with Artifact has expired, and the PR gets a red "Instrumentation Test app" check that says nothing about the code under review. This happened on #7209: run 30712233663 attempt 2 reran one cancelled job a day later, and downstream run 30763712786 failed at the download.

Retention goes to 7 days, which covers the reruns people actually do. Past that, a new check_inputs job skips emulator_wtf instead of failing it, so no check appears on the PR at all and the workflow log explains that re-running all jobs of Pull Request rebuilds the inputs. That mirrors what #7093 settled on for the "triggering workflow didn't succeed" case. I stopped at 7 rather than 30 days because the artifact is ~100 MB and Pull Request runs ~19 times a day, so 30-day retention would leave ~45 GB standing permanently to serve a rare edge case.

Two smaller fixes fell out of the same failure:

  • emulator_wtf now downloads the inputs before creating the check run. The preflight can't close the window entirely, since the job then waits on the ui-test approval and the artifact can expire while that's pending. Downloading first means such a failure happens before any check run exists, so the PR still stays clean.
  • publish_test_results had the same bug through another door: its download step had no name and no pattern, so it pulled every artifact including the expired one. It failed on the run above too, so the unit and screenshot results never published either. It now excludes emulator-wtf-inputs by pattern, which is lossless (that artifact holds only APKs and devices.txt) and saves 100 MB per run.

Git history doesn't record why retention was 1 day originally. If that was deliberate for a reason not visible in the diff, let me know and I'll adjust.

Fixes #7297

Checklist

  • New or updated tests have been added to cover the changes following the testing guidelines.
  • The code follows the project's code style and best_practices.
  • The changes have been thoroughly tested, and edge cases have been considered.
  • Changes are backward compatible whenever feasible. Any breaking changes are documented in the changelog for users and/or in the code for developers depending on the relevance.
  • I have read the Open Home Foundation AI Policy.

Select exactly one option that describes AI usage in this contribution:

  • I have not used AI for this contribution.
  • AI assistance was used for this contribution.
  • AI fully generated the code for this contribution, but I've reviewed and understood it before submitting and will respond without AI during review.

Any other notes

No changelog entry (CI-only, not user-visible) and no tests (workflow YAML, no Kotlin surface), so the first box is unticked. Validated with yamllint --strict, the same check the yamllint job runs, plus checking the artifacts API query and download-artifact's pattern matching against real runs of both workflows.

The emulator-wtf-inputs artifact was retained for one day, but GitHub
allows reruns for 30 days and a partial rerun of Pull Request re-triggers
PR Emulator.wtf without rebuilding it. Past the first day the downstream
run failed at download with "Artifact has expired", turning the PR check
red before any test ran.

Raise retention to seven days to cover realistic rerun behaviour, and add
a check_inputs preflight that skips emulator_wtf when the artifact is
gone, so nothing red appears on the PR. Download the inputs before
creating the check run as well, since the artifact can still expire while
the job waits on the ui-test approval.

Also exclude the artifact from the bulk download in publish_test_results,
which pulled every artifact of the triggering run and so failed on the
same expiry, taking the unit and screenshot results down with it.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 2, 2026 21:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the CI wiring between the Pull Request workflow and the downstream PR Emulator.wtf workflow to avoid false failures when rerunning older workflow attempts whose input artifacts have expired.

Changes:

  • Increase emulator-wtf-inputs artifact retention from 1 to 7 days to better cover typical rerun windows.
  • Add a preflight check_inputs job in PR Emulator.wtf to detect expired/missing inputs and skip instrumentation instead of failing (avoiding a red PR check).
  • Ensure publish_test_results no longer attempts to download the emulator-wtf-inputs artifact when collecting artifacts from the triggering run.
Show a summary per file
File Description
.github/workflows/pr.yml Extends retention for the emulator-wtf-inputs artifact to reduce rerun failures due to expiry.
.github/workflows/pr-emulator-wtf.yml Adds an inputs availability preflight, downloads inputs before creating the PR check run, and excludes inputs from artifact collection to prevent expired-artifact failures.

Review details

  • Files reviewed: 2/2 changed files
  • Comments generated: 0
  • Review effort level: Lite

@jpelgrom

jpelgrom commented Aug 3, 2026

Copy link
Copy Markdown
Member

Git history doesn't record why retention was 1 day originally. If that was deliberate for a reason not visible in the diff, let me know and I'll adjust.

I think the idea was that it would always run immediately after a 'normal' pr workflow run (which it was originally a part of, but split to allow forks to also use it), so it didn't need long retention. However with some failures recently being due to the runner being out of RAM (perfect example: this PR's initial run 😅) not an actual build issue, we (maintainers) sometimes manually restart and that might not be within a day.

Changing the retention to be a bit longer is fine but it doesn't need to be very long. Maybe 3 days? What do you think @TimoPtr ?

@TimoPtr

TimoPtr commented Aug 4, 2026

Copy link
Copy Markdown
Member

Git history doesn't record why retention was 1 day originally. If that was deliberate for a reason not visible in the diff, let me know and I'll adjust.

I think the idea was that it would always run immediately after a 'normal' pr workflow run (which it was originally a part of, but split to allow forks to also use it), so it didn't need long retention. However with some failures recently being due to the runner being out of RAM (perfect example: this PR's initial run 😅) not an actual build issue, we (maintainers) sometimes manually restart and that might not be within a day.

Changing the retention to be a bit longer is fine but it doesn't need to be very long. Maybe 3 days? What do you think @TimoPtr ?

The retention of 3 days could make sense for the weekend but I don't think 7d make sense.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prevent emulator.wtf workflow reruns from using expired artifacts

4 participants