[saltas888]

Superseded by #11289. The six INFP-507 PRs are consolidated there into one commit targeting develop, with the complete design, code-flow diagrams and contract-based review guide. This PR is closed to keep one active review; its branch and previous discussions are preserved.

Stacked PR 5/6 for INFP-507 (task history and activity log retention), in GitHub stack #10873 on develop. Review and merge bottom-up.

1. feat(backend): add task manager retention settings [INFP-507] #10862: retention settings applied to Prefect
2. feat(backend): clean up old task history in the task manager and during the upgrade [INFP-507] #10869: task history cleanup command, and the upgrade's task history hint
3. fix(backend): keep the Activities page fast with a year of activity log [INFP-507] #10876: Activities page queries
4. test(backend): guard the Prefect event-type list on the full stack [INFP-507] #10877: guard for Prefect's event types on the full stack
5. This PR: infrahub tasks background-services
6. docs: document task history and activity log retention [INFP-507] #10879: documentation

Why

Deployments that scale the task manager out run Prefect's background services (the cleanups, the event persister) in a separate container with prefect server services start. That command never loads Infrahub's configuration, so the task_manager.retention settings do not reach the services that enforce them: the flow-run cleanup stays off and events follow Prefect's defaults.

Closes part of INFP-507.

What changed

• New command infrahub tasks background-services [CONFIG_FILE]. It loads Infrahub's configuration, applies the same derived Prefect settings as the task manager (same code path: a pre-set PREFECT_* variable wins with a warning, an invalid retention stops it), refreshes Prefect's settings, logs the values in effect, then calls Prefect's own start_services() in the same process, as prefect server services start does.
• The test compose files (python_testcontainers/.../docker-compose.test.yml and docker-compose-cluster.test.yml) run the new command for task-manager-background-svc, and pass the three INFRAHUB_TASK_MANAGER_RETENTION_* variables through to that service and to task-manager.
• apply_infrahub_settings_to_prefect takes an optional configuration file, so the command and the task manager share it.

Behaviour to note: the background services now log through Infrahub's logging at INFRAHUB_LOG_LEVEL (INFO by default) instead of Prefect's WARNING, which adds about two scheduler lines every 5 seconds in that container.

The Helm chart change (the background-services deployment runs this command, retention values in values.yaml) is opsmill/infrahub-helm#100, a draft until the chart's images move to Infrahub 1.13; the chart's upgrade hook needs no extra option.

How to test

uv run pytest backend/tests/unit/cli/test_tasks_background_services.py backend/tests/unit/prefect_server/test_app_retention.py

Checked on a scale-out test stack (INFRAHUB_TESTING_TASKMGR_SCALEOUT=1, image built from this branch, INFRAHUB_TASK_MANAGER_RETENTION_TASK_HISTORY=12d): the task-manager-background-svc container ran infrahub tasks background-services, logged Task manager retention: PREFECT_SERVER_SERVICES_DB_VACUUM_RETENTION_PERIOD=P12D, and scheduled vacuum_old_flow_runs, which Prefect does not run by default; test_prefect_event_types.py passed on that stack. /pre-ci passed.

Impact & rollout

• Config/env changes: none new; the existing retention settings now reach separate background services.
• Deployment notes: deployments that run prefect server services start themselves should switch to infrahub tasks background-services (changelog).

Checklist

• Tests added/updated
• Changelog entry added
• External docs updated (PR 6 of this stack)
• Internal .md docs updated (PR 6 of this stack)
• I have reviewed AI generated content

🤖 Generated with Claude Code

[codspeed]

Merging this PR will not alter performance

✅ 13 untouched benchmarks

---

_{Comparing background-services-infp-507 (872ed77) with activity-log-retention-infp-507 (e14701a)}