[saltas888]

Superseded by #11289. The six INFP-507 PRs are consolidated there into one commit targeting develop, with the complete design, code-flow diagrams and contract-based review guide. This PR is closed to keep one active review; its branch and previous discussions are preserved.

Stacked PR 1/6 for INFP-507 (task history and activity log retention), in GitHub stack #10873 on develop. Review and merge bottom-up; each PR targets the one below it.

1. This PR: retention settings applied to Prefect
2. feat(backend): clean up old task history in the task manager and during the upgrade [INFP-507] #10869: task history cleanup command, and the upgrade's task history hint
3. fix(backend): keep the Activities page fast with a year of activity log [INFP-507] #10876: Activities page queries
4. test(backend): guard the Prefect event-type list on the full stack [INFP-507] #10877: guard for Prefect's event types on the full stack
5. feat(backend): run the task manager background services with the retention settings [INFP-507] #10878: infrahub tasks background-services
6. docs: document task history and activity log retention [INFP-507] #10879: documentation

Ship with #10869 in the same release. On its own, this PR switches on Prefect's hourly deletion, which works through a large backlog slowly and gives no disk space back; #10869 adds the one-off cleanup command, its table rewrite, and the upgrade's hint that names that command.

Why

The task manager never deletes finished task runs, so task history grows until the disk fills, and the activity log is fixed at Prefect's 7 days with no Infrahub setting. Design doc: Task history and Activity log retention; spec set in dev/specs/infp-507-task-history-retention/.

Goal: one Infrahub setting per kind of record, translated into the Prefect settings that enforce it, so Prefect's built-in hourly cleanup bounds task history from this PR on.

Non-goals here: deleting the existing backlog and giving disk space back (PR 2), the Activities query changes (PR 3).

Closes part of INFP-507.

How retention works

Infrahub defines the retention policy; Prefect's background services enforce it. Task history and activity are kept independently, while Infrahub and Prefect events share the same storage.

flowchart TB
    CONFIG["Infrahub retention settings<br/>Task history · Activity log · Prefect events"]
    TYPES["Infrahub-maintained list<br/>of Prefect event types"]

    subgraph TM["Task manager"]
        APPLY["At startup<br/>Translate Infrahub settings into Prefect settings"]

        subgraph SERVICES["Prefect background services — every hour"]
            TASKS["Task-history cleanup<br/>Remove finished runs older than task_history<br/>together with their logs and artifacts"]
            EVENTS["Event cleanup<br/>Keep Infrahub events for activity_log<br/>Keep listed Prefect events for prefect_own_events"]
        end

        APPLY --> TASKS
        APPLY --> EVENTS
    end

    subgraph DB["Prefect database"]
        RUNS[("Task history<br/>Runs, states, logs and artifacts")]
        LOG[("Shared event storage<br/>Infrahub activity + Prefect internal events")]
    end

    CONFIG --> APPLY
    TYPES --> APPLY
    TASKS -->|"Delete expired task history"| RUNS
    EVENTS -->|"Delete expired events and related resources"| LOG

Loading

Increasing activity-log retention can preserve a year of Infrahub activity without also keeping a year of Prefect's internal events. When background services run in their own deployment, #10878 applies the same settings before starting them.

The operator cleanup in #10869 is a separate recovery step for existing instances: clear the old task-history backlog and recover disk space after an upgrade or retention reduction. The upgrade prints guidance; ongoing retention is handled automatically by the flow above.

What changed

Behavioural changes:

• New settings task_manager.retention.task_history (30 days), activity_log (7 days, unchanged from today) and prefect_own_events (7 days); env vars INFRAHUB_TASK_MANAGER_RETENTION_*; 30d or ISO 8601; 1 to 36,500 days, or the task manager refuses to start.
• Finished task runs older than 30 days are now deleted every hour with their logs and artifacts (Prefect's flow-run vacuum is switched on). Previously they were never deleted.
• Prefect's own events (103 event types from Prefect 3.8.6: run and task-run states, heartbeats, workers, work pools, automations...) follow prefect_own_events; Infrahub events follow activity_log. With the defaults both are 7 days, as today.
• A PREFECT_* variable already set (under any name Prefect accepts, including legacy aliases) takes precedence, with one warning per variable. The effective values are logged at startup.

Implementation notes:

• backend/infrahub/prefect_server/retention.py builds the four Prefect variables; create_infrahub_prefect now loads the Infrahub configuration in every mode (it only did in distributed mode), applies them, then refreshes Prefect's global settings context (Prefect reads its settings once at import).
• The integration-test Prefect container runs the stock Prefect image without the infrahub package, so it now uses create_prefect_app_with_infrahub_routes, which skips the configuration step.
• The Compose task-manager service does not merge the shared Infrahub anchor, so the three variables are declared on it explicitly (root docker-compose.yml and the dev deps files).
• tasks/docs.py: the configuration reference showed bare field names for a nested settings model with its own env prefix; it now shows the INFRAHUB_TASK_MANAGER_RETENTION_* names.

What stayed the same: no change to Prefect's database schema; Infrahub events are untouched; the component-test harness still turns the vacuum off (its preset variable wins by design).

How to review

1. backend/infrahub/config.py (settings and validation), then prefect_server/retention.py and prefect_server/app.py.
2. Tests: tests/unit/config/test_task_manager_retention_settings.py, tests/unit/prefect_server/ (one test feeds the derived env into Prefect's own settings model, so a format mistake fails CI), tests/component/task_manager/test_event_retention.py (runs Prefect's event vacuum with a 365-day activity log and 7-day own events).
3. dev/specs/infp-507-task-history-retention/ is the spec set for the whole stack.

Worth extra scrutiny: the event-type list. Overrides match event names exactly, and an unlisted Prefect type is kept for the activity log retention. A unit test covers every built-in state name; PR 4 adds a full-stack guard on a real workload.

How to test

uv run pytest backend/tests/unit/config/test_task_manager_retention_settings.py backend/tests/unit/prefect_server/ -n 0
uv run pytest backend/tests/component/task_manager/test_event_retention.py -n 0

Manual: start the task manager with INFRAHUB_TASK_MANAGER_RETENTION_TASK_HISTORY=1d and look for the Task manager retention: lines; with 0d it refuses to start.

Impact & rollout

• Backward compatibility: irreversible deletion of finished runs older than the retention starts within an hour of the upgrade. Operators who need older runs set a longer task_history first (changelog says so).
• Performance: the first hourly passes delete the backlog in Prefect's batches; PR 2 adds the faster one-off cleanup, which operators run once after the upgrade; the upgrade prints the command.
• Config/env changes: three new settings; PREFECT_* precedence documented.
• Deployment notes: the Helm chart needs the same three variables on the task-manager deployment (infrahub-helm PR follows the stack).

Validated locally: unit and component tests, ruff, mypy, docs.validate, validate-dockercomposeenv; reviewed with cubic (two passes, findings addressed). Release evidence on restored backups comes from opsmill/infrahub-private-tests, run separately.

Checklist

• Tests added/updated
• Changelog entry added
• External docs updated (PR 6 of this stack)
• Internal .md docs updated (PR 6 of this stack)
• I have reviewed AI generated content

🤖 Generated with Claude Code

[cubic-dev-ai]

2 issues found across 30 files

Confidence score: 2/5

• spec.md brings Activities query/display performance back into this feature, even though that work moved to INFP-727 and retention must wait for IFC-2933. Align the spec with those dependencies.
• plan.md allows any caller who can reach the task-manager API to trigger cleanup and request the optional rewrite without authentication. Add authentication or obtain an approved security exception.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="dev/specs/infp-507-task-history-retention/spec.md">

<violation number="1" location="dev/specs/infp-507-task-history-retention/spec.md:22">
P2: According to linked Jira issue INFP-507, Activities query/display performance moved to INFP-727, and retention must not ship ahead of IFC-2933. This sentence brings that work back into this feature; align the spec's scope and delivery order with the current decision.</violation>
</file>

<file name="dev/specs/infp-507-task-history-retention/plan.md">

<violation number="1" location="dev/specs/infp-507-task-history-retention/plan.md:49">
P2: Do not ship this unauthenticated cleanup mutation without an approved security exception: any caller that can reach the task-manager API can trigger cleanup and request the optional rewrite. Add authentication or obtain and record maintainer approval with an enforced network boundary.</violation>
</file>

<sub>Shadow auto-approve: would not auto-approve because issues were found.

Re-trigger cubic</sub>

[cubic-dev-ai]

P2: According to linked Jira issue INFP-507, Activities query/display performance moved to INFP-727, and retention must not ship ahead of IFC-2933. This sentence brings that work back into this feature; align the spec's scope and delivery order with the current decision.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At dev/specs/infp-507-task-history-retention/spec.md, line 22:

<comment>According to linked Jira issue INFP-507, Activities query/display performance moved to INFP-727, and retention must not ship ahead of IFC-2933. This sentence brings that work back into this feature; align the spec's scope and delivery order with the current decision.</comment>

<file context>
@@ -0,0 +1,242 @@
+
+Product decided that each instance keeps both kinds of records for a configured period, deletes older records automatically without affecting the instance, keeps the activity log for its own retention whatever the task manager keeps of its own records, and gets the disk space back. A year or more of activity log is supported in every edition, built so that long retention can become an Enterprise feature later.
+
+The Activities page gets slower as the activity log grows: with a year of activity log, one page takes minutes today. The Activities page work therefore ships with this feature, ahead of the activity log retention.
+
+## User Scenarios & Testing *(mandatory)*
</file context>

[codspeed]

Merging this PR will not alter performance

✅ 13 untouched benchmarks

---

_{Comparing task-history-retention-infp-507 (ff6a8ce) with develop (2ff7c1f)¹}

Footnotes

1. No successful run was found on develop (1920cd1) during the generation of this report, so 2ff7c1f was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩