Skip to content

Fix "Long running tasks" logging when dynamic_ui is disabled - #23604

Open
lowvoltage wants to merge 1 commit into
pantsbuild:mainfrom
lowvoltage:dk-fix-straggler-logging
Open

Fix "Long running tasks" logging when dynamic_ui is disabled#23604
lowvoltage wants to merge 1 commit into
pantsbuild:mainfrom
lowvoltage:dk-fix-straggler-logging

Conversation

@lowvoltage

Copy link
Copy Markdown
Contributor

This addresses #23603

Problem

In Pants v2.32, when dynamic_ui = false is configured (the default on CI where no TTY is present), Pants would periodically print Long running tasks: lines to the console. This output serves as a heartbeat that prevents CI systems from timing out long-running builds.

In v2.33, this console output no longer appears, causing CI timeout failures.

Root Cause

Commit 52d5fa0 (#23485, "Skip queueing workunit messages for consumers that never poll to avoid unbound memory growth") introduced a regression.

The WorkunitStore constructor gained a ui_consumers flag that controls whether messages are sent to the heavy_hitters_data channel:

// session.rs line 164
let workunit_store = WorkunitStore::new(!dynamic_ui, max_workunit_level, false, dynamic_ui);
//                                                                              ^^^^^^^^^^
//                                                                              ui_consumers

When dynamic_ui = false, ui_consumers = false, so self.ui_sender = None in the WorkunitStore. This means workunit StoreMsg messages (Started/Completed/Canceled) are never sent to the heavy_hitters_data receiver channel.

When the SessionDisplay::Logging variant calls straggling_workunits() to print "Long running tasks:", HeavyHittersData::refresh_store() finds nothing because no messages were ever delivered to its receiver — the sender was None.

The incorrect assumption

The comment in the original commit says: "the heavy hitters consumer only exists for the dynamic UI". This is incorrect:

  • SessionDisplay::ConsoleUI (dynamic UI on) → uses heavy_hitters() to render the dynamic task display
  • SessionDisplay::Logging (dynamic UI off) → uses straggling_workunits() to log long-running tasks

Both read from the same heavy_hitters_data channel. Both need messages delivered to function.

Fix

Always enable the heavy hitters channel (ui_consumers = true), since it is needed by both ConsoleUI (via heavy_hitters()) and Logging (via straggling_workunits()).

-let workunit_store = WorkunitStore::new(!dynamic_ui, max_workunit_level, false, dynamic_ui);
+let workunit_store = WorkunitStore::new(!dynamic_ui, max_workunit_level, false, true);

This is a one-line change in src/rust/engine/src/session.rs.

Impact on the original optimization

The original PR aimed to avoid unbounded memory growth from messages queued for consumers that never poll. This fix means the heavy hitters channel will always have messages sent to it, but this is safe because:

  • When dynamic_ui = true: ConsoleUI::render() polls heavy_hitters() regularly
  • When dynamic_ui = false: SessionDisplay::Logging polls straggling_workunits() regularly

In both cases, the channel is actively polled, so messages do not accumulate.

Commit 52d5fa0 (pantsbuild#23485) optimized WorkunitStore to skip queueing
messages for consumers that would never poll them. It set ui_consumers
(the heavy hitters channel) to match the dynamic_ui flag, assuming only
the dynamic UI ConsoleUI uses that data.

However, the Logging display variant (used when dynamic_ui = false) also
reads from the same heavy_hitters_data channel via straggling_workunits()
to print "Long running tasks:" lines. With ui_consumers = false, the
ui_sender was None and no workunit messages were delivered to that
channel, so straggling_workunits() always returned empty results.

Fix: always enable the heavy hitters channel (ui_consumers = true) since
it is needed by both ConsoleUI (heavy_hitters) and Logging
(straggling_workunits).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@lowvoltage
lowvoltage force-pushed the dk-fix-straggler-logging branch from cf2b107 to 81d5526 Compare August 5, 2026 11:52
@cburroughs
cburroughs requested a review from tobni August 5, 2026 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant