Fix "Long running tasks" logging when dynamic_ui is disabled - #23604
Open
lowvoltage wants to merge 1 commit into
Open
Fix "Long running tasks" logging when dynamic_ui is disabled#23604lowvoltage wants to merge 1 commit into
lowvoltage wants to merge 1 commit into
Conversation
Commit 52d5fa0 (pantsbuild#23485) optimized WorkunitStore to skip queueing messages for consumers that would never poll them. It set ui_consumers (the heavy hitters channel) to match the dynamic_ui flag, assuming only the dynamic UI ConsoleUI uses that data. However, the Logging display variant (used when dynamic_ui = false) also reads from the same heavy_hitters_data channel via straggling_workunits() to print "Long running tasks:" lines. With ui_consumers = false, the ui_sender was None and no workunit messages were delivered to that channel, so straggling_workunits() always returned empty results. Fix: always enable the heavy hitters channel (ui_consumers = true) since it is needed by both ConsoleUI (heavy_hitters) and Logging (straggling_workunits). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
lowvoltage
force-pushed
the
dk-fix-straggler-logging
branch
from
August 5, 2026 11:52
cf2b107 to
81d5526
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This addresses #23603
Problem
In Pants v2.32, when
dynamic_ui = falseis configured (the default on CI where no TTY is present), Pants would periodically printLong running tasks:lines to the console. This output serves as a heartbeat that prevents CI systems from timing out long-running builds.In v2.33, this console output no longer appears, causing CI timeout failures.
Root Cause
Commit 52d5fa0 (#23485, "Skip queueing workunit messages for consumers that never poll to avoid unbound memory growth") introduced a regression.
The
WorkunitStoreconstructor gained aui_consumersflag that controls whether messages are sent to theheavy_hitters_datachannel:When
dynamic_ui = false,ui_consumers = false, soself.ui_sender = Nonein theWorkunitStore. This means workunitStoreMsgmessages (Started/Completed/Canceled) are never sent to theheavy_hitters_datareceiver channel.When the
SessionDisplay::Loggingvariant callsstraggling_workunits()to print "Long running tasks:",HeavyHittersData::refresh_store()finds nothing because no messages were ever delivered to its receiver — the sender wasNone.The incorrect assumption
The comment in the original commit says: "the heavy hitters consumer only exists for the dynamic UI". This is incorrect:
SessionDisplay::ConsoleUI(dynamic UI on) → usesheavy_hitters()to render the dynamic task displaySessionDisplay::Logging(dynamic UI off) → usesstraggling_workunits()to log long-running tasksBoth read from the same
heavy_hitters_datachannel. Both need messages delivered to function.Fix
Always enable the heavy hitters channel (
ui_consumers = true), since it is needed by bothConsoleUI(viaheavy_hitters()) andLogging(viastraggling_workunits()).This is a one-line change in
src/rust/engine/src/session.rs.Impact on the original optimization
The original PR aimed to avoid unbounded memory growth from messages queued for consumers that never poll. This fix means the heavy hitters channel will always have messages sent to it, but this is safe because:
dynamic_ui = true:ConsoleUI::render()pollsheavy_hitters()regularlydynamic_ui = false:SessionDisplay::Loggingpollsstraggling_workunits()regularlyIn both cases, the channel is actively polled, so messages do not accumulate.