Skip to content

feat: trace postprocessors - #14352

Merged
datokrat merged 14 commits into
leanprover:masterfrom
datokrat:trace_view
Jul 14, 2026
Merged

datokrat merged 14 commits into
leanprover:masterfrom
datokrat:trace_view

Conversation

@datokrat

@datokrat datokrat commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

This PR provides the experimental postprocess_traces tracePostprocessor in cmd command, which is useful for working with large trees of trace nodes. It runs the command cmd and then transforms the traces using a function tracePostprocessor. The transformation can affect which nodes are expanded or collaped by default, it can change messages of the trace nodes, and it can add or delete nodes.
Example:

module
meta import Lean.PostprocessTraces
-- Expand all ancestors of `synthInstance` trace nodes
-- for better discoverability in large trace trees
postprocess_traces exposeSubtrees (ofClass `Meta.synthInstance) in
set_option trace.Meta.isDefEq true in
set_option trace.Meta.synthInstance true in
def x ...

@github-actions github-actions Bot added toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN mathlib4-nightly-available A branch for this PR exists at leanprover-community/mathlib4-nightly-testing:lean-pr-testing-NNNN labels Jul 10, 2026
@leanprover-bot leanprover-bot added the builds-manual CI has verified that the Lean Language Reference builds against this PR label Jul 10, 2026
@leanprover-bot

leanprover-bot commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Reference manual CI status:

@mathlib-lean-pr-testing mathlib-lean-pr-testing Bot added the builds-mathlib CI has verified that Mathlib builds against this PR label Jul 10, 2026
@mathlib-lean-pr-testing

mathlib-lean-pr-testing Bot commented Jul 10, 2026

Copy link
Copy Markdown

Mathlib CI status (docs):

mathlib-nightly-testing Bot pushed a commit to leanprover-community/batteries that referenced this pull request Jul 10, 2026
mathlib-nightly-testing Bot pushed a commit to leanprover-community/mathlib4-nightly-testing that referenced this pull request Jul 10, 2026
leanprover-bot added a commit to leanprover/reference-manual that referenced this pull request Jul 10, 2026
@datokrat datokrat changed the title feat: trace preprocessors feat: trace postprocessors Jul 14, 2026
@datokrat

Copy link
Copy Markdown
Contributor Author

!radar

@leanprover-radar

leanprover-radar commented Jul 14, 2026

Copy link
Copy Markdown

Benchmark results for d2a15cd against da19ea0 are in. There are significant results. @datokrat

Warning

These warnings may indicate that the benchmark results are not directly comparable, for example due to changes in the runner configuration or hardware.

  • Bench repo commit hashes for run build differ between commits.
  • Bench repo commit hashes for run other differ between commits.
  • 🟥 build//instructions: +4.2G (+0.04%)

Large changes (3✅)

  • compiled/incr_header_load//instructions: -444.9M (-38.18%)
  • compiled/incr_header_load//maxrss: -62MiB (-9.85%)
  • compiled/incr_header_load//wall-clock: -76ms (-51.41%)

Small changes (1✅)

  • compiled/incr_header_load//task-clock: -74ms (-39.99%)

@datokrat datokrat added the changelog-language Language features and metaprograms label Jul 14, 2026
datokrat and others added 12 commits July 14, 2026 11:03
This PR adds trace postprocessors: functions that transform the trace
messages of a command before they are reported, e.g. by filtering out
irrelevant subtrees, focusing on a single trace class, or pre-expanding
the paths to failures. The new `trace_view post in cmd` command applies
a postprocessor to the traces of `cmd`, and `store_trace_as t in cmd`
stores the traces of a slow command so they can be inspected with
`#trace_roots t` and re-rendered with `#trace_view t post` without
re-running `cmd`.

Postprocessors have type `Lean.TraceView.TracePostprocessor`
(`Array TraceTree → CoreM (Array TraceTree)`, applied to the trace
roots of each trace message); they can be composed with `>=>` and
defined by users as ordinary functions. The `Lean.TraceView` namespace
provides basic combinators: `focusOn`, `hideSucceeded`, `maxDepth`,
`minTimeMs`, `grep`, `expandAll`, `collapseAll`, `expandFailures`,
`onRoots`, `onClass`, and `onRootIdx`. `TraceTree` is a structured view
of trace `MessageData` that handles the context wrappers around trace
nodes. Stored traces live in a non-persistent environment extension and
are not exported to `.olean` files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ew` at its own range

This PR makes `store_trace_as t in cmd` add an actual declaration
`t : CoreM Lean.TraceView.StoredTrace` to the environment, so the
stored trace can be inspected by arbitrary metaprograms, e.g.
`#eval do return (← t).roots.size` or
`(← t).postprocess (focusOn `cls)`. Stored trace names now live in the
current namespace and resolve like any other constant. The output of
`#trace_view t post` is now anchored at the `#trace_view` command
itself rather than at the source range of the traced command; the
original positions remain inspectable via `#trace_roots t`.

The declaration body merely references the trace data via
`Lean.TraceView.findStoredTrace`, which reads it from the in-memory
environment extension at evaluation time. Declaring a stored trace is
therefore O(1) regardless of trace size: no serialization, no
embedding of the trace in the declaration value, and no kernel
checking of large terms. The data itself consists of the already
materialized in-memory trace messages, shared rather than copied.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This PR adds the `expandMatches pat` trace postprocessor: it expands all transitive parents of the trace nodes whose trace class or head message contains `pat` as a substring, so that the trace opens in the editor already showing all matches. Unlike `grep`, no nodes are removed, and all other nodes — including the matches themselves — keep their expansion state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@datokrat

Copy link
Copy Markdown
Contributor Author

!radar

@leanprover-radar

leanprover-radar commented Jul 14, 2026

Copy link
Copy Markdown

Benchmark results for 434109f against 12c859a are in. No significant results found. @datokrat

  • 🟥 build//instructions: +4.6G (+0.04%)

Medium changes (1🟥)

  • 🟥 build/profile/interpretation//wall-clock: +732ms (+0.69%)

Small changes (2✅)

  • lake/inundation/config tree//maxrss: -3MiB (-0.36%)
  • lake/inundation/startup//maxrss: -5MiB (-4.06%)

mathlib-nightly-testing Bot pushed a commit to leanprover-community/batteries that referenced this pull request Jul 14, 2026
mathlib-nightly-testing Bot pushed a commit to leanprover-community/mathlib4-nightly-testing that referenced this pull request Jul 14, 2026
leanprover-bot added a commit to leanprover/reference-manual that referenced this pull request Jul 14, 2026
@mathlib-lean-pr-testing mathlib-lean-pr-testing Bot added breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan and removed builds-mathlib CI has verified that Mathlib builds against this PR labels Jul 14, 2026
@datokrat

Copy link
Copy Markdown
Contributor Author

!radar

@leanprover-radar

leanprover-radar commented Jul 14, 2026

Copy link
Copy Markdown

Benchmark results for 5b7031b against 12c859a are in. There are significant results. @datokrat

  • 🟥 build//instructions: +6.1G (+0.05%)

Large changes (1🟥)

  • 🟥 build/profile/interpretation//wall-clock: +1s (+1.85%)

Small changes (1✅, 1🟥)

  • 🟥 elab/big_beq//maxrss: +13MiB (+0.72%)
  • lake/inundation/startup//maxrss: -5MiB (-3.91%)

mathlib-nightly-testing Bot pushed a commit to leanprover-community/batteries that referenced this pull request Jul 14, 2026
mathlib-nightly-testing Bot pushed a commit to leanprover-community/mathlib4-nightly-testing that referenced this pull request Jul 14, 2026
leanprover-bot added a commit to leanprover/reference-manual that referenced this pull request Jul 14, 2026
@datokrat
datokrat marked this pull request as ready for review July 14, 2026 12:11
@datokrat
datokrat added this pull request to the merge queue Jul 14, 2026
Merged via the queue into leanprover:master with commit 3c080a4 Jul 14, 2026
22 checks passed
pull Bot pushed a commit to DaviRain-Su/lean4 that referenced this pull request Jul 15, 2026
leanprover#14386)

This PR is a follow-up to leanprover#14352 (introducing `postprocess_traces`). It
provides a new command `store_traces_as myTraces in cmd` that runs the
command `cmd` and stores its traces in-memory under the name `name`. The
stored traces can be transformed and viewed using `#postprocess_traces
tracePostprocessor myTraces`.
robsimmons pushed a commit that referenced this pull request Jul 29, 2026
#14386)

This PR is a follow-up to #14352 (introducing `postprocess_traces`). It
provides a new command `store_traces_as myTraces in cmd` that runs the
command `cmd` and stores its traces in-memory under the name `name`. The
stored traces can be transformed and viewed using `#postprocess_traces
tracePostprocessor myTraces`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaks-mathlib This is not necessarily a blocker for merging: but there needs to be a plan builds-manual CI has verified that the Lean Language Reference builds against this PR changelog-language Language features and metaprograms mathlib4-nightly-available A branch for this PR exists at leanprover-community/mathlib4-nightly-testing:lean-pr-testing-NNNN toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants