Skip to content

Trim shared frames from error cause backtraces - #1559

Merged
unflxw merged 2 commits into
wip-otel-reworkfrom
dedupe-error-cause-backtraces
Aug 7, 2026
Merged

Trim shared frames from error cause backtraces#1559
unflxw merged 2 commits into
wip-otel-reworkfrom
dedupe-error-cause-backtraces

Conversation

@unflxw

@unflxw unflxw commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

See also https://github.com/appsignal/appsignal-processor-rs/pull/2236 for the change to the processor that does something with the omitted lines attribute.

Trim shared frames from error cause backtraces

In collector mode, an error's causes are sent as one JSON span-event
attribute, appsignal.error_causes, and each cause carried its full
backtrace.

A cause is raised inside the frames that led to the reported error, so
its backtrace ends with the same lines as the reported error's own
backtrace. For a web request that shared tail is the whole framework
and web server stack, and it was repeated for every cause. In a
measured three-deep cause chain from a Rails controller action, 88 of
each cause's 92 lines were a verbatim repeat, and the attribute came
to 21,080 characters.

The AppSignal Collector caps a span-event attribute value at 20,000
characters. When it truncates, the JSON is no longer valid, so the
collector could not read the causes and dropped the whole chain.
Nothing was shown under the error's causes in the trace.

Drop from each cause's backtrace the trailing lines it shares with the
reported error's backtrace. Those lines are already sent once, in
exception.stacktrace. What is left is where the cause was raised,
which is the part worth showing. On the measured payload this brings
the attribute down to about 904 characters. If a cause shares every
line, its first line is kept, because a cause with no lines leaves
nothing to show.

Report how many cause lines were omitted

Each error cause in appsignal.error_causes no longer carries the
trailing backtrace lines it shares with the reported error's own
backtrace. A consumer of the attribute could not tell that lines were
left out, or how many of them there were.

Add a lines_omitted key to each cause, holding the number of trailing
lines that were dropped for that cause. The AppSignal Processor uses
the count to add a line reading "[88 repeated lines omitted]" to the
end of the cause's backtrace, so the trace shows how much is missing.

The count is the number of lines actually removed. When a cause shares
every line with the reported error, its first line is kept, so that
kept line does not count as removed. The key is left out entirely when
nothing was dropped, because a consumer reads a missing key as zero.

@backlog-helper

backlog-helper Bot commented Jul 28, 2026

Copy link
Copy Markdown

Hi @unflxw,

We've found some issues with your Pull Request.

  • This Pull Request does not include a changeset. Add a changeset if the change impacts users and should be included in the changelog upon release. Read more about changesets.
    Ignore this rule by adding [skip changeset] to your Pull Request body. - (More info)

New issue guide | Backlog management | Rules | Feedback

@unflxw
unflxw force-pushed the dedupe-error-cause-backtraces branch 2 times, most recently from d18f046 to 6019d2a Compare July 28, 2026 14:44
@unflxw
unflxw requested a review from Copilot July 28, 2026 15:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces the size of appsignal.error_causes payloads in collector mode by trimming each cause鈥檚 backtrace to remove the suffix shared with the reported error鈥檚 backtrace, preventing the collector鈥檚 20,000-character attribute limit from truncating the JSON and dropping the entire cause chain.

Changes:

  • Trim shared trailing frames from each cause backtrace before serializing appsignal.error_causes.
  • Add spec coverage for shared-tail trimming behavior, including edge cases (full overlap, no overlap, missing backtraces).
  • Add a patch changeset describing the restored visibility of error causes in collector mode.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
lib/appsignal/transaction/opentelemetry_backend.rb Introduces lines_without_shared_tail and applies it when serializing error causes to avoid duplicating shared backtrace tails.
spec/lib/appsignal/transaction/opentelemetry_backend_spec.rb Adds test cases validating the trimming logic across overlap and nil/empty backtrace scenarios.
.changesets/show-error-causes-in-collector-mode.md Documents the fix as a patch release item for collector-mode error causes display.

馃挕 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

@unflxw
unflxw force-pushed the wip-otel-rework branch from 935d9ba to 666148b Compare July 29, 2026 07:37
@unflxw
unflxw force-pushed the dedupe-error-cause-backtraces branch from 2fac3f6 to 7f89e5a Compare July 29, 2026 09:18
@unflxw
unflxw force-pushed the wip-otel-rework branch from 666148b to d2be844 Compare July 29, 2026 13:35
@unflxw
unflxw force-pushed the dedupe-error-cause-backtraces branch from 7f89e5a to fb1c1a0 Compare July 29, 2026 13:40
@unflxw

unflxw commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto wip-otel-rework, which was rewritten when #1550, #1551, #1552, #1553 and #1554 landed on it. Before the rebase this pull request compared against the wrong merge base and appeared to change 188 files.

The two commits applied without conflicts, and the full test suite passes on the new base.

I also dropped the changeset here. The collector mode work is staying under the single changeset on wip-otel-rework until the whole feature is done, and this is another part of that same unreleased work.

This still needs a human review, which is why it was not included in the batch that just landed.

@backlog-helper

Copy link
Copy Markdown

Hi @unflxw,

We've found new issues for this Pull Request. Please see the main comment on this issue for a list of all current warnings. This comment will not be updated to reflect resolved warnings.

  • This Pull Request does not include a changeset. Add a changeset if the change impacts users and should be included in the changelog upon release. Read more about changesets.
    Ignore this rule by adding [skip changeset] to your Pull Request body. - (More info)

New issue guide | Backlog management | Rules | Feedback

@unflxw
unflxw requested review from lipskis and tombruijn July 30, 2026 10:01
@backlog-helper

This comment has been minimized.

5 similar comments
@backlog-helper

This comment has been minimized.

@backlog-helper

This comment has been minimized.

@backlog-helper

This comment has been minimized.

@backlog-helper

This comment has been minimized.

@backlog-helper

backlog-helper Bot commented Aug 7, 2026

Copy link
Copy Markdown

This is a message from the daily scheduled checks.

New issue guide | Backlog management | Rules | Feedback

unflxw added 2 commits August 7, 2026 11:51
In collector mode, an error's causes are sent as one JSON span-event
attribute, `appsignal.error_causes`, and each cause carried its full
backtrace.

A cause is raised inside the frames that led to the reported error, so
its backtrace ends with the same lines as the reported error's own
backtrace. For a web request that shared tail is the whole framework
and web server stack, and it was repeated for every cause. In a
measured three-deep cause chain from a Rails controller action, 88 of
each cause's 92 lines were a verbatim repeat, and the attribute came
to 21,080 characters.

The AppSignal Collector caps a span-event attribute value at 20,000
characters. When it truncates, the JSON is no longer valid, so the
collector could not read the causes and dropped the whole chain.
Nothing was shown under the error's causes in the trace.

Drop from each cause's backtrace the trailing lines it shares with the
reported error's backtrace. Those lines are already sent once, in
`exception.stacktrace`. What is left is where the cause was raised,
which is the part worth showing. On the measured payload this brings
the attribute down to about 904 characters. If a cause shares every
line, its first line is kept, because a cause with no lines leaves
nothing to show.
Each error cause in `appsignal.error_causes` no longer carries the
trailing backtrace lines it shares with the reported error's own
backtrace. A consumer of the attribute could not tell that lines were
left out, or how many of them there were.

Add a `lines_omitted` key to each cause, holding the number of trailing
lines that were dropped for that cause. The AppSignal Processor uses
the count to add a line reading "[88 repeated lines omitted]" to the
end of the cause's backtrace, so the trace shows how much is missing.

The count is the number of lines actually removed. When a cause shares
every line with the reported error, its first line is kept, so that
kept line does not count as removed. The key is left out entirely when
nothing was dropped, because a consumer reads a missing key as zero.
@unflxw
unflxw force-pushed the dedupe-error-cause-backtraces branch from fb1c1a0 to e13e18c Compare August 7, 2026 09:53
@unflxw
unflxw merged commit e13e18c into wip-otel-rework Aug 7, 2026
432 checks passed
@tombruijn tombruijn added the chore A small task that takes a day or two at the most. label Aug 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore A small task that takes a day or two at the most.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants