Skip to content

AP_HAL_SITL: do not truncate the stack dump on a short write - #34077

Open
peterbarker wants to merge 1 commit into
ArduPilot:masterfrom
peterbarker:pr-claude2/pr/dumpstack-short-write
Open

AP_HAL_SITL: do not truncate the stack dump on a short write#34077
peterbarker wants to merge 1 commit into
ArduPilot:masterfrom
peterbarker:pr-claude2/pr/dumpstack-short-write

Conversation

@peterbarker

Copy link
Copy Markdown
Contributor

Summary

Overlook short writes when taking stack traces (system is perfectly entitled to do that)

Classification & Testing (check all that apply and add your own)

  • Checked by a human programmer
  • Non-functional change
  • No-binary change
  • Infrastructure change (e.g. unit tests, helper scripts)
  • Automated test(s) verify changes (e.g. unit test, autotest)
  • Tested manually, description below (e.g. SITL)
  • Tested on hardware
  • Logs attached
  • Logs available on request

Description

dump_stack_trace() and dump_core_file() both run a script on our own pid and copy its output to stderr a block at a time. The copy ended on any write() which did not place the whole block:

if (write(2, buf, ret) != ret) {
    // *sigh*
    break;
}

write() is entitled to do that. stderr is a pipe when SITL runs under autotest, so a full pipe shortens a write, and a timer signal can cut one short with EINTR. Either way the copy stopped silently, part-way through, with no "end dumpstack.sh output" line to show that anything was missing.

Seen in CI: a backtrace ended at frame #8, in the middle of the frame which would have named the caller - the one thing the dump exists to provide. A truncated core dump goes the same way and is even easier to miss.

Keep writing until the block is out, retry EINTR on both the read and the write, and say so if a write really does fail.

dump_stack_trace() and dump_core_file() both run a script on our own pid
and copy its output to stderr a block at a time.  The copy ended on any
write() which did not place the whole block:

    if (write(2, buf, ret) != ret) {
        // *sigh*
        break;
    }

write() is entitled to do that.  stderr is a pipe when SITL runs under
autotest, so a full pipe shortens a write, and a timer signal can cut one
short with EINTR.  Either way the copy stopped silently, part-way
through, with no "end dumpstack.sh output" line to show that anything was
missing.

Seen in CI: a backtrace ended at frame #8, in the middle of the frame
which would have named the caller - the one thing the dump exists to
provide.  A truncated core dump goes the same way and is even easier to
miss.

Keep writing until the block is out, retry EINTR on both the read and the
write, and say so if a write really does fail.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant