Skip to content

Send each fwTPM mssim reply as one segment and disable Nagle - #664

Open
dgarske wants to merge 1 commit into
wolfSSL:masterfrom
dgarske:fwtpm_nodelay
Open

dgarske wants to merge 1 commit into
wolfSSL:masterfrom
dgarske:fwtpm_nodelay

Conversation

@dgarske

@dgarske dgarske commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

Description

Every fwTPM command over the mssim socket transport paid a fixed delay. DispatchAndRespond() sent a reply as three writes - size, response, acknowledgement - and that last four-byte write goes out while the response is still unacknowledged, so Nagle holds it until the peer's delayed-ACK timer expires. The time is spent waiting on a kernel timer, which is why a measurement taken around FWTPM_ProcessCommand() looked normal while the client round trip did not.

The SWTPM client sets TCP_NODELAY on the socket it connects, but that governs the client's own sends. The server never set it on the sockets it accepts.

  • SendMssimReply() sends size, response and acknowledgement as one segment with writev(). A custom I/O HAL has no vectored form and keeps the three sequential sends; FWTPM_NO_WRITEV selects that path, as do Windows and Zephyr.
  • SetSocketNoDelay() disables Nagle on both accepted sockets, command and platform. The platform port frames its replies the same way.

Either change removes the delay on its own. Both are kept because the fallback path still writes three times, and both are
advisory in the sense SO_REUSEADDR here already is: a failure costs latency, not correctness.

Fixes #663. Thank you @jerohavaz

Testing

test_fwtpm_mssim_reply_framing drives SendMssimReply() over a socketpair and asserts the wire stream is exactly size, response, zero ack; a FWTPM_NO_WRITEV build covers the sequential path through the same test. test_fwtpm_mssim_reply_partial_writes caps writev() at every length so a short write lands in each of the three buffers, and test_fwtpm_mssim_accept_nodelay runs the server loop and checks TCP_NODELAY on the accepted command and platform sockets. make check green against the fwTPM simulator, unit.test and run_examples.sh green against ibmswtpm2, and built over eleven configurations.

@dgarske dgarske self-assigned this Nov 2, 2026
Copilot AI balanced review requested due to automatic review settings November 2, 2026 23:44

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the changes to fwTPM MSSIM reply handling.

No open issues 🎉

What changed in this PR

This PR addresses #663 by removing the server-side socket behavior responsible for delayed fwTPM MSSIM replies, without changing the wire format.

Changes:

  • Uses writev() to send reply framing and payload together, retaining sequential sends for custom HALs and unsupported or disabled configurations.
  • Enables TCP_NODELAY on accepted command and platform sockets.
  • Adds a reply-framing test shared by vectored and sequential builds.
File Description
tests/​fwtpm_unit_tests.c Adds assertions for reply length, payload, and zero acknowledgement.
tests/​fwtpm_io_test_adapter.c Exposes the reply helper to unit tests.
src/​fwtpm/​fwtpm_io.c Adds vectored replies with partial-write handling and disables Nagle on accepted sockets.

Review effort: Balanced

@dgarske dgarske assigned wolfSSL-Bot and embhorn and unassigned dgarske Nov 3, 2026
@dgarske
dgarske requested a review from embhorn November 3, 2026 00:34
embhorn
embhorn previously approved these changes Nov 3, 2026
@dgarske
dgarske force-pushed the fwtpm_nodelay branch 2 times, most recently from e0bda45 to c2553a2 Compare November 4, 2026 00:38
@dgarske dgarske changed the title fwTPM: remove the per-command delay in the mssim reply path Send each fwTPM mssim reply as one segment and disable Nagle Nov 4, 2026

@embhorn embhorn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skoll Code Review

Scan type: review

Overall recommendation: COMMENT
Findings: 2 total — 2 posted, 0 skipped
2 finding(s) posted as inline comments (see file-level comments below)

Posted findings

  • [Medium] Test partial vectored writes — tests/fwtpm_unit_tests.c:20424-20425
  • [Medium] Test TCP_NODELAY on accepted sockets — tests/fwtpm_unit_tests.c:20423

Review generated by Skoll

Comment thread tests/fwtpm_unit_tests.c
for (i = 0; i < (word32)sizeof(rsp); i++) {
rsp[i] = (byte)(i & 0xFF);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test partial vectored writes

The new framing test does not force a partial writev() result. A mistake in the PR's iovec advancement logic could therefore drop or duplicate reply bytes while this test passes.

Suggested fix: Add a deterministic test that makes writev() stop within the size, response, and acknowledgement buffers, then checks the complete byte stream.
Basis: The Linux writev(2) manual, RETURN VALUE, states that a successful call may transfer fewer bytes than requested.

Comment thread tests/fwtpm_unit_tests.c
XMEMSET(wire, 0, sizeof(wire));
for (i = 0; i < (word32)sizeof(rsp); i++) {
rsp[i] = (byte)(i & 0xFF);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test TCP_NODELAY on accepted sockets

The new test uses an AF_UNIX socketpair and calls the reply helper directly. It never exercises either new SetSocketNoDelay() call in the TCP accept paths, so a regression in the PR's latency fix would pass.

Suggested fix: Add a TCP server test that checks TCP_NODELAY on accepted command and platform sockets.
Basis: The POSIX getsockopt(3p) manual, DESCRIPTION, specifies that getsockopt() retrieves a socket option's value.

@dgarske dgarske assigned dgarske and unassigned embhorn Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fwTPM MSSIM: ~46 ms command latency resolved by server-side TCP_NODELAY

4 participants