Skip to content

bench: measure the segmented encode on compressed streams - #284

Draft
iainmcgin wants to merge 1 commit into
mainfrom
fix/277
Draft

bench: measure the segmented encode on compressed streams#284
iainmcgin wants to merge 1 commit into
mainfrom
fix/277

Conversation

@iainmcgin

Copy link
Copy Markdown
Collaborator

#271 encodes every streaming view item through Encodable::encode_segments before the framing layer knows whether the item will be compressed, and EnvelopeEncoder::encode_chained flattens a segmented body back to one buffer for the compressor. With the default policy and a client that advertises gzip, every item past the 16 KiB segment threshold pays the rope for nothing. #277 asks how much that costs, and to stop taking the segmented path under compression if the answer is "enough".

This adds stream_compressed_encode to benches/rpc. Each arm drives a real in-process ConnectRpcService server-streaming call whose handler yields 16 pre-decoded OwnedView<BloatEchoView> items, framed through the same BatchingEnvelopeStream the server uses and drained frame by frame; the views are decoded in criterion's setup so the timed region is encode, framing and compression only. The arms differ in whether the item encodes through encode_segments (segmented) or encode (a newtype leaving encode_segments at its contiguous default, contiguous), and in whether the request negotiated gzip. Two shapes: one large string field the rope captures whole, and 1 KiB strings summing to the same size, none capturable (the #278 shape). 4 KiB sits under the segment threshold as a control. A pointer check asserts that a frame aliases an item's backing buffer in exactly the one combination where a capture should reach the body, so a silently flattened segmented arm cannot pass as "no difference".

Measured on a dedicated c7i.metal-24xl (turbo off, performance governor, pinned core, 10 s measurement). Median per item, segmented vs contiguous:

shape size gzip: segmented gzip: contiguous delta identity: segmented identity: contiguous delta
one large field 4 KiB 20.00 µs 20.10 µs −0.5% 0.78 µs 0.79 µs −0.4%
one large field 32 KiB 162.4 µs 161.6 µs +0.5% 0.90 µs 1.75 µs −48.2%
one large field 256 KiB 1572.2 µs 1571.7 µs +0.03% 1.26 µs 14.48 µs −91.3%
1 KiB fields 4 KiB 20.41 µs 20.41 µs +0.03% 0.94 µs 0.93 µs +1.1%
1 KiB fields 32 KiB 164.2 µs 163.5 µs +0.5% 3.56 µs 2.40 µs +48.3%
1 KiB fields 256 KiB 1605.4 µs 1589.6 µs +1.0% 27.83 µs 18.88 µs +47.4%

The threshold for "costs more" was 5%, the run-to-run floor this setup resolves; the 4 KiB rows, where both arms take the identical path, agree within 1.1%. No compressed arm clears it: the segmented encode adds at most 1.0% (about 16 µs per 256 KiB item) to a compressor that costs 1.6 ms for the same item. For the shape whose field the rope captures, the flatten is one payload copy the contiguous encode paid anyway, and the rope's bookkeeping is below what the pair can resolve. The only visible cost is the 1 KiB-fields shape, and it appears without compression too (+47–48%): that is the rope's doubling tail capturing nothing — #278's gate problem, not a compression problem — and #278's fix takes it out of both columns.

So no change to the encode choice. Threading the negotiated encoding to the dispatcher's encode closure without touching the Dispatcher trait or generated code would need an ambient hint set around the framing layer's poll, and the measurement does not justify a hidden coupling for a sub-1% saving. The EncodedStream doc, which said the segmented encode under compression "was then wasted work", now bounds how much.

Fixes #277

No changelog fragment: a benchmark and a doc sentence are not user-visible.

Deferred review findings

  • Two casts in the bench's text generator would trip clippy::cast_possible_truncation under pedantic (not enabled); both are provably in range.

Gates: nightly fmt --check; clippy 1.95.0 -D warnings clean; cargo test --workspace 936 passed; cargo test -p connectrpc --no-default-features 513 passed; rustdoc -Dwarnings clean.

Every streaming view item is encoded through `encode_segments` before
the framing layer knows whether it will be compressed, and a compressor
needs one contiguous input, so `encode_chained` flattens a segmented
body back into one buffer. Under the default policy with a client that
advertises gzip that is every item past the 16 KiB threshold, and no
benchmark covered it.

`stream_compressed_encode` drives a real in-process `ConnectRpcService`
server-streaming call: a handler yields 16 pre-decoded `OwnedView`
items, framed through the batching body the server uses and drained
frame by frame, with the views decoded in criterion's setup so the timed
region is encode, framing and compression only. Arms pair the generated
`encode_segments` against a newtype that takes the contiguous default,
with and without gzip negotiated, over one large `string` field and over
1 KiB `string` fields summing to the same size, at 4, 32 and 256 KiB.
A pointer check before each arm asserts a frame aliases an item's buffer
in exactly the one combination where a capture should reach the body,
so a silently flattened segmented arm cannot pass as "no difference".

On a dedicated bare-metal box the segmented encode adds at most 1% to
the compressed arms, against a 5% resolution floor; the flatten is the
payload copy a contiguous encode makes anyway, and what the rope adds
around it disappears next to the compressor's cost. The one visible
cost is the 1 KiB-fields shape, which pays the rope's doubling tail
without capturing anything, and it shows up uncompressed too; that is
the gate's problem rather than compression's. So the encode choice is
left alone, and the `EncodedStream` doc now bounds the waste it already
described instead of implying it is worth avoiding.

Signed-off-by: Iain McGinniss <309153+iainmcgin@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

server: measure (and avoid) the segmented encode on compressed streams

1 participant