Skip to content

Single-threaded (!Send) client & server: tonic::local - #2806

Open
artsiomkaltovich wants to merge 5 commits into
grpc:masterfrom
artsiomkaltovich:single-threaded-tonic
Open

Single-threaded (!Send) client & server: tonic::local#2806
artsiomkaltovich wants to merge 5 commits into
grpc:masterfrom
artsiomkaltovich:single-threaded-tonic

Conversation

@artsiomkaltovich

@artsiomkaltovich artsiomkaltovich commented Aug 14, 2026

Copy link
Copy Markdown

Prototype for #2790 (see also #2171, #844, #1505, #784 — the same request keeps coming back).

Motivation

Tonic's generated servers require Send + Sync + 'static handlers, and the client path boxes
futures/bodies as Send. That makes tonic hard to adopt in single-threaded async code: anything
holding Rc, RefCell, or a non-Send third-party handle has to be redesigned or wrapped, even
though the service will only ever run on one thread. Not every service needs a multi-threaded
runtime — a current_thread runtime is often the right tool (lower latency variance, no
cross-thread synchronisation, simpler state).

This PR makes that case work natively, without touching the multi-threaded path.

What this adds

  • tonic::local:: — a !Send client, server, router, body, streaming and a minimal HTTP/2
    transport, all driven by tokio::task::spawn_local.
  • Codegen support: tonic_prost_build::Builder::local(true) (and CodeGenBuilder::local) emits
    #[async_trait(?Send)] service traits, Rc<T> handlers and non-Send futures.
  • Works on either a tokio::task::LocalSet (any tokio 1.x) or tokio::runtime::LocalRuntime
    (stable since tokio 1.51). No tokio version bump: the library only uses spawn_local.

Design

Feature-gated, additive, zero changes to existing public API.

  • Runtime: local (pure — no new dependencies, wasm-friendly) and local-transport
    (adds the hyper/tokio bits for the provided server + channel).
  • Codegen: a local feature on tonic-build / tonic-prost-build gates the local() builder
    method, mirroring the existing transport-feature precedent.
  • No existing pub item changed signature or behaviour. Everything new lives under
    tonic::local::; the default build is byte-identical in API terms.
  • Duplication is kept out of the way by sharing the heavy logic through private generics:
    BodyKind<B>, StreamingImpl<T, DecoderStore, B>, classify_response,
    set_grpc_response_headers, ServerGrpcConfig. The Send and !Send types are thin wrappers
    over the same cores, so the decode state machine, framing, compression and codec handling exist
    once. Decode benches are at parity with the core path.
  • Errors stay Send + Sync (Status, BoxError) — required by hyper's HTTP/2 signatures, and
    harmless: Send on an error value never constrains handlers, futures or state.

Trade-offs of this approach

Pros

  • Zero risk to existing users: nothing on the multi-threaded path changes, no semver impact.
  • Opt-in at both layers (cargo feature + codegen flag).
  • The !Send types are honestly !Send — misuse is a compile error, not a runtime surprise.
  • Heavy logic is shared, so bug fixes land in one place.

Cons

  • The Grpc<T> method bodies exist twice (client ~230 lines, server ~290): identical code whose
    only difference is Send/'static tokens in where clauses. See below for why.
  • Two parallel type families to learn (tonic::Streaming vs tonic::local::Streaming, …).
  • The local transport is deliberately minimal: HTTP/2 only, no TLS, no graceful shutdown, no
    reconnect/balancing. Enough for a prototype, not feature parity. (Good enough for prototype).

Alternatives considered

  1. Full duplication (a parallel local implementation). Simplest to write and to reason about
    in isolation, but every fix has to be applied twice and the two copies drift. Rejected as the
    primary strategy — though see the point below: for the Grpc method bodies it is what remains.
  2. Unification via traits ("traitification"): one implementation generic over Send-ness.
    The attractive option, and the one we probed hardest. It does not work on stable Rust: a shared
    trait seam must fix its method bounds once for all impls, so the core impl's internal Send
    requirements (Body::new, Streaming::new_response) would leak into the trait signature and
    force Send back onto the local call sites — defeating the feature. There is no way to make a
    bound conditionally present per instantiation. A macro_rules! template could cut ~150 lines,
    but at the cost of tt-munged where clauses and materially worse compiler errors: a bad trade
    for code that is otherwise plainly readable and diffable. What could be shared has been, as
    private generics (above).
  3. Make the existing types generic over their storage (e.g. Streaming<T, Store = …>).
    Removes most duplication, but changes tonic's core public types and their inference/rustdoc
    identity — too invasive for a prototype, and a call for maintainers rather than a contributor. (To be honest, I prefer this, but it is you to decide)
  4. A defaulted streaming parameter on the two core service traits
    ClientStreamingService<R, St = Streaming<R>> (same for StreamingService). Backward
    compatible (the default keeps every existing impl, including generated ones, compiling
    unchanged) and would delete ~58 lines plus a codegen branch, letting local reuse the core
    traits. Deliberately not applied here — it modifies public traits, so it is offered as a
    ready follow-up if you want it.

Example / tests

End-to-end echo service on a current_thread runtime:
tests/local/proto/echo.proto,
tests/local.rs.

It covers all four RPC shapes with Rc<RefCell<_>> handler state and non-Send streams in both
directions, an interceptor round-trip, an unimplemented-path check, and the same flow on
LocalRuntime with no LocalSet. Static assertions pin the guarantee that the local types are
!Send, so tokio::spawn of them cannot compile.

Status

All existing tests pass unchanged; new unit + e2e + codegen tests added; rustdoc clean under
-D warnings; feature combinations (local alone, local-transport alone, no-default-features)
all build. Happy to split this into smaller PRs, or to adopt alternatives above, if that suits
review better.

@linux-foundation-easycla

linux-foundation-easycla Bot commented Aug 14, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant