Add socket activation support for TCP and UDS servers - #2734
Add socket activation support for TCP and UDS servers#2734AndreeaDrehuta wants to merge 3 commits into
Conversation
|
|
Adds an opt-in "socket-activation" feature that lets tonic servers accept pre-opened listeners passed by a supervisor (e.g. systemd) for both TCP and Unix domain sockets. Includes examples, systemd unit files, test scripts, and unit tests.
8468066 to
9ea4aa1
Compare
5bd8c0e to
33a0062
Compare
|
Hello! @gu0keno0 @dfawley @sauravzg @LYZJU2019 Thanks in advance for your time and feedback! |
I'm OOO this week and can take a deeper look at the code once back. Just want to better understand the motivation, what's the memory overhead you observed in production? Tonic is highly memory-efficient. A mostly dormant server / client should consume less than 100 MB of RSS if you use system glibc / mialloc. Also, cc: @LucioFranco |
We haven't observed any significant memory issues with Tonic. Our service currently consumes roughly ~30 MB, which is already quite low. However, since we're hardware-constrained and run multiple services on the same infrastructure, we're paying close attention to resource utilization and looking for opportunities to optimize further. |
|
@AndreeaDrehuta this feels quite custom and probably something that we don't want to include in tonic but this is probably a good candidate for providing a custom transport that allows you to use this feature. |
Since gRPC already supports this deployment model in C++, I was hoping Rust could offer a comparable experience for users deploying gRPC services in systemd-managed environments. |
|
@dfawley how do you feel about adding this to tonic, I believe if we are adding this feature set to tonic we need to as well to grpc-rust? |
I think this should be reasonable, given the server API we're targeting. Even if we don't support it in grpc-rust as a first-party feature, the traits we expose should allow for the same functionality without any extra work on our end. Does that sound right to you, @sauravzg? |
|
@dfawley Based on my initial look at the implementation, I'd assume so. Although, I don't think we'll make the needed traits public for now. What I am struggliing to understand right now is why the change needs to be in core tonic and can't be in the application layer using https://docs.rs/tonic/latest/tonic/transport/server/struct.Router.html#method.serve_with_incoming (given that it essentially needs a regular tcp/unix listener) and injecting a custom TCP/Unix listener. |
The issue is less about using serve_with_incoming and more about finding the correct TCP/Unix listener in the first place. You need to match each fd against the address/path you actually want, which means every server that wants this feature has to write and maintain that search itself. Centralizing it in tonic means it's implemented and tested once. |
|
Hi, @sauravzg, just checking in on this PR. Are there any remaining concerns or questions about the rationale for handling the listener discovery in tonic rather than requiring each application to implement it separately? Happy to provide more details or explore alternative approaches if needed. Thanks! |
|
@AndreeaDrehuta Sorry. I missed responding earlier. I had an internal discussion about it. |
5 participants
Adds an opt-in "socket-activation" feature that lets tonic servers accept pre-opened listeners passed by systemd for both TCP and Unix domain sockets. Includes examples, systemd unit files and unit tests.
Fixes #2733
Motivation
Under systemd, a service can be started on-demand when a client connects. The manager creates and listens on the socket itself, then passes the already-listening file descriptor to the spawned process via the
LISTEN_FDS/LISTEN_PIDenvironment variables.Consider a gRPC server that receives requests only occasionally. Keeping it running around the clock just to answer the rare request wastes memory and other resources. A better fit for that situation is to let the process run only while it is actually serving requests: the service manager (systemd) holds the socket, starts the server on an incoming connection, and the server can stop itself once it goes idle, after which the manager re-activates it on the next connection.
Solution
Introduces a new, opt-in socket-activation cargo feature (Unix only, disabled by default, so there is no impact on existing users).
When the feature is enabled, socket binding checks for an inherited listener before opening a new one:
TCP: TcpIncoming::bind(addr) adopts an inherited listening socket whose address matches addr, if none was passed in, it falls back to binding normally.
UDS: the Unix domain socket path binding adopts a matching inherited listener when present, otherwise binds as usual.
Implementation details:
A helper (socket_activation.rs) reads LISTEN_PID / LISTEN_FDS, verifies the fds were intended for this process, and returns the first inherited fd that is a listening stream socket matching the requested address.
Behavior is fully backward compatible: with the feature off, or when no fds are passed in, binding works exactly as before.
Also included to make the feature usable and testable:
Example TCP and UDS client/server binaries under examples/src/socket_activation/.
Ready-to-use systemd .socket / .service unit files.
Shell test scripts (test_tcp.sh, test_uds.sh) that exercise the activation flow end to end.
Unit tests covering fd scanning and listening-socket detection.