fix(service): prevent signoz broken-pipe errors - #11640
Merged
peaklabs-dev merged 1 commit intoSep 8, 2026
Merged
Conversation
The ClickHouse healthcheck used `wget --spider`, which closes the connection before it reads the body. ClickHouse then fails to write the /ping response and logs an Error with a full stack trace, roughly every 30 seconds. Using `-O-` reads the body instead. Measured on clickhouse-server 25.5.6-alpine, 25 requests each: --spider 22/25 errors -O- 0/25 errors
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
The SigNoz ClickHouse healthcheck used
wget --spider. That closes the connection before reading the body, so ClickHouse fails to write the/pingresponse and logs anErrorwith a full stack trace. At a 30s interval that is ~5,700 stack traces a day. The container stays healthy throughout (the check still exits 0), so nothing surfaces it. The noise buries real ClickHouse errors.This switches it to
wget -q -O-, which reads the body. Only theclickhousehealthcheck changes; thesignozone keeps--spider, as it targets a Go server that closes cleanly.templates/service-templates.jsonis deliberately untouched, matching earlier changes to this template.Issues
Category
AI Assistance
If AI was used:
Testing
Verified on a live Coolify instance running this template, on
clickhouse-server:25.5.6-alpine, the image it pins. That is a real deployment, not a local dev instance.Each command ran 25 times in the container, counting new
StaticRequestHandlererrors:After the change the errors stopped entirely.
One caution: do not fix this by switching the host to
localhost. In the ClickHouse image it resolves to::1first, so with IPv6 disabled the connection is refused (wget exits 1). That shows no errors only because the request never reaches ClickHouse, and would make the check fail.Contributor Agreement
Important