Skip to content

app/vlinsert: consistently increase vl_http_errors_total on failed inserts - #1811

Open
func25 wants to merge 3 commits into
masterfrom
metrics-http-error
Open

func25 wants to merge 3 commits into
masterfrom
metrics-http-error

Conversation

@func25

@func25 func25 commented Sep 23, 2026 •

Copy link
Copy Markdown
Member

Previously vl_http_errors_total doesn't increase when inserts were rejected because vlstorage was in read-only mode (e.g. out of disk space), so users had to compute failures as vl_http_requests_total - vl_http_request_duration_seconds_count.

  • increase vl_http_errors_total when CanWriteData() fails for jsonline, native, multitenant native, internal insert, splunk and opentelemetry (interestingly journald already does this)
  • add the missing vl_http_errors_total for loki, elasticsearch and datadog

I keeps the minimal fix, we can consider unifying error handling, but the problem is jsonline counts error per invalid log line inside a request, not per request. Not sure this is expected.

@func25
func25 marked this pull request as ready for review September 23, 2026 10:30

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 11 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread app/vlinsert/datadog/datadog.go
Comment thread docs/victorialogs/CHANGELOG.md Outdated
@func25 func25 changed the title app/vlinsert: consistently increase vl_http_errors_total on failed inserts app/vlinsert: increment vl_http_errors_total when the storage is in read-only mode Sep 23, 2026
@func25 func25 changed the title app/vlinsert: increment vl_http_errors_total when the storage is in read-only mode app/vlinsert: consistently increase vl_http_errors_total on failed inserts Sep 23, 2026

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files (changes from recent commits).

Tip: Review your code locally with the cubic CLI to iterate faster.

Re-trigger cubic

Comment thread docs/victorialogs/metrics.md Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants