Skip to content

feat(debugger): add an evaluation time budget - #10156

Draft
watson wants to merge 1 commit into
watson/DEBUG-6081/evaluation-errorsfrom
watson/DEBUG-6081/evaluation-timeout
Draft

feat(debugger): add an evaluation time budget#10156
watson wants to merge 1 commit into
watson/DEBUG-6081/evaluation-errorsfrom
watson/DEBUG-6081/evaluation-timeout

Conversation

@watson

@watson watson commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds DD_DYNAMIC_INSTRUMENTATION_EVALUATION_TIMEOUT_MS (default 10 ms). Conditions are timed inside the breakpoint condition; an overrun is reported like a condition error regardless of the result, the probe is skipped at entry for the throttle window, and skipped hits are counted as events.skipped{reason:evaluationTimeout}. Templates and capture expressions are timed around their evaluation on the paused frame; the result is still sent with an evaluationErrors entry and the probe is throttled afterwards.

Motivation

Nothing bounded how long user expressions could run on the application thread. V8 can't interrupt an evaluation, so this measures wall-time and treat overruns as guardrail input. Enables Test_Debugger_Evaluation_Timeout_Line_Probe in system tests.

Additional Notes

The end-to-end test uses a ReDoS regex condition: the first hit paid ~900 ms once, the following hits were skipped at probe entry in ~7 ms each.

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Sep 4, 2026

Copy link
Copy Markdown

Pipelines  Tests

Unblock PR with BitsAI

⚠️ Warnings

Your PR has failed checks. Please review the issues below and take necessary action before merging.

🚦 6 Pipeline jobs failed

DataDog/apm-reliability/dd-trace-js | benchmark: [24, 1] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-js | benchmark: [26, 1] — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

DataDog/apm-reliability/dd-trace-js | validate_supported_configurations_v2_local_file — 🔧 Needs a code fix, caused by this PR

View more details · View in GitLab

View all 6 failed jobs.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 98.05% (-0.00%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 7ffa5dc | Docs | View more details | Give us feedback!

@github-actions github-actions Bot added semver-minor debugger Dynamic Instrumentation & Live Debugger labels Sep 4, 2026
@dd-octo-sts

dd-octo-sts Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 8.87 MB
Deduped: 9.53 MB
No deduping: 9.53 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.4.0 | 127.33 kB | 447.04 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.65%. Comparing base (2f97506) to head (7ffa5dc).

Additional details and impacted files
@@                           Coverage Diff                            @@
##           watson/DEBUG-6081/evaluation-errors   #10156       +/-   ##
========================================================================
+ Coverage                                86.46%   98.65%   +12.19%     
========================================================================
  Files                                      997     1007       +10     
  Lines                                   153305   158247     +4942     
  Branches                                    74       74               
========================================================================
+ Hits                                    132562   156126    +23564     
+ Misses                                   20743     2121    -18622     
Flag Coverage Δ
ai-guard 65.00% <ø> (ø)
apm-capabilities 63.03% <14.04%> (-0.05%) ⬇️
apm-integrations 81.41% <ø> (ø)
appsec 58.71% <ø> (ø)
debugger 70.45% <100.00%> (+0.12%) ⬆️
instrumentation 53.72% <ø> (+<0.01%) ⬆️
llmobs 79.00% <ø> (ø)
master-coverage 98.65% <100.00%> (+12.19%) ⬆️
openfeature 66.38% <ø> (ø)
platform 69.31% <ø> (ø)
profiling 66.75% <ø> (ø)
serverless 65.71% <ø> (ø)
test-optimization 82.67% <22.72%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@watson
watson force-pushed the watson/DEBUG-6081/evaluation-timeout branch 2 times, most recently from 7c30420 to b70e96f Compare September 4, 2026 10:24
@watson
watson force-pushed the watson/DEBUG-6081/evaluation-timeout branch from b70e96f to 9dba288 Compare September 4, 2026 12:02
@watson
watson force-pushed the watson/DEBUG-6081/evaluation-timeout branch from 9dba288 to 253b80c Compare September 4, 2026 12:29
@watson
watson force-pushed the watson/DEBUG-6081/evaluation-timeout branch from 253b80c to 7c25280 Compare September 4, 2026 15:41
@watson
watson force-pushed the watson/DEBUG-6081/evaluation-timeout branch from 7c25280 to af7c8b6 Compare September 6, 2026 09:05
Probe conditions, log message templates and capture expressions run
user-provided expressions on the application thread, and nothing
bounded how long they could take. A condition with a catastrophically
backtracking regular expression or a filter over a large collection
stalled the application on every hit.

Add `DD_DYNAMIC_INSTRUMENTATION_EVALUATION_TIMEOUT_MS` (default 10 ms,
the canonical default from the Live Debugger Circuit-Breakers RFC). V8
can't interrupt an evaluation, so the budget is enforced the way the RFC
allows for runtimes without early abort: the elapsed wall-time is
measured and an overrun is treated as guardrail input.

- Conditions are timed inside the breakpoint condition on the main
  thread. An overrun is reported like a condition error (one error
  result per probe per throttle window, no capture) regardless of the
  condition's result, and the probe is skipped at entry for the rest of
  the window. Skipped hits are counted as
  `events.skipped{reason:evaluationTimeout}`.
- Templates and capture expressions are timed around their evaluation
  on the paused frame in the worker. The result is still reported, with
  an `evaluationErrors` entry describing the overrun, and the probe is
  throttled the same way afterwards.
@watson
watson force-pushed the watson/DEBUG-6081/evaluation-timeout branch from af7c8b6 to 7ffa5dc Compare September 7, 2026 10:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

debugger Dynamic Instrumentation & Live Debugger semver-minor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant