Skip to content

fix(opentelemetry): wait for trace delivery in forceFlush - #10131

Draft
BridgeAR wants to merge 1 commit into
masterfrom
BridgeAR/2026-09-03-otel-force-flush
Draft

fix(opentelemetry): wait for trace delivery in forceFlush#10131
BridgeAR wants to merge 1 commit into
masterfrom
BridgeAR/2026-09-03-otel-force-flush

Conversation

@BridgeAR

@BridgeAR BridgeAR commented Sep 3, 2026

Copy link
Copy Markdown
Member

This fixes OpenTelemetry TracerProvider.forceFlush() resolving before Datadog trace delivery completes. The provider now waits for the exporter's completion-aware flush boundary and every configured span processor, including delayed HTTP responses and overlapping flush generations.

Exporter failures remain observable through the OpenTelemetry promise contract, while legacy background flushes keep swallowing errors; OTLP and serverless delivery paths use the same boundary without retaining completed failures.

This fixes OpenTelemetry TracerProvider.forceFlush() resolving before Datadog trace delivery completes. The provider now waits for the exporter's completion-aware flush boundary and every configured span processor, including delayed HTTP responses and overlapping flush generations.

Exporter failures remain observable through the OpenTelemetry promise contract, while legacy background flushes keep swallowing errors; OTLP and serverless delivery paths use the same boundary without retaining completed failures.
@dd-octo-sts

dd-octo-sts Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 8.74 MB
Deduped: 9.41 MB
No deduping: 9.41 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.4.0 | 127.33 kB | 447.04 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Sep 3, 2026

Copy link
Copy Markdown

Tests

All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 98.57% (-0.06%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 87d24e7 | Docs | View more details | Give us feedback!

@codecov

codecov Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.57%. Comparing base (6e65a91) to head (87d24e7).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #10131      +/-   ##
==========================================
- Coverage   98.63%   98.57%   -0.07%     
==========================================
  Files        1002     1003       +1     
  Lines      154897   155032     +135     
  Branches    13358    13548     +190     
==========================================
+ Hits       152779   152819      +40     
- Misses       2118     2213      +95     
Flag Coverage Δ
aiguard 63.82% <65.21%> (-0.04%) ⬇️
aiguard-integration 58.91% <31.88%> (-0.04%) ⬇️
apm-capabilities-tracing 62.91% <100.00%> (+0.05%) ⬆️
apm-integrations-aerospike 61.24% <65.21%> (-0.03%) ⬇️
apm-integrations-aerospike-_integrat-1d185c06 63.20% <65.21%> (-0.04%) ⬇️
apm-integrations-confluentinc-kafka--d0bf0045 67.30% <65.21%> (-0.04%) ⬇️
apm-integrations-couchbase 61.73% <65.21%> (-0.03%) ⬇️
apm-integrations-elasticsearch_integ-31b69669 68.55% <65.21%> (-0.04%) ⬇️
apm-integrations-http 66.83% <65.21%> (-0.04%) ⬇️
apm-integrations-http2_integrations--9aea12f5 ?
apm-integrations-kafkajs 67.84% <65.21%> (-0.08%) ⬇️
apm-integrations-net_integrations-or-43576e84 65.71% <65.21%> (?)
apm-integrations-next 64.89% <65.21%> (-0.04%) ⬇️
apm-integrations-prisma 62.45% <50.67%> (-0.07%) ⬇️
apm-integrations-restify_integration-b312dd16 ?
apm-integrations-sharedb_integrations-tedious 62.27% <65.21%> (?)
appsec 77.28% <65.21%> (-0.05%) ⬇️
appsec-express_fastify_graphql 72.68% <65.21%> (-0.04%) ⬇️
appsec-integration 49.54% <34.78%> (-0.02%) ⬇️
appsec-kafka_ldapjs_lodash 68.00% <65.21%> (-0.01%) ⬇️
appsec-mongodb-core_mongoose_mysql 71.11% <65.21%> (-0.04%) ⬇️
appsec-next 57.69% <65.21%> (-0.02%) ⬇️
appsec-node-serialize_passport_postgres 70.59% <65.21%> (-0.04%) ⬇️
appsec-sourcing_stripe_template 69.04% <65.21%> (-0.04%) ⬇️
debugger 69.77% <62.31%> (-0.02%) ⬇️
instr-ai_instrumentation-anthropic-l-e862da55 56.60% <62.31%> (-0.03%) ⬇️
instr-bluebird_instrumentation-body--02ae4b41 64.83% <65.21%> (-0.04%) ⬇️
instr-confluentinc-kafka-javascript 56.01% <62.31%> (-0.03%) ⬇️
instr-connect_instrumentation-cookie-0bbb44d9 57.70% <62.31%> (-0.03%) ⬇️
instr-couchbase 55.23% <62.31%> (-0.03%) ⬇️
instr-cypress-config_instrumentation-4db1de46 58.02% <62.31%> (-0.03%) ⬇️
instr-express-multi-version_instrume-f5caf487 64.30% <65.21%> (-0.04%) ⬇️
instr-fetch_instrumentation-fs_instr-43dfe6a2 50.56% <62.31%> (-0.02%) ⬇️
instr-hono_instrumentation-http_inst-026a5d4d 66.15% <65.21%> (-0.04%) ⬇️
instr-kafkajs_instrumentation-knex_i-ca6f5b19 56.66% <62.31%> (-0.03%) ⬇️
instr-light-my-request_instrumentati-8c9fceca 63.61% <65.21%> (-0.03%) ⬇️
instr-mysql_instrumentation-mysql2_i-05f94c42 59.11% <65.21%> (-0.03%) ⬇️
instr-openai-lifecycle_instrumentati-783580bd 65.96% <65.21%> (-0.04%) ⬇️
instr-passport-http_instrumentation--2b6c8eb6 66.57% <65.21%> (-0.04%) ⬇️
instr-promise_instrumentation-promis-8479bae3 56.44% <62.31%> (-0.03%) ⬇️
instr-restify_instrumentation-router-5641467c 56.58% <62.31%> (-0.03%) ⬇️
instr-url_instrumentation-when_instr-22ca7078 56.46% <62.31%> (-0.03%) ⬇️
instrumentations-integration-esbuild 34.68% <ø> (ø)
llmobs-ai_anthropic_bedrock 66.78% <50.67%> (-0.08%) ⬇️
llmobs-claude-agent-sdk_google-genai-da2c851e 64.97% <65.21%> (-0.03%) ⬇️
llmobs-openai 66.79% <65.21%> (-0.04%) ⬇️
llmobs-openai-agents_vertex-ai 64.14% <65.21%> (-0.03%) ⬇️
llmobs-sdk 76.56% <73.33%> (-0.05%) ⬇️
master-coverage 98.57% <100.00%> (?)
openfeature 60.07% <31.88%> (-0.05%) ⬇️
openfeature-unit 58.85% <62.31%> (-0.03%) ⬇️
platform-core_esbuild_instrumentations-misc 41.36% <62.31%> (+<0.01%) ⬆️
platform-integration 65.14% <40.44%> (-0.09%) ⬇️
platform-shimmer_unit-guardrails_webpack 39.11% <62.31%> (+<0.01%) ⬆️
plugins-aws-durable-execution-sdk-js-2bc88c07 61.13% <65.21%> (-0.03%) ⬇️
plugins-azure-event-hubs_azure-servi-50d5b0fc 58.49% <34.78%> (-0.05%) ⬇️
plugins-browser-bunyan_bullmq_cassandra 66.36% <65.21%> (-0.04%) ⬇️
plugins-cookie_cookie-parser_crypto 56.21% <62.31%> (-0.03%) ⬇️
plugins-dd-trace-api_express-mongo-s-83827a5c 61.79% <65.21%> (-0.04%) ⬇️
plugins-fastify_fetch_fs ?
plugins-fetch_fs_generic-pool 63.46% <65.21%> (?)
plugins-generic-pool_google-cloud-pubsub_grpc ?
plugins-google-cloud-pubsub_grpc_handlebars 69.54% <65.21%> (?)
plugins-handlebars_hapi_hono ?
plugins-hapi_hono_ioredis 65.52% <65.21%> (?)
plugins-ioredis_knex_langgraph ?
plugins-knex_langgraph_ldapjs 60.16% <65.21%> (?)
plugins-ldapjs_light-my-request_limitd-client ?
plugins-light-my-request_limitd-client_lodash 64.12% <65.21%> (?)
plugins-lodash_mariadb_memcached ?
plugins-mariadb_memcached_mercurius 68.00% <65.21%> (?)
plugins-mercurius_microgateway-core_-b9bd83c4 ?
plugins-microgateway-core_modelconte-ca2fdeba 66.21% <65.21%> (?)
plugins-moleculer_mongodb_mongodb-core ?
plugins-mongodb_mongodb-core_mongoose 64.70% <65.21%> (?)
plugins-mongoose_multer_mysql ?
plugins-multer_mysql_mysql2 64.36% <65.21%> (?)
plugins-mysql2_nats_node-serialize ?
plugins-nats_node-serialize_opensearch 66.22% <65.21%> (?)
plugins-opensearch_passport-http_pino ?
plugins-passport-http_pino_postgres 64.08% <65.21%> (?)
plugins-postgres_process_pug ?
plugins-process_pug_redis 62.91% <65.21%> (?)
plugins-redis_router_sequelize ?
plugins-router_sequelize_test-and-up-32bf768a 67.11% <65.21%> (?)
plugins-test-and-upstream-amqp10_tes-914101e2 ?
plugins-test-and-upstream-amqplib_te-47de9104 65.99% <65.21%> (?)
plugins-test-and-upstream-avsc_test--408e8d3a ?
plugins-test-and-upstream-bunyan_tes-870c2d6f 68.58% <65.21%> (?)
plugins-test-and-upstream-graphql_te-59dd94de ?
plugins-test-and-upstream-koa_test-a-e7fc792b 69.75% <65.21%> (?)
plugins-test-and-upstream-rhea_undici_url ?
plugins-undici_url_valkey 63.11% <65.21%> (?)
plugins-valkey_vm_winston ?
plugins-vm_winston_ws 65.39% <65.21%> (?)
plugins-ws ?
profiling 66.67% <62.31%> (-0.04%) ⬇️
serverless-aws-sdk-aws-sdk 55.97% <65.21%> (-0.02%) ⬇️
serverless-aws-sdk-base-inject-field 55.78% <62.31%> (-0.03%) ⬇️
serverless-aws-sdk-bedrockruntime 59.31% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-client 60.67% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-dynamodb 59.74% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-eventbridge 58.14% <65.21%> (-0.02%) ⬇️
serverless-aws-sdk-kinesis 63.68% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-lambda 61.67% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-s3 59.79% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-serverless-peer-service 64.24% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-sns 64.57% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-sqs 65.02% <65.21%> (-0.04%) ⬇️
serverless-aws-sdk-stepfunctions 59.65% <65.21%> (-0.03%) ⬇️
serverless-aws-sdk-util 56.40% <62.31%> (-0.03%) ⬇️
serverless-azure-durable-functions_a-776237ee 57.65% <31.88%> (-0.04%) ⬇️
serverless-azure-functions-eventhubs-61ee6313 64.31% <65.21%> (-0.04%) ⬇️
test-optimization-cucumber 70.43% <84.61%> (+0.04%) ⬆️
test-optimization-cypress 64.81% <23.07%> (-0.13%) ⬇️
test-optimization-jest 71.82% <43.47%> (-0.07%) ⬇️
test-optimization-mocha 71.87% <46.37%> (+0.02%) ⬆️
test-optimization-playwright-playwri-363e40a6 59.71% <20.00%> (+0.10%) ⬆️
test-optimization-playwright-playwri-5141d36d 61.22% <23.07%> (-0.03%) ⬇️
test-optimization-playwright-playwri-5b35813b 59.59% <20.00%> (-0.03%) ⬇️
test-optimization-playwright-playwri-924f4516 60.88% <20.00%> (-0.03%) ⬇️
test-optimization-playwright-playwri-d6f5a99b 59.47% <23.07%> (+0.02%) ⬆️
test-optimization-playwright-playwright-atr 59.20% <23.07%> (-0.03%) ⬇️
test-optimization-playwright-playwright-efd 60.12% <20.00%> (-0.03%) ⬇️
test-optimization-selenium 58.78% <23.07%> (-0.07%) ⬇️
test-optimization-testopt 61.42% <34.78%> (-0.01%) ⬇️
test-optimization-vitest 72.47% <23.07%> (+0.07%) ⬆️
test-optimization-vitest-browser 58.85% <20.00%> (+0.49%) ⬆️
test-optimization-webdriverio 65.87% <62.31%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter

pr-commenter Bot commented Sep 3, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-09-03 11:45:41

Comparing candidate commit 87d24e7 in PR branch BridgeAR/2026-09-03-otel-force-flush with baseline commit 6e65a91 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2314 metrics, 14 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

  • unstable max_rss_usage [-43.557MB; +40.626MB] or [-14.769%; +13.775%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-20

  • unstable max_rss_usage [-12.111MB; +21.736MB] or [-5.294%; +9.502%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-20

  • unstable max_rss_usage [-18.937MB; +25.405MB] or [-4.973%; +6.672%]

scenario:debugger-line-probe-with-snapshot-minimal-24

  • unstable max_rss_usage [-15.908MB; +11.612MB] or [-6.335%; +4.624%]

scenario:dogstatsd-with-tags-20

  • unstable cpu_user_time [-364.046ms; +366.300ms] or [-7.228%; +7.273%]
  • unstable execution_time [-367969.637µs; +366836.221µs] or [-7.192%; +7.169%]
  • unstable throughput [-122040.157op/s; +123343.788op/s] or [-7.433%; +7.513%]

scenario:encoders-0.4-immediate-flush-20

  • unstable max_rss_usage [-11.436MB; +5.790MB] or [-14.472%; +7.326%]

scenario:llmobs-encode-unicode-mixed-20

  • unstable max_rss_usage [-13.369MB; -2.788MB] or [-17.888%; -3.731%]

scenario:plugin-graphql-long-with-depth-off-20

  • unstable max_rss_usage [-5.155MB; +10.293MB] or [-4.163%; +8.313%]

scenario:plugin-pg-service-20

  • unstable cpu_usage_percentage [-10.992%; +6.580%]
  • unstable execution_time [-175.713ms; +292.495ms] or [-10.606%; +17.656%]
  • unstable throughput [-385244.779op/s; +236483.976op/s] or [-10.458%; +6.420%]

scenario:test-optimization-large-suite-20

  • unstable max_rss_usage [-4.500MB; +7.516MB] or [-5.564%; +9.293%]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant