Skip to content

feat(realtime): adapt semantic endpointing controls - #107

Draft
kvnloo wants to merge 2 commits into
TheSmokeDev:mainfrom
kvnloo:feat/realtime-semantic-endpoint-adapter
Draft

feat(realtime): adapt semantic endpointing controls#107
kvnloo wants to merge 2 commits into
TheSmokeDev:mainfrom
kvnloo:feat/realtime-semantic-endpoint-adapter

Conversation

@kvnloo

@kvnloo kvnloo commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

What and why

Mirror the provider-neutral turn-detection contract in Talk's wire boundary: map OpenAI server/semantic VAD through mint and live update paths, advertise Grok native/server support and fail fast on semantic mode, advertise Gemini native-only support and fail fast on server/semantic modes. The semantic turn-detection names are probed as an optional capability, so pre-semantic core heads (today's #101808) keep the full core lane with provider-native turn detection instead of losing it. Ships a deterministic, content-free endpointing benchmark module.

Pairs with TheSmokeDev/hermes-agent#4 (stacked on NousResearch/hermes-agent#101808).

How to test

  • Full suite: 1580 passed, 56 skipped, 5 xfailed
  • tests/test_core_provider.py: 8 passed (was 7), 38 skipped without a core checkout
  • Ruff passes on every changed Python file
  • Wheel build succeeds and includes talk_endpointing_benchmark.py
  • Benchmark smoke reports endpoint p50/p95/max, playback latency, timeout, cutoff/split, and false activations without retaining audio or transcripts

Platforms

  • Linux (CI + developer box)
  • Windows/macOS: untouched paths only; no platform-specific code in this PR

Live receipt

No live-provider run in this PR: provider lanes are exercised through contract-shaped unit coverage and the benchmark smoke module. Live-lane receipt stays with hermes talk check (#97) on a configured box.

Checklist

  • Failing behavior reproduced first (old head dropped the whole core lane with a bare ImportError)
  • Regression test binds the #101808-shaped contract and fails without the fix
  • Full suite green (1580 passed, 56 skipped, 5 xfailed)
  • Ruff clean on every changed file
  • CHANGELOG carries the endpointing bridge entry, not just the benchmark
  • No credentials, transcripts, or audio retained in tests or artifacts

@kvnloo
kvnloo force-pushed the feat/realtime-semantic-endpoint-adapter branch from b99b0a2 to a001abf Compare September 3, 2026 06:05

@TheSmokeDev TheSmokeDev left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Kevin — this is a clean design: neutral RealtimeTurnDetection, exact GA semantic_vad shape, and honest Grok/Gemini refusals. One blocker before I can take it: talk_core_provider hard-imports RealtimeSemanticEagerness / RealtimeTurnDetection / RealtimeTurnDetectionMode inside the contract try, so on today's #101808 head (without your hermes-agent#4) the whole core lane disappears — core_contract_available() goes False with a bare ImportError, and the widened _REAL_CONTRACT_MARKERS makes every contract test skip instead of fail. Could you probe those three names as an optional capability (own try/except, getattr on validate_setup / setup.turn_detection, default to native), and add a test that binds the #101808-shaped contract? Also a CHANGELOG line for the endpointing bridge itself, not just the benchmark. Happy to merge right after. — SmokeDev

- Import RealtimeSemanticEagerness/RealtimeTurnDetection/
  RealtimeTurnDetectionMode in their own try/except so a pre-semantic
  core head keeps the full lane instead of dropping it on ImportError
- Default missing setup.turn_detection to provider-native via getattr;
  refuse non-native modes before resources open
- Advertise native-only mode sets on old heads via _contract_turn_modes
- test_old_head_contract binds a #101808-shaped contract and proves
  degraded-mode behavior
- CHANGELOG: endpointing bridge entry
@kvnloo

kvnloo commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 412c673 (full suite 1580 passed, 56 skipped, 5 xfailed, ruff clean): the three semantic names are now imported in their own try/except with a turn_detection_available() probe, mode sets degrade through _contract_turn_modes, and a missing setup.turn_detection defaults to provider-native via getattr. test_old_head_contract_keeps_core_lane_with_native_only_turn_detection binds a synthetic #101808-shaped contract (API v2, no turn-detection names) and proves the lane stays up degraded with native-only advertisement and a clean ValueError on semantic. Also added the bridge CHANGELOG entry and remapped the body to the repo template.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants