Smart retry/prototype - #12
Open
louiscrc wants to merge 157 commits into
Open
Conversation
…ion control Level 1: GoodputCoupledRetryBudget gates retries at alpha * goodput using asymmetric EWMA (fast decay, slow growth) with per-tenant isolation. Level 2: EndToEndRetryBudgetMiddleware stamps each request with budget B, bounding total retries across multi-hop chains. Includes 17 experiment configs (8 SYSNAME + 9 baselines), 22 unit tests, and paper figure generation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…budget Track per-window request rate and cap the retry budget at max_retry_ratio * request_rate to prevent initial burst overshoot when the goodput EWMA is stale at fault onset.
Add /api/param_compare endpoint that sweeps a shared parameter across all strategies, computing fault-phase-only metrics. Frontend renders grouped bar charts (Success Rate, Goodput, Amplification, Retry Efficiency, P99) for side-by-side strategy comparison.
Adds admission counters in LoadLimiterMiddleware and ServiceRuntime's pre-queue check; exports to admission_stats.json (run) and admission_summary.json (sweep).
Adds two new models : - v2: 10 s observation, 10 s decision interval - v3: 10 s observation, 2 s decision interval Both configs reuse the May-3 v1-a sweep as their primary baseline. Script to analyse the new models: - plot_rl_comparison.py: align the decision plot's x-axis with the success-rate / latency plots via timeline.json (new `t_ref` arg in `read_decisions`), shade the fault window on the decision plot, and add `plot_resource_comparison` overlaying per-policy CPU / memory alongside the RL controller's own process footprint. - resource_sampler.py: finally land the file `run-experiment.sh` has been calling since the v1 implementation, with an optional `--pid` flag (psutil) so the controller's local process can be sampled alongside the cluster-side pod metrics. - run_rl_v1.sh → run_rl.sh: script is the generic RL-experiment runner (it dispatches whichever config you pass via --config, not just v1).
Wall-clock breakdown of every RL controller tick into the five existing phases — observation fetch / parse / build, model inference, and DR patch — written one record per tick to a new rl-timings.jsonl alongside rl-observations.jsonl. No behavioural change; no relocation. Becomes the baseline to compare optimization against.
Move the RL controller off the laptop and into the cluster. The controller now runs as a per-policy Kubernetes Job, observations come from a tiny aiohttp /window endpoint on each loader shard, and patches go through the in-cluster apiserver via a service-account token. The laptop is out of the hot loop entirely.
Move the controller's observation path from per-attempt JSON rows (~10k/tick) to per-second pre-aggregated buckets (~40/tick) served from a new /buckets aiohttp endpoint on the loader. Cuts controller-tick total_ms p50 from 488ms to 24ms (but p95 still high because it doesn't handle spikes well).
Move the in-cluster controller's observation source from the loader's /buckets endpoint to each caller-pod's Envoy sidecar. The controller fans out parallel HTTP fetches to pilot-agent's /stats/prometheus endpoint on port 15020, parses the prom text, and reconstructs the same 18-feature vector previously derived from the loader buckets.
Converts the design to HTML, drops all plot-viewer changes, and adds a standalone HTML report builder writing to outputs/proto-report. Sets the 4-policy set (no-control, envoy-retry-budget, arolla, rb-rl-v5) and a two-phase plan: Phase A desk-analysis of existing full-sweep + v5 data to validate the metric/report, Phase B a 30-scenario x 4-policy x 3-repeat re-run with across-run averaging. Co-authored-by: Cursor <cursoragent@cursor.com>
louiscrc
force-pushed
the
smart-retry/prototype
branch
from
June 25, 2026 17:31
ac334b2 to
0f07d96
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RL retry budget: prototype adaptation to fit the in-cluster RL controller & new sustained-recovery metric to evaluate it
Summary
Adds an RL-driven retry-budget policy (
rb-rl) to the Online Boutique Kubernetes/Istio prototype, alongside the harness needed to train, deploy, benchmark, and analyze it against some of the existing baselines (no-control,envoy-retry-budget,arolla).The controller observes per-service retry pressure from Envoy sidecar stats, maps it onto an observation space aligned with the discrete-event simulator, and self-tunes the Istio
retryBudgetat runtime. A new sustained-recovery metric replaces the old first-crossing.All RL implementation lives under
prototype/experiments/rl/. The pre-existing sweep/fairness tooling onmainis retained unchanged so existing paper figures and sweeps still run.What's new
RL controller (
prototype/experiments/rl/)rl_controller.py— agent that tunes the policy; runs as an in-cluster Kubernetes Job.rl_obs_envoy.py/rl_obs_schema.py— Envoy/statsscraper and theobservation schema shared with the loader's
/bucketsendpoint.rl_configs/v1–v5— versioned training configs and trained models.rl_controller_image/— Dockerfile + build/distribute scripts for thein-cluster image.
ensure_rl_configmaps.sh— idempotent ConfigMap setup.Sustained-recovery metric
sustained_recovery.py— recovery counted only when goodput and successrate hold above their thresholds continuously for a window (default 30 s).
analyze.pyandclassify_runs.py(Recovered / Metastable /Ambiguous labelling).
tests/test_sustained_recovery.py,tests/test_classify_decision.py.30-scenario sweep
run_full_sweep.sh— drives all 30 scenarios inscenarios/sweep_scenarios.csvacrossall four policies, with per-row RL callee/caller-label retargeting.
run_rl.sh— single-scenario dev/debug runner + comparison plots.plot_rl_comparison.py— multi-policy RL comparison figures (adapted forrun_full_sweep.shoutput).Overhead analysis
measure_overhead.sh+resource_sampler.py— CPU/memory overhead of thecontroller via
kubectl topsampling.bench_decision_diff.py/bench_summarize.py— comparises differences of RLdecision traces across runs.
Manifests
manifests/online-boutique/rl-controller/{job,rbac}.yaml— in-clustercontroller Job + RBAC.
manifests/istio/telemetry-rl-profile.yaml— promotes the loader'sx-rl-profileheader to an Envoyrl_profilemetric dimension.