Skip to content

fix: use kimi-k3-base-picker for GitHub Copilot - #493

Merged
kisernl merged 1 commit into
computesdk:masterfrom
alexneyler:fix-copilot-kimi-model
Oct 8, 2026
Merged

kisernl merged 1 commit into
computesdk:masterfrom
alexneyler:fix-copilot-kimi-model

Conversation

@alexneyler

@alexneyler alexneyler commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Change the GitHub Copilot Kimi model name from kimi-k3 to kimi-k3-base-picker in benchmarks/ai-gateway/providers-kimi.ts. Other providers are unchanged.

Validation

  • git diff --check passed.
  • Type checking could not run because npm is not available in this environment.

Devin Review

@open-cla

open-cla Bot commented Oct 7, 2026

Copy link
Copy Markdown

Contributor License Agreement

All contributors are covered by a CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

Devin Review

@kisernl
kisernl merged commit 9317bf0 into computesdk:master Oct 8, 2026
6 checks passed
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

AI Gateway Benchmark Results — OpenAI

# Gateway Score Cold E2E Warm TTFT Tok/sec Status
1 GitHub Copilot 90.2 565ms 522ms 69.9 20/20
2 OpenRouter 89.5 555ms 455ms 65.5 20/20
3 Concentrate AI 89.1 764ms 640ms 56.4 20/20
4 Pydantic AI Gateway 88.5 842ms 650ms 52.3 20/20
5 Vercel AI Gateway 88.2 795ms 778ms 51.4 20/20
6 OpenAI (direct) 87.9 904ms 590ms 57.1 20/20
7 LLM Gateway 87.2 1080ms 945ms 63.0 20/20
8 Neon AI Gateway 86.6 1049ms 823ms 48.0 20/20
9 LLM API 85.3 1313ms 809ms 46.3 20/20
10 ngrok AI Gateway 85.0 823ms 667ms 57.3 20/20
11 Cloudflare AI Gateway 84.4 1400ms 791ms 40.8 20/20

View full run · SVG available as a build artifact

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

AI Gateway Benchmark Results — Anthropic

# Gateway Score Cold E2E Warm TTFT Tok/sec Status
1 Anthropic (direct) 91.8 335ms 316ms 71.2 20/20
2 ngrok AI Gateway 91.1 437ms 340ms 67.9 20/20
3 Pydantic AI Gateway 90.8 478ms 426ms 64.7 20/20
4 OpenRouter 90.4 493ms 465ms 57.1 20/20
5 GitHub Copilot 90.2 556ms 502ms 57.2 20/20
6 Vercel AI Gateway 90.0 509ms 496ms 59.6 20/20
7 LLM Gateway 89.8 569ms 483ms 60.7 20/20
8 Concentrate AI 89.7 655ms 552ms 56.9 20/20
9 Cloudflare AI Gateway 89.5 603ms 377ms 64.7 20/20
10 Neon AI Gateway 89.2 815ms 596ms 53.1 20/20
11 LLM API 88.9 889ms 559ms 55.7 20/20
12 blazerail 76.1 1043ms 963ms 41.4 18/20

View full run · SVG available as a build artifact

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

AI Gateway Benchmark Results — Gemini

# Gateway Score Cold E2E Warm TTFT Tok/sec Status
1 OpenRouter 89.2 1547ms 1509ms 127.0 20/20
2 Neon AI Gateway 85.2 2006ms 1981ms 97.3 20/20
3 Concentrate AI 84.9 1997ms 2063ms 91.5 20/20
4 Vercel AI Gateway 59.4 6117ms 6508ms 30.4 20/20
5 LLM API 55.6 8190ms 5960ms 24.5 20/20
6 Gemini (direct) 50.1 8248ms 7326ms 26.0 19/20
7 LLM Gateway 49.7 8078ms 8070ms 24.2 20/20
8 Cloudflare AI Gateway 44.4 7659ms 7350ms 27.0 17/20
9 ngrok AI Gateway 40.7 7374ms 6891ms 26.9 14/20
10 blazerail 0.0 0ms 0ms 0.0 0/20

View full run · SVG available as a build artifact

@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

AI Gateway Benchmark Results — Kimi

# Gateway Score Cold E2E Warm TTFT Tok/sec Status
1 Neon AI Gateway 90.7 972ms 529ms 101.1 20/20
2 GitHub Copilot 90.3 374ms 367ms 113.3 20/20
3 ngrok AI Gateway 77.3 3468ms 2140ms 32.5 20/20
4 Kimi (direct) 77.3 3466ms 2161ms 33.0 20/20
5 LLM API 76.8 3679ms 2202ms 32.3 20/20
6 Vercel AI Gateway 76.7 3298ms 2278ms 31.9 20/20
7 LLM Gateway 76.6 3597ms 2304ms 32.3 20/20
8 Cloudflare AI Gateway 75.5 3537ms 2331ms 32.2 20/20
9 OpenRouter 74.5 3397ms 3464ms 31.6 20/20
10 Concentrate AI 70.7 3640ms 2843ms 30.9 19/20
11 blazerail 0.0 0ms 0ms 0.0 0/20
12 Novita 0.0 0ms 0ms 0.0 0/20

View full run · SVG available as a build artifact

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants