Skip to content

Leaderboard: re-price every router at standard prices and add six routers - #278

Open
yl231 wants to merge 1 commit into
mainfrom
leaderboard/reprice-2026-11-10
Open

yl231 wants to merge 1 commit into
mainfrom
leaderboard/reprice-2026-11-10

Conversation

@yl231

@yl231 yl231 commented Nov 10, 2026

Copy link
Copy Markdown
Contributor

For review; not merged. This is the next public ranking.

Ranks 1–28

Rank Router Arena Cost / 1K
1 (new) Liner Router 76.33 $0.34
1 → 2 Divyam 77.19 → 76.22 $0.66 → $1.10
4 → 3 R**2: Reasonometry Router 76.16 $0.48
3 → 4 Sqwish Router 76.34 → 76.13 $0.69 → $0.77
5 KT-ModelRouter 76.08 → 76.03 $0.34 → $0.35
6 LLM Router 75.77 $0.49
7 SapientAI Auto Router 75.58 → 75.55 $0.47 → $0.48
2 → 8 BizRouter 77.08 → 75.47 $1.00 → $2.00
9 (new) ATRouter 75.45 $0.98
10 (new) IQ Routing 75.19 $1.29
8 → 11 Cross-Router 75.25 → 74.97 $0.59 → $0.70
9 → 12 nadir-caliper 74.63 → 74.54 $0.25 → $0.27
10 → 13 vLLM‑SR 74.24 → 73.96 $0.67 → $0.79
11 → 14 BARouter 73.67 → 73.26 $0.54 → $0.70
12 → 15 NuRoute Artemis 73.63 → 73.02 $0.54 → $0.79
13 → 16 OrcaRouter‑Adaptive 72.48 → 72.47 $0.83
17 (new) Krusch Cascade Router 72.32 $0.50
14 → 18 Nadir Router 72.25 → 72.21 $0.67 → $0.69
15 → 19 Hybrid Router 71.20 → 70.96 $0.13 → $0.16
20 (new) f-router 70.95 $0.19
16 → 21 chuzom-solo-v32 70.64 → 70.56 $0.10 → $0.11
17 → 22 Azure-Model-Router 70.37 $0.75
19 → 23 AsiaInfo-Router 70.00 → 69.98 $2.86 → $2.88
21 → 24 Auto Router 69.80 → 69.90 $0.21 → $0.19
25 (new) Corealis Router 69.80 $3.14
20 → 26 cruq-router 69.89 → 69.52 $0.41 → $0.55
18 → 27 Weave Router 70.00 → 69.50 $3.21 → $3.80
22 → 28 Lynkr 67.97 → 67.96 $0.31

What moved most:

  • Gemini 3.7 and 3.8 Flash at Google's standard $1.50 / $7.50: BizRouter (more than 99% of its cost on Gemini 3.7 Flash), Divyam, IQ Routing and ATRouter.
  • GLM-5.3 Flash at Z.AI's $0.15 / $0.50: Corealis.
  • DeepSeek V4 Pro at DeepSeek's $1.32 / $3.96: Weave.

Not included yet:

🤖 Generated with Claude Code

…ters

- Price table: standard prices (#263), DeepSeek's own prices and
  OpenRouter's data of 2026-11-10 (#277).
- Six newly merged submissions, graded with main's evaluator on GitHub's
  runners (re-grade run 42906558950): Liner Router (#274), ATRouter (#273),
  IQ Routing (#209), Krusch Cascade Router (#169), f-router (#270),
  Corealis Router (#235, #268). README rows and manifest entries added;
  robustness from each router's robustness file.
- Every other router re-costed with scripts/pricing/reprice.py --all
  --write: only cost values change in their prediction files. The README
  table is re-ranked; external baselines keep their numbers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant