[XM-Dong]

This PR updates DT-Router following #241. It replaces the previous draft-and-escalate pipeline with query-only JEV routing, revises the candidate model pool, and records routing usage for each query.
How routing works
For each request, DT-Router calls typesafe/jev-1.13 once to assign labels to the query across several dimensions, including task type, domain, difficulty, and likely error types. DT-Router combines these signals to select a model from the candidate pool, and the selected model generates the final answer directly. Each record’s router_usage field includes the JEV call and its input and output token counts.

Why we changed the pipeline
We switched to a single JEV call to reduce the overhead of the previous LLM routing step. Removing preliminary answer generation also eliminates the cost of drafts that would be discarded after escalation.
We reassessed the candidate pool after RouterArena changed LingDT’s benchmark price to its public base-model rate. Under the updated price table, the previous pool and draft-first strategy had a different cost–quality trade-off.
We also reduced the Gemini thinking levels. These settings are recorded in the full prediction file.

[XM-Dong]

/evaluate

[github-actions]

Router Evaluation Results

Router: DT-Router
Dataset Split: full
Evaluator: main @ 18383b4

RouterArena Metrics

Metric	Value
RouterArena Score	0.7684
Accuracy	80.90%
Total Cost	$6.937199
Avg Cost per Query	$0.000826
Avg Cost per 1K Queries	$0.8259
Number of Queries	8400
Abnormal Entries	2
Robustness Score	0.7881

⚠️ 2 of 8400 queries (0.0%) had no valid generation (inference failed / empty answer) and were scored as incorrect (0). These queries still count toward the denominator, so accuracy and cost reflect the full query set. Please regenerate predictions for these queries and resubmit for a complete evaluation.

---

Evaluation completed by RouterArena automated workflow

[XM-Dong]

Hi @yl231 , thank you for your earlier guidance on #241. The /evaluate run for this follow-up PR has completed, and the updated submission records the JEV call and token usage for each query in router_usage.
The two empty responses flagged by the evaluator were due to safety filtering. We retained both entries as failed generations, and both were counted as incorrect. Could you please take a look when you have a chance? Thank you!