Skip to content

Fix LLM benchmark health-check timeouts and C# references - #6408

Open
bradleyshep wants to merge 11 commits into
masterfrom
bradley/llm-benchmark-health-check-fix
Open

bradleyshep wants to merge 11 commits into
masterfrom
bradley/llm-benchmark-health-check-fix

Conversation

@bradleyshep

Copy link
Copy Markdown
Contributor

Description of Changes

Fix blocking scoring and cleanup that can delay benchmark health checks and incorrectly report the server as unavailable.

  • Move scoring and cancellation cleanup off the async task.
  • Finish blocking work before stopping the server.
  • Upload server logs and resource diagnostics as CI artifacts.
  • Fix Register name collisions in two C# references and their prompts.
  • Add regression tests.

API and ABI breaking changes

No product API or ABI changes. Two C# benchmark tasks now use RegisterUser instead of Register.

Rollback safety impact

n/a

Expected complexity level and risk

2/5. Limited to the benchmark runner, diagnostics, and C# reference tasks. No production server behavior changes.

Testing

  • All 69 local unit tests pass.
  • Regression tests fail with the old blocking behavior and pass with the fix.
  • Both C# fixes compile against the failed run's SDK.
  • Clippy, formatting, and diff checks pass.
  • Validate on Linux CI.
  • Run a small benchmark without uploads and inspect diagnostics.

Benchmarks have not been rerun. The weekly workflow remains disabled.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant