[Sydney Lister (slister1001)]

Summary

Updates the azure-ai-projects rubric evaluator samples for the scoring contract already emitted in feature/azure-ai-projects/vnext. Changes are limited to samples, documentation, and focused sample tests; there are no generated SDK changes.

Sample changes

• Demonstrate manually authored Likert, Boolean, and safety rubric evaluators, including failure-safe cleanup.
• Explain Likert scoring and demonstrate safety generation from prompt and agent sources.
• Iterate on a generated rubric while preserving always-on dimensions and using all-or-none weights.
• Update the evaluations sample README and changelog.

Validation

• Nine focused offline sample tests passed, along with targeted mypy and Black checks.
• Live generation and iteration succeeded, including creation of both evaluator versions. Manual Likert and Boolean evaluator creation also succeeded. The created evaluator versions were deleted and verified absent.
• Live manual safety creation returned 400 because the deployed service expects permissible while the SDK sends allowed. The manual evaluation run cannot proceed past that step until the service is aligned.
• Safety and trace generation were not exercised live because an agent was unavailable.

Companion .NET PR: Azure/azure-sdk-for-net#64775

Howie Leung (@howieleung), could you review and sign off for the Ignite documentation deadline?

[azure-sdk-automation]

The sdk-api-review-ready label cannot create a review PR for this working PR because its branch is in a fork repository. Prefer a working branch in the registered upstream repository, or reach out to the ARH maintainers for help. Fork reviews are supported through the Azure SDK CLI (azsdk apireview create), but the fork requires its own ARH registration and the review PR is created in the fork rather than the upstream repository.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Sydney Lister (slister1001)]

Initial live run results against a test project:

• sample_rubric_evaluator_generation_basic.py: passed (generation job, evaluator, eval run, cleanup).
• sample_rubric_evaluator_generation_iterate.py: passed.
• sample_rubric_evaluator_generation_lifecycle.py: passed.
• sample_rubric_evaluator_generation_all_sources.py: passed; traces and safety jobs were skipped because no agent was configured.
• sample_rubric_evaluator_manual.py: Likert and Boolean evaluators were created successfully. Safety creation returned 400 because the service expected a Boolean permissible value.

The deployed service expects permissible, while the TypeSpec and SDK use allowed. The PR description summarizes the latest iteration and manual reruns.