Skip to content

docs: document evaluating a pinned rego policy version - #486

Merged
mbevc1 merged 1 commit into
mainfrom
autopilot/b0663185
Nov 3, 2026
Merged

mbevc1 merged 1 commit into
mainfrom
autopilot/b0663185

Conversation

@mintlify

@mintlify mintlify Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Shows how to pin a published policy version with --identifier <id>@v<n> and adds --server-side for evaluate trail/trails (beta: controls page already carries the beta banner; pinning is on prod and in CLI v2.52.0).

Based on evaluate a pinned policy version with --identifier @v, Evaluate a pinned version of a published rego policy and unhide server-side evaluation commands and --server-side flag.

@mintlify
mintlify Bot requested a review from a team as a code owner November 3, 2026 09:39
@mintlify
mintlify Bot requested a review from dangrondahl November 3, 2026 09:39
@mintlify

mintlify Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor Author

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
kosli 🟢 Ready View Preview Nov 3, 2026, 9:39 AM

@claude

claude Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @mintlify[bot]'s task in 1m 7s —— View job


Doc review: PR #486

  • Read the diff and both changed files at the branch head
  • Check the prose against client_reference/kosli_evaluate_{policy,trail,trails}.md
  • Sweep the site for wording that pinning makes stale
  • Post findings

policy-reference/rego_policy.mdx

Improvement: the version-status table still says evaluations always run the newest version

  • Location: policy-reference/rego_policy.mdx:148
  • Issue: The valid row says "Evaluations run the newest valid version." Once --identifier <id>@v<n> exists, that's only the default. This is the policy reference page, and the PR touches it, but pinning is documented only in the tutorial. Someone reading the reference won't learn that pinning exists.
  • Recommendation: Change it to e.g. "The version can be evaluated. Evaluations run the newest valid version unless one is pinned with --identifier <id>@v<n>; see Pinning a policy version." Fix this →

Improvement: "the same policy file" contradicts the list that follows it (:407, see the inline comment). The next bullets say local and server input have different shapes and that a policy can compile locally but fail on the server. So adding --server-side to a working local check may change the verdict.

Upstream: does not block this merge

  • kosli-dev/cli → cmd/kosli/evaluatePolicy.go: caret escaping bug in the --identifier text. client_reference/kosli_evaluate_policy.md:27 renders Append `@v`n and `:83` renders identifier@vn ``. The ^ markers around `n` inside `@v^n^` became broken nested backticks. The tutorial in this PR spells it correctly as `@v`. Fix the description, or the generator's `^` handling, upstream.

The links resolve: #pinning-a-policy-version matches the new heading. I couldn't run gh pr checks or read the kosli-dev/cli source from this job, so I haven't confirmed the link-rot result or the server-side input shape.

2 Improvements, 0 Critical, 1 upstream. Safe to merge once the two wording fixes are in.

@mbevc1
mbevc1 merged commit 4b603e8 into main Nov 3, 2026
6 checks passed
@mbevc1
mbevc1 deleted the autopilot/b0663185 branch November 3, 2026 09:40

[`kosli evaluate trail`](/client_reference/kosli_evaluate_trail) and [`kosli evaluate trails`](/client_reference/kosli_evaluate_trails) fetch trail data from Kosli and evaluate a local policy file in the CLI, passed with `--policy`. A local evaluation records nothing: it cannot record a decision against a control. To try a policy on the server's input before publishing it, use `kosli evaluate policy --path` instead.

To evaluate the same policy file on the Kosli server instead, add `--server-side` (beta) to either command. Kosli builds the input from the trails you name, so the CLI reads no trail data. The flag can't be combined with `--attestations`, `--show-input` or `--output-rule`, and a server-side evaluation takes at most 100 trails.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvement: "the same policy file" reads as a promise that a local policy works unchanged with --server-side. The next list says otherwise: local input is the trails-API shape, "not a trail moment", and fields differ (name vs trail_name, flow.name vs flow_name, see the table under Local input data). Also, a policy that passes locally "can still fail to compile on the server". A reader who adds --server-side to a working local check could get a different verdict or a compile failure.

Suggest dropping "the same" and saying which input the server builds, e.g.:

Suggested change
To evaluate the same policy file on the Kosli server instead, add `--server-side` (beta) to either command. Kosli builds the input from the trails you name, so the CLI reads no trail data. The flag can't be combined with `--attestations`, `--show-input` or `--output-rule`, and a server-side evaluation takes at most 100 trails.
To evaluate a policy file on the Kosli server instead, add `--server-side` (beta) to either command. Kosli builds the input from the trails you name as [trail moments](#trail-moment), so the CLI reads no trail data, and a policy written for local input may need its field paths changed. The flag can't be combined with `--attestations`, `--show-input` or `--output-rule`, and a server-side evaluation takes at most 100 trails.

(I couldn't read the CLI source from this job to confirm the server-side input shape for evaluate trail, so check this against kosli-dev/cli#1318 before you accept it.)

dangrondahl pushed a commit that referenced this pull request Nov 3, 2026
Applies the Claude review findings left on #485 and #486: the skip-tags
PUT example now keeps dry run explicit, the Lambda template warning
matches its example, and the rego reference covers version pinning and
server-side input shape.

Based on [#485](#485) and
[#486](#486).

---------

Co-authored-by: mintlify[bot] <109931778+mintlify[bot]@users.noreply.github.com>

This branch was successfully deployed

1 active deployment
staging — 27a929cf Deployed Nov 3, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants