docs: add proposed server-side policy evaluation tutorial

[pbeckham]

What

Documentation-driven development for the server-side Kosli Evaluate project (kosli-dev/server#6267). Adds a tutorial describing the proposed V1 kosli evaluate policy flow, published so it can be shared internally and with design-partner customers for feedback before the feature is built.

This is a design preview — nothing in it is runnable today. The page is marked as such throughout.

Scope (this draft)

Core happy path for a governance platform engineer:

• Author a policy locally — Rego with name/purpose/input-context schema (allow mandatory).
• Ship a versioned policy into Kosli — CLI/API, identifier + version.
• Evaluate and record a decision — kosli evaluate policy runs server-side and records a decision against a control; synchronous.
• Trace and replicate — evaluation context pins trail moments, attestation_ids, policy version, params, timestamp; includes the T / T+1 / T+2 temporal-correctness case.
• Evaluate a local policy file — decision still recorded server-side.

An explicit out-of-scope-for-V1 section keeps reviewer feedback on target (managing policies, testing/validation, reporting/audit views, unhappy-path inspection, composable policies, non-trail targets, scheduled evals, risk measurement).

Design notes surfaced for feedback

• Command surface verified against the real kosli attest decision and kosli evaluate trail flags. --compliant and evidence (--attachments/--user-data) are intentionally not flags — the server derives compliance and captures the policy/report/violations as output.
• The publish command reuses kosli create policy, which collides with the existing environment-policy command — flagged inline as an open question.
• Rego input shape is borrowed from the client-side tutorial and may differ server-side.

Draft because

Content describes proposed behavior and will change as the design firms up; opening as draft to gather review before it's merged/published.

🤖 Generated with Claude Code

[claude]

Claude finished @pbeckham's task in 4m 18s —— View job

---

Review: tutorials/evaluate_policies_server_side.mdx and related changes

• Read CLAUDE.md and doc-review skill
• Identify changed docs files
• Review tutorial page content
• Review navigation config
• Check for broken links and relative paths
• Post findings

---

Navigation and structure

New page registered in config/navigation.json — confirmed. tutorials/evaluate_policies_server_side is added to the "Evaluation" group alongside evaluate_trails_with_opa. No issues.

All internal links resolve — verified all 14 root-relative links (/getting_started/install, /tutorials/working_with_controls#creating-a-control, etc.) against actual files. All exist. No relative links (../) found anywhere.

working_with_controls.mdx changes — two small fixes: a broken prerequisite link (/getting_started/service-accounts → /getting_started/authenticating_to_kosli) and improved archive/unarchive wording. Both are clean improvements.

---

Findings

Suggestion — Schema/capabilities URLs reference files that don't exist yet

evaluate_policies_server_side.mdx:518-519, 612

Three URLs are presented as published JSON Schema assets readers can download:

• https://docs.kosli.com/schemas/evaluation-input/trail/v1.json
• https://docs.kosli.com/schemas/evaluation-input/trails/v1.json
• https://docs.kosli.com/rego/capabilities/v1.json

None exist in the schemas/ directory or anywhere in this repo. The page shows curl commands that download them (lines 542, 612), which will 404 today. Since the page already carries a prominent "not yet available" banner, this is fine for the design preview, but worth noting for when this moves out of draft — the schema files need to be created or hosted before the feature ships.

Suggestion — Diataxis: page is a hybrid tutorial/reference

The page is filed under "Tutorials" and follows a teach-by-doing structure in the first half (Steps component, progressive examples). The second half (The Rego authoring contract, Manage policies, View the decisions) shifts into reference material. This is acknowledged in the intro's <CardGroup> which routes different audiences to different sections, and the PR description calls it out. For a design preview gathering feedback, the hybrid form works well — it gives every reviewer one page to read. When the feature ships, splitting the authoring contract and management sections into separate reference pages would align better with Diataxis.

Suggestion — --context syntax has no error examples

The --context trail=<trail_id> and --context trail=<flow_id>/<trail_id> syntax is documented, but there's no mention of what happens if the format is wrong (e.g., missing the trail= prefix, or naming a nonexistent trail). Since --context is now required and carries unusual typed-reference syntax, a brief note on the error the CLI returns would help readers debug a mistyped flag.

---

Overall assessment

This is a thorough, well-structured design preview for a significant feature. The 40-commit history shows extensive iteration incorporating review feedback from multiple reviewers — the page is in strong shape:

• Frontmatter — present and correct, description is within 160 chars.
• Writing style — active voice, American spelling, correct product terminology ("Kosli", "attest", "audit trail"). The AI-generation tells from tooky's review (rhetorical flourishes, chatty tone) have been cleaned up.
• Components — good use of <Steps>, <Accordion>, <Info>/<Warning>/<Tip>/<Note>, <CardGroup>, <Frame>. Callouts are used purposefully, not excessively.
• Images — all four policy-*.png files are real UI mockups (120-220KB), replacing the earlier 1×1 placeholders.
• Links — all root-relative, all resolve. No relative links. Anchor links to headings within working_with_controls.mdx are correct.
• No TODOs remaining — both {/* TODO */} comments flagged in prior reviews have been removed.
• No --dry-run — dropped per the latest commit.

The open review threads from tooky and AlexKantor87 raise substantive design questions (timestamps, SARIF handling, policy bundles, roles) that are mostly about the feature design rather than the docs quality — the right kind of feedback for a design preview.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
kosli	🟢 Ready	View Preview	Jul 23, 2026, 1:43 PM

[tooky]

Thoughts

• “Running evaluation inside Kosli changes what a decision means”
  • Why does it change what it means?
• I’m not sure why it keeps talking about “a core loop”?
• Probably need a new role or two for control and policy management
• I’m not sure if we should assume --flow is defaulted into context on kosli evaluate policy
  • The note “a command that names its source and its destination is easier to read back months later than one relying on a default.” Is describing a potential footgun??
• What’s the behaviour without sync? How can the customer check for status of evaluation that is async?
• “Be deliberate about which flows may satisfy a production control.” - wondering if this feels a bit alarmist to include at this stage. I think maybe there’s an FAQ section we could add?
• “A decision carries two timestamps. “ Really keen that we make sure this has been seen widely by the team. I might need to see some examples to help understand what makes sense.
  • Maybe it would need stating explicitly that request time would be used as the query time for the trail data
  • Alternative we could consider is that if an evaluation request time doesn’t return the latest moment we invalidate the request somehow?
• Did you consider any alternatives for trail and flow “co-ordinates” that the <flow>/<trail> format (e.g. trail=build-ci/build-41)?
• “summarize it into JSON when you make the attestation and pass it as --user-data” <- this may depend on attestation type, its probably part of the main attestation_data for a custom attestation type schema?
• Have been experimenting with some different policy output ideas for richer feedback, we may be able to provide “libarary” rego they can choose to use along with their rego, maybe we should think about how this would work now. e.g see Claude conversation:
• Probably not v1 - but we might have to support multiple Engine and Rego versions? For backwards compatibility purposes? Perhaps the policy should be published with a version it is tested against? Compatible with?
• --dry-run in Kosli cli usually does not hit the server? Should we use a different flag
  • Can we evaluate with --assert but no decision recorded
• Evaluate a local policy file – wondering about recording the sha256 of the policy?
• Thinking about the “Review a policy change” section and whether we should help people create trails for their policy management processes
• I really like that we’ve gone straight for policy bundles - what does that require from us for validating the bundle on upload?
  • Does the “Re-run a decision offline” section need to be updated to account for policy bundles?
  • Out of scope says: Composable policies — composing many authored policies into one versioned policy? Is that different from a policy bundle

AI generation tells

• Opening para has a few rhetorical flourishes (e.g. “one it can vouch for”)
• “The whole loop in two commands”, “Publish a policy, then evaluate it. That is the feature:”
• Some of the language feels a bit chatty “worth reading before you send feedback, so you don’t spend it on something already deferred.”
• “Note that --context is doing real work here:”