[FEAT]: Add structured prompt template validation#120
Merged
nina-msft merged 5 commits intoJul 24, 2026
Conversation
Return an immutable PromptTemplate from the YAML loader, validate declared render parameters against both call arguments and the Jinja AST, and add focused unit coverage for metadata and parameter errors.
Replace manual prompt-template schema checks with a strict private Pydantic model, wrap validation failures in a domain exception, reject unknown fields and duplicate parameters, and declare Pydantic as a direct dependency.
Adds PromptTemplate.from_yaml(), private loading and compilation stages, generic uniqueness validation, explicit module exports, migrated consumers, and raw YAML coverage.
nina-msft
previously approved these changes
Jul 21, 2026
Align the structured template dataclass with repository conventions and prevent positional construction from exposing the private compiled-template field.
Expose one PromptTemplateDefinitionError for invalid YAML, schema, Jinja, encoding, and parameter-contract content while preserving native filesystem errors and underlying exception causes.
There was a problem hiding this comment.
Pull request overview
This PR introduces a structured PromptTemplate API for loading YAML-backed Jinja2 prompts with strict schema validation and render-time parameter enforcement, and migrates the LLM driver/evaluator to use it.
Changes:
- Added
PromptTemplatewith YAML loading via a strict private Pydantic schema plus render argument validation. - Updated
LLMDriverandLLMJudgeto load prompt templates viaPromptTemplate.from_yaml(). - Added unit tests covering YAML parsing, schema failures, parameter mismatch, and Jinja syntax errors; added
pydanticas a dependency.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Adds pydantic to the locked dependency set. |
| pyproject.toml | Declares pydantic as a runtime dependency. |
| rampart/common/templates.py | Implements PromptTemplate + strict YAML/schema/parameter validation and rendering contract. |
| rampart/drivers/llm.py | Switches system prompt loading to PromptTemplate.from_yaml(). |
| rampart/evaluators/llm_judge.py | Switches judge system prompt loading to PromptTemplate.from_yaml(). |
| tests/unit/common/test_templates.py | Adds comprehensive unit coverage for YAML/template validation and render argument enforcement. |
nina-msft
approved these changes
Jul 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a structured
PromptTemplatemodel with validated YAML loading and rendering.jinja2.Templateloading withPromptTemplate.from_yaml().LLMDriverandLLMJudgeto the structured template API.Breaking changes
load_prompt_template()has been removed in favor ofPromptTemplate.from_yaml().The loader now returns a
PromptTemplaterather than ajinja2.Template; callers using Jinja-specific APIs must migrate to the structured API.Checklist
pre-commit run --all-filespasses