Prompt Evaluation Framework: Metrics Before You Ship

Prompt evaluation is how you decide whether an instruction string is good enough to reach users. You fix a small set of inputs, write pass rules a stranger can score, and run the candidate prompt against that sheet before you flip traffic. Vibes and three lucky demos do not count. You leave with a practical prompt evaluation framework: which metrics to pick, how to write rubrics that survive handoff, a weekly workflow from draft to ship gate, failure modes that fake a green score, and 2026 notes for GPT-5.6 Sol, Claude Opus 5 / Sonnet 5, and Gemini 3.1 Pro / 3.5 Flash. Soft sell later: PromptMake /text helps you draft a clean baseline before you score it.

What prompt evaluation is (and who needs it)

Prompt evaluation means you measure model behavior on a fixed contract. The contract names the task, the output shape, and the fail cases. Each eval row pairs an input with a pass rule. You change the prompt, re-run the rows, and compare scores to the last production baseline. If the score drops past a threshold you chose in calm times, you keep the candidate off the default path.

Teams that skip prompt evaluation ship on confidence. One engineer likes the new tone. A stakeholder likes a single happy path. Production then fails on empty tickets, conflicting facts, or out-of-scope asks. Support opens a queue of surprises. Nobody can prove the old wording was safer because nobody scored it. A prompt evaluation sheet turns that argument into a table: old score, new score, delta.

This guide is for engineers, applied AI owners, and PMs who already call an LLM in a product. If you only chat for personal drafts, a checklist in your head may be enough. If two people edit the same system prompt, or if a wrong answer costs money or trust, you need written rubrics. Pair this framework with prompt versioning so every score attaches to a named release. Pair it with automatic prompt optimization only after you trust the metric; search without a honest score optimizes the wrong thing.

The metrics that matter before you ship

Pick metrics that map to user harm or product promise. A support router cares about label accuracy and wrong-escalation rate. A RAG answer bot cares about grounded citations and refuse-when-missing. A JSON extractor cares about schema validity and field completeness. Fancy research scores (BLEU, ROUGE, open-ended LLM beauty contests) belong only when they predict a business outcome you can name. For most product prompts, start with four buckets: task correctness, format obedience, safety or scope refusals, and cost or latency budgets.

Score what you can check without debate. Exact match for short labels. Set membership for enums. JSON Schema validation for structured output. Substring or regex for required disclaimers. Presence of citation ids that appear in the retrieved notes. Human or LLM judges enter when the answer is free prose and you cannot reduce the rule further. Pin the judge prompt, temperature, and model id the same way you pin the candidate under test. Otherwise the judge drifts and your trend line lies.

Write thresholds in the runbook before the first scary PR. Example: schema valid on ≥98% of rows, citation precision ≥0.90 on RAG rows, out-of-scope refuse rate ≥95%, p95 latency under your SLO. Soft gates produce debates. Hard numbers produce merges or written waivers. Review thresholds when the product promise changes; do not move them mid-incident to make a red job green.

Rubric rows you can hand to a stranger

Each row needs five fields: id, input (and retrieval context if RAG), expected behavior in one sentence, pass rule a scorer can apply in under a minute, and a note for when the row entered the set. Example pass rule: "Category is billing AND confidence is a number in [0,1] AND rationale is under 40 words." Example refuse rule: "Output starts with REFUSE and names the missing policy; no invented refund amount." Vague goals like "be helpful" fail as pass rules because two reviewers will disagree.

Keep rows short. Long novel inputs hide which clause of the prompt failed. Prefer one failure mode per row: empty body, contradictory facts, jailbreak-style ignore instruction, multilingual input, tool error string. Tag rows by risk (P0 safety, P1 core path, P2 polish) so a drop on P0 blocks ship even if average accuracy looks fine. Average scores hide concentrated risk.

Deterministic checks vs LLM-as-judge

Use deterministic checks whenever the product contract is discrete. Classifiers, routers, extractors, and tool-argument builders rarely need a judge. Free-form summaries, coaching replies, and tone-sensitive customer copy often do. When you use a judge, score one axis per call (faithfulness, tone, completeness) with a 1-5 scale and clear anchors for 1, 3, and 5. Aggregate with a floor: one P0 faithfulness fail fails the release even if tone averages 4.5.

Calibrate judges on ten human-labeled rows before you trust them in CI. If the judge disagrees with humans on more than one or two, fix the judge prompt or drop that axis. Prefer cheaper Flash-class models for high-volume judging once calibration holds; reserve Opus-class or Sol for hard disagreements you sample weekly. Log judge version next to candidate prompt version so postmortems stay honest.

A step-by-step prompt evaluation workflow

Build the loop once, then reuse it on every prompt PR. The shape is the same whether you edit a support macro or a multi-tool agent brief: freeze a draft, attach a sheet, score baseline and candidate, gate the release, watch production for new failure modes, and fold those modes back into the sheet. Skip a step and you recreate vibe-driven shipping under a prettier name.

Start from production pain, not from a blank spreadsheet. Pull the last twenty tickets, failed tool calls, or human escalations. Each pain point becomes one or two rows. Add five happy paths so you catch accidental regressions on the core job. Cap the first sheet at thirty to fifty rows. Bigger sets help later; a small set you run every PR beats a thousand-row graveyard nobody executes.

Draft the candidate prompt in a branch. Keep the change small enough that a score drop has one likely cause. If you must swap model and rewrite instructions in one release, say so in the PR and expect a longer canary. Run the suite against the model id you will serve. Staging against a different model trains you to trust the wrong green check.

Build the golden set

Store the golden set in Git next to the prompt files. Use CSV, JSONL, or YAML; pick one format your CI already parses. Include the prompt version hash and model id in every run artifact. Give the set an owner: product adds rows for new features, support adds rows for ticket gaps, applied AI retires rows when the product promise changes. Dead rows that encode last year's policy will block good releases or force silent threshold hacks.

Split a holdout if you use automatic prompt optimization or heavy few-shot search. The optimizer may train on part of the set; the holdout stays sacred for ship decisions. For hand edits without search, a single gated set is enough if you refuse to cherry-pick which rows count after you see the scores. Cherry-picking after the fact is how teams fake improvement.

Score, gate, and canary

Score the current production prompt first. That baseline is your comparison. Score the candidate next. Publish a table in the PR: metric, baseline, candidate, delta, gate. Fail CI when P0 metrics drop or when any hard threshold breaks. Allow a product owner waiver in writing for known tradeoffs (for example, a one-point accuracy dip that cuts latency in half on a non-safety path).

After CI green, canary the new prompt version on a small slice of traffic. Watch refusal rate, schema error rate, human escalation rate, and user complaints for a fixed window. Promote when the window stays flat. Roll back with one config flip to the prior prompt version. Practice that flip in a game day so on-call does not learn the load path during an incident. Nightly jobs against the live model id catch vendor drift when your Git tree did not move.

Practical rubric templates you can copy

Templates below stay short on purpose. Adapt the pass rules to your product language. Keep the structure: risk tag, input sketch, pass rule. You can paste these into a sheet today and fill real payloads after standup.

Classifier / router template. Risk P1. Input: messy ticket text. Pass: predicted label matches gold label; optional confidence within ±0.15 of a calibrated band; no second label in the rationale. Add P0 rows for "ignore previous instructions" and empty body: pass means a safe default label or an explicit NEED_MORE_INFO, never a random high-stakes category.

RAG answer template. Risk P0 for medical, legal, or finance claims. Input: question plus retrieved chunks. Pass: every factual sentence cites a chunk id that exists; if chunks lack the answer, output is REFUSE with a short reason; no URLs or statute numbers absent from chunks. Score citation precision (cited ids that exist / all citations) and refuse recall on known-missing rows.

Tool-calling template. Risk P0 when tools mutate state. Input: user goal plus tool catalog. Pass: chosen tool name is in the allow list; arguments validate against JSON Schema; stop condition fires when the goal is met or when the tool returns an error the prompt must surface. Fail rows where the model invents a tool or retries a failed charge without a human confirm step.

Tone and policy rubrics without endless debate

Tone fights eat meetings. Reduce tone to observables. Example pass rule: "No sarcasm markers; opens with the user's goal in the first sentence; offers at most one clarifying question." Example brand rule: "Uses product name Acme once; avoids competitor names; reading level near eighth grade." Score with a checklist, not "sounds on brand." When legal owns refusal copy, the pass rule is exact string match or approved paraphrase list, not a vibe judge.

Policy rubrics need severity. A soft style miss can warn. A disallowed advice miss must fail the release. Keep a separate P0 policy pack that never averages with tone. Mix policy rows into every suite so a "clarity rewrite" cannot ship a quieter refusal. If your product spans locales, add at least two non-English rows early; many prompts that look green in English leak policy in other languages.

How many rows and how often to refresh

Thirty solid rows beat three hundred stale ones. Grow when a production incident reveals a gap you cannot map to an existing row. Refresh when you change model family, tool schema, or product policy. Schedule a monthly prune: delete rows for retired features, merge duplicates, and re-label anything the team still argues about. Annotate each change in the set's CHANGELOG so score jumps after a sheet edit do not look like model miracles.

Run the full suite on every prompt PR and on a nightly schedule against production pins. Sample human review on five random rows each week even when CI is green. Humans catch contract drift the sheet missed. Log time-to-score so the suite stays under your patience budget; if CI takes forty minutes, people will skip it. Split P0 smoke (two minutes) from full pack (longer) if you must, but never ship on smoke alone.

Common mistakes in prompt evaluation

Scoring only happy paths. The model looks brilliant and then fails the first empty input. Seed the sheet with empties, contradictions, and out-of-scope asks from day one.

Changing the sheet and the prompt in the same PR without saying so. You cannot tell whether the product improved or the test got easier. Split PRs or label the sheet delta in the release note.

Using an unpinned LLM judge as the only gate. Judge drift masquerades as prompt regression. Pin judge assets and keep a deterministic subset that must stay green.

Reporting average accuracy while P0 refuse rows collapse. Lead with risk-weighted metrics. A 2% average gain that cuts refuse recall in half is a failed release.

Testing on a different model than you serve. A prompt that wins on Claude Opus 5 can wobble on Gemini 3.5 Flash. Score the serve id.

Treating one offline green run as forever. Vendor model updates move behavior under a stable model name. Nightly evals and canaries catch silent drift.

Optimizing for the judge's taste instead of the user contract. If the judge loves long answers, the prompt will grow padding. Align judge anchors with product length and cite rules.

Model and tooling notes for 2026

As of mid-2026, treat a model id change as a prompt evaluation event even when the template string stays fixed. GPT-5.6 Sol, Claude Opus 5 / Fable 5 / Sonnet 5 / Haiku 4.5, and Gemini 3.1 Pro / 3.5 Flash share contracts with different failure modes. Re-run the full golden set before you flip traffic. Keep the prior model pin ready for rollback the same way you keep the prior prompt version.

Reasoning-class models want short contracts: goal, constraints, output schema. Drop old "think step by step" scaffolding from candidates aimed at those models; it adds tokens and can confuse your latency budget without raising task scores. Fast chat and Flash tiers need sharper format examples and clearer refuse lines. Maintain sibling prompts per tier when dual-running, and score each sibling on the same sheet.

Prompt caching rewards a stable system prefix. Evaluation still needs to re-score when you bump that prefix, because cache hits do not prove quality. Budget a short cold-cache cost after large prompt edits. For tool-heavy agents, include tool-error rows; models differ on whether they surface errors or invent success. Your sheet should punish invention the same across vendors.

Draft faster, then evaluate (PromptMake /text)

Use PromptMake /text when you need a clean first system prompt, refuse ladder, or few-shot pack from a messy brief. Pick the model family you will call in production so the draft matches that dialect. Paste the result into your branch, attach the golden set, and score before anyone debates adjectives in Slack. Soft CTA: the tool speeds the blank page; prompt evaluation decides whether the page ships.

Skip generators when legal already locked exact copy, or when you only need a typo PATCH on a tagged release. Do not chat-iterate the live production string after the first score. Iterate on a branch, gate on metrics, then promote. Next steps this week: write twenty rows from real tickets, define four thresholds, wire a CI job that prints baseline vs candidate, and refuse to merge on a red P0 pack.

FAQ

What is prompt evaluation in plain terms?

Prompt evaluation means you run a candidate instruction set on a fixed list of inputs and score the outputs with written pass rules. You compare those scores to the current production baseline before you ship. The goal is a yes/no release decision backed by metrics, not a vibe check from one demo.

How is prompt evaluation different from automatic prompt optimization?

Prompt evaluation measures a prompt you already wrote. Automatic prompt optimization searches for better instructions or demos against a metric. You still need evaluation either way: APO needs a honest score to search, and hand edits need the same score to ship. Build the sheet first; automate later if the task is worth the token spend.

Which metrics should I start with for prompt evaluation?

Start with task correctness on core paths, format or schema obedience, refusal quality on out-of-scope rows, and a latency or cost budget. Add citation precision for RAG. Add tool-schema validity for agents. Expand only when a metric maps to a failure users feel. Skip academic leaderboard scores that do not predict your product outcomes.

How large should a golden set be?

Thirty to fifty well-owned rows work for a first production gate. Grow when incidents reveal gaps. Prefer risk coverage over raw count. A huge set you never run in CI is weaker than a small set that blocks every prompt PR. Refresh monthly and prune retired product rules.

Do I need an LLM-as-judge for every prompt evaluation?

No. Prefer deterministic checks for labels, JSON, citations, and banned phrases. Use a pinned judge for free-form quality axes you cannot reduce further, and calibrate it against humans. Many teams run deterministic P0 gates in CI and sample judged prose weekly.

How do I start if our team has no evals yet?

Pick one live prompt. Collect twenty failures from support or logs. Write pass rules for those cases plus five happy paths. Score the current prompt and save the numbers. Draft improvements with PromptMake /text or by hand, re-score, and block merge on drops. Free tier on PromptMake covers light drafting practice while you build the sheet in Git.

Which model should I score against in 2026?

Score against the exact model id you will serve, including GPT-5.6 Sol, Claude Sonnet 5 or Opus 5, or Gemini 3.5 Flash / 3.1 Pro as applicable. If you dual-run tiers, score each prompt sibling on the same golden set. Re-run after vendor updates even when your prompt text did not change.