Skip to content
Eval Suites50

Deterministic Assertions vs. LLM-as-a-Judge: Eliminating Flaky Evals in CI

Published Sources checked 2 min read · By the Eval Suites editors

At-a-glance answer (TL;DR)

Most flaky evals come from using an LLM judge where a deterministic check would do. If the output has a shape, a value, or a side effect you can verify, such as JSON schema, exact tool arguments, regex, or database state, check it in code. Save LLM judges for qualities like tone or helpfulness, pin the judge model and rubric, and calibrate it against human labels before it can fail a build.

The rule: verify in code first

Write a deterministic assertion whenever the correct output can be checked by a program. It's faster, it's free, and it gives the same verdict every run.

Google's IFEval benchmark shows how far this goes. It focuses on "verifiable instructions" such as word counts, required keywords, and output formats, with 25 instruction types checked by code instead of a judge.[1] Many production requirements fit the same pattern.

Which checks can be deterministic?

Choosing a check type by requirement
RequirementDeterministic checkNeeds a judge?
Valid JSON or schemais-json / JSON Schema validationNo
Correct tool and argumentsExact or structural match on the tool callNo
Contains required disclosurecontains / regexNo
Final state is correctQuery the database or file system after the runNo
Length or format limitsCode check, as in IFEvalNo
Answer is factually rightExact match when there's a short gold answerSometimes
Tone, empathy, helpfulnessNone reliableYes

Promptfoo exposes most of these as built-in assertion types,[2] and Inspect AI ships scorers for exact match, includes, pattern, and model-graded grading.[3]

What goes wrong with LLM-as-a-judge

LLM judges have measurable biases. The MT-Bench paper found strong judges agreed with human preferences over 80% of the time, about the level humans agree with each other, but also documented position bias, verbosity bias, and self-enhancement bias.[4]

  • Position bias: in pairwise grading, the judge favors one slot. Swap the order and grade twice.
  • Verbosity bias: longer answers score higher. Put length expectations in the rubric.
  • Self-enhancement: a judge may prefer outputs from its own model family. Use a different family when you can.
  • Drift: an unpinned judge model changes under you. Pin the exact version.

How to make an LLM judge safe for CI

  1. Write a rubric with concrete pass and fail examples and keep it in version control.
  2. Label 50 to 100 outputs by hand and measure how often the judge agrees.
  3. Pin the judge model and set temperature to 0 where the provider allows it.
  4. Gate on the judge only after agreement is acceptable to you, and re-check after any judge change.
  5. Report scores with standard errors so normal variance doesn't fail builds.[5]
Deterministic first, judge last (Promptfoo)
assert:
  - type: is-json
  - type: javascript
    value: output.length < 1200
  - type: llm-rubric
    value: Apologizes once, gives the refund timeline, no blame on the customer
    provider: openai:gpt-4.1   # pinned judge

Promptfoo documents llm-rubric and other model-graded types, including how to override the grading provider.[6]

Frequently asked questions

Suites mentioned in this guide

Related guides

Sources

  1. 1. Instruction-Following Evaluation for Large Language Models · arXiv:2311.07911 (Google) · Paper · Sources checked
  2. 2. Promptfoo: Assertions and metrics · Promptfoo · Official docs · Sources checked
  3. 3. Inspect AI: Scorers · UK AI Security Institute · Official docs · Sources checked
  4. 4. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena · arXiv:2306.05685 · Paper · Sources checked
  5. 5. Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations · arXiv:2411.00640 (Anthropic) · Paper · Sources checked
  6. 6. Promptfoo: Model-graded metrics · Promptfoo · Official docs · Sources checked