Guides for building AI evals that hold up
Each guide answers one question up front, then shows the evidence, tradeoffs, and a setup you can copy. Claims link to official docs or papers.
How to Build a Production AI Agent Eval Suite in 2026 (CI/CD Gating Playbook)
A working playbook for agent evals that gate merges: what to test, how many trials to run, which assertions to trust, and how to keep CI from going flaky.
Published 4 min read
Promptfoo vs. Inspect AI vs. DeepEval: Which Eval Harness Should You Pick?
A plain comparison of three open-source eval harnesses: config style, sandboxing, metrics, red-teaming, and CI. Pick by your team's language and your riskiest failure.
Published 3 min read
The Complete Checklist for RAG Evaluation Metrics (Faithfulness, Recall & Hallucination Traps)
Test retrieval and generation separately, then add unanswerable questions on purpose. A checklist of RAG metrics, what each one catches, and where each one lies to you.
Published 3 min read
How to Evaluate LLM Security Against Prompt Injection & Dual-Control Attacks
Test direct and indirect prompt injection where your agent reads untrusted text, measure utility and attack success together, and cover cases where the user also acts.
Published 3 min read
Deterministic Assertions vs. LLM-as-a-Judge: Eliminating Flaky Evals in CI
Use deterministic checks wherever the answer can be verified, LLM judges only for what can't, and pin both. How to cut flaky eval failures without losing coverage.
Published 2 min read