About the guide
Useful evidence.
Plain English.
WTF Evals is an independent guide to evaluating AI applications and agents.
01 / Purpose
Start with a failure you care about.
The goal is to help you turn a vague concern about an AI system into an observable check and a decision. The guide emphasizes real tasks, explicit expectations, inspection of failures, and honest limits on what a result can establish.
The five-minute introduction gives you the basic model. The hands-on exercise runs checks on two stored patches and shows the evidence. Edit and export its JSON cases, then use them with the downloadable grader to run locally.
02 / How this is made
Follow the evidence.
The research foundation was assembled with AI assistance for research, drafting, coding, and checks. It records sources, distinguishes observed facts from interpretation, and includes small reproducible examples. AI assistance does not make a claim reliable by itself.
The initial research pass was completed on September 26, 2026. The public reading list links to selected original papers and documentation so you can examine the evidence and its limits. It is a selection from the research, not a claim that every topic or source has been covered.
Examples described as illustrative are teaching examples, not measurements of a named commercial product. A method that works for one task does not establish the quality of an entire AI system.
03 / Editorial independence
No vendor owns the answer.
Sources are chosen for the questions they help answer. Mentioning a tool or linking to a vendor is not an endorsement. Advertising, when present, is labeled and does not buy favorable conclusions or control the guide.
The guide and exercise are free to use. See the privacy policy for the current status of analytics and advertising.