AI Evals
A 16-lesson, step-by-step tutorial on the mindset of checking whether your AI does the right thing on a given question — not whether the answer just sounds nice — using short, repeatable test cards. Goes all the way through building a Study Eval Pack.
18 Lessons · 180 min · Learning Platform Editorial Team
What you need on the desk
- A modern browser
- A notebook or text editor
- Optional: an AI chat to try your cards on
Know this much before you start
- You've opened an AI chat and asked it a question before
- Some exposure to Prompt Engineering, RAG Chatbots, or Context Engineering helps, but isn't required
- Optional: your own study bot or a handful of prompt cards
By the end, you can
- Explain an eval as a short, rerunnable test card
- Tell a vibe check apart from an eval
- Score with must / must-not and Pass/Fail
- Write a happy, missing, and trap card
- Rerun after every fix, keep secrets out, and score coverage honestly
- Finish a working Study Eval Pack