Changed the Prompt? Rerun Everything

Relax. We'll talk through this in plain words — no textbook voice.

What you'll walk away with

Understand why you need to rerun after changing a prompt, so it stops feeling intimidating
Get hands-on practice trying it yourself
Learn to spot and sidestep guesswork and secrets with a smile

Take a breath. Let's break this chapter down like a conversation, not a textbook, so it's easy to follow.

Let's think about it this way for a sec

'Regression' isn't a scary word. It just means a card that passed yesterday fails today. Trimming a prompt shorter, adding context, adding a new rule — all of these can improve the happy path while quietly nudging the missing path toward guessing. So when you edit a prompt, don't just skip ahead — call your three cards back and rerun them. Like a chef who reworks the fish dish and never tastes the soup again — that's how things go wrong.

Let's connect it to everyday life

Log it: which card, which date, pass/fail, which prompt version. File name: eval-log.md. Your first log doesn't need to be a big table — three lines is fine. If an edit makes something fail, keep your version notes short enough that you can roll the prompt back. Don't have git set up yet? Just write 'v2 - added don't guess.'

After every change, call all three cards back
Log the result and the version
Don't skip checking the rest just because one thing improved

Let's try it together, hands-on

markdown

# eval-log.md
v2  added “don't guess”
- Cite handbook   PASS
- Missing due date PASS
- Trap predict    FAIL  (still guessed 2027)

You should see

You'll be able to rerun and log all three cards after every change.

Try it in 5 minutes

Write a short prompt. Run it against three cards. Add one rule, rerun, and note what changed.

A quick word of caution

Passing yesterday doesn't mean it automatically passes today. If you don't rerun it, you'll never know.

Easy traps

After editing, only rechecking the happy path
Not tracking versions, so you can't roll back

Now try it yourself

Write a short prompt. Run it against three cards. Add one rule, rerun, and note what changed.

You'll know it worked when: You'll be able to rerun and log all three cards after every change.