4.1. Evaluating Agent Performance
💡 First Principle: You cannot chat your way to confidence. A conversation with the test pane samples one path through a probabilistic system; evaluation runs a repeatable battery — the same test cases, scored the same way, before and after every change — so quality becomes a measured trend instead of an anecdote.
The payoff is regression detection: when you swap a knowledge source, rewrite instructions, or add ten tools, the battery tells you what improved and what broke — before users do. Exam scenarios in this section usually hand you a response type and ask which evaluation approach fits; the method names and their scoring shapes are the required vocabulary.
⚠️ Common Misconception: Testing an agent means chatting with it in the test pane. The test pane debugs a single conversation; evaluation with test sets measures the agent. One is a screwdriver, the other is a dashboard — the exam expects you to know which scenario calls for which.
