4.3. Reflection Checkpoint

Key Takeaways

• Evaluation replaces anecdotes with a repeatable battery: test cases pair inputs with success criteria, covering intent phrasings, edge cases, and routing — not just answer text.
• Match method to answer variability: Exact/Keyword for fixed content, Text Similarity then Compare Meaning as wording loosens, General Quality when no expected answer exists, Tool Use for the route, Custom for your own rubric — and combine them per case.
• Ship in solutions: custom solution with publisher prefix, unmanaged in dev, managed downstream, dependencies packaged together, environment variables (text/data source/secret) and connection references externalizing everything environment-specific.
• Pipelines make promotion procedural — stages, deploy-time values, extensions for approval gates and validations, service principals for unattended runs — and the loop is build → evaluate → deploy → evaluate → approve → prod.

Connecting Forward

The content phases are complete. Phase 5 (Exam Readiness) turns this material into exam-day tactics — strategy, quick-reference decision tables, and mixed practice questions — followed by the Glossary and Conclusion.

Self-Check Questions

• An agent passes every text-based evaluation yet answers from general knowledge instead of the mandated compliance index. Which method catches this, and what does the test case configure?
• Your manager asks why the prod agent can't just be "fixed quickly in place." Make the ALM argument in three sentences, using the words managed, environment variable, and pipeline.
• A nightly automated deployment to test must run with no human signed in, and prod promotion needs Legal's click. Name the two pipeline features involved.