3.4. Monitoring and Operationalizing (GenAIOps)
💡 First Principle: Operating a generative app adds AI-specific signals on top of ordinary app monitoring — token usage and latency (cost/performance), quality/groundedness scores (is it still answering well?), and traces across tool calls (what actually happened inside a run). GenAIOps is the lifecycle that closes the loop from deploy → monitor → evaluate → improve.
Why care: the exam's production framing means "how do you know it's healthy in production?" recurs across domains. Standard infra metrics don't tell you the model started hallucinating or that an agent is looping through tools — AI observability does. This connects directly to Phase 2's evaluators, now applied continuously rather than once.
⚠️ Common Misconception: "Monitoring an AI app is the same as monitoring any web app." Infra metrics (CPU, request rate, errors) are necessary but blind to AI-specific failures — quality drift, ungrounded answers, token blowups, and tool-call misbehavior need AI observability layered on top.
