4.3.1. Risk Controls and Monitoring in Production

💡 First Principle: Pre-launch testing proves a system was acceptable on the data you tested; production monitoring proves it is still acceptable on the data it actually sees. Controls in production must detect problems early, limit their impact, and enable rapid response, because conditions, users, and the underlying models all change after launch.

Production risk controls a strategist should require:

• Performance and outcome monitoring: model quality metrics alongside business outcomes and guardrail metrics (complaints, escalations, overrides).
• Responsible AI monitoring: fairness metrics across groups, hallucination or grounding rates, harmful content detections.
• Thresholds and alerts: defined limits that trigger investigation or automatic fallback.
• Audit logging: records of inputs, outputs, decisions, and agent actions sufficient to investigate incidents and satisfy regulators.
• Human feedback channels: easy ways for users and customers to flag problems, routed to owners.
• Kill switch and fallback: the ability to disable an AI feature quickly and revert to a manual or rules-based process (controllability).
• Incident response plan: who is notified, who decides, how affected people are informed, and how lessons are captured.
• Change control: re-evaluation before new model versions, prompts, or data sources go live.
• Periodic review: scheduled re-assessment by the governance body, frequency proportional to risk tier.

Third-party and foundation model changes deserve special attention. When a provider updates or retires a model version, behavior may shift. Production governance should include version pinning or planned upgrade testing, as well as monitoring vendor notices.

⚠️ Exam Trap: "Testing was completed before launch" does not satisfy production risk management. When a scenario asks what is needed for an AI system already in production, choose continuous monitoring with thresholds, logging, feedback channels, and the ability to intervene (fallback or shutdown), with named owners.

Reflection Question: Your AI claims-triage system has been live for six months. List five production controls you would expect to see in place, and identify which ones would have detected a sudden change in the underlying model's behavior after a vendor update.