6.3.3. Audit Trails for Models and Data
💡 First Principle: AI audit trails must answer a question that traditional audit logs can't: why did the AI make this decision? This requires tracking not just who did what (standard audit), but which model version was running, what data it retrieved, what prompt it used, and how those factors combined to produce the output.
AI-Specific Audit Trail Components:
| Component | What to Track | Why It's Needed |
|---|---|---|
| Model version | Which model version produced each output | Reproduce and investigate decisions |
| Training data version | Which dataset version trained the active model | Trace bias or accuracy issues to training data |
| Prompt template version | Which system prompt was active | Explain behavior changes after prompt updates |
| Grounding data accessed | Which documents/records the model retrieved | Verify the AI had access to correct information |
| Decision inputs | User input + retrieved context + model parameters | Full reproducibility of the decision |
| Decision output | The AI's response and any actions taken | Record of what the AI said or did |
| Change log | Who changed model, prompt, data, or configuration — and when | Accountability for configuration changes |
Audit Trail Architecture:
The architect must design audit trails that are tamper-resistant (immutable log storage), queryable (can answer regulatory inquiries efficiently), and retention-compliant (stored for required periods, deleted when mandated).
Why immutable and append-only: the purpose is evidentiary. Records that could be edited after the fact cannot be trusted as proof of what the AI actually did, so regulated decisions need write-once storage (for example, Azure Blob Storage immutability policies); it is a design choice you configure, not a default, and it is not about cost.
Retention under multiple regulations: map each regulation to its retention requirement and keep each data type for the longest applicable period, which satisfies the shorter ones too (Purview retention applies the same rule: the longest retention period wins). Automate both retention and purge, and send conflicts (for example, a GDPR erasure request versus a financial-record retention duty) to legal review. Keeping everything forever creates its own exposure, and applying the shortest period risks deleting records still required.
In the Microsoft stack these are concrete products, not abstractions, and the exam names them: Microsoft Purview Audit captures agent prompts and responses in the unified audit log; Purview eDiscovery preserves, searches, and exports them for legal matters; and Purview Data Lifecycle Management retention policies (location: Microsoft Copilot Experiences) enforce how long they are kept. Model-side lineage — model version, training data version, prompt template version — lives with the model registry and ALM process covered in 6.1, not with Purview. Designing the audit trail means deciding which half of the record each requirement lands in. See 6.2.4 for the full Purview capability map.
Decision Lineage:
For high-stakes AI decisions (financial approvals, customer classifications, compliance determinations), the architect designs decision lineage — a complete chain from user input through model processing to output, including all intermediate steps. This isn't real-time logging; it's structured recording that enables after-the-fact investigation.
⚠️ Common Misconception: AI audit trails are the same as standard application audit logs. AI audit trails must additionally track model version changes, training data modifications, prompt template updates, decision lineage, and output quality metrics.
Troubleshooting Scenario: A financial services company's AI model makes an investment recommendation that causes significant client losses. During the subsequent investigation, regulators request a complete decision audit trail. The company can show which model version was deployed and when, but cannot demonstrate why the model recommended that specific action — the decision lineage is missing. Decision lineage tracks the chain from input data through model reasoning to output recommendation, including which knowledge sources were consulted, what confidence scores were generated, and what fallback logic was triggered.
AI audit trails must capture six categories beyond standard application logs: model version changes, training data modifications, prompt template updates, decision lineage (input → reasoning → output chain), output quality metrics over time, and human override records. Each category serves a different compliance purpose and has different retention requirements.
Reflection Question: A financial services regulator asks your company to explain why an AI agent denied a specific customer's loan application six months ago. Design the audit trail architecture that enables you to answer this question with full decision lineage.
