5.3.4. AI and Generative AI Security
💡 First Principle: AI and Generative AI introduce unique attack surfaces—from prompt injection to data poisoning. Securing AI requires a shift from traditional infrastructure protection to monitoring the integrity of models, data, and user interactions in real-time.
Scenario: Your organization is deploying a customer-facing AI chatbot powered by Azure OpenAI. You must ensure that users cannot bypass safety filters through clever prompts, and that sensitive company data used for grounding isn't exfiltrated through the model's responses.
Defender for Cloud AI Protection
Defender for Cloud has expanded to provide specialized protection for AI workloads, focusing on visibility and proactive risk management.
| Feature | Protection Capability |
|---|---|
| AI Agent Discovery | Automatically identifies AI agents created in Microsoft Copilot Studio to assess their security posture. |
| Azure AI Foundry Protection | Monitors AI workloads for vulnerabilities and misconfigurations in the development environment. |
| Model Security Assessment | Evaluates the security configuration of deployed models against industry best practices. |
| Training Data Protection | Ensures that data used for fine-tuning or RAG (Retrieval-Augmented Generation) is encrypted and access-controlled. |
Generative AI Threat Landscape
Securing GenAI requires understanding threats that go beyond standard web vulnerabilities.
- Prompt Injection (Direct & Indirect): Attackers craft prompts to bypass safety filters (Direct) or place malicious instructions in data the AI retrieves (Indirect). Modern controls include real-time detection and blocking of suspicious prompt patterns.
- Training Data Poisoning: Maliciously modifying the training or grounding data to introduce biases or backdoors into the model.
- Model Extraction: Attempting to reverse-engineer or steal the underlying model weights and architecture through repeated querying.
- Jailbreaking: Using adversarial techniques to force the model to ignore its system instructions and provide restricted content.
- Data Exfiltration: Tricking the AI into revealing sensitive information from its training data or grounded documents.
Visual: AI Security Architecture
Azure AI/OpenAI Security Controls
Implementing defense-in-depth for AI services involves configuring platform-native security features.
- Content Filtering: Configurable safety levels for Hate, Violence, Self-harm, and Sexual content. These filters operate at both input (prompt) and output (completion) stages.
- Private Endpoints: Ensures that communication with Azure OpenAI or Azure AI Search occurs over a private IP, removing exposure to the public internet.
- Managed Identity: Use system-assigned or user-assigned managed identities for AI services to access data (e.g., Azure Storage or AI Search) without managing long-lived secrets.
- Data Residency: Azure OpenAI allows pinning workloads to specific regions to meet strict data residency and compliance requirements.
- Responsible AI Principles: Aligning security controls with fairness, reliability, safety, privacy, and accountability.
Example: Azure CLI - Enabling Content Filters
# Update an Azure OpenAI account to use specific safety settings
# This is a conceptual representation of applying security profiles
az cognitiveservices account update \
--name MyAIInstance \
--resource-group MyRG \
--custom-subdomain my-ai-subdomain \
--set properties.publicNetworkAccess="Disabled"
⚠️ Exam Trap: Assuming that a Private Endpoint alone secures an AI model. While Private Endpoints secure the network path, they do nothing to prevent Prompt Injection or Jailbreaking. Content filtering and guardrails are required for application-layer security.
Key Trade-offs:
- Security vs. Latency: Aggressive real-time content filtering and multi-step guardrails improve safety but increase the "Time to First Token" for users.
- Data Utility vs. Privacy: Stronger data masking in grounding sets improves privacy but may degrade the AI's ability to provide contextually accurate answers.
