3.2.4. Monitoring and Troubleshooting AKS and Container Apps
💡 First Principle: Troubleshooting orchestrated containers is walking the request path — image pull → container start → probe pass → service wiring → ingress — and letting the failure signature tell you which segment broke. Orchestrators emit precise states; learn to read them and diagnosis becomes lookup, not guesswork.
On AKS, three commands cover most of the diagnostic surface: kubectl logs pod-name (what the app said — add --previous for the pre-crash container), kubectl describe pod pod-name (what Kubernetes did: events, probe failures, pull errors), and kubectl get events --sort-by=.lastTimestamp (cluster-wide recent history). The signatures the exam tests:
| Signature | Meaning | First move |
|---|---|---|
ImagePullBackOff | Can't fetch image — bad tag/registry auth | Verify tag exists; check ACR attach/AcrPull |
CrashLoopBackOff | Container starts then dies repeatedly | kubectl logs --previous for the crash output |
Pending | Unschedulable — no node fits | describe events: resource requests vs. capacity |
| Running, no traffic | Probe or label wiring | Readiness probe status; Service selector match |
OOMKilled | Exceeded memory limit | Raise limits or fix the leak — common with model-loading containers |
Container Apps, having no kubectl, surfaces the same truths through the platform: az containerapp logs show --follow streams the console; revision status (Provisioned/Failed, replica counts, restart counts) plays the role of describe; and every app in an environment ships logs to that environment's Log Analytics workspace — recall from 3.2.1 that the workspace is attached at environment level — where system and console tables answer historical questions ("what did the canary log during the error spike?") via KQL, the language Section 5.2.2 teaches.
End-to-end connectivity checks close the loop on both platforms: confirm ingress/Service reachability first, then walk inward — is the ingress target port the port the container listens on (the 3.1.3 contract again), does the Service have endpoints, do probes pass? The discipline is always the same: locate the first broken segment on the path; everything downstream is noise.
⚠️ Exam Trap: CrashLoopBackOff with clean current logs means the evidence died with the previous container — kubectl logs --previous is the specific incantation. Distractors offer describe (shows the restart count, not the crash reason) or deleting the pod (destroys evidence, changes nothing).
Two habits sharpen the toolkit. kubectl get pods -o wide adds node placement and pod IPs — often enough to spot every crashing pod sharing one node (a node problem wearing an application costume). And on the ACA side, az containerapp revision restart is the operational verb that completes secret rotation from 3.2.1; log retention follows the Log Analytics workspace's policy, so incident forensics depend on retention being set before the incident, not after.
Reflection Question: An embedding worker shows OOMKilled only under KEDA scale-out to max replicas. Walk the path: what's the likely interaction between scale rules and resource limits?
