5.1. Visual Understanding with Multimodal Models

💡 First Principle: A multimodal model treats an image as just another input alongside text — you can ask it questions about a picture the same way you prompt about a document. This collapses a whole category of former "train a vision model" tasks into "write a good prompt with an image attached," which is the central reframing of this domain.

Why care: the exam tests whether you've internalized the shift. Captioning, visual Q&A, reading a chart, describing a scene — these are now prompt-and-image tasks for a GPT-4o-class model, not jobs requiring a trained classifier. Defaulting to "train a custom vision model" for general understanding is the AI-102-era reflex this domain probes.

⚠️ Common Misconception: "You always need a custom-trained vision model to analyze images." For general understanding (what's in this image, describe it, answer a question about it), a multimodal generative model handles it via prompt. Custom training is reserved for narrow domain classification where the model needs to learn categories it doesn't already know.