5.1.2. Interpreting Visual Input with a Multimodal Model
💡 First Principle: A multimodal model lets you put an image into a prompt and ask questions about it — "what's in this photo?", "read the sign," "is this chart trending up?". The model reasons over the image and text together, so vision understanding becomes just another kind of prompt.
This is how Foundry handles "interpret visual input": you deploy a multimodal-capable model and include the image (typically as a URL or base64 data) alongside your text question in the same request. The model returns text describing, classifying, or answering questions about the image. It's the Phase 3 computer-vision workload (image in, information out) expressed through a generative model.
⚠️ Exam Trap: Interpreting an image (multimodal understanding) is the opposite direction from generating an image. If a scenario says "describe what's in this picture" or "extract the text from this photo," that's a multimodal understanding task, not image generation.
Reflection Question: A user uploads a photo of a handwritten note and asks "what does this say?" Is this a multimodal-understanding task or an image-generation task? Which direction is the data flowing?
