5.1.2. When to Use Dedicated Vision Services and Custom Models
💡 First Principle: Dedicated vision services and custom models earn their place when the task is specialized or high-volume in a way prompting handles poorly — precise OCR at scale, face detection, spatial/video analysis, or classifying into your own trained categories. Match the specialization to the service.
Azure Vision (in Foundry Tools) provides prebuilt capabilities that a general model doesn't optimize for: OCR/Read for precise printed and handwritten text extraction at scale (also shared with Document Intelligence), image tagging and smart crop, face detection (responsible facial recognition), and video analysis (spatial analysis, video retrieval). Custom vision/image classification is for when you must classify images into categories specific to your domain that a general model wasn't trained on — e.g., distinguishing your product's defect types. The decision: general understanding → multimodal prompt; precise/standardized capability (OCR, faces, video) → dedicated Vision service; your own categories → custom model.
| Need | Tool | Why |
|---|---|---|
| Open-ended understanding | Multimodal model (prompt) | No training, flexible |
| Precise OCR at scale | Azure Vision Read / Document Intelligence | Optimized, structured output |
| Face detection, video/spatial | Azure Vision services | Specialized capability |
| Your own image categories | Custom vision model | Learns domain-specific classes |
⚠️ Exam Trap: OCR is a specialized capability, not a generic "ask the model to read it" task when precision and scale matter — the Read model is purpose-built for printed/handwritten extraction with location and confidence per word. For a high-volume document-reading pipeline, the requirement-correct answer is the Read/OCR capability, not ad-hoc multimodal prompting.
Reflection Question: A pipeline must extract handwritten amounts from thousands of scanned forms with per-field confidence scores. Why is the Read/OCR model a better fit than prompting a multimodal model image-by-image?
