6.1.2. Extracting Information from Images
💡 First Principle: Image extraction applies the same analyzer idea to photos: the analyzer generates or classifies structured field values from what the picture shows. A shelf photo can yield product counts and details; a production-line photo can yield a description of a visible defect. A photo that is mostly text, such as a business card, is the exception: Microsoft recommends a document analyzer for it. The analyzer defines what to look for.
This overlaps with computer vision but is aimed at structured output: the goal isn't just "classify this image" but "give me these specific pieces of information from it." Content Understanding returns the fields you define in one step, generating or classifying each value from the image, so your application gets data it can act on directly. Fields do not have to come from printed text: an image analyzer can generate a value from what the picture shows, such as a description of a scratch or crack on a manufactured part. (When the job is mainly reading text from an image, Microsoft recommends a document analyzer instead.)
⚠️ Exam Trap: Image extraction and image classification differ in output. Classification gives a label ("this is a store shelf"); extraction gives the values ("cereal boxes: 14, empty slots: 3"). When a scenario needs the data inside the image, that's extraction.
Reflection Question: A parts supplier photographs every returned component and wants each photo to come back with the defect type (scratch, crack, or misalignment), where on the part it is, and a one-line description. Is classification, OCR alone, or information extraction the right fit, and why?