2.3.3. Data Labeling and Annotation with SageMaker Ground Truth

💡 First Principle: Supervised learning requires labeled data, and label quality directly determines model quality. Poor labels create a ceiling on model performance that no amount of algorithm tuning can break through. The exam tests how you create, manage and quality-control the labeling process — who labels, how much of the work a model can take over, and how sensitive data is protected while people look at it.

Getting accurate labels at scale is one of the most expensive and time-consuming parts of ML. Consider labeling 100,000 medical images for tumor detection: you need domain experts (radiologists), quality controls (agreement between annotators), and scale (one radiologist cannot label everything). Every labeling design balances those three pressures.

Amazon SageMaker Ground Truth manages labeling workflows: built-in task types (image classification, bounding boxes, semantic segmentation, text classification and more), custom labeling UIs, and a choice of workforce. You store the data to label, an input manifest file and the output manifest in Amazon S3. AWS has announced that Ground Truth is no longer open to new customers; existing customers can continue to use it as normal, and AWS does not plan new features for it. The public Amazon Mechanical Turk crowd that older material lists as a third workforce is gone, because AWS permanently closed Amazon Mechanical Turk at the end of September 2026.

Automated data labeling (active learning). For four built-in task types — single-label image classification, semantic segmentation, object detection (bounding boxes) and single-label text classification — Ground Truth can train a model on an initial set of human labels, auto-label the objects it is confident about, and send only low-confidence objects to people. It repeats this loop until the dataset is labeled or a stopping condition (such as the human annotation budget) is reached. Key facts:

• The minimum dataset is 1,250 objects, and AWS strongly suggests at least 5,000 — the auto-labeling model needs volume to reach the required accuracy.
• Accuracy targets are fixed by Ground Truth, not configurable: at least 95% expected label accuracy for image and text classification, a mean IoU of 0.6 for bounding boxes and 0.7 for semantic segmentation.
• Streaming labeling jobs and custom task types do not support automated labeling.
• You pay SageMaker training and inference costs for the auto-labeling model in addition to human labeling.

Label quality controls. Whatever the tool, the same controls apply: clear written instructions with examples of edge cases, several annotators per item with their labels consolidated, gold-standard items with known answers mixed into the queue to measure each annotator, and a review pass on a random sample before the labels are used for training. For text generation data (prompt-response pairs), the equivalent is a human or LLM-as-a-judge review of response quality — covered in 2.4.8.

⚠️ Exam Trap: Automated labeling reduces how many objects people must label; it does not change who the people are. When the data is sensitive, pick the workforce first (private or a contracted vendor), then turn on automated labeling to cut volume. And automated labeling only helps with large datasets of a supported built-in task type — a 600-image job or a custom labeling template gets no benefit from it.

Reflection Question: A company needs to label 500,000 product images across 50 categories. Budget is limited, accuracy requirements are high, and the images include unreleased product designs. Which workforce and labeling features would minimize cost while protecting the designs?