2.3.2. Multimodal Data Processing

💡 First Principle: Multimodal data cannot be passed directly to an FM — images must be base64-encoded, audio must be transcribed to text, PDFs must have text extracted, and all inputs must be assembled into the model-specific JSON structure before the API call. The FM never touches your raw files.

Processing pipelines by modality:

Knowledge Base parsing: when a Knowledge Base's default parser garbles tables, switch the data source to the Bedrock Data Automation or foundation-model parser, or add a custom transformation Lambda (e.g., calling Textract) to the ingestion pipeline. The same Lambda hook cleans transcripts: strip filler words and speaker labels, keep only the turns worth indexing (e.g., agent answers), and split on topic boundaries — raw, noisy transcripts dilute retrieval.

Bedrock Data Automation — the managed service for multimodal document processing at scale. It handles PDFs, images, audio, and video with standardized output formats, eliminating the need to build and maintain custom extraction pipelines:

# Bedrock Data Automation for PDF batch processing
bedrock_data_auto = boto3.client('bedrock-data-automation')

response = bedrock_data_auto.invoke_data_automation_async(
    inputConfiguration={
        's3Uri': 's3://my-bucket/invoices/',
        'documentConfiguration': {'parsingStrategy': 'AUTO'}
    },
    outputConfiguration={'s3Uri': 's3://my-bucket/extracted/'},
    dataAutomationConfiguration={
        'dataAutomationProjectArn': 'arn:aws:bedrock:...:data-automation-project/invoice-extraction'
    }
)

⚠️ Exam Trap: Images are encoded as base64 and embedded directly in the Bedrock API request body — not referenced by S3 URL. When exam scenarios describe "sending an image to Bedrock for analysis," the correct architecture includes a Lambda function that reads the image from S3, base64-encodes it, and constructs the multimodal API payload. The FM cannot read from S3 independently.

Reflection Question: You need to build a system that processes 10,000 scanned invoices per day, extracts line items, and generates a structured JSON summary using an FM. What is the complete processing pipeline, naming each AWS service involved?