2.4.1. Ingesting and Storing Text, Image, and Audio Data
💡 First Principle: Every modality must eventually become either text or a vector, and you choose which conversion happens at ingestion time. That choice decides what users can search for later: speech that was never transcribed cannot be found by a text query, and an image reduced to its OCR text cannot be found by visual similarity.
Amazon S3 remains the landing zone for unstructured AI data, exactly as it is for tabular data in 2.1.2: it stores objects of any type, scales without provisioning, and is the data source that every multimodal ingestion path supports. Amazon Bedrock Knowledge Bases, for example, processes multimodal files (images, audio, video) from S3 data sources (custom data sources can also pass inline content up to 10 MB); files in Confluence, SharePoint, Salesforce or web-crawler sources are skipped for multimodal processing. Keep the original objects in S3 as the system of record and treat everything derived from them (transcripts, extracted text, embeddings) as rebuildable outputs.
The design decision is how each modality is converted:
| Modality | Convert to text | Embed natively | Typical AWS path |
|---|---|---|---|
| Scanned PDFs, forms, tables | OCR and layout extraction | — | Amazon Textract; Bedrock Data Automation (BDA) or an FM parser in a knowledge base |
| Images | OCR text and generated descriptions | Visual similarity | BDA for text; Amazon Titan Multimodal Embeddings G1 or Amazon Nova Multimodal Embeddings for vectors |
| Audio (calls, meetings) | Speech-to-text transcript | Audio embedding | Amazon Transcribe or BDA for transcripts; Nova Multimodal Embeddings for audio vectors |
| Video | Transcript plus scene summaries | Segment embeddings | BDA; Nova Multimodal Embeddings (segments up to 30 seconds) |
| Streaming video | — | — | Amazon Kinesis Video Streams for ingestion, then batch processing |
Bedrock Knowledge Bases makes the trade-off explicit with two multimodal processing approaches:
| Characteristic | Nova Multimodal Embeddings | Bedrock Data Automation (BDA) |
|---|---|---|
| Processing | Embeds images, audio and video directly, no intermediate text | Converts media to text (ASR transcripts, scene summaries, OCR), then embeds the text |
| Query types | Text queries or image queries | Text queries only |
| Best for | Product catalogs, visual similarity, diagrams | Meeting and call recordings, training videos, support calls |
| Weak spot | Spoken content is not searchable | No image-to-image similarity search |
| RAG generation | RetrieveAndGenerate limited to text content | Full RetrieveAndGenerate support |
Parsing choices for documents. The Bedrock default parser extracts only text (from .txt, .md, .html, .doc/.docx, .xls/.xlsx and .pdf) and has no usage charge. If documents contain figures, charts, tables or images you want to query, choose BDA (priced per page) or a foundation model parser (priced per input and output token, with a customizable extraction prompt). Choosing either advanced parser applies it to all PDFs in the data source, including text-only ones, which is a cost consideration for large mixed corpora.
Storage layout. Organize the landing bucket by modality and processing stage (for example raw/audio/, derived/transcripts/), keep derived artifacts next to a pointer to their source object, and store extracted multimodal assets where retrieval can return them. Bedrock multimodal knowledge bases use a multimodal storage destination (an S3 location) that holds copies of the original media so results stay retrievable even if a source file changes; it is required for Nova Multimodal Embeddings and required for BDA whenever non-text input must be processed.
⚠️ Exam Trap: When a scenario says users must search recorded calls by what was said, native multimodal embeddings are the wrong answer — Nova Multimodal Embeddings cannot make spoken content meaningfully searchable. Transcription (BDA or Amazon Transcribe) is required. Conversely, when users must upload a photo and find similar products, text conversion is wrong, because BDA-processed knowledge bases accept text queries only.
