3.1.1. Criteria for Choosing Pre-trained Models

First Principle: Selecting the right pre-trained model is a critical design decision that requires a multi-faceted evaluation of the model's capabilities against the specific technical and business requirements of the application.

This extends the general selection factors from Phase 2 into more concrete design criteria.

Key Design Criteria:

• Modality: What kind of data does the application handle? If it's only text, a pure LLM is sufficient. If it needs to understand or generate images, a multi-modal model is required.
• Customization: Does the business problem require the model to learn company-specific jargon, styles, or information? If so, you need to choose a model that supports fine-tuning.
• Model Complexity & Size: Larger models generally have more knowledge and better reasoning capabilities but are slower and more expensive. A key design choice is finding the smallest model that can effectively perform the required task to optimize for cost and latency.
• Input/Output Length (Context Window): How much text can the model process at once? For tasks like summarizing long documents, a model with a large context window is essential. For simple Q&A, a smaller context window may be adequate.
• Latency: How quickly does the application need a response? For a real-time interactive chatbot, low latency is critical. For an offline report generation tool, latency is less of a concern.
• Cost: The cost per token (or cost for provisioned throughput) is a major design factor that must be weighed against the model's performance and the application's budget.

Scenario: You are designing an AI application that must read a 50-page legal document and provide a detailed summary.

Reflection Question: Which design criterion is most critical for this task? Why would a model with a small context window be completely unsuitable, regardless of its other capabilities?

💡 Tip: There is often a direct trade-off between model capability, speed, and cost. Your job as a practitioner is to understand the business requirements well enough to make the right compromise.