2.2.3. Factors for Selecting a Generative AI Model

First Principle: The selection of a generative AI model is a strategic decision that involves balancing multiple factors, including the specific task requirements, performance needs, compliance constraints, and cost.

There is no single "best" model; there is only the best model for a specific use case.

Key Selection Factors:

• Performance & Capabilities: Does the model excel at the required task (e.g., code generation, creative writing, summarization)? Some models are better at certain tasks than others.
• Model Type & Modality: Is it a text-only model, or is it multi-modal (can it handle images, audio, etc.)?
• Constraints (Latency & Size): How fast does the response need to be? Larger models are often more capable but slower (higher latency) and more expensive to run. Smaller models are faster and cheaper but may be less powerful.
• Compliance & Data Privacy: Where is the model hosted? How is your data used? For sensitive data, you may need a model that can be hosted in your own private environment and that guarantees your prompts and outputs are not used for training.
• Cost: Costs are often calculated per token. The pricing model, including the cost for input (prompts) and output (responses), must be factored into the total cost of ownership.
• Customization: Does the model support fine-tuning if you need to adapt it for a specialized task?

Cost, latency and model complexity: Version 1.1 of the exam guide names these three selection factors explicitly, and they move together. A larger, more complex model can reason better but usually costs more per token and responds more slowly. A practical rule: choose the smallest model that meets the quality bar, then confirm cost and latency at your expected volume. Remember that cost and latency depend on tokens as well as on the model: long outputs cost the most and take the longest (see 2.1.5). Sometimes the right choice is not a foundation model at all (see 1.2.4).

Scenario: A company is choosing an LLM for two different applications: a real-time chatbot for its website and an offline tool for summarizing very long legal documents.

Reflection Question: Why might the company choose a smaller, faster model for the chatbot (prioritizing low latency) and a larger, more powerful model for the legal summarization task (prioritizing capability and context length)?

💡 Tip: Start by defining your business requirements first. Then, map those requirements (performance, cost, compliance) to the available models to find the best fit.