2.1.3. Token Economics and Cost Drivers
💡 First Principle: Generative AI costs are driven by usage, not just licensing—specifically by tokens, which are the units of text the model processes. Think of tokens like metered electricity: you pay for what you consume, and both your question (input) and the AI's answer (output) consume tokens. Understanding this helps you forecast costs, optimize usage, and make ROI arguments that finance teams will accept.
What are tokens? A token is roughly Âľ of a word in English. The sentence "What are the quarterly sales figures?" is about 7 tokens. When you send a prompt to an AI model, you pay for:
- Input tokens — your prompt, any context or documents included
- Output tokens — the AI's response (typically more expensive per token)
This means a short question with a long answer costs more than a short question with a short answer. And a prompt that includes a 50-page document for context costs significantly more than one without.
| Cost Factor | What Drives It | How to Optimize |
|---|---|---|
| Input tokens | Prompt length + context provided | Be specific; don't include unnecessary documents |
| Output tokens | Response length and detail | Request concise formats; set length limits |
| Model tier | GPT-4 costs more than GPT-3.5 | Use the simplest model that meets the need |
| Volume | Number of requests per day/month | Prioritize high-value use cases first |
| Fine-tuning | Custom model training compute | Only fine-tune when pretrained + grounding isn't enough |
ROI calculation framework: AI ROI isn't just "cost of AI" vs. "savings." The full picture includes:
- Direct savings: Time saved Ă— hourly cost Ă— number of users
- Quality improvements: Fewer errors, faster turnaround, better decisions
- Opportunity cost: What could employees do with reclaimed time?
- AI costs: Licensing + token consumption + administration
The key insight: AI ROI scales with adoption. Deploying to 5,000 users at 10% adoption produces less value than 500 users at 90% adoption—but the token costs are similar. This is why targeted rollouts with high adoption often beat broad rollouts with low adoption.
⚠️ Exam Trap: When asked about AI costs, "licensing fees" alone is incomplete. Token consumption is a major cost driver for generative AI. Also, ROI should be measured by business outcomes (time saved, quality improved), not just cost reduction.
Reflection Question: A CFO asks you to justify the cost of Microsoft 365 Copilot. How would you structure the ROI argument, and what costs beyond licensing would you include?
