3.2.1. Defining KPIs: Tangible and Intangible Benefits
💡 First Principle: A KPI is only useful if it reflects the outcome the initiative exists to change and if someone will act on it. AI benefits come in two forms: tangible ones that show up directly in financial or operational numbers, and intangible ones that matter just as much but must be measured through proxies. A balanced set of both tells the true story.
| Benefit type | Examples named or implied by the exam guide | Example KPIs | How measured |
|---|---|---|---|
| Tangible | Cost reduction, revenue growth, time savings | Cost per transaction, revenue per customer, average handle time, cycle time, error rate, throughput | Financial and operational systems |
| Intangible | Customer satisfaction, employee productivity and experience, decision quality, brand trust | CSAT, Net Promoter Score, employee satisfaction or engagement scores, time spent on high-value work, retention | Surveys, behavioral proxies, sampling |
Good AI KPIs share several properties:
- Outcome-linked: they measure the business result (claims settled faster, customers retained), not only AI activity (number of prompts, model accuracy).
- Layered: pair a model-quality metric (accuracy, grounding rate) with an operational metric (handle time) and a business outcome (cost per claim, CSAT). Model accuracy without business impact is not value.
- Owned: a named business owner is accountable for each KPI.
- Balanced with guardrail metrics: track what could get worse, such as complaint rates, escalation rates, or fairness measures. A contact center that cuts handle time while customer satisfaction falls has not created value.
Intangible benefits often drive adoption decisions and long-term value. If an AI assistant removes tedious work and employees report higher engagement, that can reduce attrition, which has a real (if indirect) financial value. The disciplined approach is to measure intangibles through credible proxies and present them alongside tangible results, labeled honestly.
⚠️ Exam Trap: When a scenario asks how to measure an AI initiative whose main benefit is customer or employee experience, the answer is not "exclude it because it is intangible." Choose options that define proxy KPIs (CSAT, NPS, employee engagement, time reallocated to higher-value work) and track them alongside tangible metrics.
Reflection Question: An AI coding assistant is rolled out to 300 developers. Propose two tangible and two intangible KPIs, plus one guardrail metric that would warn you if the assistant was creating hidden costs.
