Measure AI cost per resolved request
Cost per resolved request needs a stable business-request identifier and a clear resolution definition. Attribute all attempts, retries, tools, and review effort to that request, then divide cohort cost by requests that meet the resolution rule while showing unresolved work and late costs separately.
Inside this article 5 sections +
A dashboard can report falling token cost while the cost of resolving a customer's request rises. Retries, extra tool calls, repeated review, and reopened cases may all sit outside the chart. The problem is usually the metric's boundary, not the arithmetic.
Define the unit as a business request that survives model attempts and provider changes. A support case may use three model calls, two retrieval queries, and one human correction. Those are costs of one attempted resolution, not six independent successes.
Write the resolution rule before the query
Specify what resolved means for the workflow. It might require a confirmed answer delivered to the customer, a successful account update, or an accepted document record. A model response returning HTTP 200 is not a business resolution.
Define how reopening affects the result. For example, a request closed and reopened within an agreed observation window may remain unresolved for that cohort's final metric. The window should match the product's behavior; a hypothetical seven-day window is a reporting choice, not a universal standard.
Keep provisional and mature cohorts distinct. Today's requests have had less time to finish or reopen than last month's. Comparing their cost per resolved request without showing maturity can make a slow workflow look artificially cheap or expensive.
Attach every cost to a durable request identity
Use a request ID at the workflow boundary and propagate it through attempts, tools, review sessions, and completion events. Each cost event needs its own unique ID for deduplication. Provider request IDs support reconciliation, but they should not replace the business identifier because one business request can span several providers.
{
"event_id": "cost-demo-104",
"business_request_id": "support-demo-31",
"attempt_id": "attempt-2",
"event_type": "model_usage_recorded",
"provider_request_id": "provider-reference",
"usage": {"input_tokens": 1800, "output_tokens": 260},
"price_version": "standard-usd-2026-11-02",
"cost_usd": "0.003100",
"cost_status": "estimated",
"occurred_at": "2026-11-02T10:20:00Z"
}
The amount is an illustrative record, not a claim about a particular request or model. Store decimal monetary values or integer minor units with enough precision for small usage costs. Retain the price version and usage quantities so an estimate can be recomputed without rewriting the original event.
Record cache writes, cache reads, ordinary input, output, and billable tool usage in the categories the provider actually reports. Their pricing can differ. Avoid calculating all input tokens at one rate or counting cached tokens twice when the usage response separates them.
Keep numerator and denominator in the same population
All attempts, including failed and retried calls.
Attributed labor estimate under a stated rate.
Meet the completion rule after the observation window.
($18 + $72) ÷ 80 resolved requests.
The same cohort costs $0.90 per incoming request. Both figures are valid, but they answer different questions. The incoming-request measure supports demand budgeting; the resolved-request measure connects cost to the delivered outcome. Label them clearly instead of presenting whichever looks lower.
If five of the eighty requests reopen before the observation window closes, the mature denominator becomes seventy-five under this example's rule. With unchanged cost, the ratio becomes $1.20. Later recovery work may increase the numerator as well. Preserve the provisional history so operators can understand why the number changed.
Distinguish direct costs from allocated overhead. Shared indexing, monitoring, and platform capacity may be allocated by request volume, storage, or another driver. Publish the method and show the direct-cost view too. An allocation policy should not disguise itself as precise per-request measurement.
Reconcile missing, duplicate, and late usage
Deduplicate usage events by a stable event or provider-attempt key. A retried telemetry delivery must not double the bill. Conversely, a retry of the actual model request is a new billable attempt and should remain in the total. The event schema must make those two forms of retry distinguishable.
When a request times out before usage is returned, mark cost as pending or estimated rather than zero. Reconcile it against provider usage or billing exports when available. Preserve the original estimate and post an adjustment with a reason, so finance and engineering can trace the difference.
Monitor unallocated cost: usage that cannot be linked to a business request. A rising unallocated share may indicate missing propagation in a new tool path or an incorrect provider mapping. Do not silently spread it across successful requests just to make the dashboard total balance.
Show the dimensions that change a decision
Break the metric out by workflow version, request type, language, model path, and review outcome where sample sizes support interpretation. A small number of expensive cases can dominate the total. Show counts and distributions alongside averages, and avoid exposing tenant-sensitive data through unrestricted dashboard filters.
Pair cost with quality and completion. A cheap route that declines every difficult request can reduce model spend while increasing human work. A higher-cost route may be justified if it resolves a materially different class of requests. The dashboard should reveal that tradeoff rather than compress it into one success-colored number.
Use the planning cost model for estimates and review-time measurement for labor inputs. Our AI integration services can connect those assumptions to production event lineage and reconciliation.