GPU Inference Cost per Token Anomaly Monitor

GPU Inference Cost per Token Anomaly Monitor

Watch AI infrastructure efficiency as traffic, models, and GPU prices change. The workflow queries hourly usage and cost data, calculates cost per million tokens against a baseline, and explains only material regressions.

What this workflow does

1. GpuFinOpsSchedule starts the check every four hours
2. QueryGpuInferenceUsage reads cost, token, model, and GPU-hour data from BigQuery
3. CalculateInferenceUnitCost derives current and baseline unit economics
4. GpuCostAnomalyGate detects regressions above the configured threshold
5. ExplainGpuCostAnomaly identifies likely demand, model, batching, and utilization drivers
6. EmailAiFinOpsOwners sends the investigation brief
7. AppendGpuCostHistory records the observation in Google Sheets
8. Separate receipts cover alert and normal states

Use cases

• GPU FinOps monitoring
• LLM inference cost per token tracking
• AI infrastructure unit economics
• Model serving efficiency alerts

Setup

Connect BigQuery, an LLM credential, email, and Google Sheets. Update the table and field names in the query to match your billing export and inference telemetry. Validate the cost allocation model with finance and platform teams before using the metric for chargeback.