How to Evaluate AI Projects ROI

In the rush to adopt GenAI and advanced analytics, organizations often skip the boring part: the math. We see "proof of concepts" that prove the technology works but fail to prove it creates value. As an AI leader, my job isn't just to deploy models—it's to ensure every deployment pays for itself. Here is the framework I use to evaluate AI projects, separating the vanity metrics from the true ROI drivers.

1. The "Value Bucket" Approach

Not all AI value looks the same on a P&L statement. I categorize every potential project into one of three buckets:

• Cost Reduction (Hard ROI): Direct labor savings, reduced software licensing, lower cloud spend through optimization.
  Example: An automated document processing bot saving 2,000 analyst hours/year.
• Revenue Generation (Hard ROI): Improved conversion rates, churn reduction, cross-sell lift.
  Example: A recommender system increasing average order value by 4%.
• Strategic Capability (Soft ROI): Faster time-to-market, improved customer satisfaction (NPS), risk mitigation.
  Example: A predictive maintenance model preventing reputation-damaging outages.

The Rule: Strategic capability is great, but your portfolio should be weighted 70% toward Hard ROI to maintain executive support.

2. Calculating the True Cost (TCO)

ROI is (Value - Cost) / Cost. Most teams underestimate the denominator. A realistic AI budget must include:

• Development: Data engineering, data science, and UI/UX time.
• Infrastructure: GPU/cloud compute, API token costs (which scale linearly with GenAI usage).
• Maintenance (The "AI Tax"): Model retraining, monitoring, and fixing data drift. We typically estimate this at 20-30% of the initial build cost annually.
• Change Management: Training users and redesigning workflows. If the users don't adopt it, the value is zero.

3. Define "Success" Before You Code

I never approve a line of code until the stakeholder signs off on the "Metric of Truth." This avoids the common trap where a model has 95% accuracy but zero business impact.

"Don't measure the model. Measure the process the model improves."

Instead of "Model Accuracy > 90%," use "Reduction in manual review time by 50%."
Instead of "Chatbot containment rate," use "Cost per ticket resolution."

4. Case Study: The "Perfect" Customer Churn Model

I once saw a team build a churn predictor with incredible precision. They identified customers at risk with 90% accuracy. The ROI? Negative using the basic calculation. Why?

Because the cost of the retention offer (a 20% discount) was higher than the profit margin on those customers, and the marketing team lacked the resources to call them all. The model worked, but the project failed.

The Fix: We re-scoped the project to focus only on "high-value, saveable" customers. The accuracy dropped, but the ROI skyrocketed because the intervention was now profitable.

5. The "Go/No-Go" Gate Review

Every quarter, review your portfolio. Be ruthless. If a pilot has been stuck in "validation" for 6 months, kill it. If a deployed model's maintenance cost exceeds its value, deprecate it. Freeing up resources from zombie projects is the only way to have capacity for the next big breakthrough.

Conclusion

Evaluating AI ROI isn't about stifling innovation; it's about steering it. By rigorously defining value, accounting for full costs, and measuring business outcomes, you transform your data team from a cost center into a strategic profit engine.

To see how these principles have played out in practice, browse my AI project portfolio — each case study includes the business case, the cost model, and the measured outcome. For a deeper conversation about your own portfolio, the AI Strategy & Consulting page outlines how I engage with leadership teams.