Low-value AI in production needs a retirement gate

Volume is not a value measure

Gartner's 19 October survey release says 13% of 2,002 surveyed CIOs and technology executives described their enterprise's AI value as significant or transformational. It says 31% had formal decision criteria for when an AI pilot should not move to production, while 17% reported having guidelines for abandoning low-value AI initiatives already in production. These are respondent reports using Gartner's categories, not audited returns or causal estimates.

The practical signal is the asymmetry. Many organisations have processes for starting AI work, but fewer describe a comparable process for revising or ending it after deployment. A pilot can remain “promising” while its target changes, review labour moves off budget, or a business process absorbs retries and exceptions. A deployed system can survive because the original sponsor has moved on and no one owns a fresh comparison with the non-AI baseline.

Independent coverage by Express Computer repeats the survey's main findings and does not independently validate the returns. Gartner also reports that only 4% of respondents had plans for a human-agent workforce. That figure should not be read as an effectiveness threshold: it shows how respondents classified their planning, not whether a particular design works.

Make three decisions explicit

Every proposed use case should begin with three different evidence questions. Scale asks whether the task result, risk and cost meet a predeclared threshold under representative load. Revise asks whether a bounded change—data, workflow, model, controls or training—has a plausible route to close a named gap. Stop asks whether the remaining gap, opportunity cost or ownership burden makes further work unjustified.

Do not compress these into one traffic light. Record the baseline process, target population, acceptance measures, exposure volume, full cost, review burden, incident class and workforce consequence. Add a decision date, accountable owner and the evidence that would reopen the decision. A result may be technically acceptable but uneconomic; another may save time yet create a supervision queue that the receiving team cannot staff.

Portfolio review needs a common unit. “Number of use cases” mixes experiments, features and business services. Use a versioned workload record that identifies the task, model and tool boundary, affected users, decision authority and cost window. Compare candidate systems on accepted outcomes after retries and review, not raw model calls or demo completion.

Report the decision distribution as well as the portfolio total. A healthy review can produce scales, revisions and stops in the same quarter. If every item advances, test whether the gate is measuring evidence or merely documenting sponsorship. If every item stops, inspect whether intake is too broad or whether teams lack the evaluation capability needed to produce an answer.

The counterargument is that fixed criteria can kill exploration too early. The remedy is not to omit criteria but to separate discovery from commitment. A short discovery phase can use learning goals, small exposure and a time box. Scaling requires stronger outcome and risk evidence. Revision receives a bounded budget and a specific hypothesis. Stopping preserves the evidence and lessons so the same failed assumption is not quietly repackaged.

Put deployed systems back into the queue

An exit rule is not only for pilots. Review deployed systems when price, model, data, authority, user population or failure consequence changes. Include the cost of monitoring, escalation, retraining and vendor switching. A lower unit price can be offset by more calls or human review; a better model can expand the task boundary and therefore the risk population.

This extends the newsroom's earlier pilot-baseline analysis. The immediate decision is to add a one-page scale–revise–stop record to the portfolio review and require it for both new pilots and existing services. Gartner's survey does not prove that this record creates value. It does show a large declared gap between running AI work and having explicit criteria for what happens next.