The Question No One Wants to Ask: Is Your Vision AI Still Solving the Right Problem?
There is a particular kind of institutional courage that separates technology organizations that persist from those that merely endure. It is not the courage to invest — most companies manage that. It is the courage to stop, look directly at what has been built, and ask whether any of it still belongs in the future the business is actually moving toward.
Cost audits are common. Accuracy benchmarks are routine. Architecture reviews happen on quarterly cycles at well-run organizations. But there is one audit that almost never gets scheduled, never makes it onto the engineering roadmap, and rarely surfaces in board-level technology discussions: the relevance audit. The structured, unsentimental examination of whether a vision AI system is still solving a problem the business genuinely needs solved.
The omission is not accidental.
Why Organizations Avoid the Hardest Evaluation
Computer vision deployments carry a specific kind of organizational weight. They are visible investments — cameras, edge hardware, data pipelines, labeling contracts, model training infrastructure. They represent months or years of engineering effort, often championed by leaders who staked professional credibility on their success. When a system reaches production, the organization exhales. The battle to get there was difficult enough that very few people are eager to reopen it.
This creates a structural blind spot. The metrics tracked after deployment — inference latency, model accuracy, uptime — are all operational. They measure how well the system is performing its function. None of them measure whether that function still deserves to exist.
Market conditions shift. Business priorities realign. The operational bottleneck a vision system was designed to address in 2021 may have been reorganized away, automated by a different process, or simply rendered irrelevant by a change in how the company operates. The system keeps running. The dashboards stay green. And no one asks the uncomfortable question because asking it implies that a significant investment may have been misallocated — and in most organizations, that implication lands as an accusation.
The psychological barrier is compounded by organizational dynamics. Teams that built the system have an understandable attachment to it. Procurement relationships, vendor contracts, and infrastructure commitments create financial inertia. And in environments where AI investment is still treated as a signal of organizational sophistication, admitting that a vision deployment no longer serves a strategic purpose can feel like admitting failure in public.
So the audit doesn't happen. Resources continue flowing into systems that are technically functional but strategically hollow.
What a Relevance Assessment Actually Examines
A genuine relevance audit is distinct from a performance review. It does not ask whether the model is accurate. It asks whether accuracy in this domain still produces business value. The questions are different in kind, not just degree.
The assessment begins with a return to original intent. Every vision AI deployment was justified by a specific business case — a problem statement, a projected outcome, a measurable improvement in some operational dimension. The first step is to locate that original documentation and read it without the benefit of hindsight. What problem was being solved? What would success look like? What assumptions were embedded in that framing?
The second step is to evaluate whether those assumptions still hold. Has the underlying operational process changed? Has the market context shifted in ways that alter the value of the insight the system produces? Have competitors solved the same problem differently, neutralizing whatever advantage the deployment was expected to create? Has the organization's strategic direction moved in ways that make this capability peripheral rather than central?
Third, and most critically, the assessment must examine what decisions the vision system is actually informing today — not what it was designed to inform, but what it is genuinely influencing in practice. Many deployed systems produce outputs that are technically consumed but operationally ignored. Reports are generated. Dashboards are maintained. But if the downstream decisions those outputs were designed to support are being made through different channels or not being made at all, the system's operational footprint has become a form of sophisticated theater.
The Organizational Conditions That Make Honest Assessment Possible
Running this audit honestly requires conditions that most organizations have to deliberately create. The evaluation cannot be led by the team that built the system — not because those engineers lack integrity, but because the cognitive investment required to build a production vision system makes genuine detachment nearly impossible. The assessment needs an internal sponsor with sufficient authority to hear an unfavorable conclusion and act on it without political consequence.
It also requires separating the evaluation of the system from the evaluation of the people who built it. This distinction is harder to maintain than it sounds. In practice, when a relevance audit concludes that a vision deployment is no longer strategically justified, the instinct in many organizations is to treat that conclusion as a verdict on the team responsible. That conflation is both unfair and counterproductive. Markets change. Priorities shift. A system that was the right answer to a real problem in a prior business context can become obsolete without any failure of execution on anyone's part.
Organizations that conduct these audits effectively tend to frame them not as investigations but as strategic calibration exercises — routine checkpoints in the lifecycle of any significant technology investment. When the relevance audit is normalized rather than exceptional, it loses most of its political charge. It becomes simply what responsible technology stewardship looks like.
What to Do With the Findings
The outcome of a relevance assessment is rarely binary. Few systems are completely irrelevant, and few are as strategically central as their original business cases claimed. Most fall somewhere in between — partially aligned with current needs, solving a subset of the original problem, or producing value in a domain adjacent to where they were aimed.
This spectrum of outcomes calls for a corresponding spectrum of responses. A system that is substantially misaligned with current strategic priorities deserves serious consideration for decommissioning or significant reorientation, even if it is technically performing well. A system that is partially relevant may warrant restructuring — redefining its scope, redirecting its outputs, or integrating it with different downstream processes — rather than wholesale replacement or continuation as-is.
In either case, the findings should feed directly into forward-looking planning. The patterns that caused a vision investment to drift from strategic relevance — the assumptions that aged poorly, the market dynamics that weren't anticipated, the organizational changes that weren't reflected in the technology roadmap — are exactly the inputs needed to design the next deployment with more durable alignment.
Persistence Requires Honest Accounting
At PersistVision, we hold a specific view of what it means to build technology that lasts. Persistence is not the same as stubbornness. Enduring systems are not systems that simply refuse to be decommissioned — they are systems that remain genuinely useful because the organizations that built them were willing to question that usefulness honestly and repeatedly.
The relevance audit is not a threat to vision AI investment. It is the mechanism by which that investment earns the right to continue. Companies that run it regularly, honestly, and without political self-protection are the companies whose AI deployments still matter five years after launch — because they were willing to find out, at every stage, whether they still should.
