Uncertainty Is Hidden At The Point Of Decision

AI uncertainty becomes operational at the moment of action

Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.

Every AI model produces outputs at varying levels of confidence. Some AI outputs are grounded in strong signal and are highly reliable. Other AI outputs are extrapolations from sparse or ambiguous evidence and should be treated cautiously. The distinction exists in the model's probability distribution over outputs, but whether the interface communicates it to the user is a design decision.

Uncertainty is present throughout an AI-assisted workflow, but it becomes operationally costly at decision points. A decision point is the moment when the user must act on the AI's output. At that moment, uncertainty is no longer an abstract property of the system; it becomes a direct input to the user's decision.

Uniform AI output presentation creates a false confidence signal

Many AI product interfaces present all AI outputs in the same visual container, with the same formatting and the same apparent weight. This creates a uniform confidence signal: every AI output appears equally authoritative, even when the model's own confidence varies across outputs.

When uncertainty is hidden, users face a choice they cannot make well. They can trust each output enough to act without verification, or they can verify everything. Verifying everything removes the efficiency gain the AI was meant to provide. Trusting everything creates a predictable failure pattern: low-confidence outputs are accepted without scrutiny, errors become embedded in decisions, and confidence in the system declines when those errors surface.

In this situation, the system may be blamed for unreliability even when the immediate failure is not the model's inability to estimate uncertainty. The failure is that the interface does not communicate the model's uncertainty about specific outputs at the moment when the user needs that information.

Decision points require users to allocate scrutiny

AI uncertainty matters most when the user must approve, override, accept, reject, or investigate an AI output. A reviewer approving or overriding an AI screening decision is at a decision point. A fraud analyst acting on an AI-generated risk assessment is at a decision point. A clinician responding to an AI-surfaced finding is at a decision point. An information specialist accepting or rejecting an AI-generated inclusion criterion is at a decision point.

At each decision point, the user needs to know whether the AI output is high-confidence enough to act on quickly or low-confidence enough to require verification. If the interface does not answer that question, the user has to treat every decision point as if it requires the same level of scrutiny.

The cost compounds under volume. In a systematic review involving hundreds of publications, a reviewer cannot realise the efficiency benefit of AI-assisted screening if every AI recommendation requires the same verification effort. The benefit appears when high-confidence outputs can be handled quickly and attention can be concentrated on low-confidence outputs.

A confidence number alone does not provide enough basis for oversight

Displaying AI confidence is necessary but not sufficient. A confidence score without context is not actionable enough for informed human judgment. A 70% confidence rating attached to a clearly wrong inference is less useful than a 70% confidence rating accompanied by the specific evidence the model used to reach its conclusion.

The design problem is not only whether to show confidence. The design problem is how to show enough of the AI's reasoning for the user to evaluate whether the confidence signal is warranted. Users need a basis for oversight, not only a number.

This creates a specific design tension. Full reasoning supports informed human judgment, but dense reasoning information can slow review. Minimal displays support scanning speed, but can remove the evidence that makes meaningful oversight possible. A usable AI interface at the decision point has to preserve both properties: the user must have the information needed for epistemic control, and the information must be presented at the pace the work requires.

Puraite case: AI-assisted systematic review screening

In the Puraite case, Creative Navy worked on a web application for conducting AI-assisted systematic literature reviews. Puraite integrates AI assistance into review stages including initial screening decisions, suggested inclusion and exclusion criteria, and structured data extraction from qualifying publications.

The specific design challenge Creative Navy addressed was the AI screening screen. This was the interface where reviewers encountered AI inclusion and exclusion decisions and decided whether to accept or override them. The decision point mattered because reviewers could be working through hundreds of publications, while each acceptance or override remained a substantive judgment about whether a publication qualified for inclusion in a research synthesis used in academic, clinical, and pharmaceutical research contexts.

When Creative Navy arrived at the engagement, the AI behaviour model had not been fully specified as a design problem. The product needed decisions about what the AI should surface, how confidence should be communicated, and where human override should sit. The issue was not simply that confidence information was absent. The issue was that the interface did not yet present confidence and evidence in a form that supported screening pace while preserving epistemic control.

Four option space mapping cycles tested the AI suggestion display

Creative Navy applied option space mapping across four design cycles for the Puraite AI suggestion display. Each cycle committed to a different theory of how to hold scan speed and epistemic control together.

The repeated failure mode was specific. Designs that gave reviewers enough criteria evidence to make informed decisions often required extra interaction steps, such as expanding a panel, navigating to a detail view, or loading supporting text. Each step was small in isolation, but across hundreds of screening decisions those steps accumulated into friction. Designs that compressed the display for fast scanning removed the evidence that made override decisions meaningful rather than reflexive.

The resolved design placed the direct quote from the publication visible in the side panel from the outset of the decision interaction. The reviewer could see the AI's decision, the criteria applied, and the specific text evidence used by the AI simultaneously, without an additional interaction step.

The difference between evidence that is available after clicking through and evidence that is visible at the decision point is operational. In the first case, oversight exists in principle. In the second case, oversight is possible within the pace of the review task.

Confidence display in Puraite data extraction

Puraite's data extraction component required a further uncertainty communication decision. AI extractions vary in confidence, and reviewers and project managers scanning extracted data need to identify which extractions warrant attention without examining every extraction in the same way.

Creative Navy's design communicated AI confidence as an explicit percentage with colour-coded visual scanning support. Lower-confidence extractions were made easier to notice. This made the AI's reliability model visible and scannable, so verification effort could be allocated to the outputs most likely to need attention.

The behavioural implication is direct. A project manager reviewing an extraction table does not need to treat a 95%-confidence extraction and a 52%-confidence extraction with the same scrutiny. The interface communicates which is which, making the allocation of verification effort explicit rather than forcing uniform scrutiny.

Blinded screening mode changes when AI judgment is shown

The Puraite engagement also identified a product-level uncertainty communication question: when should AI decisions be shown to reviewers at all? A blinded screening mode was identified as a requirement in which AI recommendations are withheld during initial screening to prevent the AI's judgment from biasing the human reviewer's independent assessment.

The detailed design of the blinded screening mode was not in scope for the engagement. Its identification still matters because it shows that uncertainty communication is not limited to how AI outputs are displayed when visible. In some workflows, showing the AI output changes the epistemic status of the human decision.

Creative Navy's Critical Systems Design method addresses the task-specific tension

Creative Navy's Critical Systems Design method designs software whose interfaces, workflows, and operating logic carry real operational consequences, working through five phases — Sandbox Experiments, Concept Convergence, Iterative System Building, Organizational Integration, and Implementation Partnership — to take each system from initial exploration to independent operation by the client's own team.

In the situation of hidden uncertainty at the decision point, Creative Navy's Critical Systems Design method addresses a task-specific tension rather than applying a generic confidence-display pattern. The right interface depends on the task, user, volume and pace of decisions, and form of AI reasoning in the domain.

In the Puraite engagement, domain learning was constrained because direct user research access was not available within the timeline. Creative Navy used a team member with firsthand experience using systematic review software as a domain learning proxy. This substitution was presented to the client explicitly with its tradeoffs acknowledged, rather than treated as equivalent to observed user research.

The four-iteration option space mapping process was the mechanism for finding the direct-quote-in-side-panel resolution. The resolution was not derived from a general principle alone. It emerged from evaluating candidate designs and identifying how each failed to preserve both review speed and epistemic control.

Evidence basis and limits for the Puraite example

The Puraite engagement ran for 7 months as an Implementation Partnership. The documented outcome is client-reported and indirect: users who had previously perceived Puraite as theoretical or prototype-stage began actively using it after the redesign, and the client launched into a user acquisition and growth phase on that basis.

The available quote is from a single user and was relayed by the client: "Jetzt passt das tool in meine Arbeit" — "Now the tool fits my work." The quote is not independently verified.

No task-time data was collected. No error-rate data was collected. No baseline research was conducted before the engagement. The Puraite example therefore supports a calibrated design claim about how uncertainty was presented at the decision point, but it does not establish a measured effect on screening speed or decision accuracy.