Uncertainty Is Not Communicated Usefully

Summary

Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.

Uncertainty is not communicated usefully when an AI system produces outputs with varying reliability, but the interface presents those outputs as if they had uniform apparent reliability. The user sees an output that appears plausible and complete, but does not see enough information to decide whether to accept it, override it, or verify it.

The failure is not that uncertainty exists. AI systems that perform non-trivial reasoning produce outputs with varying confidence, varying evidence basis, and varying structural reliability. The failure is that this variation remains inside the model or system and is not surfaced in a form the user can act on.

Failure pattern: AI outputs appear uniformly reliable

An AI interface creates this failure when it displays outputs as results rather than as estimates. The user must then choose between over-trusting outputs the AI itself has low confidence in or treating every output with equal scepticism and verifying everything.

Both responses are miscalibrated relative to the actual reliability profile of the system. Over-trust ignores low-confidence outputs. Uniform scepticism removes the efficiency that AI assistance was supposed to create. The interface has not given the user the distinction needed to allocate attention and verification effort appropriately.

This failure is a specific AI-context instance of a wider interpretation failure: users can see information but cannot judge what matters. In the AI uncertainty case, the issue is that users cannot determine which AI outputs warrant verification because the interface does not communicate the AI system's confidence and evidence basis.

Useful confidence information must be present at the decision point

Confidence information must be visible at the moment the user decides whether to accept or override an AI output. If confidence information exists only behind a detail view, explanation panel, or separate confidence summary screen, the cost of accessing it compounds across high-volume review work.

This is a design failure rather than a user diligence failure. When the cost of exercising oversight is higher than the benefit under real use conditions, the oversight mechanism becomes nominally available and operationally unused. Users may open the confidence detail for edge cases and skip it for routine-seeming decisions, even though the AI system's confidence variation does not necessarily track the user's sense of what is routine.

The design condition is direct availability. Confidence information needs to appear in the same view from which the user makes the accept or override decision, without requiring an additional interaction before the user can decide whether verification is warranted.

Useful confidence signals distinguish graded reliability

A binary uncertainty flag communicates less than the variation it represents. AI systems do not produce only two classes of output, such as reliable and unreliable. They produce a distribution of confidence levels.

For verification work, a 95%-confidence extraction and a 52%-confidence extraction require different responses. The higher-confidence extraction may need quick review. The lower-confidence extraction may need scrutiny closer to the treatment of an unreliable draft.

A useful interface communicates confidence as a graded signal that can be scanned. The user should be able to identify low-confidence outputs without examining every output individually. This supports efficient allocation of verification effort while preserving human oversight where it is needed.

Useful uncertainty communication includes evaluable reasoning

A confidence score without context is not fully actionable. A 70%-confidence output based on a misread publication is not equivalent to a 70%-confidence output based on a genuinely ambiguous evidence match. The score may be the same while the epistemic basis differs.

Useful uncertainty communication therefore includes enough reasoning or evidence for the user to evaluate the AI system's basis for the output. The user needs to see not only the confidence level, but also the information that explains why the AI system reached that confidence level.

The design tension is that complete reasoning can slow down review, while too little reasoning prevents meaningful oversight. The structural requirement is to make reasoning visible without additional interaction, organised so that users can scan the confidence signal quickly and examine the underlying reasoning without leaving the decision context.

Puraite example: AI confidence in systematic literature review

Puraite is a web application for AI-assisted systematic literature reviews in academic, clinical, and pharmaceutical research contexts. The review process involves screening large volumes of publications against inclusion and exclusion criteria, then extracting structured data from qualifying studies.

Puraite integrates AI assistance into screening decisions, suggested inclusion and exclusion criteria, and structured data extraction. Reviewers encounter AI outputs at decision points and must decide whether to accept each output or override it.

In the Puraite engagement, uncertainty communication was the central design problem in two areas: the AI screening screen and the data extraction flow. The AI system produced outputs with varying reliability, and the interface needed to communicate that variation in a way that supported calibrated review at the pace required by systematic literature review.

No dedicated user research was possible within the engagement timeline. A member of the Creative Navy team had direct firsthand experience using systematic review software, and this operational knowledge served as the domain learning proxy. The tradeoff was presented to the client explicitly: the proxy was not equivalent to observed user research, but it was the available approach under the engagement constraints.

Puraite screening design: direct evidence at the decision point

Creative Navy's design work on the Puraite screening interaction addressed two failure modes. A display that was too compact would allow reviewers to move quickly but would not support informed override decisions. A display that was too detailed would support evaluation but would slow review to the point where AI assistance no longer provided the intended efficiency.

Four iteration cycles on the suggestion display were required before a resolution was found. Iterations that provided full criteria and evidence required expansion or navigation, adding an interaction cost across hundreds of decisions. Iterations that compressed the display for scan speed removed the evidence needed for substantive override decisions.

The resolved design placed the direct quote from the publication visible in the side panel from the outset of the decision interaction. The quote was the specific text the AI used to reach its inclusion or exclusion conclusion. The reviewer could see the AI decision, the criteria applied, and the text evidence simultaneously.

This design made confidence information and reasoning available at the decision point without an additional step. A reviewer could accept quickly when appropriate, or evaluate the AI system's basis for its conclusion without navigating away from the decision context.

Puraite extraction design: confidence as a scannable graded signal

The Puraite data extraction flow required a separate uncertainty communication decision. AI-extracted values appeared across a table that project managers and reviewers needed to scan to identify which extractions warranted attention.

The design communicated AI confidence as an explicit percentage with colour-coded scanning support. Lower-confidence extractions were visually marked for attention without requiring the reviewer to examine every extraction individually. A 52%-confidence extraction and a 95%-confidence extraction received different visual treatment.

This supported allocation of verification effort based on the AI system's confidence rather than on uniform scrutiny or uninformed judgment.

Blinded screening was identified as a product-level requirement

The Puraite engagement also raised the question of when AI decisions should be withheld from reviewers entirely. A blinded screening mode would prevent the AI system's recommendation from biasing the reviewer's independent judgment.

The detailed design of blinded screening was not in scope for the engagement. Its identification was a finding, and it shows that uncertainty communication in human-AI interaction can include decisions about when not to show an AI output, not only decisions about how to show confidence.

eToro analogue: market uncertainty at the point of commitment

eToro is a non-AI analogue for this failure pattern. The uncertainty in the eToro buy flow was market and outcome uncertainty, not AI confidence. The analogue is bounded: it demonstrates the broader interpretation principle of communicating uncertainty at the decision point, but it is not evidence about AI confidence communication.

The pre-redesign buy flow presented a trade as a price-and-quantity confirmation, which communicated an implied single expected result. The redesigned flow introduced structured scenario framing, showing how a position might behave under different market movements. Downside exposure was visible at the moment of commitment rather than discoverable afterwards.

The regulatory framing in the eToro example was also bounded. The uncertainty was presented as ranges rather than predictions, which the documented case connects to MiFID II / SEC-FINRA rules for risk communication that must not imply a guaranteed outcome.

The behavioural confirmation was a randomised A/B result: conversion changed from 5.1% to 7.4%, time-to-trade changed from 11.8 minutes to 8.6 minutes, with no increase in drop-off and no reduction in exploration depth. These figures measure decision efficiency and coherence, not uncertainty-communication quality directly. The connection to this failure pattern is the removed single-value framing and the structured-range replacement.

How Creative Navy's Critical Systems Design method addresses the failure

Creative Navy's Critical Systems Design method treats good AI behaviour as a design problem as well as a model problem. In this failure pattern, the central question is not only whether the AI system can produce a confidence value. The design question is what confidence, reasoning, and evidence the user needs at the decision point to make a calibrated judgment.

The correct implementation depends on the task, user, decision volume, work pace, and form of AI reasoning in the domain. In systematic review, the relevant evidence unit was a publication and the specific text passage used by the AI. The transferable design principle was reasoning visible at the decision point without additional interaction. The specific implementation was the direct-quote-in-side-panel resolution.

The Puraite iterations show why this is a structural design problem. Compressing the display violated the requirement for informed override decisions. Expanding the display violated the requirement for scan speed. The resolved design emerged from iterating through those failure modes under conditions that approximated real review pace and volume.

Evidence basis and known limits

The strongest evidence on this page is the documented Puraite design record: the engagement ran for 7 months as an Implementation Partnership, the suggestion display went through four iteration cycles, and the final design made the AI decision, applied criteria, and specific publication quote visible in the decision context.

The Puraite outcome evidence is weaker. The primary documented outcome is client-reported: users who had previously perceived Puraite as theoretical or prototype-stage began actively using it after the redesign, and the client launched into a user acquisition and growth phase on that basis. A single user quote was relayed by the client: “Jetzt passt das tool in meine Arbeit” — “Now the tool fits my work.” This quote is client-reported and not independently verified.

No task-time, error-rate, or baseline research data was collected during or before the Puraite engagement. No dedicated user research was possible within the engagement timeline. The systematic review domain learning proxy was based on direct firsthand experience from a Creative Navy team member, not on observed user research.

The eToro analogue has a different evidence boundary. The randomised A/B figures report decision efficiency and coherence, not direct uncertainty-communication quality. eToro should therefore be treated as a non-AI demonstration of the general decision-point uncertainty principle, while Puraite remains the grounded AI-confidence example.