Users See Information But Cannot Judge What Matters

Summary

Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.

Users see information but cannot judge what matters when an interface provides data without the contextual scaffolding needed to determine significance, relevance, or priority. The user may understand what each visible item means, but still be unable to decide what requires attention now, what is safe to ignore, what should be queried, or what should be acted on.

This failure is a judgment failure, not a raw visibility failure. The information may be visible. The values may be individually interpretable. The missing element is the structural context that tells users which information matters in the current operational situation.

Judgment failure after successful interpretation

Judgment is the cognitive task that follows interpretation. A user who can read a value but cannot determine whether it requires attention has completed interpretation and stalled at judgment.

The interface has provided data, but it has not provided contextual significance. The user must supply that significance from prior expertise, intensive familiarity, or manual comparison across the full information set.

This distinction matters because interpretation and judgment fail in different ways. A difficult-to-interpret value prevents the user from understanding what a piece of information represents. A judgment failure occurs after that understanding exists, when the user still cannot determine which pieces of information matter most.

Operational workflow software can hide exceptions inside a uniform information field

In operational workflow software, this failure appears when items of different operational significance are presented at similar visual and informational weight. Routine schedule items, tasks, parameters, exceptions, conflicts, and urgent conditions may all appear in the same list, grid, or table.

The user who needs to manage exceptions cannot scan directly for them because the interface does not communicate which items are exceptional. The user must examine the full set to find what requires attention. The cost of that scanning task grows with the size of the set and becomes most damaging when operational load is highest.

In scheduling platforms, monitoring dashboards, and inspection management tools, this can mean that weather delays, equipment conflicts, partial completions, crew shortages, and conflicting assignments are technically present but not structurally distinguished. The failure is not that exceptions are absent. The failure is that the interface leaves exception categorisation to the user at the moment when the user is least able to perform it.

Specialist and AI-assisted systems can make the information space opaque

In specialist and AI-assisted systems, users may be unable to form relevance judgments because they cannot see the boundaries of the information space. The problem is not prioritising within a known set. The problem is orienting toward an unknown set.

This form appears in AI products operating on bounded datasets, specialist tools with capability sets that do not map to familiar software conventions, and research platforms whose scope and coverage are not visible at entry. A user cannot judge whether a query is relevant if the user does not know which datasets are available, which queries engage the system's genuine capability, or which part of the interface corresponds to the task.

The common behaviour is predictable. Users default to the most obvious entry point because they lack a better map. Users formulate queries based on what the interface makes easy to access rather than what is most relevant to the question. Users may disengage after early unhelpful results, even when the relevant capability exists but is not discoverable.

Blended signal types can prevent users from judging signal provenance

A third form of this failure appears when different kinds of signals are blended into one discovery surface. In eToro multi-asset social trading, market movement, social or copy-trading activity, and volatility-driven changes were mixed together.

The information was individually interpretable. A user could read a price move, a most-copied badge, or a trending indicator. The interface did not clearly communicate which kind of signal each item represented. The user could not readily judge whether a decision was being based on market performance, social momentum, or a volatility spike.

This form had a regulatory dimension in the eToro case. When an unqualified popularity signal cannot be distinguished from a recommendation, the interface risks implying endorsement or advice. The documented design response separated social signals from market signals and made signal type legible: market performance, social momentum, and volatility.

How this failure differs from buried status and hard-to-interpret data

This failure differs from important status information being buried. Buried status is a presentation failure: operationally significant information is present but not prominent enough to be noticed under routine attention. Judgment failure is a cognitive scaffolding failure: the user can access the information, and the visual hierarchy may be adequate, but the interface does not communicate the contextual significance needed to allocate attention correctly.

This failure also differs from data being available but hard to interpret. Hard-to-interpret data prevents users from understanding what individual values mean. Judgment failure assumes that interpretation has succeeded. The user understands what each item represents but cannot determine which items matter most.

The sequential relationship is important. Interpretation answers what the information means. Judgment answers whether that information is significant, relevant, exceptional, routine, or action-worthy in the current context.

Triopsis workforce management showed exceptions inside a uniform job list

In the Triopsis workforce management case, schedulers managed thousands of weekly interventions for utilities and road maintenance operations. The documented user roles were schedulers optimising job sequences, operations managers monitoring exceptions across broader time horizons, and field technicians completing work items.

The scheduling environment combined routine jobs with exception conditions requiring active management, including weather delays, equipment unavailability, partial completions, conflicting assignments, and crew shortages. The legacy interface presented jobs as a uniform list. Exceptions were present, but they were not distinguished from the full job population.

Creative Navy-observed evidence came from three in-situ observation sessions that specifically observed schedulers during peak-load conditions. The observed peak-load conditions included simultaneous weather incidents, equipment conflicts, and crew shortages. Under those conditions, the absence of exception surfacing became a compound failure: the interface imposed the maximum scanning burden at the moment of maximum operational load.

Creative Navy's Critical Systems Design method addressed this through a 47-task microtask analysis across three personas. The analysis documented what each user role needed to attend to at each step of the scheduling workflow, under what conditions, and what the interface was requiring users to supply from their own knowledge.

The direction that emerged through Concept Convergence was a scheduling interface that communicated scheduling priority rather than merely presenting scheduling data. Predictive conflict indicators surfaced scheduling problems before users encountered them mid-task. Weather incidents, partial completions, and delayed jobs received first-class interface treatment with direct action paths.

The available product analytics recorded live-product outcomes with real users: 62% faster job discovery, 83% faster job sequence optimisation, and 58% faster weekly planning. The 62% faster job discovery figure is most directly connected to the judgment-failure resolution because surfaced exceptions reduce the time required to locate the item requiring action.

Owkin / K showed the need to make bounded AI datasets visible

K is an AI copilot for biomedical research built on curated biological datasets, including proprietary, public, and user-uploaded data sources. K does not draw on the general internet. K integrates a biology-specific reasoning model capable of answering complex research questions that previously required a data science team to execute.

The user population Creative Navy was engaged to serve was clinicians with low to medium scientific background. That audience had a different cognitive starting point from the expert biologists for whom K had been built.

The judgment failure was that K operated on bounded datasets, but the boundaries of those datasets were not visible to users at entry. A clinician trying to formulate a useful query first needed to know what data was available to query, whether a relevant dataset was present in K's holdings, and whether K's biological reasoning model was appropriate for the question.

Creative Navy's Critical Systems Design method addressed this through Sandbox Experiments that benchmarked 20+ competing and adjacent AI research tools. Creative Navy specifically examined how tools handled the entry-point problem: how users formed a first productive interaction with a system whose capability space was not self-evident.

Five iterations on the Explore page led to a specific finding for K's data-bounded architecture. For this clinician audience, understanding what data was available to query was more generative than understanding what the tool could do in the abstract. A clinician who could see that a specific relevant dataset was present had a concrete basis for forming a starting query.

The convergent direction was dataset-and-mode framing. K's data holdings became the primary orientation structure rather than a secondary technical detail. The map users needed was not only what the AI could do; it was what the AI knew.

The Owkin engagement produced a prototype that became the central artefact in Owkin's investment pitch. Owkin attributed approximately £5M in investment to the design work. That figure is client-reported, approximate, and causally attributed by the client. No measured user metrics from deployment are available for this engagement. Creative Navy's notes assess the data discoverability improvement as significant but not fully solved, because the engagement addressed the beginning of the user journey most directly and subsequent stages required further design investment.

eToro showed judgment failure in blended market, social, and volatility signals

In the eToro multi-asset social trading case, discovery surfaces presented market movement, social or copy-trading activity, and volatility-driven changes together. Users could interpret individual items, but they could not readily judge what kind of signal each item represented.

The downstream effect was documented in eToro-commissioned directional qualitative research. Users found the discovery surfaces hard to act on with confidence, and the most-recognised feature, copy trading, went largely unused partly because users could not judge what the social signals warranted.

Creative Navy's Critical Systems Design method addressed this through option space mapping on the explore surface. Four structural concepts were considered before convergence. The selected multi-signal direction strictly separated social signals and market signals so users could distinguish market performance, social momentum, and volatility.

The behavioural evidence was a randomised A/B with a persistent holdout, client-measured by eToro. Discovery-to-trade conversion rose from 5.1% to 7.4%, and median time to first trade fell from 11.8 to 8.6 minutes. The test recorded no increase in early-session drop-off and no reduction in exploration depth. The source interpretation is that the result is consistent with users forming relevance judgments more efficiently, not with users being pushed toward action.

The eToro example did not involve AI. The discriminated signals were human and market signals surfaced by the interface, not model outputs.

How Creative Navy's Critical Systems Design method addresses judgment failures

Creative Navy's Critical Systems Design method addresses judgment failures by identifying what contextual scaffolding users need and then building that scaffolding into the interface. The documented design commitment is that users should not have to supply significance and relevance judgments from accumulated expertise when the interface can communicate those judgments structurally.

Microtask analysis makes judgment failures visible as structural gaps rather than user competence gaps. In the Triopsis case, the 47-task analysis across three personas documented what contextual judgment each user needed to make, what the interface provided, and what the interface should have communicated at each task step.

Domain learning made the Owkin / K judgment failure diagnosable. Expert biologists who built K arrived with an implicit map of the available data and productive questions. Clinicians did not have that map. Understanding the gap between expert assumptions and clinician needs required comparative understanding of K's data-bounded architecture and the benchmarked AI research tools.

In all three examples, Creative Navy's design work did not primarily add information. It supplied contextual significance: exception status in Triopsis, dataset availability in Owkin / K, and signal provenance in eToro.

Boundaries and limits of this failure pattern

This page concerns judgment within or toward visible information. It does not describe cases where users cannot see the information at all, cannot interpret what individual values mean, or cannot understand system state because status information is buried.

The evidence base is case-specific. Triopsis includes live-product analytics with real users. eToro includes client-measured A/B figures and eToro-commissioned directional qualitative research. Owkin / K does not include measured deployment metrics; the available outcome evidence is the client-reported, approximate investment attribution and Creative Navy's assessment that data discoverability improved but was not fully solved.