What Is Observed But Not Quantified
This evidence standard defines outcomes confirmed through direct observation, prototype testing, deployment patterns, or structured feedback where the direction is known but no numerical metric exists. It separates Creative Navy-observed findings from client-reported, inferred, measured, and explicitly non-claimed outcomes.
Creative Navy uses this category when the direction of a change was directly observed but not quantified.
The absence of a number indicates that no measurement instrument or controlled comparison was applied, not that the direction is unknown.
This category is distinct from client-reported outcomes because the source is direct observation rather than the client's account alone.
This category is distinct from inferred outcomes because the finding comes from watching the system in use, prototype testing, support patterns, or deployment behaviour.
Measured outcomes with a figure produced by a defined methodology belong in the measured category, not here.
Examples include Callsign analyst workflow improvement, Polymatica support-pattern change, Gexcon training-format change, Beissbarth repeated-measurement reduction, Elsner prototype validation, Hudex non-expert access, and Squaremind field and demo observations.
The Owkin / K finding is the most bounded example in this category because it is directional and based partly on client characterisation.
The Squaremind examples include 4 unstructured field observation sessions in France and Creative Navy attendance at 5 of 9 clinic purchase demos.
Definition of observed-but-not-quantified evidence
Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.
An observed-but-not-quantified outcome is an outcome where Creative Navy directly witnessed a change, or where engineers working with Creative Navy confirmed the change in prototype testing, structured user sessions, or deployment patterns. The direction of the change can be stated with confidence, but no measurement methodology was applied and no figure was captured.
The absence of a number is not treated as permission to estimate a number. It means the observation conditions did not include a measurement instrument, controlled comparison, or comparable baseline. The finding is retained as directional evidence only.
How observed-but-not-quantified evidence differs from other evidence categories
Observed-but-not-quantified evidence is distinct from client-reported evidence because the source is direct observation rather than the client's account alone. Creative Navy, or engineers working with Creative Navy, saw the relevant behaviour, prototype performance, support-pattern change, or operational change occur.
Observed-but-not-quantified evidence is distinct from inferred evidence because it comes from watching the system in use or seeing a deployment-level pattern, not from reasoning only from the system's structure. An inferred claim reasons from how a design is likely to affect behaviour. An observed finding records something witnessed in use, testing, support data, or operational practice.
Observed-but-not-quantified evidence is distinct from measured evidence because no defined measurement methodology produced a figure. If a figure exists from a defined methodology, the outcome belongs in the measured category. If Creative Navy explicitly does not claim an outcome, it belongs in the non-claimed category.
Callsign analyst workflow improvement was observed without task timing
In the Callsign engagement, the redesign of the policy engine and fraud scenario configuration interface was followed by early internal testing with Callsign's analysts. Creative Navy-observed evidence from those sessions confirmed that the new journeys reduced the time required to express a common fraud scenario in the tool and made explanations during client calls more straightforward.
No task completion times were recorded. No controlled before-and-after comparison was conducted. The evidence supports the direction of the finding: fraud scenarios were faster to express and easier to explain during client calls. It does not support a numerical performance claim.
The reason no figure is attached is methodological. The testing was structured to confirm direction and surface interaction problems, not to produce a performance metric. A quantified claim would have required a controlled comparison with the previous interface under matched conditions.
Polymatica support patterns showed that the messy-data failure mode largely disappeared
In the Polymatica engagement, early users of the analytical platform commonly encountered a messy-data problem: real imported data contained formatting inconsistencies, non-standard characters, or structural irregularities, and users received errors they could not diagnose. This failure mode generated a significant share of early support requests.
After the redesign introduced a data preparation and preview step, the messy-data failure mode largely disappeared from support requests. The evidence is Creative Navy-observed at deployment level through changes in support request patterns, not through a controlled experiment.
No before-and-after count is available because support request volume was not tracked against categories before the redesign in a way that produced a comparable baseline. The supported claim is categorical: a problem that had been common stopped appearing in support data.
Owkin / K discoverability improved directionally at the start of the journey
In the Owkin / K engagement, the redesign of K's entry experience included the Explore page, prompt suggestions, and dataset presentation. After this redesign, discoverability improved at the start of the user journey: users arriving on the platform could identify where to start and what data was available to query.
The evidence is directional and bounded. It comes from internal product launch feedback observed and reported by the engagement team, together with Owkin's characterisation of the outcome. Owkin reported that complaints from the internal product launch matched what Creative Navy had identified and proposed solutions for.
No measured user metrics are available. There are no task completion times and no independent usability test. The engagement was scoped to produce a design direction for implementation, not to measure post-deployment user performance.
Gexcon onboarding changed from instructor-led training to webinars and video materials
In the Gexcon engagement, the CFD simulation platform redesign was followed by a change in how Gexcon delivered onboarding. Formal 3-day instructor-led training events were replaced with short webinars and video materials.
This is an observed operational change in deployment practice, not a measured reduction in training time per session. The supported claim is that Gexcon changed the training model. The evidence does not support a specific cost saving or quantified training-time reduction.
The Gexcon metrics for simulation time, error rate, and corrective load are measured outcomes and belong in the measured evidence category. The training-format change belongs here because it is a structural change in deployment practice rather than a captured metric from the live system.
Beissbarth repeated measurements were reduced, but the figure was not shared
In the Beissbarth engagement, technicians performing calibrations with the redesigned interface made fewer repeated measurements. The direction was confirmed by Beissbarth's own deployment measurement across 8 locations, but the exact figure was not shared with Creative Navy.
This finding sits at the boundary between client-measured and observed-but-not-quantified evidence. Beissbarth tracked the outcome, so the direction was produced by defined measurement rather than by post-hoc account. Creative Navy cannot state the figure, so the finding does not belong in the measured category on this site.
The documented mechanism is clear: the redesign produced unambiguous state communication across three device classes under real workshop conditions, including movement, variable lighting, and viewing distance. Technicians who previously remeasured because they were uncertain whether a reading had stabilised stopped doing so because the interface communicated state clearly enough to remove that uncertainty.
Elsner prototype testing confirmed firmware-aligned behaviour and sensor fault handling
In the Elsner engagement, the redesigned interface surfaced sensor fault states explicitly rather than hiding them. The relevant states included delayed readings, contradictory values from the sensor network, and calibration drift.
The behaviour of the interface under those conditions was confirmed in prototype testing with Elsner's engineers. This evidence is Creative Navy-observed in functional validation sessions, not a formal usability test with scored tasks.
The supported claim is directional and functional: the firmware-aligned behaviours performed as intended under the operating conditions the interface was designed for. No performance metric was attached.
Elsner product managers could iterate the interface independently at engagement close
At the close of the Elsner engagement, Elsner's product managers confirmed that they were able to iterate the UI without Creative Navy's involvement. The outcome is an observed result of Organizational Integration: the client team had the capability to extend the design system without Creative Navy remaining involved.
No metric is attached because capability transfer is not a quantity in this evidence standard. The supported claim is that the condition existed at a named point in the engagement: the end of the engagement.
Hudex non-expert access improved after an observed orientation failure
In the Hudex engagement, research during the Sandbox Experiments phase confirmed that users without training could not orient themselves in the platform on arrival. The dondogram was effective for expert analysts, but it had no entry point for non-experts.
The redesign introduced a project overview layer and progressive disclosure architecture. Following the redesign, non-expert users could extract value from the platform independently.
The directional outcome belongs here because it is grounded in an observed structural failure and an observable structural change that addressed it. Hudex user reception data is treated separately as client-reported evidence because it was gathered for Hudex's internal marketing effort.
Squaremind field observations recorded failure patterns before the redesign
In the Squaremind engagement, Creative Navy conducted 4 unstructured observation sessions in France during Sandbox Experiments. Creative Navy watched patients attempt the existing scanning interface.
The observation was deliberately not structured as measurement because the existing system was performing so poorly that systematic measurement would have produced noise rather than signal. The sessions produced qualitative findings: the physical pattern of confusion before abandonment, the absence of any recovery structure in the interface, and the way patients who got stuck stopped and waited because the interface gave them no next step.
The evidence is Creative Navy-observed in field sessions. The supported direction is that the existing interface had no recovery path and that patients who encountered difficulty had no route back. The 14-user pre-engagement test with 2 completions was conducted by Squaremind before Creative Navy's involvement and is treated as client-reported background, not as the basis for this observed finding.
Squaremind clinic purchase demos showed unassisted completion in 5 observed demos
After the Squaremind redesign and the London/Paris testing, Squaremind ran demonstrations for 9 clinics. Creative Navy attended 5 of the 9 demonstrations as silent observers and took notes.
The demonstrations were not usability testing sessions. The buyers were clinic staff who walked through the patient scanning experience to evaluate whether their patients would be able to complete it without assistance. In all 5 observed demos, the clinic staff completed the patient experience without assistance or intervention.
The evidence is Creative Navy-observed across 5 demos. The supported claim is qualitative: clinic buyers could complete the patient experience unassisted and found it convincing. The commercial outcome that all 9 clinics purchased is client-reported and belongs in the client-reported evidence category.
Boundaries of the observed-but-not-quantified category
An outcome does not belong in this category if it was communicated by the client without Creative Navy directly observing it. Those outcomes belong in the client-reported category.
An outcome does not belong in this category if it is reasoned from system structure rather than observed in use. Those outcomes belong in the inferred category.
An outcome does not belong in this category if it carries a figure produced by a defined measurement methodology. Those outcomes belong in the measured category.
An outcome does not belong in this category if it describes something Creative Navy explicitly does not claim. Those items belong in the non-claimed category.
The defining criterion is direct observation of direction without a quantified figure. The page preserves what was witnessed and avoids inventing numbers where no measurement was captured.
Related evidence standards and case examples
This evidence standard should be read alongside the standards for client-reported evidence, inferred evidence, measured evidence, and outcomes Creative Navy does not claim. The named examples on this page include Callsign, Polymatica, Owkin / K, Gexcon, Beissbarth, Elsner, Hudex, and Squaremind.
- Observed-but-not-quantified outcomes are directly witnessed directional changes where no measurement methodology was applied and no figure was captured.
- The Callsign redesign was observed in internal testing to make common fraud scenarios faster to express and easier to explain during client calls, without recorded task completion times.
- The Polymatica messy-data failure mode largely disappeared from support requests after a data preparation and preview step was introduced.
- Gexcon changed onboarding from formal 3-day instructor-led training events to short webinars and video materials after the CFD simulation platform redesign.
- Beissbarth technicians made fewer repeated measurements, with direction confirmed by Beissbarth deployment measurement across 8 locations but no figure shared with Creative Navy.
- Elsner prototype testing confirmed that sensor fault handling and firmware-aligned behaviours performed as intended under the designed operating conditions.
- Creative Navy directly observed Squaremind failure patterns in 4 unstructured field observation sessions in France and later observed unassisted completion in 5 of 9 clinic purchase demos attended as silent observers.
- Owkin / K discoverability improved directionally at the beginning of the user journey after redesign of the entry experience.
- At engagement close, Elsner product managers confirmed they could iterate the UI without Creative Navy's involvement.
- Hudex research confirmed that untrained users could not orient themselves on arrival, and the redesign gave non-expert users an independent starting point.
- Observed-but-not-quantified outcomes do not support numerical claims unless a measurement methodology and figure exist elsewhere.
- The category records direction, not magnitude.
- Client-reported outcomes are excluded unless Creative Navy directly observed the relevant behaviour, pattern, or operational change.
- Inferred outcomes are excluded unless the finding was observed in use, prototype testing, deployment patterns, or structured sessions.
- The Owkin / K example is bounded because it is based partly on client characterisation and no measured user metrics are available.
- The Beissbarth example does not state the reduction figure because the exact figure was not shared with Creative Navy.
- The Squaremind demo observations were not usability testing sessions and did not apply a performance metric.