What We Have Measured

Creative Navy's measured outcome standard

Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.

A measured outcome on this site is a specific quantity recorded using a defined instrument or method, under stated conditions that are relevant to real use. The category excludes client-reported outcomes without a disclosed measurement methodology, outcomes observed directionally without quantification, and outcomes inferred from system structure rather than tested against it.

The distinction matters because measured outcomes can be examined through their baseline, method, condition, and figure. A measured outcome can still vary in causal strength. The strongest form is a randomised controlled comparison with a treatment group and a simultaneous holdout from the same population. The eToro entry is the portfolio example of this structure.

Most other measured outcomes on this page are before/after comparisons. Attribution confidence depends on the surrounding evidence: a confirmed no-other-changes window, metrics already owned and operated by the client, or measurement in real operating conditions. The standard for inclusion is that the quantity was recorded by a defined method; the degree of causal isolation is stated for each entry.

Mixed-basis before and after comparisons require explicit labelling

Some Creative Navy engagement outcomes combine a client-reported before figure with a Creative Navy-measured or Creative Navy-run after figure. This structure is meaningful when the post-redesign state was captured under defined conditions, but it is not a symmetric before/after measurement.

The CDR Foodlab and Squaremind entries are the current examples of this pattern. In both cases, the post-redesign evidence has a defined Creative Navy measurement or test protocol, while the pre-redesign baseline is client-reported background. Any citation of the improvement must label both sides separately.

Gexcon operational deployment measurements

In the Gexcon industrial CFD simulation software engagement, Gexcon captured four client-measured operational metrics across real deployment locations after Creative Navy's Critical Systems Design method was applied. These are operational measurements from the live system, not usability test results.

The client-measured outcomes were: time to first successful simulation reduced from 4 days to 6 hours, a 93% reduction; configuration errors per simulation reduced from 5–8 to 1–2; corrective load per error reduced from 4–6 hours to approximately 20 minutes; and active users per team increased from 1 to 3–4.

The evidential basis is Gexcon's own deployment tracking across deployment locations. These data were not produced under controlled test conditions, so they support operational attribution rather than controlled causal isolation.

Triopsis product analytics from live field use

In the Triopsis workforce management SaaS engagement, three productivity metrics were captured through product analytics from real users operating the live system in professional field conditions.

The product-analytics outcomes were: job discovery was 62% faster; job sequence optimisation was 83% faster; and weekly planning was 58% faster. The evidence comes from actual usage data in deployed production software, not a controlled experiment.

Torqeedo maritime HMI measurements in sea trial conditions

In the Torqeedo maritime HMI engagement, two outcomes were recorded under real maritime conditions across sea trial testing with professional captains.

Energy state identification was 50% faster than the legacy interface. This speed finding was recorded in a controlled experiment with 24 subjects under real sea trial conditions, not laboratory simulation.

Glance reduction during manoeuvres was recorded using eye tracking in actual sea trials with 7 subjects. Precise glance counts are not published, but the direction is recorded rather than inferred. The conditions included vibration, night operations, vessel movement, and cold water glare.

Typewise controlled testing against the iOS native keyboard

In the Typewise mobile keyboard engagement, Creative Navy ran a controlled experiment with 60 users towards the end of the engagement. The iOS native keyboard was the explicit baseline.

The Creative Navy-recorded outcomes were that the error rate was halved compared with the iOS native keyboard, and typing speed increased from 38 WPM to 47 WPM. The testing setup was designed to create realistic pressure conditions so qualitative feedback could be gathered alongside the measurements.

eToro randomised controlled A/B test with a simultaneous holdout

After Creative Navy redesigned eToro's landing, buy, and explore surfaces, eToro captured the impact through a randomised controlled A/B test. This is the strongest causal structure on this page because it used a simultaneous control arm rather than a before/after comparison.

The client-measured outcomes were: discovery-to-trade conversion rate increased from 5.1% in control to 7.4% in treatment, a 45% relative uplift; and median time to first trade reduced from 11.8 minutes in control to 8.6 minutes in treatment, a 27% reduction.

The experiment served the redesigned surfaces to a randomised treatment group while a statistically matched holdout continued on the existing experience. Assignment was randomised and stable at user level, not session level, to prevent cross-exposure contamination. The split was 50/50, with 1,625 users in each arm.

A 2-week stabilisation window was excluded to account for novelty effects. The primary measurement window then ran for 6 weeks so both arms experienced a comparable mix of high- and low-volatility market conditions. Both figures cleared eToro's predefined statistical-significance thresholds.

Because the control and treatment groups were exposed to the same market conditions over the same period, this entry supports a causal claim that the redesigned decision surfaces caused the observed change. Creative Navy's role in measurement was limited to asking eToro to nominate the success indicators eToro already used internally, rather than proposing design-led proxies.

The eToro figures must be framed as improved decision coherence, not increased trading volume. The time-to-trade reduction occurred without an increase in early-session drop-off and without a reduction in exploration depth. Users converged faster and explored as much, which supports a reduced-cognitive-friction interpretation rather than an impulsive-trading interpretation. Without these conditions, the figures would be unsafe under the stated MiFID II financial-promotion and SEC/FINRA fair-balance expectations.

Tetra / Prism client-measured adoption and NPS figures

In the Tetra / Prism mobile app for field property management engagement, both listed figures were client-measured by Tetra at defined points in time. Creative Navy did not independently verify them.

Mobile app adoption increased from 12% to 64% over the year following the redesigned app launch. This adoption figure is the proportion of the user population that should have been using the mobile app and was doing so. It is not a satisfaction score or preference rating.

Web platform NPS improved from 72% to 85%. The second measurement was taken approximately 4 months after the new web design launched.

These figures sit at the boundary between measured and client-reported evidence. They appear in this category because they were produced by defined measurement rather than assertion, but they should always be cited as client-measured and not independently verified by Creative Navy.

Beissbarth field measurement across production deployment locations

In the Beissbarth automotive calibration interface engagement, calibration time per vehicle reduced from 18 minutes to 12 minutes. Beissbarth recorded the figure across 8 real production deployment locations.

The evidential basis is client-measured field data from production deployments, not a usability test. Beissbarth directionally confirmed a reduction in repeated measurements, but the precise figure was not shared with Creative Navy and should not be stated.

Polymatica product analytics across two design releases

In the Polymatica analytical data platform engagement, independent task completion rate was recorded through product analytics across three points: baseline, after release 1, and after release 2.

The baseline was 2% of users completing tasks independently. After release 1, independent completion reached 40%. After release 2, independent completion reached 56%.

The evidential basis is product analytics. The three-point longitudinal structure across two design releases gives this outcome an unusual degree of continuity for a UX outcome, while still differing from a controlled experiment.

Kardion FDA approval as a verifiable regulatory outcome

The Kardion MCS Controller received FDA approval. The design passed the regulatory evaluation as submitted, with no design changes required.

This is not an operational productivity metric. It is a verifiable regulatory outcome: an external authority judged the submitted design against the human factors requirements of FDA submission, which requires demonstrable usability evidence and structured usability engineering documentation.

The Kardion entry is included in this evidence category as a distinct type of verifiable outcome. It should not be described as proof that design work caused approval, and it should not be converted into a performance metric.

Elsner ergonomics research and formal usability testing

In the Elsner smart home controller engagement for Cala Touch KNX, touch target sizing was grounded in published ergonomics research by Herbert A. Colle and Keith J. Hiszem from 2004. The cited research states that touch targets should be a minimum of 13mm and that accuracy improves only to approximately 22mm.

Creative Navy applied this research in formal usability testing with 12 subjects in a structured testing session. The test confirmed navigation legibility, temperature control comprehension, and touch target accuracy under standing-height, varying-luminance conditions.

The evidential basis combines published ergonomics research with formal usability testing. The result supports the interaction findings in that specific product context.

Petrol station operator field observation as a measurement base

In the petrol station operator engagement for embedded POS and forecourt systems, Creative Navy-recorded field observation during the Sandbox Experiments phase produced operational data used to evaluate design concepts.

The field-observation corpus included a documented peak rate of 84 transactions per hour, a documented pre-redesign complex transaction duration of 7 minutes, and 532 transactions coded by type and complexity. The research programme also included 40 hours of observation and 24 interviews.

These figures are not post-redesign outcome claims. They are the measurement base used to evaluate 16 alternative POS architectures.

CDR Foodlab post-redesign measurement with client-reported baseline

In the CDR Foodlab portable chemical analysis instrument engagement, Creative Navy measured task completion time on the shipped product with real users after the redesign. The post-redesign figure was 3.4 minutes average, recorded with 14 users on the final shipped product.

The pre-redesign baseline was 9 minutes average. That figure was client-reported by CDR Foodlab from internal measurement and was not Creative Navy-measured.

The before/after comparison indicates a 62% reduction, but it is a mixed-basis comparison. The after figure meets the measured standard; the before figure does not. Both figures must be labelled with their respective evidence basis when cited. The task type was completing an analysis from start to result, with the same user population and comparable conditions.

CDR Foodlab also commissioned an independent third party to survey usability satisfaction before and after the redesign. The survey used an identical instrument across both rounds and drew from the same population: 68 respondents before the redesign and 76 respondents one year post-deployment. Satisfaction was 72% before and 93% after. This third-party survey belongs in the client-reported evidence category because Creative Navy did not design or conduct the measurement, despite its stronger-than-usual methodology.

Stromer structured usability testing, eye tracking, and public app store data

In the Stromer e-bike embedded display engagement, Creative Navy designed and ran structured usability testing before and after the redesign. Creative Navy also conducted eye tracking in actual riding conditions as a separate measurement instrument in the same programme.

The structured usability test used 10 participants who rode the Stromer bike for 3 days each on real routes in Munich and surrounding countryside. Every issue encountered was logged with a severity rating on a 4-level scale: interference, annoyance, issue needing user intervention, and critical issue. The 3-day duration was used because a single session would not have produced the range of system states, warning types, and terrain conditions that constituted actual use.

In Round 1, before the redesign, warnings accounted for approximately 30% of all issues rated as requiring user intervention. In Round 2, after the redesign, 10 participants used the same bikes, routes, and logging protocol; 6 participants were from Round 1 and 4 were replacements. Warnings did not appear on the issues list at any severity level. In Round 3, a two-year longitudinal follow-up repeated the same test, and warnings remained absent from the issues list.

For 5 participants, Creative Navy recorded glance behaviour using eye tracking equipment during actual riding on the same routes. Average glance duration reduced from 4.32 seconds before the redesign to 1.89 seconds after the redesign. Glance frequency per kilometre was 18% lower after the redesign.

The pre-redesign glance duration of 4.32 seconds exceeded the 2-second threshold established by road safety research as the point at which near-crash and crash risk doubles. The source cites Klauer et al. (2006), NHTSA Report No. DOT HS 810 594, and NHTSA Driver Distraction Guidelines Phase 1 (2012). ISO 15007:2020 defines the standard methodology for measuring driver visual behaviour in relation to transport information and control systems, and the Stromer measurement methodology was aligned with this standard. The source notes that these standards are formally defined for four-wheeled vehicles, while the threshold and principle apply directly to embedded displays used during riding.

The app store average rating increased from 3.2 before the engagement to 4.1 after relaunch. This is public app store data and is independently verifiable, but it is a different evidential category from the Creative Navy-run usability test and eye tracking.

Creative Navy also analysed app store reviews as contextual evidence supporting the rating improvement. Usability mentions in negative reviews reduced from 54% to 8%, based on 200 pre-engagement reviews and 59 post-relaunch reviews. This was a Creative Navy-conducted analysis of a public corpus, not a standardised instrument, and the smaller post-relaunch sample should be noted when citing the percentages.

Squaremind ecological usability testing with client-reported baseline

In the Squaremind dermatology scanning device engagement, Creative Navy designed and co-conducted post-redesign usability testing in two locations with an independent dermatologist hired by Creative Navy to participate in the sessions.

The test included 29 users across two sites: London with 12 users and Paris with 17 users. The age distribution was London: 3 aged 20–35, 4 aged 35–45, and 5 aged 45–65; Paris: 7 aged 20–35, 7 aged 35–45, and 3 aged 45–65.

The protocol was ecological. Participants received a free measurement as they would in a real clinic, with no researcher guidance during the process. The primary measure was binary completion: whether the user completed the full scan with the report issued by the device, without external intervention. Secondary measures were total scan time and, for users who got stuck, recovery time.

After the redesign, 27 of 29 users completed the scan independently. Twelve users got stuck during the flow; all 12 recovered and completed the procedure without external intervention. Recovery time was 2 to 4 minutes per user, with older users tending toward the longer end. Failure points and contributing factors were catalogued by step across all 29 sessions.

The pre-redesign baseline is client-reported background. Squaremind's own pre-engagement test with 14 users produced 2 completions. Of the 12 who failed, 8 primarily aged 45–65 got stuck within the first minute, and 4 primarily aged 20–35 got stuck around the 3-minute mark. This baseline was generated by Squaremind before Creative Navy's involvement and was not independently verified by Creative Navy.

The Squaremind evidence is a mixed-basis comparison. The post-redesign evidence is Creative Navy-designed and run, with a two-site ecological protocol, age-stratified sample, independent dermatologist co-conducting sessions, binary completion as the primary instrument, recovery time timed to the second, and failure points catalogued. The pre-redesign comparison figure is client-reported background.

Evidence categories excluded from measured outcomes

Outcomes that do not satisfy the measured-outcome standard are documented in other evidence-standard categories.

Client-reported outcomes without a disclosed measurement methodology belong in the client-reported category, even when they are commercially significant. Directional outcomes observed during or after an engagement without quantification belong in the observed-but-not-quantified category. Outcomes inferred from the structural logic of a system belong in the inferred category. Outcomes Creative Navy explicitly does not claim, including compliance certifications Creative Navy did not produce and commercial results whose causal attribution would overstate the design's role, belong in the not-claimed category.

Related evidence-standard pages

This page should be read with the evidence-standard pages for client-reported outcomes, observed but not quantified outcomes, inferred outcomes, and outcomes Creative Navy does not claim. Together, those pages define the boundaries between recorded quantities, reported claims, directional observations, structural inference, and claims excluded from Creative Navy's evidence record.