Research In Complex Operational Contexts

Summary

Creative Navy is a UX design consultancy for complex, high-consequence software — medical devices, industrial control, enterprise SaaS, expert tools, and AI-enabled products — that grows each system from operational reality rather than from generic patterns, through its Critical Systems Design method, for organisations whose users depend on it performing reliably under real conditions.

Research in complex operational contexts is needed when the conditions that shape interface use cannot be reproduced accurately through interviews, laboratory sessions, or standardised research protocols. The capability covers domain learning, in-situ observation, participant access, representative participant selection, real-time protocol adaptation, triangulation-not-confirmation, and the construction of a research corpus that can support design decisions.

The central distinction is that domain learning is the prerequisite for research, not an outcome of it. In expert tools and operational environments, Creative Navy must acquire enough operational understanding to ask valid questions, observe the right behaviours, and distinguish temporary learning-curve issues from structural system problems.

When complex operational research is needed

Creative Navy uses complex operational research when interface behaviour depends on real work conditions, expert judgement, multi-role coordination, physical environment, or institutional governance. These conditions include vessel control at sea, clinical and reception work in veterinary clinics, workforce scheduling under operational pressure, CFD simulation work, petrol forecourt transactions under peak load, multi-institutional health research governance, embedded consumer controls, customs administration across countries, and patient scanning in dermatology workflows.

Interviews are not treated as sufficient evidence in these contexts. Stakeholder interviews and user interviews have different purposes and different validity. Stakeholders reveal organisational priorities; users reveal operational reality. Neither alone is sufficient when organisational expectations, user self-report, observed behaviour, and benchmarked competitor behaviour conflict.

Representative participant selection is also part of the research problem. In expert tools, representative users are domain experts. Non-expert participants produce invalid findings when the work requires professional captains, surgeons, CFD engineers, schedulers, cashiers under peak load, or institutional research staff operating under specific governance paths.

What the capability does

Creative Navy's research in complex operational contexts starts by establishing the domain vocabulary, operational structure, and evidence sources required for valid investigation. The work may include field research, in-situ observation, structured interviews, stakeholder interviews, controlled experiments, eye tracking, surveys, benchmarking, dealer reviews, focus groups, unstructured observation, ecological testing, and expert co-validation.

The research does not use multiple evidence sources simply to confirm the same finding. Triangulation-not-confirmation uses evidence sources against each other to find discrepancies. In the documented engagements, discrepancies between stakeholder priorities, user self-report, observed behaviour, benchmarking, governance requirements, and clinical judgement became findings that the design work had to answer.

The output is a research corpus: a documented body of evidence from the research programme. The source examples include 102 tasks in Gexcon, 532 transactions in the Swiss petrol forecourt research, and 47 microtasks in Triopsis. These corpora made the design work tractable by converting observed operational complexity into documented task, transaction, role, and behaviour structures.

Domain learning before formal research in expert tools

Creative Navy's Gexcon CFD simulation research shows domain learning as a prerequisite to valid research. Before user sessions began, the team studied calibration manuals, watched YouTube tutorials, reviewed Gexcon's internal training videos, and ran controlled tests inside the application. Two intensive four-hour stakeholder sessions established the underlying workflow logic.

The depth of domain learning was directly commented on by Franz Zdravistch, Chief Training Engineer: "I can't believe how much you learned on your own in three days, even some of the experts I train need more time." In this engagement, domain learning was necessary before the formal research programme because the product concentrated expert work in a single environment.

The formal Gexcon research programme included 24 user interviews, 23 workplace observations, 9 stakeholder interviews, and benchmarking of 12 competitor products. The documented research corpus contained 102 individual tasks, including goals, frequency, difficulty, actions, and hierarchy of needs within sequences. The 102-task scope followed from the product's expert-work concentration, rather than the 20–30 tasks described as typical in enterprise software audits.

In-situ observation reveals tacit work and workaround patterns

Creative Navy uses in-situ observation when users cannot fully articulate the rationale for their own practice or when operational behaviour changes under real conditions. This includes the blanks phenomenon: participants who cannot explain the rationale for their own practice, where tacit knowledge must be surfaced through observation and structured teaching methods rather than direct questioning.

In the Swiss petrol forecourt research, 40 hours of structured observation documented and coded 532 transactions. The research observed 36 cashiers and included 24 interviews. The peak transaction rate was 84 transactions per hour, documented during field observation rather than from interview self-report. Complex mixed transactions ran up to 7 minutes. The transaction corpus became a baseline for later design evaluation against real operating conditions.

In the Triopsis workforce management research, the in-situ observation sessions took place under real operational pressure. Schedulers were handling weather incidents, conflicting locations, overlapping jobs, and sudden crew shortages. Interviews had not produced an accurate picture of this work; observation did. The combined evidence mapped 47 microtasks.

Operational immersion in maritime HMI research

Creative Navy's Torqeedo maritime HMI research was operationally immersive because the product operates in conditions that cannot be replicated elsewhere. The research included 12 sea trials over 6 months with 15 professional captains. The temperature range was −5°C to +35°C, and the conditions included vibration, hull slamming, spray, glare from cold water, rain, gloved interaction, and night operations through early morning.

The sea trials produced a finding that interviews or laboratory observation could not have produced: the emotional dimension of vessel control interfaces. The research identified the relief crews experience when information remains stable while the vessel behaves unpredictably. This finding became a structural design requirement.

The research used the physical blueprint of the vessel as a reference structure, including propulsion motors, the central control centre, the power balancing system, and auxiliary loads. Grounding the research in the vessel's architecture ensured correspondence between what was discovered and how the vessel actually works.

The Torqeedo research used five evidence sources: legacy system analysis, sea trial observation with 15 captains across 12 sessions, a controlled experiment with 24 subjects, eye tracking with 7 subjects, and structured participant feedback from 15 captains. These sources had different roles and were used against each other rather than treated as parallel confirmation.

Real-time protocol adaptation during live fieldwork

Creative Navy uses real-time protocol adaptation when early field sessions reveal patterns that the original research protocol did not anticipate. This differs from research programmes that fix protocols in advance and keep them unchanged throughout fieldwork.

The IDEXX Animana field research covered 35 clinics, 150+ participants, 2 weeks, and 3 countries: the Netherlands, the UK, and Germany. The clinic sample included urban single-practitioner clinics, suburban group practices, and large corporate networks. The participant roles were vets, nurses, reception staff, and administrative staff, ranging from first-week users to ten-year veterans.

The range of experience levels was deliberate. Including both first-week users and ten-year veterans helped separate temporary learning-curve problems from permanent structural problems. When handwritten workarounds appeared in multiple clinics, including checklists taped to monitors and printed reference sheets near terminals, field protocols were updated in real time to probe those patterns in subsequent visits.

A central coordination point allowed real-time data compilation during live fieldwork, so protocol adjustments could be made before the next visit. The central finding was that reception and clinical roles had incompatible cognitive requirements. This finding was not in any participant's explicit description; it emerged from observing the same system used by different roles under genuinely different conditions.

Stakeholder research exposes organisational tensions that user research cannot

Creative Navy treats stakeholder interviews and user interviews as different evidence sources with different validity. In Triopsis, 5 stakeholder interviews surfaced competing organisational priorities that user research could not. The founder's understanding of the product differed from developers', developers' understanding differed from sales', and sales' understanding differed from key clients'. These differences were treated as structural tensions in the organisation, not as noise to filter out.

The Triopsis research held five evidence sources in tension: stakeholder priorities conflicted with each other; user self-report conflicted with observed behaviour; benchmarking revealed gaps the client had not identified as problems. Each conflict became a question the design had to answer.

In Akrivia Health, Creative Navy conducted 14 individual interviews and 3 focus groups with 24 participants across NHS analysts, academic researchers, and pharmaceutical research staff. These groups were not aggregated into a single researcher persona. The research treated them as three distinct institutional contexts with different governance paths through the same system.

The Akrivia Health differences were the primary research finding. NHS analysts had clear governance boundaries between research and operational use. Academic teams faced lengthy ethics and data access approvals. Pharmaceutical teams had audit obligations and regulatory reporting requirements. Triangulation across groups showed conflicts between what NHS analysts needed from the interface, what pharmaceutical teams needed, and what academic researchers assumed about the system.

Multi-source evidence prevents single-method overreach

Creative Navy uses multiple evidence sources when one method cannot reach all relevant behaviour, market variation, engineering constraints, or operational context. In Elsner Elektronik, four evidence sources were used against each other: ergonomics research by Colle and Hiszem 2004 on touch targets, a 20-user survey distributed and collected by the client, formal usability testing with 12 subjects in one structured session, and dealer prototype reviews with 30 responses from 10 countries.

Each evidence source had a different role. The ergonomics research determined exact touch target sizing parameters. The survey confirmed known issues and yielded two undocumented use cases and one new mental model. Dealer feedback revealed behaviour differences across markets. Benchmarking disqualified gestural patterns that survey enthusiasm might otherwise have favoured. Engineering sessions revealed firmware behaviour that neither user research nor ergonomics literature addressed.

In WCO/IPM, the research involved 47 participants across Italy, Romania, Uzbekistan, Algeria, and Spain. The geographic spread was deliberate validation that a design for 107 governments held under genuinely different operational conditions. Remote observation combined with workshops allowed research across member administrations where on-site visits were not feasible for all markets.

Unstructured observation is appropriate when measurement would produce noise

Creative Navy's Squaremind research includes a methodological case where unstructured observation was selected deliberately because structured measurement would not have produced useful signal. Before redesign decisions were made, Creative Navy conducted 4 observation sessions of the existing patient interface in France. The sessions were not structured as measurement.

The reason was that the existing system had produced 2 completions from 14 patients in Squaremind's own internal test. Applying a structured measurement protocol to a system performing at that level would have produced noise rather than insight. The observation sessions were designed to produce situated understanding of the physical relationship between patient and screen, the way confusion appeared in body movement before abandonment, the scan sequence and its physical demands, and the absence of a guidance structure when a step became unclear.

This establishes a boundary for measurement. When a system is performing so poorly that systematic measurement produces noise rather than insight, unstructured observation can be the right research instrument. Observation finds the terrain. Structured measurement becomes useful once the terrain is understood well enough to know what to measure.

Clinical co-validation adds expert judgement to usability completion data

Creative Navy's Squaremind post-redesign testing added an independent dermatologist hired and paid by Creative Navy to co-conduct the sessions. The testing involved 29 users across London and Paris, with 12 participants in London and 17 in Paris. Participants were age-stratified across three cohorts: 20–35, 35–45, and 45–65.

The dermatologist's participation added clinical judgement that binary completion data could not contain. A patient could complete the scan while being incorrectly positioned for clinically significant body regions. That result would register as completion in usability data, but not in clinical assessment.

The age stratification was a deliberate participant selection choice. The commercial claim required the interface to work for the full clinical patient population, not only for digitally fluent younger users. Age-stratified sampling was the methodology used to produce evidence credible to a clinical buyer about whether the interface held across that range.

The Squaremind post-redesign evidence basis records ecological, age-stratified testing co-conducted by an independent dermatologist, with 27/29 completions, 12/12 recoveries, and recovery times timed to the second.

Outputs of complex operational research

Creative Navy's research in complex operational contexts produces documented evidence structures rather than only summary findings. These structures include task maps, microtask maps, coded transaction corpora, role differences, governance differences, observed workaround patterns, triangulated conflicts, participant feedback, and evidence distinctions between stakeholder priorities and user behaviour.

The output depends on the operational context. Gexcon produced a 102-task corpus. Swiss petrol forecourt research produced a 532-transaction corpus coded by type and complexity. Triopsis produced 47 mapped microtasks. Torqeedo produced five evidence sources that included sea trial observation, controlled experimentation, eye tracking, legacy system analysis, and structured participant feedback.

These outputs are used to make design work tractable in systems where the work cannot be understood from interface screens alone. The evidence identifies what users do, what users cannot articulate, where self-report conflicts with observed behaviour, and which constraints come from the physical, organisational, institutional, or expert context.

Boundaries and limits

Research in complex operational contexts does not mean that every engagement uses the same research method. The documented examples vary by domain, access, evidence need, system maturity, and operational risk. Some programmes used structured observation; some used unstructured observation; some used controlled experiments; some used focus groups; some used stakeholder interviews, benchmarking, or dealer reviews.

The documented evidence is engagement-specific. Findings from sea trials, clinics, scheduling teams, petrol forecourts, CFD engineers, dermatology patients, customs administrations, or institutional health research groups should not be generalised to all operational systems without further evidence.

The evidence also shows that interviews, surveys, and tests can be valid only within their role. Interviews can reveal priorities or self-report; observation can reveal behaviour and workaround patterns; benchmarking can reveal gaps the client did not identify; clinical co-validation can identify issues that completion data cannot contain. No single evidence source is treated as sufficient across the documented engagements.

What this produces

Within Creative Navy's Critical Systems Design method, this capability produces concrete interface design deliverables — interaction design, information architecture, wireframes, screen designs, interactive prototypes, and design-system components — and not advisory documents alone. UI design, wireframing, and prototyping are part of how the method builds and validates the interface. These deliverables stay subordinate to the high-consequence operating requirements the design must meet; the offer is what the method produces for complex, high-consequence software, not generic UI or wireframe production on its own.