The Art of Developing New Products
The essential link between ethnographic research, human factors engineering and risk management is explained by Alun Wilcox of PDD Group.
The increasing demand by regulatory bodies to demonstrate that sound usability engineering practices have been applied to medical device development appears to have started a minor panic within the industry to find out how to apply them. This article attempts to clarify how, when and where and explain the value of an integrated approach to the process of risk management, which utilises the skills of researchers, human factors engineers, designers and engineers throughout.
Usability engineering activities come under the overall banner of risk management, but there are important differences between what is defined in ISO 14971 and in ISO 62366.
ISO 14971:2009, Application of Risk Management to Medical Devices, is the systematic application of management policies, procedures and practices to the task of analysing, evaluating and controlling risk. It is a decision-making process for determining acceptable risk, but the control measures are only applied on risks identified as being unacceptable.
In contrast, ISO 62366:2007, Medical Devices, Application of Usability Engineering to Medical Devices, is a design and development process that addresses risks associated with usability and it applies to all identified risks.
Since March 2010, compliance with ISO 62366 is required by European regulatory authorities. Figure 1 shows where the two ISO standards overlap and how they interact.
Compliance with the standard’s predecessor, ANSI/AAMI HE74:2001, Human Factors Design Process for Medical Devices, has been required by the US Food and Drug Administration (FDA) for more than ten years. ANSI/AAMI HE75:2009, Human Factors Engineering, Design of Medical Devices, is also now formally recognised by FDA.
Figure 1: Where the two ISO standards overlap and how they interact
The value of research
The starting point of risk management and usability engineering is to identify hazards by understanding the relationship between people, products and places. However, research does much more than identify hazards. It can be used to decide what products or services should be developed and to commercially justify those decisions. In a fiercely competitive and increasingly financially squeezed medical marketplace, companies are proving commercial viability by identifying points of differentiation.
Differentiation can be achieved in a number of ways: cost, service, superior technology, and in drug delivery clinically proven improved drug efficacy or improved therapy compliance. It can also be achieved through enhancing stakeholder (patient, carer or clinician) experience. HE75 states, "Human factors are not only about safe and effective task performance, but also about user satisfaction.”
Research helps to identify opportunities for differentiation that have the potential to add real value, and to understand what user satisfaction means to all stakeholders. Different skill sets are required to gain this holistic understanding.
We use ethnographic research methods to understand the current state-of-play surrounding a product, service or experience: how people live, what they think, why they make the choices they do, what problems they encounter, what aspects they enjoy, and how they interact with other people, products, technology, systems and cultures. Ethnographic methods differ from other research methods in that the research is conducted in the context of use. This allows us to see the familiar artefacts and natural unpredictability of the participants’ real world, which cannot be replicated in an artificial environment.
Human factors engineers define the skills and the physical, perceptual and cognitive abilities of individuals making up the distinct user groups for the device, whether patients, carers or clinicians. Any special circumstances that could affect device use, including variability of abilities among the stakeholder groups, should be described, as well as any special training required.
Understanding how the various stakeholders interact with products in different environments of use also enables human factors engineers to conduct comparative task analyses and establish baselines with which to compare new products. The environment in which a device is used (operating room, hospital bed or home) can affect the actual use of the device because of variations in conditions such as noise, lighting and temperature.
Designers and engineers also need to be immersed in the challenges of people, products and places so that they can consider holistic solutions and apply appropriate technology, not just technology for technology’s sake. There are other, softer elements of environments that need to be explored such as storage of devices and drugs and whether patients are self-conscious about drawing attention to their condition.
Dispatch to disposal mapping
A useful framework for structuring research and identifying opportunities is to map the product or therapy journey. Mapping journeys from start to finish (dispatch to disposal) ensures that the interaction touch points of all stakeholders are defined and their needs and issues at each of these touch points are understood. Stakeholders will include prescribing clinicians, nurses, pharmacists, purchasing officers, biomedical engineers, carers and patients.
Figure 2 shows a map produced for BOC Medical (now a member of the Linde Group) as part of a review of handling of medical gas cylinders. The map highlighted that half of the life of the product took place away from the final usage within the hospital and with completely different stakeholders: BOC fillers and delivery drivers.
Figure 2: Medical gas cylinder journey map.
Courtesy of BOC Medical
Tangible benefits of mapping
Mapping can be conducted prior to any research activities to help clarify how much a company knows about the target market and to highlight information gaps. When targeting international markets, it is important to understand the cultural differences of product or therapy journeys. For example, when we were researching the use of urine meters for Unomedical (now part of ConvaTec) across several markets we found significant differences in attitude and practices in relation to infection control and the handling of urine. Another important finding was that these products were seen as commodity products and not something that warranted a higher cost, even if they had more advanced infection control features. This realisation redirected the product development away from additional and more costly infection control features and moved the focus to the numerous other opportunities the research uncovered.
Although the nature of this product development was evolutionary, the value of the research has been demonstrated by the commercial success. Since its launch in 2007, the UnoMeter Safeti Plus (Figure 3) was the single biggest significant driver of Unomedical’s innovation rate. In just two and a half months, Unomedical broke even on its investment in research, design and innovation and sales have, on average, exceeded expectations by more than 50%.
Another framework that is critically important in the pharmaceutical world is emotional mapping. By spending time with patients and families in their home environments, one can gain an understanding of what it is like to live with a particular condition, how it affects their day-to-day lives on practical and emotional levels, and what drives attitudes and behaviour. Understanding these issues and being innovative in developing solutions can present rich opportunities for differentiation.
Figure 3: UnoMeter Safeti Plus
The next steps
Thorough research enables companies to evaluate opportunities against the differentiation potential, value added options, commercial criteria, technical risk and freedom to operate. Having decided what to do next, the well-trodden and understood path of design inputs, design outputs, verification and validation, and risk management can begin. What is perhaps not so well understood is the role that usability engineering is now required to play in this process.
As shown in Figure 1, usability engineering runs in parallel and feeds the risk management process. Having defined an application specification, and through an understanding of the people and places identified potential use related hazards, now begins the process of risk control and demonstration that risks have been mitigated. This involves usability testing, which should be conducted iteratively throughout the development of any device. It is a formal method of systematically observing and recording representative users performing real tasks with a real or simulated product.
Usability testing can verify that design outputs match design inputs and it is appropriate for measuring observable user actions or assessing whether usability objectives are attainable. Specifying usability objectives enables metrics to be defined that can be applied during testing to provide quantitative test acceptance criteria. The actual target values of usability objectives should be established with input from users through research.
Formative usability testing
Formative testing supports the process of improvement and is often iterative: a round of evaluation followed by a round of design refinement, followed by another round of evaluation, and so forth. Formative evaluations do not require large numbers of testers because we are not seeking statistical validity, but rather design guidance. That is, understanding what testers find easy or difficult so that we can focus on those concepts that have the most potential and refine them further before taking them into the final stages of development.
The resulting test reports become components of the usability engineering file or, for FDA the human factors engineering file. This should happen throughout the product development process with the design/prototype fidelity increasing at each stage.
There are no fixed numbers required, but it is generally recommended that five to eight participants of each target user group (patients, nurses and doctors) are involved. Ideally, the testing is conducted in the actual use environments, but this can be staged in research facilities or offices.
Summative usability testing
Summative testing is the last usability testing a medical device manufacturer completes. The goal is to validate the final design to prove that all use-related hazards have been mitigated. There are more precise requirements for conducting validation tests including the definition of formal acceptance criteria.
It is recommended that there are 15-20 participants in each user group. Again the testing can happen outside actual use environments, but these will have to accurately simulate conditions of the use environment. Location does not need to be geographically specific providing there are no significant cultural differences in user groups.
Testing must be conducted with production equivalent units. This means the actual hardware, software, labelling, instructions-for-use and training materials are used. The tasks performed must include the primary operating functions (frequent and critical tasks) and any tasks with associated safety risks. It is critical to relate tasks to the hazards and risks analysis.
Advantages of an interdisciplinary team
Human factors engineers and researchers are ideally qualified to structure and facilitate usability testing. Designers and engineers are involved in the risk management process and should also be involved in usability testing to understand any issues first hand so that they can react appropriately to further mitigate risks.
Involving an interdisciplinary team of researchers, human factors engineers, designers and engineers throughout the process of device research and development offers important advantages. The whole team being immersed in the holistic challenge prevents misinterpretation of the aims and parameters and generates real opportunities to add value and differentiate.
Alun Wilcox is Director of Medical at PDD Group Ltd,
87 Richford Street
W6 7HJ, UK,
tel. +44 (0)20 8735 1111,
If you found this article interesting please
Sign Up Now for your free subscription to