Human factors engineering for medical devices: European regulation and current issues

Detectability of use errors in summative usability tests of medical devices: Impact of the test environment

The regulations on summative usability evaluations of medical devices (MDs) emphasize that the test environment must have sufficient ecological validity for generalization to real-life use. Here, we examined the influence of environmental fidelity (a component of ecological validity) on the detectability of MD use errors. A total of 140 participants participated in a summative usability evaluation of an anaphylactic shock auto-injector device under either a high-fidelity condition or a condition acceptable from the manufacturer's perspective, lower-fidelity condition. The numbers of errors detected in each condition were compared by applying descriptive statistics and logistic and Poisson multivariate regressions. We found that the level of fidelity did not influence the overall number of use errors detected but did influence the detection of certain use errors. To optimize the test environment and increase the detection of use errors, each environmental feature's role in the test task should first be examined.

The use of procedural kits may reduce unscheduled central line dressing changes: A matched pre-post intervention study

Background

Unscheduled dressing changes for central venous lines (CVLs) have been shown to increase the risk of bloodstream infections.

Objective

The objective of this study is to determine if the use of an innovative dressing change kit reduces the rate of unscheduled dressing changes.

Methods

This pre-post interventional study took place at a large, academic, tertiary care center in metro Detroit, Michigan, the United States. We assessed the impact of the interventional dressing change procedure kit on the rate of unscheduled dressing changes for adult patients who underwent placement of a CVL inclusive of a central catheter, peripherally inserted central catheter, or hemodialysis catheter. Data was collected for the pre-intervention cohort through electronic health records (EHRs), while data for the post-intervention cohort were collected by direct observation by trained research staff in combination with EHR data. The primary outcome was the rate of unscheduled dressing changes. Secondary outcomes included rate of unscheduled dressing changes based on admission floor type, etiology of unscheduled dressing changes, and central line-associated bloodstream infections (CLABSIs).

Results

The study included a convenience sample of 1548 CVLs placed between May 2018 and June 2022 with a matched analysis including 488 catheters in each of the pre- and post-intervention groups. The results showed that the unadjusted rate of unscheduled dressing evaluations was significantly reduced from the pre-intervention group (0.21 per day) to the post-intervention group (0.13 per day) (p < .001). The adjusted rate ratio demonstrated the same trend at 1.00 pre- and 0.60 post-intervention (p < .001). Stratifying the analysis based on the highest level of care showed that the intervention was effective in reducing the unadjusted rate of unscheduled dressing evaluations for both the advanced and regular medical floor subgroups pre- to post-intervention; the advanced subgroup had an reduction from 0.22 to 0.15 per day (p = .001), while the regular medical floor subgroup had a reduction from 0.21 to 0.09 per day (p < .001). CLABSIs were similar in both groups (0.6% vs 0.8%; p = 1.00) in pre- and post-intervention groups, respectively.

Discussion

Procedural kits for central line dressing changes are effective in reducing unscheduled dressing changes and may have a role in reducing CLABSI. Further studies assessing the impact of dressing change kits on cost, procedural compliance, and the precise impact on CLABSI are needed.

Benefits of Usability Evaluation in the Development Process of Diabetes Technologies Using the Example of a Continuous Glucose Monitoring System Prototype

BACKGROUND

Usability engineering analyzes the interaction between the intended users and a device. Its implementation is mandatory for manufacturers to obtain regulatory approval for the European market. The aim of this evaluation was assessing the role of usability testing in the development process. For this purpose, a continuous glucose monitoring (CGM) device under development was investigated to determine whether it could be used safely and effectively by the intended users.

METHODS

Conduct of the usability testing was based on the international standard IEC 62366-1. Medical device use of CGM-experienced and non-experienced users (n = 15 each) was observed without initial training in use scenarios containing 18 tasks. The success rate of task completion was determined and the System Usability Scale (SUS) score was calculated from a questionnaire. A prototype of the FiberSense CGM System (EyeSense GmbH, Großostheim, Germany), comprising of a single-use sensor and a reusable detector, was investigated.

RESULTS

Most use errors made by both user groups were related to ease of handling of the reusable detectors. The SUS scores achieved in this study were below the pre-defined SUS score acceptance criterion of ≥68. The most frequently mentioned reason for use errors was an incomprehensible and non-chronological instructions for use (IFU).

CONCLUSIONS

The evaluation provides valuable insights on how to improve usability of the prototype device and demonstrates the value of conducting structured usability testing prior to product finalization. The results reflected areas for improvement of the user interface, mainly by restructuring the IFU, provision of an additional leaflet, and device training prior to use.

Design Development of the SMARTCLIC®/CLICWISE® Injection Device for Self-Administered Subcutaneous Therapies: Findings from Usability and Human Factor Studies

INTRODUCTION

An easy-to-use, multiuse, single-patient, electromechanical autoinjector, the SMARTCLIC®/CLICWISE® injection device, was recently developed to improve the self-administration options available to patients with chronic inflammatory disease treated with biologic agents. An extensive series of studies were conducted to guide the design and development of this device and to ensure its safety and effectiveness.

METHODS

Participants in two user preference studies and three formative human factor (HF) studies evaluated evolving iterations of the autoinjector device, dose dispenser cartridge, graphical user interface, and informational materials; participants in a summative HF test subsequently assessed the final proposed commercially representative product. In the user preference studies, rheumatologists and patients with chronic inflammatory disease, interviewed online and in-person, provided feedback on the design and functionality of four prototypes. In the HF studies, the safety, effectiveness, and usability of adapted prototypes were assessed under simulated-use conditions by patients with chronic inflammatory disease, caregivers, and healthcare professionals (HCPs). The safety and effectiveness of the final refined device and system were confirmed in a summative HF test by patients and HCPs in simulated-use scenarios.

RESULTS

Rheumatologists (n = 204) and patients (n = 39) interviewed in the two user preference studies provided feedback on the device size, feature ergonomics, and usability that guided prototype development in the subsequent formative HF studies. Observations from patients, caregivers, and HCPs (n = 55) participating in the latter studies yielded additional critical design revisions that culminated in development of the final device and system. Of 106 injection simulations conducted in the summative HF test, all resulted in successful medication delivery, and no potential harms were associated with injection-related use events.

CONCLUSION

Findings from this research facilitated development of the SmartClic/ClicWise autoinjector device and demonstrated that it could be used safely and effectively by participants representative of the intended-use population of patients, lay caregivers, and HCPs.

Medical device regulation (MDR) in health technology enterprises - perspectives of managers and regulatory professionals

BACKGROUND

In the European Union (EU), there are over half a million medical devices, varying from pacemakers to software. Medical devices play an important role in health care as they are used in diagnosis, prevention, monitoring, prediction, prognosis, treatment, or to alleviate disease. Medical devices are regulated in the EU by the Medical Device Regulation (MDR), which came into force on 25 April 2017 and into application on 26 May 2021. The demand for regulation arose from the need to establish a transparent, robust, predictable, and sustainable regulatory framework. This study aims to examine how the managers and regulatory professionals in health technology enterprises perceived the application of the MDR and what were their information needs regarding the MDR.

METHODS

A link to an online questionnaire was sent to 405 managers and regulatory professionals representing health technology enterprises in Finland. The study included 74 respondents. Descriptive statistics were used to describe and summarise the characteristics of the dataset.

RESULTS

Information related to the MDR was fragmented and the necessary information was sought from multiple information sources, while the Finnish Medicines Agency (Fimea) was regarded as the most important source of information and training provider. To some extent, the managers and regulatory professionals expressed their dissatisfaction with the performance of Fimea. The managers and regulatory professionals were not very familiar with the ICT systems provided by the EU. The size of an enterprise affected how many medical devices it manufactures and generally affected the views about the MDR.

CONCLUSIONS

The managers and regulatory professionals understood the role of the MDR regarding the safety and transparency of medical devices. The available information regarding the MDR did not properly fit the needs of users and there seemed to be a gap in information quality. The managers and regulatory professionals had some difficulties understanding the available information. Based on our findings, we believe it is paramount to evaluate the challenges faced by Fimea and how it could improve its performance. To some extent, the MDR is regarded as a burden for smaller enterprises. It is important to highlight the benefits of ICT systems and to develop them to better meet the information needs of enterprises.

A three step protocol for the development of an innovative footwear (shoe and sensor based insole) to prevent diabetic foot ulceration

Background

The incidence of diabetic foot ulceration (DFU) is increasing worldwide. Therapeutic footwear is usually recommended in clinical practice for preventing foot ulcers in persons with diabetes. The project Science DiabetICC Footwear aims to develop innovative footwear to prevent DFU, specifically a shoe and sensor-based insole, which will allow for monitoring pressure, temperature, and humidity parameters.

Method

This study presents a three-step protocol for the development and evaluation of this therapeutic footwear, specifically: (i) a first observational study will specify the user requirements and contexts of use; (ii) after the design solutions were developed for shoe and insole, the semi-functional prototypes will be evaluated against the initial requirements; (iii) and a pre-clinical study protocol will enable the evaluation of the final functional prototype. The eligible diabetic participants will be involved in each stage of product development. The data will be collected using interviews, clinical evaluation of the foot, 3D foot parameters and plantar pressure evaluation. This three-step protocol was defined according to the national and international legal requirements, ISO norms for medical devices development, and was also reviewed and approved by the Ethics Committee of the Health Sciences Research Unit: Nursing (UICISA: E) of the Nursing School of Coimbra (ESEnfC).

Results

The involvement of end-users (diabetic patients) will enable the definition of user requirements and contexts of use to develop design solutions for the footwear. Those design solutions will be prototyped and evaluated by end-users to achieve the final design for therapeutic footwear. The final functional prototype will be evaluated in pre-clinical studies to ensure that the footwear meets all the requirements to move forward to clinical studies.

Discussion

The three-step study outlined in this protocol will provide the necessary insights during the product development, ensuring this new therapeutic footwear's main functional and ergonomic features for DFU prevention.

Development and Regulation of Connected Combined Products: Reflections From the Medtech & Pharma Platform Association

PURPOSE

Patients taking a medicinal product in a homecare setting typically use a medical device to facilitate the injection process. Reductions in wireless connectivity costs, combined with the rapid adoption of smartphones with connectivity to cloud-based services, are enabling these drug delivery devices to now be connected to a digital ecosystem as connected combined products (CCPs). The purposes of this article are to identify the challenges in developing and releasing these products when they straddle different regulatory frameworks and standards and to highlight gaps in the European Union regulations.

METHODS

Industry subject matter experts from pharmaceutical, medical device, and consultancy companies, who are members of the Medtech & Pharma Platform Association, formed 4 working groups to address current best practice for developing and releasing CCPs and the different relevant regulatory frameworks. The 4 areas studied were clinical and regulatory, usability and human factors engineering, development and life cycle management, and cybersecurity.

FINDINGS

Development teams require new skills to create innovative products that have a good safety profile and are simple to use, such as design thinking to understand user needs and systems engineering to manage complexity and ensure interoperability. Risk management process should integrate cybersecurity, data privacy, and data integrity, whereas design control processes should enable asynchronous development cycles for hardware and software components. Regulatory frameworks exist for individual components within the CCP. However, for a complex product, regulatory guidance is needed when combining components with different risk and safety profiles and to ensure that the responsibilities and liabilities of companies contributing components are clear. The efficient management of software changes and product updates, as well as dealing with end-of-life hardware and backward compatibility to older software versions, needs agile approaches when it comes to regulatory updates.

IMPLICATIONS

The regulatory uncertainties and development processes outlined in this article need to be addressed. We call for joint discussions among the various stakeholders in the fields of medicinal products, medical devices, and in vitro diagnostics, as well as standalone software, data protection, and cybersecurity experts, together with regulators and lawmakers in the European Union to meet in focused discussion groups with the aim of devising pragmatic solutions and regulations for the benefit of the sector and hence the patients it serves. (Clin Ther. 2022;XX:XXX-XXX) © 2022 Elsevier HS Journals, Inc.

Field Experiences with Handheld Diagnostic Devices to Triage Children under Five Presenting with Severe Febrile Illness in a District Hospital in DR Congo

As part of a field study (NCT04473768) in children presenting with severe febrile illness to Kisantu hospital (DR Congo), we retrospectively compiled user experiences (not performance) with handheld diagnostic devices assisting triage: tympanic thermometer, pulse oximeter (measuring heart rate, respiratory rate and oxygen saturation), hemoglobinometer and glucometer. Guidance documents for product selection were generic and scattered. Stock rupture, market withdrawal and unaffordable prices interfered with procurement. Challenges at implementation included environmental temperature, capillary blood sampling (antisepsis, order of multiple tests, filling microcuvettes and glucose strips), calibration (environmental temperature, cold chain) and liability-oriented communication with a manufacturer. Instructions for use were readable and contained symbol keys; two devices had printed French-language instructions. Shortcomings were poor integration of figures with text and distinct procedures for the oximeter and its sensor. Usability interview revealed appreciations for quick results, visibility of the display and memory function (three devices) but also problems of capillary blood sample transfer, cleaning, too long of a time-to-results (respiratory rate) and size, fitting and disposal of thermometer probes. Pictorial error messages were preferred over alphanumeric error codes but interpretation of symbols was poor. Alarm sounds of the oximeter caused unrest in children and caretakers perceived the device as associated with poor prognosis.

The Optimal Sample Size for Usability Testing, From the Manufacturer's Perspective: A Value-of-Information Approach

OBJECTIVES

For medical devices, a usability assessment is mandatory for market access; the objective is to detect potentially harmful use errors that stem from the device's design. The manufacturer assesses the final version of the device and determines the risk-benefit ratio for remaining errors. Nevertheless, the decision rule currently used to determine the sample size for this testing has statistical limitations and the lack of a clear decision-making perspective.

METHODS

As an alternative, we developed a value-of-information analysis from the medical device manufacturer's perspective. The consequences of use errors not detected during usability testing and the errors' probability of occurrence were embedded in a loss function. The value of further testing was assessed as a reduction in the expected loss for the manufacturer. The optimal sample size was determined using the expected net benefit of sampling (ENBS) (the difference between the value provided by new participants and the cost of their inclusion).

RESULTS

The value-of-information approach was applied to a real usability test of a needle-free adrenaline autoinjector. The initial estimate (performed on the first n = 20 participants) gave an optimal sample size of 100 participants and an ENBS of €255 453. This estimation was updated iteratively as new participants were included. After the inclusion of 90 participants, the ENBS was null for any sample size; hence, the cost of adding more participants outweighed the expected value of information, and therefore, the study could be stopped.

CONCLUSIONS

On the basis of these results, our method seems to be highly suitable for sample size estimation in the usability testing of medical devices before market access.