Assessing the usability by clinicians of VISION: A hierarchical display of patient-collected physiological information to clinicians

Visualization of Patient-Generated Health Data: A Scoping Review of Dashboard Designs

BACKGROUND

Patient-centered clinical decision support (PC CDS) aims to assist with tailoring decisions to an individual patient's needs. Patient-generated health data (PGHD), including physiologic measurements captured frequently by automated devices, provide important information for PC CDS. The volume and availability of such PGHD is increasing, but how PGHD should be presented to clinicians to best aid decision-making is unclear.

OBJECTIVES

Identify best practices in visualizations of physiologic PGHD, for designing a software application as a PC CDS tool.

METHODS

We performed a scoping review of studies of PGHD dashboards that involved clinician users in design or evaluations. We included only studies that used physiologic PGHD from single patients for usage in decision-making.

RESULTS

We screened 468 titles and abstracts, 63 full-text papers, and identified 15 articles to include in our review. Some research primarily sought user input on PGHD presentation; other studies garnered feedback only as a side effort for other objectives (e.g., integration with electronic health records [EHRs]). Development efforts were often in the domains of chronic diseases and collected a mix of physiologic parameters (e.g., blood pressure and heart rate) and activity data. Users' preferences were for data to be presented with statistical summaries and clinical interpretations, alongside other non-PGHD data. Recurrent themes indicated that users desire longitudinal data display, aggregation of multiple data types on the same screen, actionability, and customization. Speed, simplicity, and availability of data for other purposes (e.g., documentation) were key to dashboard adoption. Evaluations were favorable for visualizations using common graphing or table formats, although best practices for implementation have not yet been established.

CONCLUSION

Although the literature identified common themes on data display, measures, and usability, more research is needed as PGHD usage grows. Ensuring that care is tailored to individual needs will be important in future development of clinical decision support.

Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems: Metrics and Evaluation Methods for eHealth Systems

BACKGROUND AND OBJECTIVES

Electronic healthcare records have become central to patient care. Evaluation of new systems include a variety of usability evaluation methods or usability metrics (often referred to interchangeably as usability components or usability attributes). This study reviews the breadth of usability evaluation methods, metrics, and associated measurement techniques that have been reported to assess systems designed for hospital staff to assess inpatient clinical condition.

METHODS

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, we searched Medline, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, and Open Grey from 1986 to 2019. For included studies, we recorded usability evaluation methods or usability metrics as appropriate, and any measurement techniques applied to illustrate these. We classified and described all usability evaluation methods, usability metrics, and measurement techniques. Study quality was evaluated using a modified Downs and Black checklist.

RESULTS

The search identified 1336 studies. After abstract screening, 130 full texts were reviewed. In the 51 included studies 11 distinct usability evaluation methods were identified. Within these usability evaluation methods, seven usability metrics were reported. The most common metrics were ISO9241-11 and Nielsen's components. An additional "usefulness" metric was reported in almost 40% of included studies. We identified 70 measurement techniques used to evaluate systems. Overall study quality was reflected in a mean modified Downs and Black checklist score of 6.8/10 (range 1-9) 33% studies classified as "high-quality" (scoring eight or higher), 51% studies "moderate-quality" (scoring 6-7), and the remaining 16% (scoring below five) were "low-quality."

CONCLUSION

There is little consistency within the field of electronic health record systems evaluation. This review highlights the variability within usability methods, metrics, and reporting. Standardized processes may improve evaluation and comparison electronic health record systems and improve their development and implementation.

Technology Acceptance in Mobile Health: Scoping Review of Definitions, Models, and Measurement

BACKGROUND

Designing technologies that users will be interested in, start using, and keep using has long been a challenge. In the health domain, the question of technology acceptance is even more important, as the possible intrusiveness of technologies could lead to patients refusing to even try them. Developers and researchers must address this question not only in the design and evaluation of new health care technologies but also across the different stages of the user's journey. Although a range of definitions for these stages exists, many researchers conflate related terms, and the field would benefit from a coherent set of definitions and associated measurement approaches.

OBJECTIVE

This review aims to explore how technology acceptance is interpreted and measured in mobile health (mHealth) literature. We seek to compare the treatment of acceptance in mHealth research with existing definitions and models, identify potential gaps, and contribute to the clarification of the process of technology acceptance.

METHODS

We searched the PubMed database for publications indexed under the Medical Subject Headings terms "Patient Acceptance of Health Care" and "Mobile Applications." We included publications that (1) contained at least one of the terms "acceptability," "acceptance," "adoption," "accept," or "adopt"; and (2) defined the term. The final corpus included 68 relevant studies.

RESULTS

Several interpretations are associated with technology acceptance, few consistent with existing definitions. Although the literature has influenced the interpretation of the concept, usage is not homogeneous, and models are not adapted to populations with particular needs. The prevalence of measurement by custom surveys suggests a lack of standardized measurement tools.

CONCLUSIONS

Definitions from the literature were published separately, which may contribute to inconsistent usage. A definition framework would bring coherence to the reporting of results, facilitating the replication and comparison of studies. We propose the Technology Acceptance Lifecycle, consolidating existing definitions, articulating the different stages of technology acceptance, and providing an explicit terminology. Our findings illustrate the need for a common definition and measurement framework and the importance of viewing technology acceptance as a staged process, with adapted measurement methods for each stage.

Usability of an Educational Intervention to Overcome Therapeutic Inertia in Multiple Sclerosis Care

Background: Educational interventions are needed to overcome knowledge-to-action gaps in clinical care. We previously tested the feasibility and potential efficacy of an educational intervention that facilitates treatment decisions in multiple sclerosis care. A demonstration of the usability of such an intervention is crucial prior to demonstration of efficacy in a large trial. Objectives: To evaluate the usability of a novel, pilot-tested intervention aimed at neurologists to improve therapeutic decisions in multiple sclerosis (MS) care. Methods: We surveyed 50 neurologists from Chile, Argentina, and Canada randomized to an educational intervention arm of a pilot feasibility study using the System Usability Score (SUS) to assess the usability of a traffic light system (TLS)-based educational intervention. The TLS facilitates therapeutic decisions, allowing participants to easily recognize high-risk scenarios requiring treatment escalation. The SUS is a validated 10-item questionnaire with five response options. The primary outcome was the average and 95% confidence interval (CI) of the SUS score. Values above 68 are considered highly usable. Results: Of 50 neurologists invited to be part of the study, all completed the SUS scale and the study. For the primary outcome, the average usability score was 74.7 (95%CI 70.1-79.2). There was one outlier with a score of 35. The usability score excluding the outlier was 76.8 (95%CI 72.7-80.8). Multivariate analysis revealed no association between participants' characteristics and the SUS score. Conclusions: Our educational intervention has shown high usability among neurologists. The next step is to evaluate the effectiveness of this educational intervention in facilitating treatment decisions for the management of multiple sclerosis in a large trial.