Usability study of clinical exome analysis software: top lessons learned and recommendations

Improving Medication Information Presentation Through Interactive Visualization in Mobile Apps: Human Factors Design

Background

Despite the detailed patient package inserts (PPIs) with prescription drugs that communicate crucial information about safety, there is a critical gap between patient understanding and the knowledge presented. As a result, patients may suffer from adverse events. We propose using human factors design methodologies such as hierarchical task analysis (HTA) and interactive visualization to bridge this gap. We hypothesize that an innovative mobile app employing human factors design with an interactive visualization can deliver PPI information aligned with patients’ information processing heuristics. Such an app may help patients gain an improved overall knowledge of medications.

Objective

The objective of this study was to explore the feasibility of designing an interactive visualization-based mobile app using an HTA approach through a mobile prototype.

Methods

Two pharmacists constructed the HTA for the drug risperidone. Later, the specific requirements of the design were translated using infographics. We transferred the wireframes of the prototype into an interactive user interface. Finally, a usability evaluation of the mobile health app was conducted.

Results

A mobile app prototype using HTA and infographics was successfully created. We reiterated the design based on the specific recommendations from the usability evaluations.

Conclusions

Using HTA methodology, we successfully created a mobile prototype for delivering PPI on the drug risperidone to patients. The hierarchical goals and subgoals were translated into a mobile prototype.

Incorporating user feedback in the design of a genetics analysis tool: A two-part approach

While general usability assessment models for websites have been developed for a wide variety of contexts, research literature on incorporating user feedback in the design of online scientific tools is lacking. In this article, we present an approach that we developed and illustrate how it was used to elicit user feedback of the AnalyzeMyVariant tool, which enables geneticists to use family pedigree data to calculate pathogenicity likelihood ratios for variants of unknown significance. We reviewed existing usability literature and developed a survey instrument emphasizing concepts of importance to online, data-driven, scientific tools. The items on the survey instrument were grouped in four categories: usability, quality, privacy and security, and satisfaction. We performed a two-part evaluation using the survey and a semi-structured interview protocol. The survey instrument was used to collect data about the use experience of AnalyzeMyVariant from 57 genetic experts and trainees who were recruited via email invitations. We also conducted semi-structured interviews with six genetics experts to explore work contexts in which users might use the tool and further delve into issues faced in tool use. Interviews were inductively coded and major themes identified using the constant comparative method. We found that the needs of genetics professionals vary for research- and clinically-focused work. These differences can inform the design of tools to serve their needs. The major contribution of this work is the description of a two-part method to elicit user feedback to inform the design of online, data-driven, scientific tools, which focuses on constructs of particular relevance to these tools such as usability, quality, privacy, security, and satisfaction. The survey instrument that we developed, coupled with contextual interviews, may serve as an example that can be used by others conducting usability studies of similar tools. In addition, our results emphasize the importance of considering contextual factors such as background knowledge, situational factors, and the intended application of results, in the usability evaluation of scientific software. It is our hope that this two-part approach might be adapted to assess the usability of other online scientific tools and facilitate the design of tools to meet the needs of their target audiences.

A User-Centered design and usability testing of a web-based medication reconciliation application integrated in an eHealth network

BACKGROUND

Medication discrepancies, which are a threat to patient safety, can be reduced by medication reconciliation (MedRec). MedRec is a complex process that can be supported by the use of information technology and patient engagement. Therefore, the SEAMPAT project aims to develop a MedRec IT platform based on two applications. The application for the professionals is called: the "MedRec app".

OBJECTIVE

In the present study, we aimed to describe the development and usability testing of the MedRec app, reporting results of a three iterations user-centered usability evaluation.

METHODS

We used a three phase iterative user-centered study spread over 16 months. At each phase, the usability evaluation included several methods (observations, questionnaires, and follow-up discussions with participants) to collect quantitative and qualitative data in order to improve the current prototype and evolve to the next prototype.

RESULTS

In total, 48 healthcare professionals (25 general practitioners and 23 hospital clinicians) participated to the MedRec app evaluation. There were 14, 32 and 5 participants for phases 1, 2 and 3 respectively. At each phase, many design modifications were done to strengthen usability. Concerning usability, participants considered the prototypes as an acceptable interface with a median System Usability Score of 73 at phase 2 and 75 at phase 3. Participants emphasized the need for improvements concerning workflow integration, usefulness and interoperability.

CONCLUSION

The MedRec app was perceived as being useful, usable and satisfying. However, further improvements are required in several usability aspects. Our study demonstrates the importance of conducting usability assessments before investing time and resources in a large study evaluating the effect of an eMedRec approach on clinical outcomes. Our findings may also increase the chances of acceptability and sustained use over time by clinicians.

Testing of the assisting software for radiologists analysing head CT images: lessons learned

Background

Assessing a plan for user testing and evaluation of the assisting software developed for radiologists.

Methods

Test plan was assessed in experimental testing, where users performed reporting on head computed tomography studies with the aid of the software developed. The user testing included usability tests, questionnaires, and interviews. In addition, search relevance was assessed on the basis of user opinions.

Results

The testing demonstrated weaknesses in the initial plan and enabled improvements. Results showed that the software has acceptable usability level but some minor fixes are needed before larger-scale pilot testing. The research also proved that it is possible even for radiologists with under a year’s experience to perform reporting of non-obvious cases when assisted by the software developed. Due to the small number of test users, it was impossible to assess effects on diagnosis quality.

Conclusions

The results of the tests performed showed that the test plan designed is useful, and answers to the key research questions should be forthcoming after testing with more radiologists. The preliminary testing revealed opportunities to improve test plan and flow, thereby illustrating that arranging preliminary test sessions prior to any complex scenarios is beneficial.

Towards Usable E-Health. A Systematic Review of Usability Questionnaires

BACKGROUND

The use of e-health can lead to several positive outcomes. However, the potential for e-health to improve healthcare is partially dependent on its ease of use. In order to determine the usability for any technology, rigorously developed and appropriate measures must be chosen.

OBJECTIVES

To identify psychometrically tested questionnaires that measure usability of e-health tools, and to appraise their generalizability, attributes coverage, and quality.

METHODS

We conducted a systematic review of studies that measured usability of e-health tools using four databases (Scopus, PubMed, CINAHL, and HAPI). Non-primary research, studies that did not report measures, studies with children or people with cognitive limitations, and studies about assistive devices or medical equipment were systematically excluded. Two authors independently extracted information including: questionnaire name, number of questions, scoring method, item generation, and psychometrics using a data extraction tool with pre-established categories and a quality appraisal scoring table.

RESULTS

Using a broad search strategy, 5,558 potentially relevant papers were identified. After removing duplicates and applying exclusion criteria, 35 articles remained that used 15 unique questionnaires. From the 15 questionnaires, only 5 were general enough to be used across studies. Usability attributes covered by the questionnaires were: learnability (15), efficiency (12), and satisfaction (11). Memorability (1) was the least covered attribute. Quality appraisal showed that face/content (14) and construct (7) validity were the most frequent types of validity assessed. All questionnaires reported reliability measurement. Some questionnaires scored low in the quality appraisal for the following reasons: limited validity testing (7), small sample size (3), no reporting of user centeredness (9) or feasibility estimates of time, effort, and expense (7).

CONCLUSIONS

Existing questionnaires provide a foundation for research on e-health usability. However, future research is needed to broaden the coverage of the usability attributes and psychometric properties of the available questionnaires.

User-centered design of multi-gene sequencing panel reports for clinicians

The objective of this study was to develop a high-fidelity prototype for delivering multi-gene sequencing panel (GS) reports to clinicians that simulates the user experience of a final application. The delivery and use of GS reports can occur within complex and high-paced healthcare environments. We employ a user-centered software design approach in a focus group setting in order to facilitate gathering rich contextual information from a diverse group of stakeholders potentially impacted by the delivery of GS reports relevant to two precision medicine programs at the University of Maryland Medical Center. Responses from focus group sessions were transcribed, coded and analyzed by two team members. Notification mechanisms and information resources preferred by participants from our first phase of focus groups were incorporated into scenarios and the design of a software prototype for delivering GS reports. The goal of our second phase of focus group, to gain input on the prototype software design, was accomplished through conducting task walkthroughs with GS reporting scenarios. Preferences for notification, content and consultation from genetics specialists appeared to depend upon familiarity with scenarios for ordering and delivering GS reports. Despite familiarity with some aspects of the scenarios we proposed, many of our participants agreed that they would likely seek consultation from a genetics specialist after viewing the test reports. In addition, participants offered design and content recommendations. Findings illustrated a need to support customized notification approaches, user-specific information, and access to genetics specialists with GS reports. These design principles can be incorporated into software applications that deliver GS reports. Our user-centered approach to conduct this assessment and the specific input we received from clinicians may also be relevant to others working on similar projects.

Visual programming for next-generation sequencing data analytics

Background

High-throughput or next-generation sequencing (NGS) technologies have become an established and affordable experimental framework in biological and medical sciences for all basic and translational research. Processing and analyzing NGS data is challenging. NGS data are big, heterogeneous, sparse, and error prone. Although a plethora of tools for NGS data analysis has emerged in the past decade, (i) software development is still lagging behind data generation capabilities, and (ii) there is a ‘cultural’ gap between the end user and the developer.

Text

Generic software template libraries specifically developed for NGS can help in dealing with the former problem, whilst coupling template libraries with visual programming may help with the latter. Here we scrutinize the state-of-the-art low-level software libraries implemented specifically for NGS and graphical tools for NGS analytics. An ideal developing environment for NGS should be modular (with a native library interface), scalable in computational methods (i.e. serial, multithread, distributed), transparent (platform-independent), interoperable (with external software interface), and usable (via an intuitive graphical user interface). These characteristics should facilitate both the run of standardized NGS pipelines and the development of new workflows based on technological advancements or users’ needs. We discuss in detail the potential of a computational framework blending generic template programming and visual programming that addresses all of the current limitations.

Conclusion

In the long term, a proper, well-developed (although not necessarily unique) software framework will bridge the current gap between data generation and hypothesis testing. This will eventually facilitate the development of novel diagnostic tools embedded in routine healthcare.

Dynamic software design for clinical exome and genome analyses: insights from bioinformaticians, clinical geneticists, and genetic counselors

BACKGROUND

The transition of whole-exome and whole-genome sequencing (WES/WGS) from the research setting to routine clinical practice remains challenging.

OBJECTIVES

With almost no previous research specifically assessing interface designs and functionalities of WES and WGS software tools, the authors set out to ascertain perspectives from healthcare professionals in distinct domains on optimal clinical genomics user interfaces.

METHODS

A series of semi-scripted focus groups, structured around professional challenges encountered in clinical WES and WGS, were conducted with bioinformaticians (n = 8), clinical geneticists (n = 9), genetic counselors (n = 5), and general physicians (n = 4).

RESULTS

Contrary to popular existing system designs, bioinformaticians preferred command line over graphical user interfaces for better software compatibility and customization flexibility. Clinical geneticists and genetic counselors desired an overarching interactive graphical layout to prioritize candidate variants--a "tiered" system where only functionalities relevant to the user domain are made accessible. They favored a system capable of retrieving consistent representations of external genetic information from third-party sources. To streamline collaboration and patient exchanges, the authors identified user requirements toward an automated reporting system capable of summarizing key evidence-based clinical findings among the vast array of technical details.

CONCLUSIONS

Successful adoption of a clinical WES/WGS system is heavily dependent on its ability to address the diverse necessities and predilections among specialists in distinct healthcare domains. Tailored software interfaces suitable for each group is likely more appropriate than the current popular "one size fits all" generic framework. This study provides interfaces for future intervention studies and software engineering opportunities.