Integrated information visualization to support decision making for use of antibiotics in intensive care: design and usability evaluation

Expert gaze as a usability indicator of medical AI decision support systems: a preliminary study

Given the current state of medical artificial intelligence (AI) and perceptions towards it, collaborative systems are becoming the preferred choice for clinical workflows. This work aims to address expert interaction with medical AI support systems to gain insight towards how these systems can be better designed with the user in mind. As eye tracking metrics have been shown to be robust indicators of usability, we employ them for evaluating the usability and user interaction with medical AI support systems. We use expert gaze to assess experts' interaction with an AI software for caries detection in bitewing x-ray images. We compared standard viewing of bitewing images without AI support versus viewing where AI support could be freely toggled on and off. We found that experts turned the AI on for roughly 25% of the total inspection task, and generally turned it on halfway through the course of the inspection. Gaze behavior showed that when supported by AI, more attention was dedicated to user interface elements related to the AI support, with more frequent transitions from the image itself to these elements. When considering that expert visual strategy is already optimized for fast and effective image inspection, such interruptions in attention can lead to increased time needed for the overall assessment. Gaze analysis provided valuable insights into an AI's usability for medical image inspection. Further analyses of these tools and how to delineate metrical measures of usability should be developed.

Usability and User Experience of an mHealth App for Therapy Support of Patients With Breast Cancer: Mixed Methods Study Using Eye Tracking

BACKGROUND

Early identification of quality of life (QoL) loss and side effects is a key challenge in breast cancer therapy. Digital tools can be helpful components of therapeutic support. Enable, a smartphone app, was used in a multicenter, prospective randomized controlled trial in 3 breast cancer centers. The app simultaneously serves as a therapy companion (eg, by displaying appointments), a tool for documenting QoL (eg, by enabling data collection for QoL questionnaires), and documentation of patient-reported side effects. The need for digital tools is continually rising. However, evidence of the effects of long-term use of mobile health (mHealth) apps in aftercare for patients with breast cancer is limited. Therefore, evaluating the usability and understanding the user experience of this mHealth app could potentially contribute valuable insights in this field.

OBJECTIVE

A usability study was conducted to explore how patients with breast cancer receiving neoadjuvant, adjuvant, or palliative outpatient treatment rated their engagement with the app , the user experience, and the benefits of using the app.

METHODS

A mixed methods approach was chosen to combine subjective and objective measures, including an eye-tracking procedure, a standardized usability questionnaire (mHealth App Usability Questionnaire), and semistructured interviews. Participants were surveyed twice during the study period. Interviews were transcribed verbatim and analyzed using thematic analysis. Analysis of the eye-tracking data was carried out using the tracker-integrated software. Descriptive analysis was conducted for the quantitative data.

RESULTS

The mHealth App Usability Questionnaire results (n=105) indicated good overall usability for 2 different time points (4 wk: mean 89.15, SD 9.65; 20 wk: mean 85.57, SD 12.88). The qualitative analysis of the eye-tracking recordings (n=10) and interviews (n=16) showed that users found the Enable app easy to use. The design of the app, information about therapies and side effects, and usefulness of the app as a therapy companion were rated positively. Additionally, participants contributed requests for additional app features and suggestions for improving the content and usability of the app. Relevant themes included optimization of the appointment feature, updating the app's content regularly, and self-administration. In contrast to the app's current passive method of operation, participants expressed a desire for more active engagement through messaging, alarms, or emails.

CONCLUSIONS

The results of this study demonstrate the good usability of the Enable app as well as the potential for further development. We concluded from patients' feedback and requests that mHealth apps could benefit from giving patients a more active role (eg, being able to actively document side effects as they occur). Additionally, regular updates of app content could further contribute to encouraging continued use of mHealth apps. Our findings may also assist other researchers in tailoring their mHealth apps to the actual needs of patients undergoing breast cancer therapy.

Patient Information Summarization in Clinical Settings: Scoping Review

BACKGROUND

Information overflow, a common problem in the present clinical environment, can be mitigated by summarizing clinical data. Although there are several solutions for clinical summarization, there is a lack of a complete overview of the research relevant to this field.

OBJECTIVE

This study aims to identify state-of-the-art solutions for clinical summarization, to analyze their capabilities, and to identify their properties.

METHODS

A scoping review of articles published between 2005 and 2022 was conducted. With a clinical focus, PubMed and Web of Science were queried to find an initial set of reports, later extended by articles found through a chain of citations. The included reports were analyzed to answer the questions of where, what, and how medical information is summarized; whether summarization conserves temporality, uncertainty, and medical pertinence; and how the propositions are evaluated and deployed. To answer how information is summarized, methods were compared through a new framework "collect-synthesize-communicate" referring to information gathering from data, its synthesis, and communication to the end user.

RESULTS

Overall, 128 articles were included, representing various medical fields. Exclusively structured data were used as input in 46.1% (59/128) of papers, text in 41.4% (53/128) of articles, and both in 10.2% (13/128) of papers. Using the proposed framework, 42.2% (54/128) of the records contributed to information collection, 27.3% (35/128) contributed to information synthesis, and 46.1% (59/128) presented solutions for summary communication. Numerous summarization approaches have been presented, including extractive (n=13) and abstractive summarization (n=19); topic modeling (n=5); summary specification (n=11); concept and relation extraction (n=30); visual design considerations (n=59); and complete pipelines (n=7) using information extraction, synthesis, and communication. Graphical displays (n=53), short texts (n=41), static reports (n=7), and problem-oriented views (n=7) were the most common types in terms of summary communication. Although temporality and uncertainty information were usually not conserved in most studies (74/128, 57.8% and 113/128, 88.3%, respectively), some studies presented solutions to treat this information. Overall, 115 (89.8%) articles showed results of an evaluation, and methods included evaluations with human participants (median 15, IQR 24 participants): measurements in experiments with human participants (n=31), real situations (n=8), and usability studies (n=28). Methods without human involvement included intrinsic evaluation (n=24), performance on a proxy (n=10), or domain-specific tasks (n=11). Overall, 11 (8.6%) reports described a system deployed in clinical settings.

CONCLUSIONS

The scientific literature contains many propositions for summarizing patient information but reports very few comparisons of these proposals. This work proposes to compare these algorithms through how they conserve essential aspects of clinical information and through the "collect-synthesize-communicate" framework. We found that current propositions usually address these 3 steps only partially. Moreover, they conserve and use temporality, uncertainty, and pertinent medical aspects to varying extents, and solutions are often preliminary.

Factors influencing the implementation of decision support systems for antibiotic prescription in hospitals: a systematic review

BACKGROUND

Antibiotic resistance is a major health threat. Inappropriate antibiotic use has been shown to be an important determinant of the emergence of antibiotic resistance. Decision support systems for antimicrobial management can support clinicians to optimize antibiotic prescription.

OBJECTIVE

The aim of this systematic review is to identify factors influencing the implementation of decision support systems for antibiotic prescription in hospitals.

METHODS

A systematic search of factors impeding or facilitating successful implementation of decision support systems for antibiotic prescription was performed in January 2022 in the databases PubMed, Web of Science and The Cochrane Library. Only studies were included which comprised decision support systems in hospitals for prescribing antibiotic therapy, published in English with a qualitative, quantitative or mixed-methods study design and between 2011 and 2021. Factors influencing the implementation were identified through text analysis by two reviewers.

RESULTS

A total of 14 publications were identified matching the inclusion criteria. The majority of factors relate to technological and organizational aspects of decision support system implementation. Some factors include the integration of the decision support systems into existing systems, system design, consideration of potential end-users as well as training and support for end-users. In addition, user-related factors, like user attitude towards the system, computer literacy and prior experience with the system seem to be important for successful implementation of decision support systems for antibiotic prescription in hospitals.

CONCLUSION

The results indicate a broad spectrum of factors of decision support system implementation for antibiotic prescription and contributes to the literature by identifying important organizational as well as user-related factors. Wider organizational dimensions as well as the interaction between user and technology appear important for supporting implementation.

Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems: Metrics and Evaluation Methods for eHealth Systems

BACKGROUND AND OBJECTIVES

Electronic healthcare records have become central to patient care. Evaluation of new systems include a variety of usability evaluation methods or usability metrics (often referred to interchangeably as usability components or usability attributes). This study reviews the breadth of usability evaluation methods, metrics, and associated measurement techniques that have been reported to assess systems designed for hospital staff to assess inpatient clinical condition.

METHODS

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, we searched Medline, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, and Open Grey from 1986 to 2019. For included studies, we recorded usability evaluation methods or usability metrics as appropriate, and any measurement techniques applied to illustrate these. We classified and described all usability evaluation methods, usability metrics, and measurement techniques. Study quality was evaluated using a modified Downs and Black checklist.

RESULTS

The search identified 1336 studies. After abstract screening, 130 full texts were reviewed. In the 51 included studies 11 distinct usability evaluation methods were identified. Within these usability evaluation methods, seven usability metrics were reported. The most common metrics were ISO9241-11 and Nielsen's components. An additional "usefulness" metric was reported in almost 40% of included studies. We identified 70 measurement techniques used to evaluate systems. Overall study quality was reflected in a mean modified Downs and Black checklist score of 6.8/10 (range 1-9) 33% studies classified as "high-quality" (scoring eight or higher), 51% studies "moderate-quality" (scoring 6-7), and the remaining 16% (scoring below five) were "low-quality."

CONCLUSION

There is little consistency within the field of electronic health record systems evaluation. This review highlights the variability within usability methods, metrics, and reporting. Standardized processes may improve evaluation and comparison electronic health record systems and improve their development and implementation.

Eye Tracking-An Innovative Tool in Medical Parasitology

The innovative Eye Movement Modelling Examples (EMMEs) method can be used in medicine as an educational training tool for the assessment and verification of students and professionals. Our work was intended to analyse the possibility of using eye tracking tools to verify the skills and training of people engaged in laboratory medicine on the example of parasitological diagnostics. Professionally active laboratory diagnosticians working in a multi-profile laboratory (non-parasitological) (n = 16), laboratory diagnosticians no longer working in this profession (n = 10), and medical analyst students (n = 56), participated in the study. The studied group analysed microscopic images of parasitological preparations made with the cellSens Dimension Software (Olympus) system. Eye activity parameters were obtained using a stationary, video-based eye tracker Tobii TX300 which has a 3-ms temporal resolution. Eye movement activity parameters were analysed along with time parameters. The results of our studies have shown that the eye tracking method is a valuable tool for the analysis of parasitological preparations. Detailed quantitative and qualitative analysis confirmed that the EMMEs method may facilitate learning of the correct microscopic image scanning path. The analysis of the results of our studies allows us to conclude that the EMMEs method may be a valuable tool in the preparation of teaching materials in virtual microscopy. These teaching materials generated with the use of eye tracking, prepared by experienced professionals in the field of laboratory medicine, can be used during various training, simulations and courses in medical parasitology and contribute to the verification of education results, professional skills, and elimination of errors in parasitological diagnostics.

Improving Health Care Management in Hospitals Through a Productivity Dashboard

Health information systems have been developed to help hospital managers steer daily operations, including key performance indicators (KPIs) for monitoring on a time-aggregated basis. Yet, current literature lacks in proposals of productivity dashboards to assist hospitals stakeholders. This research focuses on two related problems: (1) hospital organizations need access to productivity information to improve access to services; and (2) managers need productivity information to optimize resource allocation. This research consists in the development of dashboards to monitor information obtained from a hospital organization to support decision makers. To develop and evaluate the productivity dashboard, the Design Science Research (DSR) methodology was adopted. The dashboard was evaluated by stakeholders of a large Portuguese hospital who contributed to iteratively improving its design toward a useful decision support tool. Additionally, it was ascertained that monitoring productivity needs more study and that the dashboards on these themes are valuable assets at a monitoring level and subsequent decision-making process.

Association of Health Record Visualizations With Physicians' Cognitive Load When Prioritizing Hospitalized Patients

IMPORTANCE

Current electronic health records (EHRs) contribute to increased physician cognitive workload when completing clinical tasks.

OBJECTIVE

To assess the association of different design features of an EHR-based information visualization tool with the cognitive load of physicians during the clinical prioritization process.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study included a convenience sample of 29 attending physicians at Seattle Children's Hospital, a large tertiary academic pediatric hospital. Data collection took place from August 2017 through October 2017, and analysis occurred from August to October 2018.

EXPOSURE

Physician participants used 3 prototypes with novel visualizations of simulated EHR data that highlighted 1 of 3 key patient characteristics, as follows: (1) acuity, (2) clinical problem list, and (3) clinical change.

MAIN OUTCOMES AND MEASURES

Cognitive workload was measured using the NASA Task Load Index (TLX) scale (range, 1-100, with lower scores indicating lower cognitive workload). Cognitive workload was assessed for the 2 following clinical prioritization tasks: (1) finding information for a specific patient and (2) comparing results among patients for each prototype. Participants ranked 5 hypothetical patients from having the highest to the lowest priority in each design.

RESULTS

A total of 29 physician participants (15 [52%] men; 14 [48%] women; mean [range] age, 43 [35-58] years; mean [range] time in practice, 11 [3-30] years) completed the study. For task 1, the prototype highlighting clinical change was associated with lower median (interquartile range) NASA TLX scores compared with the prototype highlighting acuity (30.3 [15.2-41.6] vs 48.5 [18.7-59.3]; P = .02). For task 2, the prototype highlighting clinical change was associated with lower median (interquartile range) NASA TLX scores compared with the prototype highlighting the clinical problem list (29.1 [16.3-50.8] vs 43.5 [26.6-55.9]; P = .02). The prototype highlighting clinical change had the lowest TLX score in 17 of 29 rankings (59%) for task 1 (χ24 = 24.4; P < .001) and 18 of 29 rankings (62%) for task 2 (χ24 = 17.2; P = .002).

CONCLUSIONS AND RELEVANCE

In this study, well-designed EHR-based information visualizations that highlighted and featured clinically meaningful information patterns significantly reduced physician cognitive workload when prioritizing patient needs.

Critical care information display approaches and design frameworks: A systematic review and meta-analysis

OBJECTIVE

To systematically review original user evaluations of patient information displays relevant to critical care and understand the impact of design frameworks and information presentation approaches on decision-making, efficiency, workload, and preferences of clinicians.

METHODS

We included studies that evaluated information displays designed to support real-time care decisions in critical care or anesthesiology using simulated tasks. We searched PubMed and IEEExplore from 1/1/1990 to 6/30/2018. The search strategy was developed iteratively with calibration against known references. Inclusion screening was completed independently by two authors. Extraction of display features, design processes, and evaluation method was completed by one and verified by a second author.

RESULTS

Fifty-six manuscripts evaluating 32 critical care and 22 anesthesia displays were included. Primary outcome metrics included clinician accuracy and efficiency in recognizing, diagnosing, and treating problems. Implementing user-centered design (UCD) processes, especially iterative evaluation and redesign, resulted in positive impact in outcomes such as accuracy and efficiency. Innovative display approaches that led to improved human-system performance in critical care included: (1) improving the integration and organization of information, (2) improving the representation of trend information, and (3) implementing graphical approaches to make relationships between data visible.

CONCLUSION

Our review affirms the value of key principles of UCD. Improved information presentation can facilitate faster information interpretation and more accurate diagnoses and treatment. Improvements to information organization and support for rapid interpretation of time-based relationships between related quantitative data is warranted. Designers and developers are encouraged to involve users in formal iterative design and evaluation activities in the design of electronic health records (EHRs), clinical informatics applications, and clinical devices.

Automated analysis of free-text comments and dashboard representations in patient experience surveys: a multimethod co-design study

Background

Patient experience surveys (PESs) often include informative free-text comments, but with no way of systematically, efficiently and usefully analysing and reporting these. The National Cancer Patient Experience Survey (CPES), used to model the approach reported here, generates > 70,000 free-text comments annually.

Main aim

To improve the use and usefulness of PES free-text comments in driving health service changes that improve the patient experience.

Secondary aims

(1) To structure CPES free-text comments using rule-based information retrieval (IR) (‘text engineering’), drawing on health-care domain-specific gazetteers of terms, with in-built transferability to other surveys and conditions; (2) to display the results usefully for health-care professionals, in a digital toolkit dashboard display that drills down to the original free text; (3) to explore the usefulness of interdisciplinary mixed stakeholder co-design and consensus-forming approaches in technology development, ensuring that outputs have meaning for all; and (4) to explore the usefulness of Normalisation Process Theory (NPT) in structuring outputs for implementation and sustainability.

Design

A scoping review, rapid review and surveys with stakeholders in health care (patients, carers, health-care providers, commissioners, policy-makers and charities) explored clinical dashboard design/patient experience themes. The findings informed the rules for the draft rule-based IR [developed using half of the 2013 Wales CPES (WCPES) data set] and prototype toolkit dashboards summarising PES data. These were refined following mixed stakeholder, concept-mapping workshops and interviews, which were structured to enable consensus-forming ‘co-design’ work. IR validation used the second half of the WCPES, with comparison against its manual analysis; transferability was tested using further health-care data sets. A discrete choice experiment (DCE) explored which toolkit features were preferred by health-care professionals, with a simple cost–benefit analysis. Structured walk-throughs with NHS managers in Wessex, London and Leeds explored usability and general implementation into practice.

Key outcomes

A taxonomy of ranked PES themes, a checklist of key features recommended for digital clinical toolkits, rule-based IR validation and transferability scores, usability, and goal-oriented, cost–benefit and marketability results. The secondary outputs were a survey, scoping and rapid review findings, and concordance and discordance between stakeholders and methods.

Results

(1) The surveys, rapid review and workshops showed that stakeholders differed in their understandings of the patient experience and priorities for change, but that they reached consensus on a shortlist of 19 themes; six were considered to be core; (2) the scoping review and one survey explored the clinical toolkit design, emphasising that such toolkits should be quick and easy to use, and embedded in workflows; the workshop discussions, the DCE and the walk-throughs confirmed this and foregrounded other features to form the toolkit design checklist; and (3) the rule-based IR, developed using noun and verb phrases and lookup gazetteers, was 86% accurate on the WCPES, but needs modification to improve this and to be accurate with other data sets. The DCE and the walk-through suggest that the toolkit would be well accepted, with a favourable cost–benefit ratio, if implemented into practice with appropriate infrastructure support.

Limitations

Small participant numbers and sampling bias across component studies. The scoping review studies mostly used top-down approaches and focused on professional dashboards. The rapid review of themes had limited scope, with no second reviewer. The IR needs further refinement, especially for transferability. New governance restrictions further limit immediate use.

Conclusions

Using a multidisciplinary, mixed stakeholder, use of co-design, proof of concept was shown for an automated display of patient experience free-text comments in a way that could drive health-care improvements in real time. The approach is easily modified for transferable application.

Future work

Further exploration is needed of implementation into practice, transferable uses and technology development co-design approaches.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Association of Data Integration Technologies With Intensive Care Clinician Performance: A Systematic Review and Meta-analysis

IMPORTANCE

Sources of data in the intensive care setting are increasing exponentially, but the benefits of displaying multiparametric, high-frequency data are unknown. Decision making may not benefit from this technology if clinicians remain cognitively overburdened by poorly designed data integration and visualization technologies (DIVTs).

OBJECTIVE

To systematically review and summarize the published evidence on the association of user-centered DIVTs with intensive care clinician performance.

DATA SOURCES

MEDLINE, Embase, Cochrane Central Register of Controlled Trials, PsycINFO, and Web of Science were searched in May 2014 and January 2018.

STUDY SELECTION

Studies had 3 requirements: (1) the study tested a viable DIVT, (2) participants involved were intensive care clinicians, and (3) the study reported quantitative results associated with decision making in an intensive care setting.

DATA EXTRACTION AND SYNTHESIS

Of 252 records screened, 20 studies, published from 2004 to 2016, were included. The human factors framework to assess health technologies was applied to measure study completeness, and the Quality Assessment Instrument was used to assess the quality of the studies. PRISMA guidelines were adapted to conduct the systematic review and meta-analysis.

MAIN OUTCOMES AND MEASURES

Study completeness and quality; clinician performance; physical, mental, and temporal demand; effort; frustration; time to decision; and decision accuracy.

RESULTS

Of the 20 included studies, 16 were experimental studies with 410 intensive care clinician participants and 4 were survey-based studies with 1511 respondents. Scores for study completeness ranged from 27 to 43, with a maximum score of 47, and scores for study quality ranged from 46 to 79, with a maximum score of 90. Of 20 studies, DIVTs were evaluated in clinical settings in 2 studies (10%); time to decision was measured in 14 studies (70%); and decision accuracy was measured in 11 studies (55%). Measures of cognitive workload pooled in the meta-analysis suggested that any DIVT was an improvement over paper-based data in terms of self-reported performance, mental and temporal demand, and effort. With a maximum score of 22, median (IQR) mental demand scores for electronic display were 10 (7-13), tabular display scores were 8 (6.0-11.5), and novel visualization scores were 8 (6-12), compared with 17 (14-19) for paper. The median (IQR) temporal demand scores were also lower for all electronic visualizations compared with paper, with scores of 8 (6-11) for electronic display, 7 (6-11) for tabular and bar displays, 7 (5-11) for novel visualizations, and 16 (14.3-19.0) for paper. The median (IQR) performance scores improved for all electronic visualizations compared with paper (lower score indicates better self-reported performance), with scores of 6 (3-11) for electronic displays, 6 (4-11) for tabular and bar displays, 6 (4-11) for novel visualizations, and 14 (11-16) for paper. Frustration and physical demand domains of cognitive workload did not change, and differences between electronic displays were not significant.

CONCLUSIONS AND RELEVANCE

This review suggests that DIVTs are associated with increased integration and consistency of data. Much work remains to identify which visualizations effectively reduce cognitive workload to enhance decision making based on intensive care data. Standardizing human factors testing by developing a repository of open access benchmarked test protocols, using a set of outcome measures, scenarios, and data sets, may accelerate the design and selection of the most appropriate DIVT.

Eye-tracking reveals how observation chart design features affect the detection of patient deterioration: An experimental study

Particular design features intended to improve usability - including graphically displayed observations and integrated colour-based scoring-systems - have been shown to increase the speed and accuracy with which users of hospital observation charts detect abnormal patient observations. We used eye-tracking to evaluate two potential cognitive mechanisms underlying these effects. Novice chart-users completed a series of experimental trials in which they viewed patient data presented on one of three observation chart designs (varied within-subjects), and indicated which observation was abnormal (or that none were). A chart that incorporated both graphically displayed observations and an integrated colour-based scoring-system yielded faster, more accurate responses and fewer, shorter fixations than a graphical chart without a colour-based scoring-system. The latter, in turn, yielded the same advantages over a tabular chart (which incorporated neither design feature). These results suggest that both colour-based scoring-systems and graphically displayed observations improve search efficiency and reduce the cognitive resources required to process vital sign data.

Geographical mapping of advanced practice registered nurses in Louisiana: Leveraging the power of data display

BACKGROUND AND PURPOSE

Identifying the locations of advanced practice registered nurses (APRNs) in the state is challenging without data readily available, which can adversely affect stakeholder initiatives. The purpose of this project was to develop an online interactive map reflecting distribution of APRNs licensed by the Louisiana State Board of Nursing (LSBN), heightening APRN visibility and improving usability of data for policy, education, practice, and research initiatives.

METHODS

A descriptive design with development of a graphical interactive map showing all Louisiana parishes with integrated distribution summaries of APRNs was made. Work proceeded from paper to higher fidelity simulations to the final interactive web page. Compatibility with LSBN's web page and annual data upload were ensured.

CONCLUSION

The project was done in collaboration with LSBN, an academic institution, and a privately hired web designer. The principals of Louisiana State Board of Nursing approved the final product. The product meets and exceeds the expectations for mapping. Evaluation of use and usability will be done at the discretion of LSBN.

IMPLICATIONS FOR PRACTICE

The outcome of this project will serve to heighten visibility of the distribution of APRNs in the state of Louisiana, enhancing opportunities for APRNs and for patients. Furthermore, those interested in leveraging the information in policy, education, or research initiatives will be served.

Data Visualizations to Support Health Practitioners' Provision of Personalized Care for Patients With Cancer and Multiple Chronic Conditions: User-Centered Design Study

BACKGROUND

There exists a challenge of understanding and integrating various types of data collected to support the health of individuals with multiple chronic conditions engaging in cancer care. Data visualization has the potential to address this challenge and support personalized cancer care.

OBJECTIVE

The aim of the study was to assess the health care practitioners' perceptions of and feedback regarding visualizations developed to support the care of individuals with multiple chronic conditions engaging in cancer care.

METHODS

Medical doctors (n=4) and registered nurses (n=4) providing cancer care at an academic medical center in the western United States provided feedback on visualization mock-ups. Mock-up designs were guided by current health informatics and visualization literature and the Munzner Nested Model for Visualization Design. User-centered design methods, a mock patient persona, and a scenario were used to elicit insights from participants. Directed content analysis was used to identify themes from session transcripts. Means and SDs were calculated for health care practitioners' rankings of overview visualizations.

RESULTS

Themes identified were data elements, supportive elements, confusing elements, interpretation, and use of visualization. Overall, participants found the visualizations useful and with the potential to provide personalized care. Use of color, reference lines, and familiar visual presentations (calendars, line graphs) were noted as helpful in interpreting data.

CONCLUSIONS

Visualizations guided by a framework and literature can support health care practitioners' understanding of data for individuals with multiple chronic conditions engaged in cancer care. This understanding has the potential to support the provision of personalized care.

Evidence-Based Practice in the Treatment for Antibiotic-Associated Diarrhea in the Intensive Care Unit

Unit nurses provide care to patients with serious health conditions. Often antibiotics are recommended to fight infections. Sometimes patients are on proton pump inhibitors (PPI). Antibiotics and PPIs may lead to diarrhea, causing the patient more discomfort, and possibly leading to a more serious infection. One serious infection is caused by Clostridium difficile, which causes death in some cases. About 75% of patients on antibiotics may not need antibiotics. Several studies showed less diarrhea in patients on probiotics. However, probiotics cause harm to some patients, which needs to be considered. Nurses need to ensure that antibiotic stewardship is followed.

Clinical Use of Precision Oncology Decision Support

PURPOSE

Precision oncology is hindered by the lack of decision support for determining the functional and therapeutic significance of genomic alterations in tumors and relevant clinically available options. To bridge this knowledge gap, we established a Precision Oncology Decision Support (PODS) team that provides annotations at the alteration-level and subsequently determined if clinical decision-making was influenced.

METHODS

Genomic alterations were annotated to determine actionability based on a variant's known or potential functional and/or therapeutic significance. The medical records of a subset of patients annotated in 2015 were manually reviewed to assess trial enrollment. A web-based survey was implemented to capture the reasons why genotype-matched therapies were not pursued.

RESULTS

PODS processed 1,669 requests for annotation of 4,084 alterations (2,254 unique) across 49 tumor types for 1,197 patients. 2,444 annotations for 669 patients included an actionable variant call: 32.5% actionable, 9.4% potentially, 29.7% unknown, 28.4% non-actionable. 66% of patients had at least one actionable/potentially actionable alteration. 20.6% (110/535) patients annotated enrolled on a genotype-matched trial. Trial enrolment was significantly higher for patients with actionable/potentially actionable alterations (92/333, 27.6%) than those with unknown (16/136, 11.8%) and non-actionable (2/66, 3%) alterations (p=0.00004). Actionable alterations in PTEN, PIK3CA, and ERBB2 most frequently led to enrollment on genotype-matched trials. Clinicians cited a variety of reasons why patients with actionable alterations did not enroll on trials.

CONCLUSION

Over half of alterations annotated were of unknown significance or non-actionable. Physicians were more likely to enroll a patient on a genotype-matched trial when an annotation supported actionability. Future studies are needed to demonstrate the impact of decision support on trial enrollment and oncologic outcomes.

"Usability of data integration and visualization software for multidisciplinary pediatric intensive care: a human factors approach to assessing technology"

Background

Intensive care clinicians use several sources of data in order to inform decision-making. We set out to evaluate a new interactive data integration platform called T3™ made available for pediatric intensive care. Three primary functions are supported: tracking of physiologic signals, displaying trajectory, and triggering decisions, by highlighting data or estimating risk of patient instability. We designed a human factors study to identify interface usability issues, to measure ease of use, and to describe interface features that may enable or hinder clinical tasks.

Methods

Twenty-two participants, consisting of bedside intensive care physicians, nurses, and respiratory therapists, tested the T3™ interface in a simulation laboratory setting. Twenty tasks were performed with a true-to-setting, fully functional, prototype, populated with physiological and therapeutic intervention patient data. Primary data visualization was time series and secondary visualizations were: 1) shading out-of-target values, 2) mini-trends with exaggerated maxima and minima (sparklines), and 3) bar graph of a 16-parameter indicator. Task completion was video recorded and assessed using a use error rating scale. Usability issues were classified in the context of task and type of clinician. A severity rating scale was used to rate potential clinical impact of usability issues.

Results

Time series supported tracking a single parameter but partially supported determining patient trajectory using multiple parameters. Visual pattern overload was observed with multiple parameter data streams. Automated data processing using shading and sparklines was often ignored but the 16-parameter data reduction algorithm, displayed as a persistent bar graph, was visually intuitive. However, by selecting or automatically processing data, triggering aids distorted the raw data that clinicians use regularly. Consequently, clinicians could not rely on new data representations because they did not know how they were established or derived.

Conclusions

Usability issues, observed through contextual use, provided directions for tangible design improvements of data integration software that may lessen use errors and promote safe use. Data-driven decision making can benefit from iterative interface redesign involving clinician-users in simulated environments. This study is a first step in understanding how software can support clinicians’ decision making with integrated continuous monitoring data. Importantly, testing of similar platforms by all the different disciplines who may become clinician users is a fundamental step necessary to understand the impact on clinical outcomes of decision aids.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-017-0520-7) contains supplementary material, which is available to authorized users.

Eye tracking in library and information science: a literature review

Purpose

The purpose of this paper is to present a systematic literature review of the application of eye-tracking technology within the field of library and information science. Eye-tracking technology has now reached a level of maturity, which makes the use of the technology more accessible. Subsequently, a growing interest in employing eye tracking as a methodology within library and information science research must be anticipated.

Design/methodology/approach

The review follows the guidelines set in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses recommendations. Two reference databases are searched for relevant references: Library and Information Science Abstracts and Library, Information Science and Technology Abstracts. The main selection criteria are peer-reviewed literature that describes the experimental setting, including which eye-tracking equipment was used, the number of test persons and reports on the eye-tracking measures. Furthermore, this study will report which other methods were applied in combination with eye tracking.

Findings

The number of published research utilizing eye-tracking technologies within library and information science (LIS) is still limited although an increase in the use of eye-tracking technologies is observed during recent years.

Originality/value

To the knowledge of the author, this is the first systematic review on eye-tracking technology and application in LIS.

Identifying visual search patterns in eye gaze data; gaining insights into physician visual workflow

Importance and Objectives As health information technologies become more prevalent in physician workflow, it is increasingly important to understand how physicians are using and interacting with these systems. This includes understanding how physicians search for information presented through health information technology systems. Eye tracking technologies provide a useful technique to understand how physicians visually search for information. However, analyzing eye tracking data can be challenging and is often done by measuring summative metrics, such as total time looking at a specific area and first-order transitions.

Methods In this paper, we propose an algorithmic approach to identify different visual search patterns. We demonstrate this approach by identifying common visual search patterns from physicians using a simulated prototype emergency department patient tracking system.

Results and Conclusions We evaluate and compare the visual search pattern results to first-order transition results. We discuss the benefits and limitations of this approach and insights from this initial evaluation.

EHDViz: clinical dashboard development using open-source technologies

OBJECTIVE

To design, develop and prototype clinical dashboards to integrate high-frequency health and wellness data streams using interactive and real-time data visualisation and analytics modalities.

MATERIALS AND METHODS

We developed a clinical dashboard development framework called electronic healthcare data visualization (EHDViz) toolkit for generating web-based, real-time clinical dashboards for visualising heterogeneous biomedical, healthcare and wellness data. The EHDViz is an extensible toolkit that uses R packages for data management, normalisation and producing high-quality visualisations over the web using R/Shiny web server architecture. We have developed use cases to illustrate utility of EHDViz in different scenarios of clinical and wellness setting as a visualisation aid for improving healthcare delivery.

RESULTS

Using EHDViz, we prototyped clinical dashboards to demonstrate the contextual versatility of EHDViz toolkit. An outpatient cohort was used to visualise population health management tasks (n=14,221), and an inpatient cohort was used to visualise real-time acuity risk in a clinical unit (n=445), and a quantified-self example using wellness data from a fitness activity monitor worn by a single individual was also discussed (n-of-1). The back-end system retrieves relevant data from data source, populates the main panel of the application and integrates user-defined data features in real-time and renders output using modern web browsers. The visualisation elements can be customised using health features, disease names, procedure names or medical codes to populate the visualisations. The source code of EHDViz and various prototypes developed using EHDViz are available in the public domain at http://ehdviz.dudleylab.org.

CONCLUSIONS

Collaborative data visualisations, wellness trend predictions, risk estimation, proactive acuity status monitoring and knowledge of complex disease indicators are essential components of implementing data-driven precision medicine. As an open-source visualisation framework capable of integrating health assessment, EHDViz aims to be a valuable toolkit for rapid design, development and implementation of scalable clinical data visualisation dashboards.

Development of a computerised decisions support system for renal risk drugs targeting primary healthcare

OBJECTIVES

To assess general practitioners (GPs) experience from the implementation and use of a renal computerised decision support system (CDSS) for drug dosing, developed for primary healthcare, integrated into the patient's electronic health record (EHR), and building on estimation of the patient's creatinine clearance (ClCG).

DESIGN

Qualitative research design by a questionnaire and a focus group discussion.

SETTING AND PARTICIPANTS

Eight GPs at two primary healthcare centres (PHCs).

INTERVENTIONS

The GP at PHC 1, and the project group, developed and tested the technical solution of the CDSS. Proof-of-concept was tested by seven GPs at PHC 2. They also participated in a group discussion and answered a questionnaire. A web window in the EHR gave drug and dosage in relation to ClCG. Each advice was according to three principles: If? Why? Because.

OUTCOME MEASURES

(1) The GPs' experience of 'easiness to use' and 'perceived usefulness' at PHC 2, based on loggings of use, answers from a questionnaire using a 5-point Likert scale, and answers from a focus group discussion. (2) The number of patients aged 65 years and older with an estimation of ClCG before and after the implementation of the CDSS.

RESULTS

The GPs found the CDSS fast, simple and easy to use. They appreciated the automatic presentation of the CICG status on opening the medication list, and the ability to actively look up specific drug recommendations in two steps. The CDSS scored high on the Likert scale. All GPs wanted to continue the use of the CDSS and to recommend it to others. The number of patients with an estimated ClCG increased 1.6-fold.

CONCLUSIONS

Acceptance of the simple graphical interface of this push and pull renal CDSS was high among the primary care physicians evaluating this proof of concept. The graphical model should be useful for further development of renal decision support systems.

Using Eye Trackers for Usability Evaluation of Health Information Technology: A Systematic Literature Review

Background

Eye-tracking technology has been used to measure human cognitive processes and has the potential to improve the usability of health information technology (HIT). However, it is still unclear how the eye-tracking method can be integrated with other traditional usability methodologies to achieve its full potential.

Objective

The objective of this study was to report on HIT evaluation studies that have used eye-tracker technology, and to envision the potential use of eye-tracking technology in future research.

Methods

We used four reference databases to initially identify 5248 related papers, which resulted in only 9 articles that met our inclusion criteria.

Results

Eye-tracking technology was useful in finding usability problems in many ways, but is still in its infancy for HIT usability evaluation. Limited types of HITs have been evaluated by eye trackers, and there has been a lack of evaluation research in natural settings.

Conclusions

More research should be done in natural settings to discover the real contextual-based usability problems of clinical and mobile HITs using eye-tracking technology with more standardized methodologies and guidance.