Development and refinement of a novel end-of-life planning website for patients with advanced cancer: a mixed methods approach

PURPOSE

Despite known benefits of planning for end-of-life, no digital tool exists to help patients with advanced cancer and their loved ones plan for death comprehensively. To address this unmet need, we developed a preliminary version of an innovative website to help patients with advanced cancer prepare for end-of-life tasks.

METHODS

Guided by the Obesity-Related Behavioral Intervention Trials (ORBIT) model for behavioral intervention development, patients with advanced cancer (n = 10) and their caregivers (n = 10) participated in a "Think Aloud" exercise and usability protocols to optimize the end-of-life planning website. The website was iteratively refined throughout the study in collaboration with the partnering company, Peacefully, Inc. Participants also completed the Acceptability E-Scale and System Usability Scale, with a priori benchmarks established for acceptability (scores of ≥ 24 on the Acceptability E-Scale) and usability (scores of ≥ 68 on the System Usability Scale).

RESULTS

Patients (N = 10) and caregivers (N = 10) completed usability testing. Patients were majority female (80%), White (100%), and had a mean age of 58 years. Caregivers (N = 10) were majority male (60%), spouse/partner (90%), White (90%), and had a mean age of 59 years. For patients, a priori hypotheses were met for both acceptability (mean score of 24.7, SD = 4.35) and usability (mean score of 73.8, SD = 6.15). For caregivers, acceptability was just below the cutoff (mean score of 22.9, SD = 4.07) and usability exceeded the cutoff (mean score of 70.0, SD = 8.42). Overall, patients and caregivers reported high levels of satisfaction and found the website helpful, with specific suggestions for changes (e.g., add more information about information security, improve text legibility).

CONCLUSIONS

The findings from this study will inform modifications to optimize an innovative website to support patients with advanced cancer to prepare holistically for end-of-life tasks.

Comparison of usability evaluation methods for a health information system: heuristic evaluation versus cognitive walkthrough method

Background

There are differences of opinion regarding the selection of the most practical usability evaluation method among different methods. The present study aimed to compare two expert-based evaluation methods in order to assess a nursing module as the most widely used module of a Hospital Information System (HIS).

Methods

Five independent evaluators used the Heuristic Evaluation (HE) and Cognitive Walkthrough (CW) methods to evaluate the nursing module of Shafa HIS. In this regard, the number and severity of the recognized problems according to the usability attributes were compared using two evaluation methods.

Results

The HE and CW evaluation methods resulted in the identification of 104 and 24 unique problems, respectively, of which 33.3% of recognized problems in the CW evaluation method overlapped with the HE method. The average severity of the recognized problems was considered to be minor (2.34) in the HE method and major (2.77) in the CW evaluation method. There was a significant difference in terms of the total number and average severity of the recognized problems by these methods (P < 0.001). Based on the usability attribute, the HE method identified a larger number of problems concerning all usability attributes, and a significant difference was observed in terms of the number of recognized problems in both methods for all attributes except ‘memorability’. Also, there was a significant difference between the two methods based on the average severity of recognized problems only in terms of ‘learnability’.

Conclusion

The HE method identified more problems with lower average severity while the CW was able to recognize fewer problems with higher average severity. Regarding the evaluation goal, the HE method was able to be used to improve the effectiveness and satisfaction of the HIS. Furthermore, the CW evaluation method is recommended to identify usability problems with the highest average severity, especially in terms of ‘learnability’.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01905-7.

Current state of dental informatics in the field of health information systems: a scoping review

Background

Over the past 50 years, dental informatics has developed significantly in the field of health information systems. Accordingly, several studies have been conducted on standardized clinical coding systems, data capture, and clinical data reuse in dentistry.

Methods

Based on the definition of health information systems, the literature search was divided into three specific sub-searches: “standardized clinical coding systems,” “data capture,” and “reuse of routine patient care data.” PubMed and Web of Science were searched for peer-reviewed articles. The review was conducted following the PRISMA-ScR protocol.

Results

A total of 44 articles were identified for inclusion in the review. Of these, 15 were related to “standardized clinical coding systems,” 15 to “data capture,” and 14 to “reuse of routine patient care data.” Articles related to standardized clinical coding systems focused on the design and/or development of proposed systems, on their evaluation and validation, on their adoption in academic settings, and on user perception. Articles related to data capture addressed the issue of data completeness, evaluated user interfaces and workflow integration, and proposed technical solutions. Finally, articles related to reuse of routine patient care data focused on clinical decision support systems centered on patient care, institutional or population-based health monitoring support systems, and clinical research.

Conclusions

While the development of health information systems, and especially standardized clinical coding systems, has led to significant progress in research and quality measures, most reviewed articles were published in the US. Clinical decision support systems that reuse EDR data have been little studied. Likewise, few studies have examined the working environment of dental practitioners or the pedagogical value of using health information systems in dentistry.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12903-022-02163-9.

Usability assessment of an open-source simulated electronic medical record in pharmacy education

BACKGROUND AND PURPOSE

The growth of electronic medical records (EMRs) has been a critical component of evolving pharmacy practice, catalyzed by foundational initiatives such as the HITECH Act in 2009. The objective of this study was to evaluate the usability and student perceptions of a novel, open-source, educational EMR (EdEMR) for integration in the PharmD program at the University of British Columbia (UBC).

EDUCATIONAL ACTIVITY AND SETTING

Participants were PharmD students at UBC and had to complete a series of survey questions and EdEMR tasks including a Systems Usability Scale (SUS) evaluation. Task completion time, mouse clicks, SUS score, and 5-point Likert scale rankings of student opinions on the use of the EdEMR were collected.

FINDINGS

Seven students successfully completed the study. Participants self-ranked themselves as novice EMR users who used EMRs a few times a week. The mean time to complete the tasks was 6 min, 42 seconds and was approximately double that of benchmark times (mean benchmark = 2 min, 20 seconds). The EdEMR scored within the upper quartile of SUS scores (mean = 83), indicating good usability. Participants identified that the EMR would most optimally be used for distance learning and case-based learning.

SUMMARY

The EdEMR showed good usability as demonstrated by the successfully completed tasks and SUS score. Participants were receptive to the EdEMR's application as an online learning tool and for case-based learning. The EdEMR is an enticing, open-source tool currently available to all pharmacy programs and could further support online and remote teaching.

Using patients' own knowledge of early sensations and symptoms to develop an interactive, individualized e-questionnaire to facilitate early diagnosis of lung cancer

Background

One reason for the often late diagnosis of lung cancer (LC) may be that potentially-indicative sensations and symptoms are often diffuse, and may not be considered serious or urgent, making their interpretation complicated. However, with only a few exceptions, efforts to use people’s own in-depth knowledge about prodromal bodily experiences has been a missing link in efforts to facilitate early LC diagnosis. In this study, we describe and discuss facilitators and challenges in our process of developing and initial testing an interactive, self-completion e-questionnaire based on patient descriptions of experienced prodromal sensations and symptoms, to support early identification of lung cancer (LC).

Methods

E-questionnaire items were derived from in-depth, detailed explorative interviews with individuals undergoing investigation for suspected LC. The descriptors of sensations/symptoms and the background items obtained were the basis for developing an interactive, individualized instrument, PEX-LC, which was refined for usability through think-aloud and other interviews with patients, members of the public, and clinical staff.

Results

Major challenges in the process of developing PEX-LC related to collaboration among many actors, and design/user interface problems including technical issues. Most problems identified through the think-aloud interviews related to design/user interface problems and technical issues rather than content, for example we re-ordered questions to be in line with patients’ chronological, rather than retrospective, descriptions of their experiences. PEX-LC was developed into a final e-questionnaire on a touch-screen smart tablet with one background module covering sociodemographic characteristics, 10 interactive, individualized modules covering early sensations and symptoms, and a 12th assessing current symptoms.

Conclusions

Close collaboration with patients throughout the process was intrinsic for developing PEX-LC. Similarly, we recognized the extent to which clinicians and technical experts were also important in this process. Similar endeavors should assure all necessary competence is included in the core research team, to facilitate timely progress. Our experiences developing PEX-LC combined with new empirical research suggest that this individualized, interactive e-questionnaire, developed through systematizing patients’ own formulations of their prodromal symptom experiences, is both feasible for use and has potential value in the intended group.

Data on the activation and utilization of an electronic health record patient portal in an adult inpatient population at an academic medical center

Electronic health record patient portals allow patients to access their own health data online and interact with the healthcare team. Many studies have focused on use of patient portals in the outpatient setting. Relatively less is known about use of patient portals for hospitalized patients. The data in this article include analysis of patient portal activation and utilization for adults hospitalized in 2018 at an academic medical center in a Midwestern state in the United States. Activation was assessed by percentage of patients who had activated their patient portal by the time of data review. Utilization of the patient portal was determined by whether patients or their legal proxies accessed one or more reports from diagnostic testing ordered during inpatient encounter(s) in 2018. The data include 826,843 diagnostic tests on 40,640 unique patients. Patient characteristics include sex, age, whether outpatient diagnostic tests were also performed in 2018, preferred language (English or non-English), health insurance status (private, public, or uninsured), self-declared race (White or non-White), and whether there was a legal proxy for the patient. Association of these covariates with patient portal activation and utilization were analyzed.

A qualitative research framework for the design of user-centered displays of explanations for machine learning model predictions in healthcare

BACKGROUND

There is an increasing interest in clinical prediction tools that can achieve high prediction accuracy and provide explanations of the factors leading to increased risk of adverse outcomes. However, approaches to explaining complex machine learning (ML) models are rarely informed by end-user needs and user evaluations of model interpretability are lacking in the healthcare domain. We used extended revisions of previously-published theoretical frameworks to propose a framework for the design of user-centered displays of explanations. This new framework served as the basis for qualitative inquiries and design review sessions with critical care nurses and physicians that informed the design of a user-centered explanation display for an ML-based prediction tool.

METHODS

We used our framework to propose explanation displays for predictions from a pediatric intensive care unit (PICU) in-hospital mortality risk model. Proposed displays were based on a model-agnostic, instance-level explanation approach based on feature influence, as determined by Shapley values. Focus group sessions solicited critical care provider feedback on the proposed displays, which were then revised accordingly.

RESULTS

The proposed displays were perceived as useful tools in assessing model predictions. However, specific explanation goals and information needs varied by clinical role and level of predictive modeling knowledge. Providers preferred explanation displays that required less information processing effort and could support the information needs of a variety of users. Providing supporting information to assist in interpretation was seen as critical for fostering provider understanding and acceptance of the predictions and explanations. The user-centered explanation display for the PICU in-hospital mortality risk model incorporated elements from the initial displays along with enhancements suggested by providers.

CONCLUSIONS

We proposed a framework for the design of user-centered displays of explanations for ML models. We used the proposed framework to motivate the design of a user-centered display of an explanation for predictions from a PICU in-hospital mortality risk model. Positive feedback from focus group participants provides preliminary support for the use of model-agnostic, instance-level explanations of feature influence as an approach to understand ML model predictions in healthcare and advances the discussion on how to effectively communicate ML model information to healthcare providers.

Brave New World: Challenges and Opportunities in the COVID-19 Virtual Interview Season

The COVID-19 pandemic has forced the transition of the traditional residency interview to a virtual format. This new interview format creates additional challenges and opportunities for both programs and applicants. The specific challenges of the virtual interview format are described, as well as means to mitigate those challenges. In addition, opportunities to improve residency selection from the program end are described.

Development of an electronic tool (e-AdPharm) to address unmet needs and barriers of community pharmacists to provide medication adherence support to patients

BACKGROUND

Community pharmacists are best placed to improve medication adherence because they frequently interact with patients and have been trained to manage medication-related problems. Therefore, it is essential to equip pharmacists adequately to detect non-adherent patients quickly and intervene to improve medication adherence.

OBJECTIVE

To design e-AdPharm, a tool that addresses unmet needs and barriers of community pharmacists to provide medication adherence support to patients with chronic diseases.

METHODS

A qualitative study using 4 focus group discussions with community pharmacists was conducted with a semi-structured interview guide and discussions lasting for 1-2 h. The discussions covered the barriers and needs of pharmacists related to medication adherence support provided to patients, their expectations of an electronic tool based on prescription refills to help them provide this support, and the design of the tool. Focus group data were coded and analyzed using an iterative process, with thematic and descriptive analyses.

RESULTS

Twenty-six community pharmacists participated. Lack of time and motivation from pharmacists and patients were common barriers to the provision of medication adherence support. Accordingly, community pharmacists wished to measure medication adherence quickly, provide easily interpretable data to patients on their medication use, and raise the patient's awareness of non-adherence. The pharmacists expressed their need to have an electronic tool to share medication adherence information with the treating physician. Regarding the design of e-AdPharm, the pharmacists wanted a table displaying medication adherence with a color code representing adherence level. They also stressed the importance of a structured section enabling them to continuously document the interventions made and the need for patient follow-ups.

CONCLUSIONS

e-AdPharm meet the needs and overcome the barriers of community pharmacists to provide medication adherence support to their patients. Future studies should examine the feasibility of implementing e-AdPharm in community pharmacies and test its efficacy for improving medication adherence.

Usability evaluation of a comprehensive national health information system: relationship of quality components to users' characteristics

OBJECTIVE

One of the most important methods for evaluating information systems is usability evaluation. Usability is a context-dependent qualitative feature that is measured by multiple quality components that can be related to users' characteristics. This study was conducted to evaluate the usability of a comprehensive national health information system (SIB; an abbreviation for the Persian equivalent of 'integrated health system') from the perspective of different users and to determine the relationship between quality components and users' characteristics.

METHOD

The study population were users of the national health information system (n = 309) at health centers and health homes affiliated to Kashan University of Medical Sciences, Kashan, Iran. Data were collected using Software Usability Measurement Inventory (SUMI) questionnaire which measures users' experiences of software interface in five quality components (i.e. affect, efficiency, helpfulness, control, and learnability) and provides a global usability score. SUMI scores were analyzed according to an extensive reference database (SUMISCO). The relationships between quality components and users' characteristics were investigated by one-way analysis of variance (ANOVA), independent t-test, and Pearson's correlation coefficient.

RESULTS

A total of 250 users completed the questionnaire (response rate = 81%). The mean scores of all quality components were significantly lower than the mean of SUMISCO. Learnability score had significant relationships with the user's position, education level, and field of education (P < 0.001). Physicians scored significantly lower than other users in efficiency, helpfulness and global usability (P < 0.05). Users' practice experience and age had significant linear inverse relationships with efficiency, helpfulness, and learnability (P < 0.05).

CONCLUSIONS

The national health information system which is used by a large number of users across a developing country have low usability. Given the significant relationships between the users' characteristics of such systems and quality components, it is essential to consider the characteristics and needs of various user groups during the processes of system analysis and design.

Touchless interaction with medical images based on 3D hand cursors supported by single-foot input: A case study in dentistry

Feet input can support mid-air hand gestures for touchless medical image manipulation to prevent unintended activations, especially in sterile contexts. However, foot interaction has yet to be investigated in dental settings. In this paper, we conducted a mixed methods research study with medical dentistry professionals. To this end, we developed a touchless medical image system in either sitting or standing configurations. Clinicians could use both hands as 3D cursors and a minimalist single-foot gesture vocabulary to activate manipulations. First, we performed a qualitative evaluation with 18 medical dentists to assess the utility and usability of our system. Second, we used quantitative methods to compare pedal foot-supported hand interaction and hands-only conditions next to 22 medical dentists. We expand on previous work by characterizing a range of potential limitations of foot-supported touchless 3D interaction in the dental domain. Our findings suggest that clinicians are open to use their foot for simple, fast and easy access to image data during surgical procedures, such as dental implant placement. Furthermore, 3D hand cursors, supported by foot gestures for activation events, were considered useful and easy to employ for medical image manipulation. Even though most clinicians preferred hands-only manipulation for pragmatic purposes, feet-supported interaction was found to provide more precise control and, most importantly, to decrease the number of unintended activations during manipulation. Finally, we provide design considerations for future work exploring foot-supported touchless interfaces for sterile settings in Dental Medicine, regarding: interaction design, foot input devices, the learning process and camera occlusions.

Designing Safer Use of Antipsychotics Among Youths: A Human-Centered Approach to an Algorithm-Based Solution

OBJECTIVE

Guidelines recommend against prescribing antipsychotics without an indication, yet prescription rates remain undesirably high for youths. Information technology can facilitate guideline-based prescribing, but little is known about providers' needs. The Safer Use of Antipsychotics in Youth project is implementing an algorithm-based workflow for peer consultation, care navigation, and expedited psychotherapy access. To optimize workflow for a multisite trial, we engaged providers for input.

METHODS

Guided by human-centered design, we interviewed 15 providers from Kaiser Permanente Washington and Nationwide Children's Hospital about their prescribing barriers and workflow preferences. We identified qualitative themes on barriers affecting implementation and design opportunities to optimize workflow.

RESULTS

Providers expressed two major barriers: potential disruptions to clinical practice and threats to professional autonomy. Three design opportunities emerged: à la carte orders, passive review of orders, and consultation self-acknowledgment.

CONCLUSIONS

Human-centered design offers an innovative approach to improve guideline-based prescribing with optimizations that are grounded in providers' needs.

Critical care information display approaches and design frameworks: A systematic review and meta-analysis

OBJECTIVE

To systematically review original user evaluations of patient information displays relevant to critical care and understand the impact of design frameworks and information presentation approaches on decision-making, efficiency, workload, and preferences of clinicians.

METHODS

We included studies that evaluated information displays designed to support real-time care decisions in critical care or anesthesiology using simulated tasks. We searched PubMed and IEEExplore from 1/1/1990 to 6/30/2018. The search strategy was developed iteratively with calibration against known references. Inclusion screening was completed independently by two authors. Extraction of display features, design processes, and evaluation method was completed by one and verified by a second author.

RESULTS

Fifty-six manuscripts evaluating 32 critical care and 22 anesthesia displays were included. Primary outcome metrics included clinician accuracy and efficiency in recognizing, diagnosing, and treating problems. Implementing user-centered design (UCD) processes, especially iterative evaluation and redesign, resulted in positive impact in outcomes such as accuracy and efficiency. Innovative display approaches that led to improved human-system performance in critical care included: (1) improving the integration and organization of information, (2) improving the representation of trend information, and (3) implementing graphical approaches to make relationships between data visible.

CONCLUSION

Our review affirms the value of key principles of UCD. Improved information presentation can facilitate faster information interpretation and more accurate diagnoses and treatment. Improvements to information organization and support for rapid interpretation of time-based relationships between related quantitative data is warranted. Designers and developers are encouraged to involve users in formal iterative design and evaluation activities in the design of electronic health records (EHRs), clinical informatics applications, and clinical devices.

[Display technologies for augmented reality in medical applications]

BACKGROUND

One of the main challenges for modern surgery is the effective use of the many available imaging modalities and diagnostic methods. Augmented reality systems can be used in the future to blend patient and planning information into the view of surgeons, which can improve the efficiency and safety of interventions.

OBJECTIVE

In this article we present five visualization methods to integrate augmented reality displays into medical procedures and the advantages and disadvantages are explained.

MATERIAL AND METHODS

Based on an extensive literature review the various existing approaches for integration of augmented reality displays into medical procedures are divided into five categories and the most important research results for each approach are presented.

RESULTS

A large number of mixed and augmented reality solutions for medical interventions have been developed as research prototypes; however, only very few systems have been tested on patients.

CONCLUSION

In order to integrate mixed and augmented reality displays into medical practice, highly specialized solutions need to be developed. Such systems must comply with the requirements with respect to accuracy, fidelity, ergonomics and seamless integration into the surgical workflow.

Reinterpreting the nursing record for an electronic context

BACKGROUND

This article seeks to facilitate the re-imagining of nursing records purposefully within an electronic context. It questions existing approaches to nursing documentation, critically examines existing nursing record systems and identifies new requirements.

METHODS

A comprehensive literature review was conducted to identify themes, that might meaningfully contribute to a new approach to nursing record systems development, around four key interrelated areas - standards, decision making, abstraction and summarization, and documenting. Studies were analyzed using narrative synthesis to provide a critical analysis of the current 'state of the art', and recommendations for the future.

RESULTS

Included studies collectively described aspects of current best practice, both in terms of nursing record systems themselves, and how nurses and other health professionals contribute to and engage with those systems. A number of cross-cutting themes identified more novel approaches taken by nurses to systems development: going back to basics in determining purpose; firming up informatics foundations; nuancing or tailoring to suit different requirements; and engagement, involvement and participation.

CONCLUSION

There is a paucity of research that specifically focuses on the nature of the electronic nursing record and its impact on patient care processes and outcomes. In addition to further research in these areas, there is a need: to reinterpret nurses as knowledge workers rather than as 'data collectors'; to agree on the application in practice of appropriate standards and terminologies; and to work together with system developers to change the ways in which data are captured and care is documented.

Validity assessment of a simulation module for robot-assisted thoracic lobectomy

BACKGROUND

Training for robot-assisted thoracic lobectomy remains an issue, prompting the development of virtual reality simulators. Our aim was to assess the construct and face validity of a new thoracic lobectomy module on the RobotiX Mentor, a robotic surgery simulator. We also aimed to determine the acceptability and feasibility of implementation into training.

METHODS

This prospective, observational, and comparative study recruited novice (n = 16), intermediate (n = 9), and expert (n = 5) participants from King's College London, the 25th European Conference on General Thoracic Surgery, and the Society of Robotic Surgery conference 2018. Each participant completed two familiarization tasks followed by the Guided Robotic Lobectomy module and an evaluation questionnaire. Outcome measures were compared using Mann-Whitney U tests.

RESULTS

Construct validity was demonstrated in 12/21 performance evaluation metrics. Significant differences between groups were found in all metrics including: time taken to complete module, vascular injury, respect for tissue, number of stapler firings, time instruments out of view, number of instrument collisions, and number of movements. Participants deemed aspects of the simulator (mean 3/5) and module (3/5) as realistic and rated the simulator as both acceptable (3.8/5) and feasible (3.8/5) for robotic surgical training.

CONCLUSIONS

Face validity, acceptability, and feasibility were established for the thoracic lobectomy module of the RobotiX Mentor simulator. Moderate evidence of construct validity was also demonstrated. With further work, this simulation module could help to reduce the initial part of the learning curve for trainees and decrease the risk of errors during live training.

Evaluating the agreement of users with usability problems identified by heuristic evaluation

OBJECTIVE

Heuristic evaluation (HE) is an inspection method that can be used for identifying usability problems of health information systems. There is insufficient evidence to what extent these problems are experienced by users in their working environment. The objective of this study was to investigate the agreement of real end users of a health information system with the problems identified by HE.

METHODS

First, the emergency information system of educational hospitals in Kerman, Iran was evaluated by a heuristic evaluation. Then, a valid and reliable (Cronbach's alpha = 0.95) electronic questionnaire was designed based on the results of the first phase.

RESULTS

Among 127 identified problems, the highest number of problems was related to the "consistency and standards" heuristic (18.9%). The user agreement coefficient with heuristic results was 0.5. The highest agreement of the participants was on the problems related to the "help user recognize, diagnose and recover from errors" heuristic (78%). There was a significant relationship between participants age and work experience with a computer (p < 0.05).

CONCLUSION

The agreement rate of real end users with the results of HE was at a medium level. Therefore, in some areas, the perception of evaluators in using this method is not consistent with the users experience with a system. This can result in identifying minor issues and ignoring important system-critical usability issues. The HE is appropriate for identifying specific problems, such as problems related to identifying and correcting errors, but in some areas, such as consistency and standards in the design of a system, it identifies minor or ineffective problems.

Augmentation of the think aloud method with users' perspectives for the selection of a picture archiving and communication system

OBJECTIVES

Users attitude toward a picture archiving and communication system (PACS) and their interaction with this system are among the most important factors that influence its acceptance. This study aimed to augment the user's interaction with the user's perspective to select a usable PACS among three systems available on the market.

METHODS

We augmented the think aloud (TA) usability evaluation method with the Post-Study System Usability Questionnaire (PSSUQ) to compare user interaction problems of three PACS user interfaces. Four radiologists and four internist physicians participated in this study. Usability characteristics including efficiency, effectiveness, learnability, error, and satisfaction were used to assess the usability of each PACS.

RESULTS

There was a significant difference in efficiency (p = 0.01), effectiveness (p = 0.005), learnability (p = 0.001), and satisfaction (p = 0.009). However, no significant difference in the number of errors (p = 0.18), mouse clicks and keystrokes (p = 0.12), and the number of usability problems (p = 0.6) were observed among the three PACS systems studied.

CONCLUSIONS

This study showed that applying the proposed approach to augment TA with the user's perspective addresses almost all of the theoretical aspects of usability and can be employed to select the most usable PACS.

Hippocampal activation and memory performance in schizophrenia depend on strategy use in a virtual maze

Different strategies may be spontaneously adopted to solve most navigation tasks. These strategies are associated with dissociable brain systems. Here, we use brain-imaging and cognitive tasks to test the hypothesis that individuals living with Schizophrenia Spectrum Disorders (SSD) have selective impairment using a hippocampal-dependent spatial navigation strategy. Brain activation and memory performance were examined using functional magnetic resonance imaging (fMRI) during the 4-on-8 virtual maze (4/8VM) task, a human analog of the rodent radial-arm maze that is amenable to both response-based (egocentric or landmark-based) and spatial (allocentric, cognitive mapping) strategies to remember and navigate to target objects. SSD (schizophrenia and schizoaffective disorder) participants who adopted a spatial strategy performed more poorly on the 4/8VM task and had less hippocampal activation than healthy comparison participants using either strategy as well as SSD participants using a response strategy. This study highlights the importance of strategy use in relation to spatial cognitive functioning in SSD. Consistent with a selective-hippocampal dependent deficit in SSD, these results support the further development of protocols to train impaired hippocampal-dependent abilities or harness non-hippocampal dependent intact abilities.

Interdisciplinary Approach to the Development of Accessible Computer-Administered Measurement Instruments

Principles of fairness in testing require that all test takers, including people with disabilities, have an equal opportunity to demonstrate their capacity on the construct being measured. Measurement design features and assessment protocols can pose barriers for people with disabilities. Fairness in testing is a fundamental validity issue at all phases in the design, administration, and interpretation of measurement instruments in clinical practice and research. There is limited guidance for instrument developers on how to develop and evaluate the accessibility and usability of measurement instruments. This article describes a 6-stage iterative process for developing accessible computer-administered measurement instruments grounded in the procedures implemented across several major measurement initiatives. A key component of this process is interdisciplinary teams of accessibility experts, content and measurement experts, information technology experts, and people with disabilities working together to ensure that measurement instruments are accessible and usable by a wide range of users. The development of accessible measurement instruments is not only an ethical requirement, it also ensures better science by minimizing measurement bias, missing data, and attrition due to mismatches between the target population and test administration platform and protocols.

Usability evaluation methods employed to assess information visualisations of electronically stored patient data for clinical use: a protocol for a systematic review

BACKGROUND

The use of electronic records in healthcare is increasing. To avoid errors, it is essential that the data displays used by these systems are usable: efficient, effective and satisfying. A wide variety of display techniques are used to present clinical data, but the best methods to assess the usability of these techniques have not been determined. This systematic review will answer the question: What methods are employed to assess the usability of electronic visualisations of patient data for clinical use? The results of this systematic review will then be used to inform best assessment and design practice.

METHODS

MEDLINE, EMBASE, CINAHL, OpenGrey, and the Cochrane Database of Systematic Reviews will be searched for original studies related to the usability of electronic information visualisations of patient data for clinical use. Reference lists of eligible studies and relevant reviews will be explored to identify further eligible studies.

DISCUSSION

This systematic review will identify methods used to assess the usability of electronic information visualisations of patient data for clinical use. We will summarise the similarities and differences between the methods found. Our results will inform best practice when developing new user interfaces to display electronic patient data for clinical use.

TRIAL REGISTRATION

PROSPERO CRD42016041604.

A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory

BACKGROUND

Audit and feedback is a common intervention for supporting clinical behaviour change. Increasingly, health data are available in electronic format. Yet, little is known regarding if and how electronic audit and feedback (e-A&F) improves quality of care in practice.

OBJECTIVE

The study aimed to assess the effectiveness of e-A&F interventions in a primary care and hospital context and to identify theoretical mechanisms of behaviour change underlying these interventions.

METHODS

In August 2016, we searched five electronic databases, including MEDLINE and EMBASE via Ovid, and the Cochrane Central Register of Controlled Trials for published randomised controlled trials. We included studies that evaluated e-A&F interventions, defined as a summary of clinical performance delivered through an interactive computer interface to healthcare providers. Data on feedback characteristics, underlying theoretical domains, effect size and risk of bias were extracted by two independent review authors, who determined the domains within the Theoretical Domains Framework (TDF). We performed a meta-analysis of e-A&F effectiveness, and a narrative analysis of the nature and patterns of TDF domains and potential links with the intervention effect.

RESULTS

We included seven studies comprising of 81,700 patients being cared for by 329 healthcare professionals/primary care facilities. Given the extremely high heterogeneity of the e-A&F interventions and five studies having a medium or high risk of bias, the average effect was deemed unreliable. Only two studies explicitly used theory to guide intervention design. The most frequent theoretical domains targeted by the e-A&F interventions included 'knowledge', 'social influences', 'goals' and 'behaviour regulation', with each intervention targeting a combination of at least three. None of the interventions addressed the domains 'social/professional role and identity' or 'emotion'. Analyses identified the number of different domains coded in control arm to have the biggest role in heterogeneity in e-A&F effect size.

CONCLUSIONS

Given the high heterogeneity of identified studies, the effects of e-A&F were found to be highly variable. Additionally, e-A&F interventions tend to implicitly target only a fraction of known theoretical domains, even after omitting domains presumed not to be linked to e-A&F. Also, little evaluation of comparative effectiveness across trial arms was conducted. Future research should seek to further unpack the theoretical domains essential for effective e-A&F in order to better support strategic individual and team goals.

Group-based exercise at workplace: short-term effects of neck and shoulder resistance training in video display unit workers with work-related chronic neck pain-a pilot randomized trial

Effectiveness of exercise therapy in video display unit (VDU) workers with work-related chronic neck pain (W-RCNP) is unclear. The aim of the study was to assess the efficacy of group-based neck-shoulder resistance exercises on symptoms and muscular function in VDU workers with W-RCNP. Thirty-five employees with CNP were randomly assigned to neck-shoulder resistance exercise (NSRE) group or to conventional stretching and postural exercise (SPE) group. Participants trained 45 min, twice a week for 7 weeks. Primary outcomes were pain intensity (0-10 numeric rating scale, NRS) and pain-related disability (Vernon Neck Disability Index, NDI). Secondary outcomes were cervical active range of motion (AROM), endurance of neck flexors, strength of shoulder abductors, and health-related quality of life (SF-36). Post-treatment, pain and disability decreased 2.5 and 5 points in NSRE (p = 0.001) and 5 and 8 points in SPE (p = 0.002), with no significant differences between groups. AROM improved significantly in both groups; between-group differences were not significant. Neck flexors endurance and shoulder abductors strength improved by 59.6 and 34.2% (p = 0.001) in the NSRE group and by 27.6% (p = 0.009) and 2.3% (p = 0.002) in the SPE group; between-group differences were significant (p = 0.016 and p < 0.001, respectively). SF-36 score remained unchanged. NSRE and SPE are feasible and equally effective in reducing pain and pain-related disability as group exercises in W-RCNP. NSRE may provide some advantage in improving strength and endurance in neck muscles. Further, larger studies should confirm these findings and assess whether a better muscle function confers long-term clinical advantages.

Short-term motor learning through non-immersive virtual reality task in individuals with down syndrome

Background

Down syndrome (DS) has unique physical, motor and cognitive characteristics. Despite cognitive and motor difficulties, there is a possibility of intervention based on the knowledge of motor learning. However, it is important to study the motor learning process in individuals with DS during a virtual reality task to justify the use of virtual reality to organize intervention programs. The aim of this study was to analyze the motor learning process in individuals with DS during a virtual reality task.

Methods

A total of 40 individuals participated in this study, 20 of whom had DS (24 males and 8 females, mean age of 19 years, ranging between 14 and 30 yrs.) and 20 typically developing individuals (TD) who were matched by age and gender to the individuals with DS. To examine this issue, we used software that uses 3D images and reproduced a coincidence-timing task.

Results

The results showed that all individuals improved performance in the virtual task, but the individuals with DS that started the task with worse performance showed higher difference from the beginning. Besides that, they were able to retain and transfer the performance with increase of speed of the task.

Conclusion

Individuals with DS are able to learn movements from virtual tasks, even though the movement time was higher compared to the TD individuals. The results showed that individuals with DS who started with low performance improved coincidence- timing task with virtual objects, but were less accurate than typically developing individuals.

Trial registration

ClinicalTrials.gov Identifier: NCT02719600.

Pilot evaluation of a method to assess prescribers' information processing of medication alerts

BACKGROUND

Prescribers commonly receive alerts during medication ordering. Prescribers work in a complex, time-pressured environment; to enhance the effectiveness of safety alerts, the effort needed to cognitively process these alerts should be minimized. Methods to evaluate the extent to which computerized alerts support prescribers' information processing are lacking.

OBJECTIVE

To develop a methodological protocol to assess the extent to which alerts support prescribers' information processing at-a-glance; specifically, the incorporation of information into their working memory. We hypothesized that the method would be feasible and that we would be able to detect a significant difference in prescribers' information processing with a revised alert display that incorporates warning design guidelines compared to the original alert display.

METHODS

A counterbalanced, within-subject study was conducted with 20 prescribers in a human-computer interaction laboratory. We tested a single alert that was displayed in two different ways. Prescribers were informed that an alert would appear for 10s. After the alert was shown, a white screen was displayed, and prescribers were asked to verbally describe what they saw; indicate how many total warnings; and describe anything else they remembered about the alert. We measured information processing via the accuracy of prescribers' free recall and their ability to identify that three warning messages were present. Two analysts independently evaluated participants' responses against a comprehensive catalog of alert elements and then discussed discrepancies until reaching consensus.

RESULTS

This feasibility study demonstrated that the method seemed to be effective for evaluating prescribers' information processing of medication alert displays. With this method, we were able to detect significant differences in prescribers' recall of alert information. The proportion of total data elements that prescribers were able to accurately recall was significantly greater for the revised versus original alert display (p=0.006). With the revised display, more prescribers accurately reported that three warnings were shown (p=0.002).

CONCLUSIONS

The methodological protocol was feasible for evaluating the alert display and yielded important findings on prescribers' information processing. Study methods supplement traditional usability evaluation methods and may be useful for evaluating information processing of other healthcare technologies.

Usability evaluation of the SMART application for youth with mTBI

OBJECTIVE

There is a dearth of evidence-based treatments available to address the significant morbidity associated with mild traumatic brain injury (mTBI). To address this gap, we designed a novel user-friendly, web-based application. We describe the preliminary evaluation of feasibility and usability of the application to promote recovery following mTBI in youth, the Self-Monitoring Activity-Restriction and Relaxation Treatment (SMART). SMART incorporates real-time recommendations for individualized symptom management and activity restriction along with training in cognitive-behavioral coping strategies.

METHODS

We conducted a usability evaluation to assess and modify the SMART system prior to further study and deployment. Children ages 11-18 years presenting to the emergency department were recruited after symptoms resolved. Usability was assessed using a 60-min think-aloud protocol of teens and parents describing their interaction with the application. Upon completion of the tasks, each participant also completed the system usability scale (SUS).

RESULTS

We performed tests with 4 parent/child dyads. The average age of the children was 13 years (standard deviation=1.8). The parents were an average of 41.5 years old (standard deviation=6.2). Research revealed that the participants were enthusiastic about the interactive portions of the tool particularly the video based sessions. Parents were concerned about the speed at which their child might move through the program and the children thought that the system required large amounts of reading. Based on user feedback, researchers modified SMART to include an audio file in every module and improved the system's aesthetic properties. The mean SUS score was 85, with high SUS scores (>68) indicating satisfactory usability.

CONCLUSION

High initial usability and favorable user feedback provide a foundation for further iterative development and testing of the SMART application as a tool for managing recovery from concussion.

Post-deployment usability evaluation of a radiology workstation

OBJECTIVES

To determine the number, nature and severity of usability issues radiologists encounter while using a commercially available radiology workstation in clinical practice, and to assess how well the results of a pre-deployment usability evaluation of this workstation generalize to clinical practice.

METHODS

The usability evaluation consisted of semi-structured interviews and observations of twelve users using the workstation during their daily work. Usability issues and positive usability findings were documented. Each issue was given a severity rating and its root cause was determined. Results were compared to the results of a pre-deployment usability evaluation of the same workstation.

RESULTS

Ninety-two usability issues were identified, ranging from issues that cause minor frustration or delay, to issues that cause significant delays, prevent users from completing tasks, or even pose a potential threat to patient safety. The results of the pre-deployment usability evaluation had limited generalizability to clinical practice.

CONCLUSIONS

This study showed that radiologists encountered a large number and a wide variety of usability issues when using a commercially available radiology workstation in clinical practice. This underlines the need for effective usability engineering in radiology. Given the limitations of pre-deployment usability evaluation in radiology, which were confirmed by our finding that the results of a pre-deployment usability evaluation of this workstation had limited generalizability to clinical practice, it is vital that radiology workstation vendors devote significant resources to usability engineering efforts before deployment of their workstation, and to continue these efforts after the workstation is deployed in a hospital.

Pattern mining of user interaction logs for a post-deployment usability evaluation of a radiology PACS client

OBJECTIVES

To perform a post-deployment usability evaluation of a radiology Picture Archiving and Communication System (PACS) client based on pattern mining of user interaction log data, and to assess the usefulness of this approach compared to a field study.

METHODS

All user actions performed on the PACS client were logged for four months. A data mining technique called closed sequential pattern mining was used to automatically extract frequently occurring interaction patterns from the log data. These patterns were used to identify usability issues with the PACS. The results of this evaluation were compared to the results of a field study based usability evaluation of the same PACS client.

RESULTS

The interaction patterns revealed four usability issues: (1) the display protocols do not function properly, (2) the line measurement tool stays active until another tool is selected, rather than being deactivated after one use, (3) the PACS's built-in 3D functionality does not allow users to effectively perform certain 3D-related tasks, (4) users underuse the PACS's customization possibilities. All usability issues identified based on the log data were also found in the field study, which identified 48 issues in total.

CONCLUSIONS

Post-deployment usability evaluation based on pattern mining of user interaction log data provides useful insights into the way users interact with the radiology PACS client. However, it reveals few usability issues compared to a field study and should therefore not be used as the sole method of usability evaluation.

[Identifying Usability Level and Factors Affecting Electronic Nursing Record Systems: A Multi-institutional Time-motion Approach]

PURPOSE

The usability, user satisfaction, and impact of electronic nursing record (ENR) systems were investigated.

METHODS

This mixed-method research was performed as a time-motion (TM) study and a survey which were carried out at six hospitals between August and November 2013. The TM study involved 108 nurses from medical, surgical, and intensive care units at each hospital, plus an additional 48 nurses who served as nonparticipating observers. In the survey, 1879 volunteer nurses completed the Impact of ENR Systems Scale, the System Usability Scale, and a global satisfaction scale. Qualitative and quantitative analyses were performed.

RESULTS

The mean scores for the ENR impact, system usability, and satisfaction were 4.28 (out of 6), 58.62 (out of 100), and 74.31 (out of 100), respectively, and they differed significantly between hospitals (F=43.43, p<.001, F=53.08 and p<.001, and F=29.13 and p<.001, respectively). A workflow fragmentation assessment revealed different patterns of ENR system use among the included hospitals. Three user characteristics-educational background, practice period, and experience of using paper records-significantly affected the system usability and satisfaction scores.

CONCLUSION

The system quality varied widely among the ENR systems. The generally low-to-moderate levels of system usability and user satisfaction suggest many opportunities for improvement.

Clinical safety of England's national programme for IT: a retrospective analysis of all reported safety events 2005 to 2011

OBJECTIVE

To analyse patient safety events associated with England's national programme for IT (NPfIT).

METHODS

Retrospective analysis of all safety events managed by a dedicated IT safety team between September 2005 and November 2011 was undertaken. Events were reviewed against an existing classification for problems associated with IT. The proportion of reported events per problem type, consequences, source of report, resolution within 24h, time of day and day of week were examined. Sub-group analyses were undertaken for events involving patient harm and those that occurred on a large scale.

RESULTS

Of the 850 events analysed, 68% (n=574) described potentially hazardous circumstances, 24% (n=205) had an observable impact on care delivery, 4% (n=36) were a near miss, and 3% (n=22) were associated with patient harm, including three deaths (0·35%). Eleven events did not have a noticeable consequence (1%) and two were complaints (<1%). Amongst the events 1606 separate contributing problems were identified. Of these 92% were predominately associated with technical rather than human factors. Problems involving human factors were four times as likely to result in patient harm than technical problems (25% versus 8%; OR 3·98, 95%CI 1·90-8.34). Large-scale events affecting 10 or more individuals or multiple IT systems accounted for 23% (n=191) of the sample and were significantly more likely to result in a near miss (6% versus 4%) or impact the delivery of care (39% versus 20%; p<0·001).

CONCLUSION

Events associated with NPfIT reinforce that the use of IT does create hazardous circumstances and can lead to patient harm or death. Large-scale patient safety events have the potential to affect many patients and clinicians, and this suggests that addressing them should be a priority for all major IT implementations.

Usability evaluation of pharmacogenomics clinical decision support aids and clinical knowledge resources in a computerized provider order entry system: a mixed methods approach

BACKGROUND

Pharmacogenomics (PGx) is positioned to have a widespread impact on the practice of medicine, yet physician acceptance is low. The presentation of context-specific PGx information, in the form of clinical decision support (CDS) alerts embedded in a computerized provider order entry (CPOE) system, can aid uptake. Usability evaluations can inform optimal design, which, in turn, can spur adoption.

OBJECTIVES

The study objectives were to: (1) evaluate an early prototype, commercial CPOE system with PGx-CDS alerts in a simulated environment, (2) identify potential improvements to the system user interface, and (3) understand the contexts under which PGx knowledge embedded in an electronic health record is useful to prescribers.

METHODS

Using a mixed methods approach, we presented seven cardiologists and three oncologists with five hypothetical clinical case scenarios. Each scenario featured a drug for which a gene encoding drug metabolizing enzyme required consideration of dosage adjustment. We used Morae(®) to capture comments and on-screen movements as participants prescribed each drug. In addition to PGx-CDS alerts, 'Infobutton(®)' and 'Evidence' icons provided participants with clinical knowledge resources to aid decision-making.

RESULTS

Nine themes emerged. Five suggested minor improvements to the CPOE user interface; two suggested presenting PGx information through PGx-CDS alerts using an 'Infobutton' or 'Evidence' icon. The remaining themes were strong recommendations to provide succinct, relevant guidelines and dosing recommendations of phenotypic information from credible and trustworthy sources; any more information was overwhelming. Participants' median rating of PGx-CDS system usability was 2 on a Likert scale ranging from 1 (strongly agree) to 7 (strongly disagree).

CONCLUSIONS

Usability evaluation results suggest that participants considered PGx information important for improving prescribing decisions; and that they would incorporate PGx-CDS when information is presented in relevant and useful ways.

Comparison of two kinds of interface, based on guided navigation or usability principles, for improving the adoption of computerized decision support systems: application to the prescription of antibiotics

CONTEXT

It is important to consider the way in which information is presented by the interfaces of clinical decision support systems, to favor the adoption of these systems by physicians. Interface design can focus on decision processes (guided navigation) or usability principles.

OBJECTIVE

The aim of this study was to compare these two approaches in terms of perceived usability, accuracy rate, and confidence in the system.

MATERIALS AND METHODS

We displayed clinical practice guidelines for antibiotic treatment via two types of interface, which we compared in a crossover design. General practitioners were asked to provide responses for 10 clinical cases and the System Usability Scale (SUS) for each interface. We assessed SUS scores, the number of correct responses, and the confidence level for each interface.

RESULTS

SUS score and percentage confidence were significantly higher for the interface designed according to usability principles (81 vs 51, p=0.00004, and 88.8% vs 80.7%, p=0.004). The percentage of correct responses was similar for the two interfaces.

DISCUSSION/CONCLUSION

The interface designed according to usability principles was perceived to be more usable and inspired greater confidence among physicians than the guided navigation interface. Consideration of usability principles in the construction of an interface--in particular 'effective information presentation', 'consistency', 'efficient interactions', 'effective use of language', and 'minimizing cognitive load'--seemed to improve perceived usability and confidence in the system.

Supporting medical communication for older patients with a shared touch-screen computer

OBJECTIVE

Increasingly health care facilities are adopting electronic medical record systems and installing computer workstations in patient exam rooms. The introduction of computer workstations into the medical interview process makes it important to consider the impact of such technology on older patients as well as new types of interfaces that may better suit the needs of older adults. While many older adults are comfortable with a traditional computer workstation with a keyboard and mouse, this article explores how a large horizontal touch-screen (i.e., a surface computer) may suit the needs of older patients and facilitates the doctor-patient interview process.

METHOD

Twenty older adults (age 60 to 88) used a prototype multiuser, multitouch system in our research laboratory to examine seven health care scenarios. Behavioral observations as well as results from questionnaires and a structured interview were analyzed.

RESULTS

The older adults quickly adapted to the prototype system and reported that it was easy to use. Participants also suggested that having a shared view of one's medical records, especially charts and images, would enhance communication with their doctor and aid understanding.

CONCLUSION

While this study is exploratory and some areas of interaction with a surface computer need to be refined, the technology is promising for sharing electronic patient information during medical interviews involving older adults. Future work must examine doctors' and nurses' interaction with the technology as well as logistical issues of installing such a system in a real world medical setting.