1
|
A Rodrigues D, I Plácido A, Mateos-Campos R, Figueiras A, Herdeiro MT, Roque F. Usability of APIMedOlder: A Web Application to Manage Potentially Inappropriate Medication in Older Adults. ACTA MEDICA PORT 2024; 37:609-616. [PMID: 39226560 DOI: 10.20344/amp.21537] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2024] [Accepted: 06/05/2024] [Indexed: 09/05/2024]
Abstract
INTRODUCTION Considering the increase in the proportion of the older population worldwide, the demand for health system resources also arises. These tools optimize clinical decision-making, thus avoiding iatrogenesis and thus contributing to a better quality of life for the older population. In response, we created an online web application, the APIMedOlder, that provides access to healthcare professionals to allow healthcare professionals to access potentially inappropriate medication identification criteria through a useful tool with a simplified profile, allowing its applicability in clinical practice. This study aims to assess the usability of the APIMedOlder online web application by healthcare professionals. METHODS A questionnaire, based on the System Usability Scale, was distributed among 15 healthcare professionals (five pharmacists, four physicians, three pharmacy technicians, and three nurses), to fully explore the website. RESULTS Overall, healthcare professionals' evaluation of the usability of the APIMedOlder online web application was rated as "Best imaginable" (mean score of 87.17 points), with individual scores ranging from 75 to 100 points. Internal consistency of α = 0.881 (CI 95%: 0.766 - 0.953) was achieved. Specific questionnaire items contributing to this high score included ease of use, learning efficiency, and integration of functions. CONCLUSION The overall evaluation of the developed tool was positive, with this online application being recognized as being easy to use and having well-integrated functions.
Collapse
Affiliation(s)
- Daniela A Rodrigues
- Biotechnology Research, Innovation and Design for Health Products (BRIDGES) - Research Laboratory on Epidemiology and Population Health. Instituto Politécnico da Guarda. Guarda; Health Sciences Research Centre (CICS). Universidade da Beira Interior. Covilhã; Universidad de Salamanca. Salamanca. Spain
| | - Ana I Plácido
- Biotechnology Research, Innovation and Design for Health Products (BRIDGES) - Research Laboratory on Epidemiology and Population Health. Instituto Politécnico da Guarda. Guarda. Portugal
| | - Ramona Mateos-Campos
- Area of Preventive Medicine and Public Health. Department of Biomedical and Diagnostic Sciences. Universidad de Salamanca. Salamanca. Spain
| | - Adolfo Figueiras
- Department of Preventive Medicine and Public Health. Universidade de Santiago de Compostela. Santiago de Compostela. Spain; Health Research Institute of Santiago de Compostela (IDIS). Santiago de Compostela. Spain; Consortium for Biomedical Research in Epidemiology and Public Health (CIBER Epidemiology and Public Health-CIBERESP). Madrid. Spain
| | - Maria Teresa Herdeiro
- Department of Medical Sciences. Institute of Biomedicine (iBiMED). Universidade de Aveiro. Aveiro. Portugal
| | - Fátima Roque
- Biotechnology Research, Innovation and Design for Health Products (BRIDGES) - Research Laboratory on Epidemiology and Population Health. Instituto Politécnico da Guarda. Guarda; Health Sciences Research Centre (CICS). Universidade da Beira Interior. Covilhã. Portugal
| |
Collapse
|
2
|
Alfian SD, Sania JA, Aini DQ, Khoiry QA, Griselda M, Ausi Y, Zakiyah N, Puspitasari IM, Suwantika AA, Mahfud M, Aji S, Abdulah R, Kassianos AP. Evaluation of usability and user feedback to guide telepharmacy application development in Indonesia: a mixed-methods study. BMC Med Inform Decis Mak 2024; 24:130. [PMID: 38773562 PMCID: PMC11106925 DOI: 10.1186/s12911-024-02494-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Accepted: 03/27/2024] [Indexed: 05/24/2024] Open
Abstract
BACKGROUND In Indonesia, the adoption of telepharmacy was propelled by the COVID-19 pandemic, prompting the need for a user-friendly application to support both the general population and pharmacists in accessing healthcare services. Therefore, this study aimed to evaluate usability and user feedback of a pioneering telepharmacy application known as Tanya Obat (translating to "Ask about Medications") in Indonesia, from the perspectives of the general population and pharmacists. METHODS A mixed-methods sequential study was conducted with the early-stage Tanya Obat application in Bandung City. Participants, including the general population and pharmacists, were instructed to use the application for a week. Questionnaires for the general population and pharmacists were distributed from March to May and February to June 2023, respectively. The System Usability Scale questionnaire was adopted to describe usability of the developed application. Further exploration of the quantitative results required collecting open-ended feedback to assess the impressions of the participants, difficulties encountered, and desired features for enhanced user-friendliness. The collected statements were summarized and clustered using thematic analysis. Subsequently, the association between the characteristics of participants and perceived usability was determined with the Chi-square test. RESULT A total of 176 participants, comprising 100 individuals from the general population and 76 pharmacists, engaged in this study. In terms of usability, the questionnaire showed that Tanya Obat application was on the borderline of acceptability, with mean scores of 63.4 and 64.1 from the general population and pharmacists, respectively. Additionally, open-ended feedback targeted at achieving a more compelling user experience was categorized into two themes, including concerns regarding the functionality of certain features and recommendations for improved visual aesthetics and bug fixes. No significant associations were observed between the characteristics of participants and perceived usability (p-value > 0.05). CONCLUSION The results showed that the perceived usability of Tanya Obat developed for telepharmacy was below average. Therefore, feature optimizations should be performed to facilitate usability of this application in Indonesia.
Collapse
Affiliation(s)
- Sofa D Alfian
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia.
- Drug Utilization and Pharmacoepidemiology Research Group, Center of Excellence for Pharmaceutical Care Innovation, Universitas Padjadjaran, Jatinangor, Indonesia.
- Center for Health Technology Assessment, Universitas Padjadjaran, Jatinangor, Indonesia.
| | - Jihan A Sania
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia
| | - Dzulfah Q Aini
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia
| | - Qisty A Khoiry
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia
| | - Meliana Griselda
- Drug Utilization and Pharmacoepidemiology Research Group, Center of Excellence for Pharmaceutical Care Innovation, Universitas Padjadjaran, Jatinangor, Indonesia
| | - Yudisia Ausi
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia
- Drug Utilization and Pharmacoepidemiology Research Group, Center of Excellence for Pharmaceutical Care Innovation, Universitas Padjadjaran, Jatinangor, Indonesia
| | - Neily Zakiyah
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia
- Drug Utilization and Pharmacoepidemiology Research Group, Center of Excellence for Pharmaceutical Care Innovation, Universitas Padjadjaran, Jatinangor, Indonesia
| | - Irma M Puspitasari
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia
- Drug Utilization and Pharmacoepidemiology Research Group, Center of Excellence for Pharmaceutical Care Innovation, Universitas Padjadjaran, Jatinangor, Indonesia
| | - Auliya A Suwantika
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia
- Drug Utilization and Pharmacoepidemiology Research Group, Center of Excellence for Pharmaceutical Care Innovation, Universitas Padjadjaran, Jatinangor, Indonesia
- Center for Health Technology Assessment, Universitas Padjadjaran, Jatinangor, Indonesia
| | | | - Saktian Aji
- Dienggo Kreasi Nusantara Company, Jakarta, Indonesia
| | - Rizky Abdulah
- Department of Pharmacology and Clinical Pharmacy, Faculty of Pharmacy, Universitas Padjadjaran, Jatinangor, Indonesia
- Drug Utilization and Pharmacoepidemiology Research Group, Center of Excellence for Pharmaceutical Care Innovation, Universitas Padjadjaran, Jatinangor, Indonesia
| | - Angelos P Kassianos
- Department of Nursing, School of Health Sciences, Cyprus University of Technology, Limassol, Cyprus
| |
Collapse
|
3
|
Fossouo Tagne J, Yakob RA, Mcdonald R, Wickramasinghe N. A Web-Based Tool to Report Adverse Drug Reactions by Community Pharmacists in Australia: Usability Testing Study. JMIR Form Res 2023; 7:e48976. [PMID: 37773620 PMCID: PMC10576234 DOI: 10.2196/48976] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2023] [Revised: 08/04/2023] [Accepted: 08/04/2023] [Indexed: 10/01/2023] Open
Abstract
BACKGROUND Adverse drug reactions (ADRs) are unintended and harmful events associated with medication use. Despite their significance in postmarketing surveillance, quality improvement, and drug safety research, ADRs are vastly underreported. Enhanced digital-based communication of ADR information to regulators and among care providers could significantly improve patient safety. OBJECTIVE This paper presents a usability evaluation of the commercially available GuildCare Adverse Event Recording system, a web-based ADR reporting system widely used by community pharmacists (CPs) in Australia. METHODS We developed a structured interview protocol encompassing remote observation, think-aloud moderating techniques, and retrospective questioning to gauge the overall user experience, complemented by the System Usability Scale (SUS) assessment. Thematic analysis was used to analyze field notes from the interviews. RESULTS A total of 7 CPs participated in the study, who perceived the system to have above-average usability (SUS score of 68.57). Nonetheless, the structured approach to usability testing unveiled specific functional and user interpretation issues, such as unnecessary information, lack of system clarity, and redundant data fields-critical insights not captured by the SUS results. Design elements like drop-down menus, free-text entry, checkboxes, and prefilled or auto-populated data fields were perceived as useful for enhancing system navigation and facilitating ADR reporting. CONCLUSIONS The user-centric design of technology solutions, like the one discussed herein, is crucial to meeting CPs' information needs and ensuring effective ADR reporting. Developers should adopt a structured approach to usability testing during the developmental phase to address identified issues comprehensively. Such a methodological approach may promote the adoption of ADR reporting systems by CPs and ultimately enhance patient safety.
Collapse
Affiliation(s)
- Joel Fossouo Tagne
- School of Health Sciences and Biostatistics, Swinburne University of Technology, Melbourne, Australia
- Centre for Health Analytics, Murdoch Children's Research Institute, Health Informatics, Melbourne, Australia
| | | | - Rachael Mcdonald
- MedTechVic, Swinburne University of Technology, Melbourne, Australia
- Department of Nursing and Allied Health, Occupational Therapy, Swinburne University of Technology, Melbourne, Australia
- Iverson Health Innovation Research Institute, Swinburne University of Technology, Melbourne, Australia
| | - Nilmini Wickramasinghe
- School of Health Sciences and Biostatistics, Swinburne University of Technology, Melbourne, Australia
- Iverson Health Innovation Research Institute, Swinburne University of Technology, Melbourne, Australia
- Epworth Healthcare, Melbourne, Australia
- School of Computing, Engineering & Mathematical Sciences, La Trobe University, Melbourne, Australia
| |
Collapse
|
4
|
Burningham Z, Jackson GL, Kelleher JL, Morris I, Stevens MB, Cohen J, Maloney G, Sauer BC, Halwani AS, Chen W, Vaughan CP. Use of a Medication Safety Audit and Feedback Tool in the Emergency Department Is Affected by Prescribing Characteristics. Appl Clin Inform 2023; 14:684-692. [PMID: 37648222 PMCID: PMC10468720 DOI: 10.1055/s-0043-1771393] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Accepted: 05/17/2023] [Indexed: 09/01/2023] Open
Abstract
BACKGROUND The Enhancing Quality of Prescribing Practices for Older Veterans Discharged from the Emergency Department (EQUIPPED) program developed an audit and feedback health information technology (IT) solution with the intent to replace the in-person academic detailing service provided by the program. The EQUIPPED dashboard provides emergency department (ED) providers with a personalized view of their prescribing performance. OBJECTIVES Here, we analyze the association between ED provider characteristics and viewership of the EQUIPPED dashboard, adding insight into strategies for addressing barriers to initial use. METHODS We performed a retrospective analysis of EQUIPPED dashboard viewership among four Veterans Affairs (VA) EDs. We extracted quantitative data from user interaction logs to determine evidence of dashboard use. Provider characteristics and baseline potentially inappropriate medication (PIM) prescribing rate were extracted from the VA's Corporate Data Warehouse. Logistic regression was used to examine the association between dashboard use and provider characteristics. RESULTS A total of 82 providers were invited to receive audit and feedback via the EQUIPPED dashboard. Among invited providers, 40 (48.7%) had evidence of at least 1 dashboard view during the 1-year feedback period. Adjusted analyses suggest that providers with a higher baseline PIM prescribing rate were more likely to use the dashboard (odds ratio [OR]: 1.22; 95% confidence interval [CI]: 1.01-1.47). Furthermore, providers at ED site D were more likely to use the dashboard in comparison to the other sites (OR: 9.99; 95% CI: 1.72-58.04) and reportedly had the highest site-level baseline PIM rate. CONCLUSION Providers with lower PIM prescribing rates (i.e., <5%) receive communication from an integrated dashboard reminder system that they are "optimal prescribers" which may have discouraged initial attempts to view the dashboard. Site D had the highest baseline PIM rate, but further qualitative investigation is warranted to better understand why site D had the greatest users of the dashboard.
Collapse
Affiliation(s)
- Zach Burningham
- Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
- Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States
| | - George L. Jackson
- Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, North Carolina, United States
- Medicine (Division of General Internal Medicine), and Family Medicine & Community Health, Departments of Population Health Sciences, Duke University, Durham, North Carolina, United States
| | - Jessica L. Kelleher
- Department of Veterans Affairs, Birmingham/Atlanta Geriatric Research, Education, and Clinical Center, Decatur, Georgia, United States
| | - Isis Morris
- Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, North Carolina, United States
| | - Melissa B. Stevens
- Department of Veterans Affairs, Birmingham/Atlanta Geriatric Research, Education, and Clinical Center, Decatur, Georgia, United States
- Division of General Medicine and Division of Geriatrics and Gerontology, Department of Medicine, Emory University, Atlanta, Georgia, United States
| | - Joy Cohen
- Department of Emergency Medicine, New Orleans Veterans Affairs Medical Center, New Orleans, Louisiana, United States
| | - Gerald Maloney
- Department of Emergency Medicine, Cleveland Veterans Affairs Medical Center, Cleveland, Ohio, United States
| | - Brian C. Sauer
- Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
| | - Ahmad S. Halwani
- Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
- Division of Hematology and Hematologic Malignancies, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States
| | - Wei Chen
- Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
| | - Camille P. Vaughan
- Department of Veterans Affairs, Birmingham/Atlanta Geriatric Research, Education, and Clinical Center, Decatur, Georgia, United States
| |
Collapse
|
5
|
Tungare S, Gantela S, Kookal KK, Yansane AI, Sedlock E, Jeske A, Johnson T, Walji M. Designing audit and feedback dashboards for dentists to monitor their opioid prescribing. Int J Med Inform 2023; 176:105092. [PMID: 37267811 DOI: 10.1016/j.ijmedinf.2023.105092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Revised: 02/28/2023] [Accepted: 05/06/2023] [Indexed: 06/04/2023]
Abstract
BACKGROUND AND OBJECTIVE Prescription drug abuse is a major factor leading to drug overdose deaths in the US and dentists are one of the leading prescribers of opioid pain medication. Knowing that Audit & Feedback (A&F) dashboards are an effective tool and are used as quality improvement interventions, we aimed to develop such dashboards personalized for dental providers which could allow them to monitor their own opioid prescribing performance. METHODS In this paper we report on the process for designing the A&F dashboards for dentists which were developed by using an iterative human-centered design process. The results obtained from each iteration were used to enrich the information needs analyses, provide function testing, and guide the design decisions of the next iteration. RESULTS Engaging dentists in the development and refinement of the dashboards while using the think-aloud protocol for user-testing, provided rapid feedback and identified areas that were confusing and needed either a redesign or additional explanatory content. The final version of dashboards consisted of displaying necessary information through easy to interpret visualizations and interactive features. These included providing access to current national and organizational prescribing guidelines, displaying changes in individual prescribing behavior over time, comparing individual prescribing rate to peer group rate and target rate, displaying procedure specific prescribing, integrating patient reported post-operative dental pain experience and providing navigation and interpretation tips for users. The dashboards were easy to learn and understand for the dentists and were deemed as worth using often in dental practice. CONCLUSION Our research was able to demonstrate the creation of useful and usable A&F dashboards using data from electronic dental records and patient surveys, for dentists to effectively monitor their opioid prescribing behavior. Efficacy of the dashboards will be tested in future work.
Collapse
Affiliation(s)
- Sayali Tungare
- The University of Texas Health Science Center at Houston (UTHealth Houston) School of Dentistry, 7500 Cambridge St, Houston, TX 77054, United States
| | - Swaroop Gantela
- The University of Texas Health Science Center at Houston (UTHealth Houston) School of Biomedical Informatics, 7000 Fannin St Suite 600, Houston, TX 77030, United States
| | - Krishna Kumar Kookal
- The University of Texas Health Science Center at Houston (UTHealth Houston) School of Dentistry, 7500 Cambridge St, Houston, TX 77054, United States
| | - Alfa-Ibrahim Yansane
- The University of California San Francisco School of Dentistry, 707 Parnassus Ave, San Francisco, CA 94143, United States
| | - Emily Sedlock
- The University of Texas Health Science Center at Houston (UTHealth Houston) School of Dentistry, 7500 Cambridge St, Houston, TX 77054, United States
| | - Arthur Jeske
- The University of Texas Health Science Center at Houston (UTHealth Houston) School of Dentistry, 7500 Cambridge St, Houston, TX 77054, United States
| | - Todd Johnson
- The University of Texas Health Science Center at Houston (UTHealth Houston) School of Biomedical Informatics, 7000 Fannin St Suite 600, Houston, TX 77030, United States
| | - Muhammad Walji
- The University of Texas Health Science Center at Houston (UTHealth Houston) School of Dentistry, 7500 Cambridge St, Houston, TX 77054, United States.
| |
Collapse
|
6
|
Fossouo Tagne J, Yakob RA, Mcdonald R, Wickramasinghe N. Linking Activity Theory Within User-Centered Design: Novel Framework to Inform Design and Evaluation of Adverse Drug Reaction Reporting Systems in Pharmacy. JMIR Hum Factors 2023; 10:e43529. [PMID: 36826985 PMCID: PMC10007010 DOI: 10.2196/43529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 01/02/2023] [Accepted: 01/24/2023] [Indexed: 01/26/2023] Open
Abstract
BACKGROUND Adverse drug reactions (ADRs) may cause serious injuries including death. Timely reporting of ADRs may play a significant role in patient safety; however, underreporting exists. Enhancing the electronic communication of ADR information to regulators and between health care providers has the potential to reduce recurrent ADRs and improve patient safety. OBJECTIVE The main objectives were to explore the low rate of ADR reporting by community pharmacists (CPs) in Australia, evaluate the usability of an existing reporting system, and how this knowledge may influence the design of subsequent electronic ADR reporting systems. METHODS The study was carried out in 2 stages. Stage 1 involved qualitative semistructured interviews to identify CPs' perceived barriers and facilitators to ADR reporting. Data were analyzed by thematic analysis, and identified themes were subsequently aligned to the task-technology fit (TTF) framework. The second stage involved a usability evaluation of a commercial web-based ADR reporting system. A structured interview protocol that combined virtual observation, think-aloud moderating techniques, retrospective questioning of the overall user experience, and a System Usability Scale (SUS). The field notes from the interviews were subjected to thematic analysis. RESULTS In total, 12 CPs were interviewed in stage 1, and 7 CPs participated in stage 2. The interview findings show that CPs are willing to report ADRs but face barriers from environmental, organizational, and IT infrastructures. Increasing ADR awareness, improving workplace practices, and implementing user-focused electronic reporting systems were seen as facilitators of ADR reporting. User testing of an existing system resulted in above average usability (SUS 68.57); however, functional and user interpretation issues were identified. Design elements such as a drop-down menu, free-text entry, checkbox, and prefilled data fields were perceived to be extremely useful for navigating the system and facilitating ADR reporting. CONCLUSIONS Existing reporting systems are not suited to report ADRs, or adapted to workflow, and are rarely used by CPs. Our study uncovered important contextual information for the design of future ADR reporting interventions. Based on our study, a multifaceted, theory-guided, user-centered, and best practice approach to design, implementation, and evaluation may be critical for the successful adoption of ADR reporting electronic interventions and patient safety. Future studies are needed to evaluate the effectiveness of theory-driven frameworks used in the design and implementation of ADR reporting systems.
Collapse
Affiliation(s)
- Joel Fossouo Tagne
- School of Health Sciences and Biostatistics, Swinburne University of Technology, Melbourne, Australia.,Centre for Health Analytics, Murdoch Children's Research Institute, Melbourne, Australia.,MedTechVic, Swinburne University of Technology, Melbourne, Australia
| | | | - Rachael Mcdonald
- MedTechVic, Swinburne University of Technology, Melbourne, Australia.,Department of Nursing and Allied Health, Occupational Therapy, Swinburne University of Technology, Melbourne, Australia.,Iverson Health Innovation Research Institute, Swinburne University of Technology, Melbourne, Australia
| | - Nilmini Wickramasinghe
- School of Health Sciences and Biostatistics, Swinburne University of Technology, Melbourne, Australia.,Iverson Health Innovation Research Institute, Swinburne University of Technology, Melbourne, Australia.,Epworth Healthcare, Melbourne, Australia
| |
Collapse
|
7
|
Almasi S, Bahaadinbeigy K, Ahmadi H, Sohrabei S, Rabiei R. Usability Evaluation of Dashboards: A Systematic Literature Review of Tools. BIOMED RESEARCH INTERNATIONAL 2023; 2023:9990933. [PMID: 36874923 PMCID: PMC9977530 DOI: 10.1155/2023/9990933] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/01/2023] [Revised: 01/16/2023] [Accepted: 02/04/2023] [Indexed: 02/25/2023]
Abstract
Introduction In recent years, the use of dashboards in healthcare has been considered an effective approach for the visual presentation of information to support clinical and administrative decisions. Effective and efficient use of dashboards in clinical and managerial processes requires a framework for the design and development of tools based on usability principles. Objectives The present study is aimed at investigating the existing questionnaires used for the usability evaluation framework of dashboards and at presenting more specific usability criteria for evaluating dashboards. Methods This systematic review was conducted using PubMed, Web of Science, and Scopus, without any time restrictions. The final search of articles was performed on September 2, 2022. Data collection was performed using a data extraction form, and the content of selected studies was analyzed based on the dashboard usability criteria. Results After reviewing the full text of relevant articles, a total of 29 studies were selected according to the inclusion criteria. Regarding the questionnaires used in the selected studies, researcher-made questionnaires were used in five studies, while 25 studies applied previously used questionnaires. The most widely used questionnaires were the System Usability Scale (SUS), Technology Acceptance Model (TAM), Situation Awareness Rating Technique (SART), Questionnaire for User Interaction Satisfaction (QUIS), Unified Theory of Acceptance and Use of Technology (UTAUT), and Health Information Technology Usability Evaluation Scale (Health-ITUES), respectively. Finally, dashboard evaluation criteria, including usefulness, operability, learnability, ease of use, suitability for tasks, improvement of situational awareness, satisfaction, user interface, content, and system capabilities, were suggested. Conclusion General questionnaires that were not specifically designed for dashboard evaluation were mainly used in reviewed studies. The current study suggested specific criteria for measuring the usability of dashboards. When selecting the usability evaluation criteria for dashboards, it is important to pay attention to the evaluation objectives, dashboard features and capabilities, and context of use.
Collapse
Affiliation(s)
- Sohrab Almasi
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Kambiz Bahaadinbeigy
- Digital Health Team, Australian College of Rural and Remote Medicine, Brisbane, QLD, Australia
- Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
| | - Hossein Ahmadi
- Centre for Health Technology, Faculty of Health, University of Plymouth, Plymouth PL4 8AA, UK
| | - Solmaz Sohrabei
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Reza Rabiei
- Department of Health Information Technology and Management, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| |
Collapse
|
8
|
Feldman J, Goodman A, Hochman K, Chakravartty E, Austrian J, Iturrate E, Bosworth B, Saxena A, Moussa MM, Chenouda DM, Volpicelli F, Adler N, Weisstuch J, Testa P. Novel Note Templates to Enhance Signal and Reduce Noise in Medical Documentation: a Prospective Improvement Study. JMIR Form Res 2023; 7:e41223. [PMID: 36821760 PMCID: PMC10134024 DOI: 10.2196/41223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 01/23/2023] [Accepted: 02/15/2023] [Indexed: 02/17/2023] Open
Abstract
BACKGROUND The introduction of electronic workflows has allowed for the flow of raw un-contextualized clinical data into medical documentation. As a result, many electronic notes have become replete of "noise" and deplete of clinically significant "signals". There is an urgent need to develop and implement innovative approaches in electronic clinical documentation that improve note quality and reduce unnecessary bloating. OBJECTIVE To describe the development and impact of a novel set of templates designed to change the flow of information in medical documentation. METHODS This is a multi-hospital nonrandomized prospective improvement study conducted on the Inpatient General Internal Medicine Service across three hospital campuses at the New York University (NYU) Langone Health System. A group of physician leaders representing each campus met biweekly for six months. The output of these meetings included 1) a conceptualization of the note bloat problem as a dysfunction in information flow 2) a set of guiding principles for organizational documentation improvement 3) the design and build of novel electronic templates that reduced the flow of extraneous information into provider notes by providing link outs to best practice data visualizations and 4) a documentation improvement curriculum for inpatient medicine providers. Prior to go-live, pragmatic usability testing was performed with the new progress note template, and the overall user experience measured using the System Usability Scale (SUS). Primary outcomes measures after go-live include template utilization rate and note length in characters. RESULTS In usability testing amongst 22 medicine providers, the new progress note template averaged a usability score of 90.6/100 on the System Usability Scale. 77% of providers strongly agreed that the new template was easy to use. 68% strongly agreed that they would like to use the template frequently. In the three months after template implementation, General Internal Medicine providers wrote 65% of all inpatient notes with the new templates. During this period of time the organization saw a 46%, 47%, and 32% reduction in note length for general medicine progress notes, consults, and H&Ps, respectively, when compared to a baseline measurement period prior to interventions. CONCLUSIONS A bundled intervention that included deployment of novel templates for inpatient general medicine providers significantly reduced average note length on the clinical service. Templates designed to reduce the flow of extraneous information into provider notes performed well during usability testing, and these templates were rapidly adopted across all hospital campuses. Further research is needed to assess the impact of novel templates on note quality, provider efficiency and patient outcomes. CLINICALTRIAL
Collapse
Affiliation(s)
- Jonah Feldman
- Medical Center Information Technology, NYU Langone Health, New York, US.,Department of Medicine, NYU Long Island School of Medicine, Mineola, US
| | - Adam Goodman
- Division of Gastroenterology & Hepatology, NYU Grossman School of Medicine, New York,, US
| | - Katherine Hochman
- Department of Medicine, New York University Langone Health, 550 1st avenue, New York, US
| | - Eesha Chakravartty
- Department of Medicine, New York University Langone Health, 550 1st avenue, New York, US.,Medical Center Information Technology, NYU Langone Health, New York, US
| | - Jonathan Austrian
- Medical Center Information Technology, NYU Langone Health, New York, US.,Department of Medicine, New York University Langone Health, 550 1st avenue, New York, US
| | - Eduardo Iturrate
- Medical Center Information Technology, NYU Langone Health, New York, US.,Department of Medicine, New York University Langone Health, 550 1st avenue, New York, US
| | - Brian Bosworth
- Department of Medicine, New York University Langone Health, 550 1st avenue, New York, US
| | - Archana Saxena
- Department of Medicine, New York University Langone Health, 550 1st avenue, New York, US
| | - Marwa M Moussa
- Department of Medicine, New York University Langone Health, 550 1st avenue, New York, US
| | - Dina M Chenouda
- Department of Medicine, NYU Long Island School of Medicine, Mineola, US
| | | | - Nicole Adler
- Department of Medicine, New York University Langone Health, 550 1st avenue, New York, US
| | | | - Paul Testa
- Medical Center Information Technology, NYU Langone Health, New York, US
| |
Collapse
|
9
|
Zhang Y, Chen Z, Tian F. Potentially inappropriate medications in older Chinese outpatients based on the Beers criteria and Chinese criteria. Front Pharmacol 2022; 13:991087. [PMID: 36249753 PMCID: PMC9561887 DOI: 10.3389/fphar.2022.991087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Accepted: 09/15/2022] [Indexed: 11/13/2022] Open
Abstract
Objective: Polypharmacy increases the prevalence of potentially inappropriate drugs potentially inappropriate medications among older persons, lowering their quality of life. PIMs use can lead to higher mortality in older patients. This study aimed to compare the prevalence of PIMs in older Chinese outpatients according to the Beers criteria and the Chinese criteria and to analyze the risk factors. Second, we describe the differences between the two criteria, focusing on the inappropriate prescription of drugs in older outpatients.Methods: In Chengdu, Southwest China, a cross-sectional study was undertaken using electronic medical data from 9 general hospitals s. Outpatients above the age of 60 who were treated in the Geriatrics Center of these medical institutions were included. The 2019 Beers criteria and the 2017 Chinese criteria were used to evaluate the PIM status of older outpatients, and binary logistic regression was used to identify potential risk factors for PIMs.Results: There were 44,458 prescriptions from 2016 to 2018. The prevalence of PIMs among older outpatients was 30.05% (according to the Beers criteria) and 35.38% (according to the Chinese criteria), with statistical difference. Estazolam, hydrochlorothiazide and alprazolam were the top three PIMs in the Beers criteria, while the top three PIMs in the Chinese criteria were clopidogrel, estazolam and insulin. The prevalence of PIMs was associated with age, the number of diseases and the number of drugs. PIMs were shown to be more common in patients aged 70 and above, with more than 2 kinds of diseases and more than 4 kinds of drugs.Conclusion: PIMs were shown to be common among older outpatients in China, according to this study. The detection rate of the Chinese criteria was higher than that of the Beers criteria.
Collapse
|
10
|
Burningham Z, Lagha RR, Duford-Hutchinson B, Callaway-Lane C, Sauer BC, Halwani AS, Bell J, Huynh T, Douglas JR, Kramer BJ. Developing the VA Geriatric Scholars Programs' Clinical Dashboards Using the PDSA Framework for Quality Improvement. Appl Clin Inform 2022; 13:961-970. [PMID: 36223868 PMCID: PMC9556171 DOI: 10.1055/s-0042-1757553] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/03/2022] Open
Abstract
Background
Involving clinician end users in the development process of clinical dashboards is important to ensure that user needs are adequately met prior to releasing the dashboard for use. The challenge with following this approach is that clinician end users can undergo periodic turnover, meaning, the clinicians that played a role in the initial development process may not be the same individuals that use the dashboard in future.
Objectives
Here, we summarize our Plan, Do, Study, Act (PDSA)-guided clinical dashboard development process for the VA Geriatric Scholars Program (GSP) and the value of continuous, iterative development. We summarize dashboard adaptations that resulted from two PDSA cycles of improvement for the potentially inappropriate medication dashboard (PIMD), one of many Geriatric Scholars clinical dashboards. We also present the evaluative performance of the PIMD.
Methods
Evaluation of the PIMD was performed using the system usability scale (SUS) and through review of user interaction logs. Routine end users that were Geriatric Scholars and had evidence of 5 or more dashboard views were invited to complete an electronic form that contained the 10-item SUS.
Results
The proportion of Geriatric Scholars that utilized the PIMD increased for each iterative dashboard version that was produced as a byproduct from feedback (31.0% in 2017 to 60.2% in 2019). The overall usability of the PIMD among routine users was found to be above average (SUS score: 75.2 [95% CI 70.5–79.8]) in comparison to the recommended standard of acceptability (SUS score: 68)
Conclusion
The solicitation of feedback during dashboard orientations led to iterative adaptations of the PIMD that broadened its intended use. The presented PDSA-guided process to clinical dashboard development for the VA GSP can serve as a valuable framework for development teams seeking to produce well-adopted and usable health information technology (IT) innovations.
Collapse
Affiliation(s)
- Zachary Burningham
- Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States.,Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
| | - Regina Richter Lagha
- Geriatric Research Education and Clinical Center (GRECC), Veterans Affairs Greater Los Angeles Medical Center, Los Angeles, California, United States
| | - Brittany Duford-Hutchinson
- Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
| | - Carol Callaway-Lane
- Geriatric Research Education and Clinical Center (GRECC), Veterans Affairs Tennessee Valley Health Care System, Murfreesboro, Tennessee, United States
| | - Brian C Sauer
- Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States.,Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
| | - Ahmad S Halwani
- Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States.,Department of Hematology, University of Utah, Salt Lake City, Utah, United States
| | - Jamie Bell
- Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States.,Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
| | - Tina Huynh
- Division of Epidemiology, Department of Internal Medicine, University of Utah, Salt Lake City, Utah, United States.,Informatics, Decision-Enhancement and Analytic Sciences (IDEAS) Centers of Innovation (COIN), Salt Lake City Veterans Affairs Medical Center, Salt Lake City, Utah, United States
| | - Joseph R Douglas
- Geriatric Research Education and Clinical Center (GRECC), Veterans Affairs Greater Los Angeles Medical Center, Los Angeles, California, United States
| | - B Josea Kramer
- Geriatric Research Education and Clinical Center (GRECC), Veterans Affairs Greater Los Angeles Medical Center, Los Angeles, California, United States.,Division of Geriatric Medicine, David Geffen School of Medicine, University of California at Los Angeles, Los Angeles, United States
| |
Collapse
|
11
|
Makic MBF, Stevens KR, Gritz RM, Wald H, Ouellet J, Morrow CD, Rodrick D, Reeder B. Dashboard Design to Identify and Balance Competing Risk of Multiple Hospital-Acquired Conditions. Appl Clin Inform 2022; 13:621-631. [PMID: 35675838 DOI: 10.1055/s-0042-1749598] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022] Open
Abstract
BACKGROUND Hospital-acquired conditions (HACs) are common, costly, and national patient safety priority. Catheter-associated urinary tract infections (CAUTIs), hospital-acquired pressure injury (HAPI), and falls are common HACs. Clinicians assess each HAC risk independent of other conditions. Prevention strategies often focus on the reduction of a single HAC rather than considering how actions to prevent one condition could have unintended consequences for another HAC. OBJECTIVES The objective of this study is to design an empirical framework to identify, assess, and quantify the risks of multiple HACs (MHACs) related to competing single-HAC interventions. METHODS This study was an Institutional Review Board approved, and the proof of concept study evaluated MHAC Competing Risk Dashboard to enhance clinicians' management combining the risks of CAUTI, HAPI, and falls. The empirical model informing this study focused on the removal of an indwelling urinary catheter to reduce CAUTI, which may impact HAPI and falls. A multisite database was developed to understand and quantify competing risks of HACs; a predictive model dashboard was designed and clinical utility of a high-fidelity dashboard was qualitatively tested. Five hospital systems provided data for the predictive model prototype; three served as sites for testing and feedback on the dashboard design and usefulness. The participatory study design involved think-aloud methods as the clinician explored the dashboard. Individual interviews provided an understanding of clinician's perspective regarding ease of use and utility. RESULTS Twenty-five clinicians were interviewed. Clinicians favored a dashboard gauge design composed of green, yellow, and red segments to depict MHAC risk associated with the removal of an indwelling urinary catheter to reduce CAUTI and possible adverse effects on HAPI and falls. CONCLUSION Participants endorsed the utility of a visual dashboard guiding clinical decisions for MHAC risks preferring common stoplight color understanding. Clinicians did not want mandatory alerts for tool integration into the electronic health record. More research is needed to understand MHAC and tools to guide clinician decisions.
Collapse
Affiliation(s)
| | - Kathleen R Stevens
- School of Nursing, University of Texas Health Science Center San Antonio, San Antonio, Texas, United States
| | - R Mark Gritz
- Division of Health Care Policy and Research, School of Medicine, University of Colorado Denver, Aurora, Colorado, United States
| | - Heidi Wald
- SCL Health, Denver, Colorado, United States
| | - Judith Ouellet
- Division of Health Care Policy and Research, School of Medicine, University of Colorado Denver, Aurora, Colorado, United States
| | - Cynthia Drake Morrow
- Health Systems, Management and Policy, Colorado School of Public Health, Aurora, Colorado, United States
| | - David Rodrick
- Center for Quality Improvement and Patient Safety, Agency for Healthcare Research and Quality, Rockville, Maryland, United States
| | - Blaine Reeder
- University of Missouri Health, Sinclair School of Nursing and MU Institute for Data Science and Informatics, School of Nursing, Columbia, Missouri, United States
| |
Collapse
|
12
|
Sinabell I, Ammenwerth E. Agile, Easily Applicable, and Useful eHealth Usability Evaluations: Systematic Review and Expert-Validation. Appl Clin Inform 2022; 13:67-79. [PMID: 35263798 PMCID: PMC8906994 DOI: 10.1055/s-0041-1740919] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022] Open
Abstract
Background
Electronic health (eHealth) usability evaluations of rapidly developed eHealth systems are difficult to accomplish because traditional usability evaluation methods require substantial time in preparation and implementation. This illustrates the growing need for fast, flexible, and cost-effective methods to evaluate the usability of eHealth systems. To address this demand, the present study systematically identified and expert-validated rapidly deployable eHealth usability evaluation methods.
Objective
Identification and prioritization of eHealth usability evaluation methods suitable for agile, easily applicable, and useful eHealth usability evaluations.
Methods
The study design comprised a systematic iterative approach in which expert knowledge was contrasted with findings from literature. Forty-three eHealth usability evaluation methods were systematically identified and assessed regarding their ease of applicability and usefulness through semi-structured expert interviews with 10 European usability experts and systematic literature research. The most appropriate eHealth usability evaluation methods were selected stepwise based on the experts' judgements of their ease of applicability and usefulness.
Results
Of these 43 eHealth usability evaluation methods identified as suitable for agile, easily applicable, and useful eHealth usability evaluations, 10 were recommended by the experts based on their usefulness for rapid eHealth usability evaluations. The three most frequently recommended eHealth usability evaluation methods were Remote User Testing, Expert Review, and Rapid Iterative Test and Evaluation Method. Eleven usability evaluation methods, such as Retrospective Testing, were not recommended for use in rapid eHealth usability evaluations.
Conclusion
We conducted a systematic review and expert-validation to identify rapidly deployable eHealth usability evaluation methods. The comprehensive and evidence-based prioritization of eHealth usability evaluation methods supports faster usability evaluations, and so contributes to the ease-of-use of emerging eHealth systems.
Collapse
Affiliation(s)
- Irina Sinabell
- Department of Biomedical Computer Science and Mechatronics, Institute of Medical Informatics, UMIT, Private University of Health Sciences, Medical Informatics and Technology, Hall in Tirol, Austria
| | - Elske Ammenwerth
- Department of Biomedical Computer Science and Mechatronics, Institute of Medical Informatics, UMIT, Private University of Health Sciences, Medical Informatics and Technology, Hall in Tirol, Austria
| |
Collapse
|
13
|
Jonnalagadda P, Swoboda C, Singh P, Gureddygari H, Scarborough S, Dunn I, Doogan NJ, Fareed N. Developing Dashboards to Address Children's Health Disparities in Ohio. Appl Clin Inform 2022; 13:100-112. [PMID: 35081656 PMCID: PMC8791762 DOI: 10.1055/s-0041-1741482] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2021] [Accepted: 11/27/2021] [Indexed: 01/28/2023] Open
Abstract
OBJECTIVES Social determinants of health (SDoH) can be measured at the geographic level to convey information about neighborhood deprivation. The Ohio Children's Opportunity Index (OCOI) is a composite area-level opportunity index comprised of eight health domains. Our research team has documented the design, development, and use cases of a dashboard solution to visualize OCOI. METHODS The OCOI is a multidomain index spanning the following eight domains: (1) family stability, (2) infant health, (3) children's health, (4) access, (5) education, (6) housing, (7) environment, and (8) criminal justice. Information on these eight domains is derived from the American Community Survey and other administrative datasets. Our team used the Tableau Desktop visualization software and applied a user-centered design approach to developing the two OCOI dashboards-main OCOI dashboard and OCOI-race dashboard. We also performed convergence analysis to visualize the census tracts where different health indicators simultaneously exist at their worst levels. RESULTS The OCOI dashboard has multiple, interactive components as follows: a choropleth map of Ohio displaying OCOI scores for a specific census tract, graphs presenting OCOI or domain scores to compare relative positions for tracts, and a sortable table to visualize scores for specific county and census tracts. A case study using the two dashboards for convergence analysis revealed census tracts in neighborhoods with low infant health scores and a high proportion of minority population. CONCLUSION The OCOI dashboards could assist health care leaders in making decisions that enhance health care delivery and policy decision-making regarding children's health particularly in areas where multiple health indicators exist at their worst levels.
Collapse
Affiliation(s)
- Pallavi Jonnalagadda
- CATALYST, Center for the Advancement of Team Science, Analytics, and Systems Thinking in Health Services and Implementation Science Research, College of Medicine, The Ohio State University, Columbus, Ohio, United States
| | - Christine Swoboda
- CATALYST, Center for the Advancement of Team Science, Analytics, and Systems Thinking in Health Services and Implementation Science Research, College of Medicine, The Ohio State University, Columbus, Ohio, United States
| | - Priti Singh
- CATALYST, Center for the Advancement of Team Science, Analytics, and Systems Thinking in Health Services and Implementation Science Research, College of Medicine, The Ohio State University, Columbus, Ohio, United States
| | - Harish Gureddygari
- CATALYST, Center for the Advancement of Team Science, Analytics, and Systems Thinking in Health Services and Implementation Science Research, College of Medicine, The Ohio State University, Columbus, Ohio, United States
| | - Seth Scarborough
- CATALYST, Center for the Advancement of Team Science, Analytics, and Systems Thinking in Health Services and Implementation Science Research, College of Medicine, The Ohio State University, Columbus, Ohio, United States
| | - Ian Dunn
- The Ohio Colleges of Medicine Government Resource Center, Columbus, Ohio, United States
| | - Nathan J. Doogan
- The Ohio Colleges of Medicine Government Resource Center, Columbus, Ohio, United States
| | - Naleef Fareed
- CATALYST, Center for the Advancement of Team Science, Analytics, and Systems Thinking in Health Services and Implementation Science Research, College of Medicine, The Ohio State University, Columbus, Ohio, United States
- Department of Biomedical Informatics, College of Medicine, The Ohio State University, Columbus, Ohio, United States
| |
Collapse
|
14
|
Alvarado N, McVey L, Elshehaly M, Greenhalgh J, Dowding D, Ruddle R, Gale CP, Mamas M, Doherty P, West R, Feltbower R, Randell R. Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation. J Med Internet Res 2021; 23:e28854. [PMID: 34817384 PMCID: PMC8663683 DOI: 10.2196/28854] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2021] [Revised: 07/15/2021] [Accepted: 10/05/2021] [Indexed: 11/30/2022] Open
Abstract
BACKGROUND Dashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation. OBJECTIVE This paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement. METHODS QualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard's perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement. RESULTS The observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice. CONCLUSIONS Web-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impact in different contexts; this includes using data meaningful to stakeholders in the co-design process and actively engaging staff knowledgeable about current data use and routines in the scrutiny of the dashboard metrics and functions. In addition, consideration should be given to the processes of data collection and upload that underpin the quality of the data visualized and consequently its potential to stimulate quality improvement. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) RR2-10.1136/bmjopen-2019-033208.
Collapse
Affiliation(s)
- Natasha Alvarado
- Faculty of Health Studies, University of Bradford, Bradford, United Kingdom
- Wolfson Centre for Applied Health Research, Bradford, United Kingdom
| | - Lynn McVey
- Faculty of Health Studies, University of Bradford, Bradford, United Kingdom
- Wolfson Centre for Applied Health Research, Bradford, United Kingdom
| | - Mai Elshehaly
- Wolfson Centre for Applied Health Research, Bradford, United Kingdom
- Faculty of Engineering and Informatics, University of Bradford, Bradford, United Kingdom
| | - Joanne Greenhalgh
- School of Sociology and Social Policy, University of Leeds, Leeds, United Kingdom
| | - Dawn Dowding
- School of Health Sciences, University of Manchester, Manchester, United Kingdom
| | - Roy Ruddle
- School of Computing, University of Leeds, Leeds, United Kingdom
- Leeds Institute for Data Analytics, Leeds, United Kingdom
| | - Chris P Gale
- Leeds Institute for Data Analytics, Leeds, United Kingdom
- Leeds Institute of Cardiovascular and Metabolic Medicine, University of Leeds, Leeds, United Kingdom
- Department of Cardiology, Leeds Teaching Hospitals NHS Trust, Leeds, United Kingdom
| | - Mamas Mamas
- Keele Cardiovascular Group, School of Medicine, Keele University, Keele, United Kingdom
| | - Patrick Doherty
- Department of Health Sciences, University of York, York, United Kingdom
| | - Robert West
- Leeds Institute of Health Sciences, University of Leeds, Leeds, United Kingdom
| | - Richard Feltbower
- Leeds Institute for Data Analytics, Leeds, United Kingdom
- School of Medicine, University of Leeds, Leeds, United Kingdom
| | - Rebecca Randell
- Faculty of Health Studies, University of Bradford, Bradford, United Kingdom
- Wolfson Centre for Applied Health Research, Bradford, United Kingdom
| |
Collapse
|
15
|
González Aguña A, Fernández Batalla M, Arias Baelo C, Monsalvo San Macario E, Gonzalo de Diego B, Santamaría García JM. Usability Evaluation by Nurses of a Knowledge-Based System of Care Plans for People Hospitalized by COVID-19. Comput Inform Nurs 2021; 40:186-200. [PMID: 34570005 DOI: 10.1097/cin.0000000000000830] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
The aim of this study is to analyze the usability by nurses of the Knowledge-Based System "Diagnostics care for COVID-19." A convenience sample of 16 nurses was selected, among hospital workers and external experts. The group was divided into three subgroups intentionally to obtain different usability perspectives. Usability was evaluated by the System Usability Scale questionnaire. The participants completed the questionnaire on general usability, data inputs, and information output, after completing a minimum of 12 care plans. The first subgroup used real cases and the "think aloud" technique, the second simulated cases from the same hospital, and the third subgroup performed the external simulation. The highest scores were obtained in data inputs (94.38-97.50); and the lowest, in general usability (90.00-95.00). The subgroup of external experts scored the highest (93.13-95.63), and the first subgroup, which carried out real cases, gave the lowest score (90.00-94.38). The "think aloud" technique found an improvement in including more diagnoses and being able to carry out several plans for one person at the same time. The usability obtained was "excellent" in all subgroups and questionnaires, although the application showed limitations related to its characteristics imposed in the requirements specification.
Collapse
Affiliation(s)
- Alexandra González Aguña
- Author Affiliations: Henares University Hospital (Ms González Aguña) and Torres de la Alameda Health Center (Dr Fernández Batalla), Community of Madrid Health Service, Research Group MISKC, University of Alcalá; Quality Management Unit, Gregorio Marañón Hospital, Community of Madrid Health Service (Ms Arias Baelo); and La Garena Health Center (Mr Monsalvo San Macario) and Meco Health Center (Ms Gonzalo de Diego and Dr Santamaría García), Community of Madrid Health Service, Research Group MISKC, University of Alcalá, Spain
| | | | | | | | | | | |
Collapse
|
16
|
Kernebeck S, Busse TS, Jux C, Meyer D, Dreier LA, Zenz D, Zernikow B, Ehlers JP. Participatory Design of an Electronic Medical Record for Paediatric Palliative Care: A Think-Aloud Study with Nurses and Physicians. CHILDREN-BASEL 2021; 8:children8080695. [PMID: 34438586 PMCID: PMC8392291 DOI: 10.3390/children8080695] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/15/2021] [Revised: 08/09/2021] [Accepted: 08/09/2021] [Indexed: 11/30/2022]
Abstract
Background: Electronic medical records (EMRs) offer a promising approach to mapping and documenting the complex information gathered in paediatric palliative care (PPC). However, if they are not well developed, poorly implemented EMRs have unintended consequences that may cause harm to patients. One approach to preventing such harm is the involvement of users in the participatory design to ensure user acceptance and patient safety. Therefore, the aim of this study is to evaluate the acceptance of a novel patient chart module (PCM) as part of an EMR from the perspective of potential users in PPC and to involve these professionals in the design process. Methods: A qualitative observational study with N = 16 PPC professionals (n = 10 nurses, n = 6 physicians) was conducted, including concurrent think aloud (CTA) and semi-structured interviews. A structured content analysis based on the Unified Theory of Acceptance and Use of Technology was applied. Results: The results can be summarized in terms of general observations, performance expectancy, effort expectancy and facilitating conditions, all of which are likely to have a positive influence on acceptance of the PCM from the user perspective in the context of PPC. Conclusions: The involvement of users in the development of EMRs is important for meeting the requirements in PPC. Further software adaptations are necessary to implement these requirements.
Collapse
Affiliation(s)
- Sven Kernebeck
- Chair of Didactics and Educational Research in Health Science, Faculty of Health, Witten/Herdecke University, 58448 Witten, Germany; (T.S.B.); (C.J.); (J.P.E.)
- Correspondence: ; Tel.: +49-(0)2302/926-786-13
| | - Theresa Sophie Busse
- Chair of Didactics and Educational Research in Health Science, Faculty of Health, Witten/Herdecke University, 58448 Witten, Germany; (T.S.B.); (C.J.); (J.P.E.)
| | - Chantal Jux
- Chair of Didactics and Educational Research in Health Science, Faculty of Health, Witten/Herdecke University, 58448 Witten, Germany; (T.S.B.); (C.J.); (J.P.E.)
| | - Dorothee Meyer
- PedScience Research Institute, 45711 Datteln, Germany; (D.M.); (L.A.D.); (B.Z.)
- Department of Children’s Pain Therapy and Pediatric Palliative Care, Faculty of Health, School of Medicine, Witten/Herdecke University, 58448 Witten, Germany
| | - Larissa Alice Dreier
- PedScience Research Institute, 45711 Datteln, Germany; (D.M.); (L.A.D.); (B.Z.)
- Department of Children’s Pain Therapy and Pediatric Palliative Care, Faculty of Health, School of Medicine, Witten/Herdecke University, 58448 Witten, Germany
| | - Daniel Zenz
- Smart-Q Softwaresysteme GmbH, Lise-Meitner-Allee 4, 44801 Bochum, Germany;
| | - Boris Zernikow
- PedScience Research Institute, 45711 Datteln, Germany; (D.M.); (L.A.D.); (B.Z.)
- Department of Children’s Pain Therapy and Pediatric Palliative Care, Faculty of Health, School of Medicine, Witten/Herdecke University, 58448 Witten, Germany
- Pediatric Palliative Care Centre, Children’s and Adolescents’ Hospital, 45711 Datteln, Germany
| | - Jan Peter Ehlers
- Chair of Didactics and Educational Research in Health Science, Faculty of Health, Witten/Herdecke University, 58448 Witten, Germany; (T.S.B.); (C.J.); (J.P.E.)
| |
Collapse
|
17
|
Abstract
OBJECTIVE Human factors and ergonomics (HF/E) frameworks and methods are becoming embedded in the health informatics community. There is now broad recognition that health informatics tools must account for the diverse needs, characteristics, and abilities of end users, as well as their context of use. The objective of this review is to synthesize the current nature and scope of HF/E integration into the health informatics community. METHODS Because the focus of this synthesis is on understanding the current integration of the HF/E and health informatics research communities, we manually reviewed all manuscripts published in primary HF/E and health informatics journals during 2020. RESULTS HF/E-focused health informatics studies included in this synthesis focused heavily on EHR customizations, specifically clinical decision support customizations and customized data displays, and on mobile health innovations. While HF/E methods aimed to jointly improve end user safety, performance, and satisfaction, most HF/E-focused health informatics studies measured only end user satisfaction. CONCLUSION HF/E-focused health informatics researchers need to identify and communicate methodological standards specific to health informatics, to better synthesize findings across resource intensive HF/E-focused health informatics studies. Important gaps in the HF/E design and evaluation process should be addressed in future work, including support for technology development platforms and training programs so that health informatics designers are as diverse as end users.
Collapse
|