Qualitative evaluation of a diabetes electronic decision support tool: views of users

Assessing the Utility, Impact, and Adoption Challenges of an Artificial Intelligence-Enabled Prescription Advisory Tool for Type 2 Diabetes Management: Qualitative Study

BACKGROUND

The clinical management of type 2 diabetes mellitus (T2DM) presents a significant challenge due to the constantly evolving clinical practice guidelines and growing array of drug classes available. Evidence suggests that artificial intelligence (AI)-enabled clinical decision support systems (CDSSs) have proven to be effective in assisting clinicians with informed decision-making. Despite the merits of AI-driven CDSSs, a significant research gap exists concerning the early-stage implementation and adoption of AI-enabled CDSSs in T2DM management.

OBJECTIVE

This study aimed to explore the perspectives of clinicians on the use and impact of the AI-enabled Prescription Advisory (APA) tool, developed using a multi-institution diabetes registry and implemented in specialist endocrinology clinics, and the challenges to its adoption and application.

METHODS

We conducted focus group discussions using a semistructured interview guide with purposively selected endocrinologists from a tertiary hospital. The focus group discussions were audio-recorded and transcribed verbatim. Data were thematically analyzed.

RESULTS

A total of 13 clinicians participated in 4 focus group discussions. Our findings suggest that the APA tool offered several useful features to assist clinicians in effectively managing T2DM. Specifically, clinicians viewed the AI-generated medication alterations as a good knowledge resource in supporting the clinician's decision-making on drug modifications at the point of care, particularly for patients with comorbidities. The complication risk prediction was seen as positively impacting patient care by facilitating early doctor-patient communication and initiating prompt clinical responses. However, the interpretability of the risk scores, concerns about overreliance and automation bias, and issues surrounding accountability and liability hindered the adoption of the APA tool in clinical practice.

CONCLUSIONS

Although the APA tool holds great potential as a valuable resource for improving patient care, further efforts are required to address clinicians' concerns and improve the tool's acceptance and applicability in relevant contexts.

Workload and workflow implications associated with the use of electronic clinical decision support tools used by health professionals in general practice: a scoping review

BACKGROUND

Electronic clinical decision support tools (eCDS) are increasingly available to assist General Practitioners (GP) with the diagnosis and management of a range of health conditions. It is unclear whether the use of eCDS tools has an impact on GP workload. This scoping review aimed to identify the available evidence on the use of eCDS tools by health professionals in general practice in relation to their impact on workload and workflow.

METHODS

A scoping review was carried out using the Arksey and O'Malley methodological framework. The search strategy was developed iteratively, with three main aspects: general practice/primary care contexts, risk assessment/decision support tools, and workload-related factors. Three databases were searched in 2019, and updated in 2021, covering articles published since 2009: Medline (Ovid), HMIC (Ovid) and Web of Science (TR). Double screening was completed by two reviewers, and data extracted from included articles were analysed.

RESULTS

The search resulted in 5,594 references, leading to 95 full articles, referring to 87 studies, after screening. Of these, 36 studies were based in the USA, 21 in the UK and 11 in Australia. A further 18 originated from Canada or Europe, with the remaining studies conducted in New Zealand, South Africa and Malaysia. Studies examined the use of eCDS tools and reported some findings related to their impact on workload, including on consultation duration. Most studies were qualitative and exploratory in nature, reporting health professionals' subjective perceptions of consultation duration as opposed to objectively-measured time spent using tools or consultation durations. Other workload-related findings included impacts on cognitive workload, "workflow" and dialogue with patients, and clinicians' experience of "alert fatigue".

CONCLUSIONS

The published literature on the impact of eCDS tools in general practice showed that limited efforts have focused on investigating the impact of such tools on workload and workflow. To gain an understanding of this area, further research, including quantitative measurement of consultation durations, would be useful to inform the future design and implementation of eCDS tools.

Barriers and Facilitators to the Use of Clinical Decision Support Systems in Primary Care: A Mixed-Methods Systematic Review

PURPOSE

To identify and quantify the barriers and facilitators to the use of clinical decision support systems (CDSSs) by primary care professionals (PCPs).

METHODS

A mixed-methods systematic review was conducted using a sequential synthesis design. PubMed/MEDLINE, PsycInfo, Embase, CINAHL, and the Cochrane library were searched in July 2021. Studies that evaluated CDSSs providing recommendations to PCPs and intended for use during a consultation were included. We excluded CDSSs used only by patients, described as concepts or prototypes, used with simulated cases, and decision supports not considered as CDSSs. A framework synthesis was performed according to the HOT-fit framework (Human, Organizational, Technology, Net Benefits), then a quantitative synthesis evaluated the impact of the HOT-fit categories on CDSS use.

RESULTS

A total of 48 studies evaluating 45 CDSSs were included, and 186 main barriers or facilitators were identified. Qualitatively, barriers and facilitators were classified as human (eg, perceived usefulness), organizational (eg, disruption of usual workflow), and technological (eg, CDSS user-friendliness), with explanatory elements. The greatest barrier to using CDSSs was an increased workload. Quantitatively, the human and organizational factors had negative impacts on CDSS use, whereas the technological factor had a neutral impact and the net benefits dimension a positive impact.

CONCLUSIONS

Our findings emphasize the need for CDSS developers to better address human and organizational issues, in addition to technological challenges. We inferred core CDSS features covering these 3 factors, expected to improve their usability in primary care.

Barriers and enablers to implementing and using clinical decision support systems for chronic diseases: a qualitative systematic review and meta-aggregation

BACKGROUND

Clinical decision support (CDS) is increasingly used to facilitate chronic disease care. Despite increased availability of electronic health records and the ongoing development of new CDS technologies, uptake of CDS into routine clinical settings is inconsistent. This qualitative systematic review seeks to synthesise healthcare provider experiences of CDS-exploring the barriers and enablers to implementing, using, evaluating, and sustaining chronic disease CDS systems.

METHODS

A search was conducted in Medline, CINAHL, APA PsychInfo, EconLit, and Web of Science from 2011 to 2021. Primary research studies incorporating qualitative findings were included if they targeted healthcare providers and studied a relevant chronic disease CDS intervention. Relevant CDS interventions were electronic health record-based and addressed one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolaemia. Qualitative findings were synthesised using a meta-aggregative approach.

RESULTS

Thirty-three primary research articles were included in this qualitative systematic review. Meta-aggregation of qualitative data revealed 177 findings and 29 categories, which were aggregated into 8 synthesised findings. The synthesised findings related to clinical context, user, external context, and technical factors affecting CDS uptake. Key barriers to uptake included CDS systems that were simplistic, had limited clinical applicability in multimorbidity, and integrated poorly into existing workflows. Enablers to successful CDS interventions included perceived usefulness in providing relevant clinical knowledge and structured chronic disease care; user confidence gained through training and post training follow-up; external contexts comprised of strong clinical champions, allocated personnel, and technical support; and CDS technical features that are both highly functional, and attractive.

CONCLUSION

This systematic review explored healthcare provider experiences, focussing on barriers and enablers to CDS use for chronic diseases. The results provide an evidence-base for designing, implementing, and sustaining future CDS systems. Based on the findings from this review, we highlight actionable steps for practice and future research.

TRIAL REGISTRATION

PROSPERO CRD42020203716.

Using an Integrated Framework to Investigate the Facilitators and Barriers of Health Information Technology Implementation in Noncommunicable Disease Management: Systematic Review

BACKGROUND

Noncommunicable disease (NCD) management is critical for reducing attributable health burdens. Although health information technology (HIT) is a crucial strategy to improve chronic disease management, many health care systems have failed in implementing HIT. There has been a lack of research on the implementation process of HIT for chronic disease management.

OBJECTIVE

We aimed to identify the barriers and facilitators of HIT implementation, analyze how these factors influence the implementation process, and identify key areas for future action. We will develop a framework for understanding implementation determinants to synthesize available evidence.

METHODS

We conducted a systematic review to understand the barriers and facilitators of the implementation process. We searched MEDLINE, Cochrane, Embase, Scopus, and CINAHL for studies published between database inception and May 5, 2022. Original studies involving HIT-related interventions for NCD management published in peer-reviewed journals were included. Studies that did not discuss relevant outcome measures or did not have direct contact with or observation of stakeholders were excluded. The analysis was conducted in 2 parts. In part 1, we analyzed how the intrinsic attributes of HIT interventions affect the successfulness of implementation by using the intervention domain of the Consolidated Framework for Implementation Research (CFIR). In part 2, we focused on the extrinsic factors of HIT using an integrated framework, which was developed based on the CFIR and the levels of change framework by Ferlie and Shortell.

RESULTS

We identified 51 papers with qualitative, mixed-method, and cross-sectional methodologies. Included studies were heterogeneous regarding disease populations and HIT interventions. In part 1, having a relative advantage over existing health care systems was the most prominent intrinsic facilitator (eg, convenience, improvement in quality of care, and increase in access). Poor usability was the most noted intrinsic barrier of HIT. In part 2, we mapped the various factors of implementation to the integrated framework (the coordinates are shown as level of change-CFIR). The key barriers to the extrinsic factors of HIT included health literacy and lack of digital skills (individual-characteristics of individuals). The key facilitators included physicians' suggestions, cooperation (interpersonal-process), integration into a workflow, and adequate management of data (organizational-inner setting). The importance of health data security was identified. Self-efficacy issues of patients and organizational readiness for implementation were highlighted.

CONCLUSIONS

Internal factors of HIT and external human factors of implementation interplay in HIT implementation for chronic disease management. Strategies for improvement include ensuring HIT has a relative advantage over existing health care; tackling usability issues; and addressing underlying socioeconomic, interpersonal, and organizational conditions. Further research should focus on studying various stakeholders, such as service providers and administrative workforces; various disease populations, such as those with obesity and mental diseases; and various countries, including low- and middle-income countries.

Integrating Option Grid Patient Decision Aids in the Epic Electronic Health Record: Case Study at 5 Health Systems

Background

Some researchers argue that the successful implementation of patient decision aids (PDAs) into clinical workflows depends on their integration into electronic health records (EHRs). Anecdotally, we know that EHR integration is a complex and time-consuming task; yet, the process has not been examined in detail. As part of an implementation project, we examined the work involved in integrating an encounter PDA for symptomatic uterine fibroids into Epic EHR systems.

Objective

This study aims to identify the steps and time required to integrate a PDA into the Epic EHR system and examine facilitators and barriers to the integration effort.

Methods

We conducted a case study at 5 academic medical centers in the United States. A clinical champion at each institution liaised with their Epic EHR team to initiate the integration of the uterine fibroid Option Grid PDAs into clinician-facing menus. We scheduled regular meetings with the Epic software analysts and an expert Epic technologist to discuss how best to integrate the tools into Epic for use by clinicians with patients. The meetings were then recorded and transcribed. Two researchers independently coded the transcripts and field notes before categorizing the codes and conducting a thematic analysis to identify the facilitators and barriers to EHR integration. The steps were reviewed and edited by an Epic technologist to ensure their accuracy.

Results

Integrating the uterine fibroid Option Grid PDA into clinician-facing menus required an 18-month timeline and a 6-step process, as follows: task priority negotiation with Epic software teams, security risk assessment, technical review, Epic configuration; troubleshooting, and launch. The key facilitators of the process were the clinical champions who advocated for integration at the institutional level and the presence of an experienced technologist who guided Epic software analysts during the build. Another facilitator was the use of an emerging industry standard app platform (Health Level 7 Substitutable Medical Applications and Reusable Technologies on Fast Healthcare Interoperability Resources) as a means of integrating the Option Grid into existing systems. This standard platform enabled clinicians to access the tools by using single sign-on credentials and prevented protected health information from leaving the EHR. Key barriers were the lack of control over the Option Grid product developed by EBSCO (Elton B Stephens Company) Health; the periodic Epic upgrades that can result in a pause on new software configurations; and the unforeseen software problems with Option Grid (ie, inability to print the PDA), which delayed the launch of the PDA.

Conclusions

The integration of PDAs into the Epic EHR system requires a 6-step process and an 18-month timeline. The process required support and prioritization from a clinical champion, guidance from an experienced technologist, and a willing EHR software developer team.

Mobile Clinical Decision Support System for the Management of Diabetic Patients With Kidney Complications in UK Primary Care Settings: Mixed Methods Feasibility Study

BACKGROUND

Attempts to utilize eHealth in diabetes mellitus (DM) management have shown promising outcomes, mostly targeted at patients; however, few solutions have been designed for health care providers.

OBJECTIVE

The purpose of this study was to conduct a feasibility project developing and evaluating a mobile clinical decision support system (CDSS) tool exclusively for health care providers to manage chronic kidney disease (CKD) in patients with DM.

METHODS

The design process was based on the 3 key stages of the user-centered design framework. First, an exploratory qualitative study collected the experiences and views of DM specialist nurses regarding the use of mobile apps in clinical practice. Second, a CDSS tool was developed for the management of patients with DM and CKD. Finally, a randomized controlled trial examined the acceptability and impact of the tool.

RESULTS

We interviewed 15 DM specialist nurses. DM specialist nurses were not currently using eHealth solutions in their clinical practice, while most nurses were not even aware of existing medical apps. However, they appreciated the potential benefits that apps may bring to their clinical practice. Taking into consideration the needs and preferences of end users, a new mobile CDSS app, "Diabetes & CKD," was developed based on guidelines. We recruited 39 junior foundation year 1 doctors (44% male) to evaluate the app. Of them, 44% (17/39) were allocated to the intervention group, and 56% (22/39) were allocated to the control group. There was no significant difference in scores (maximum score=13) assessing the management decisions between the app and paper-based version of the app's algorithm (intervention group: mean 7.24 points, SD 2.46 points; control group: mean 7.39, SD 2.56; t₃₇=-0.19, P=.85). However, 82% (14/17) of the participants were satisfied with using the app.

CONCLUSIONS

The findings will guide the design of future CDSS apps for the management of DM, aiming to help health care providers with a personalized approach depending on patients' comorbidities, specifically CKD, in accordance with guidelines.

Developing a Clinical Decision Support Tool for Appropriate Antibiotic Prescribing in Australian General Practice: A Simulation Study

Background. Inappropriate antibiotic prescribing can lead to antimicrobial resistance and drug side effects. Tools that assist general practitioners (GPs) in prescribing decisions may help to optimize prescribing. The aim of this study was to explore the use, acceptability, and feasibility of a clinical decision support (CDS) tool that incorporates evidence-based guidelines and consumer information that integrates with the electronic medical record (EMR). Methods. Eight GPs completed an interview and brief survey after participating in 2 simulated consultations. The survey consisted of demographic questions, perception of realism and representativeness of consultations, Post-Study System Usability Questionnaire, and System Usability Scale. Qualitative data were analyzed using framework analysis. Video data were reviewed, with length of consultation and time spent using the CDS tool documented. Results. Survey responses indicated that all GPs thought the consultations were "real" and representative of real-life consultations; 7 of 8 GPs were satisfied with usability of the tool. Key qualitative findings included that the tool assisted with clinical decision making and informed appropriate antibiotic prescribing. Accessibility and ease of use, including content (guideline and patient education resources), layout, and format, were key factors that determined whether GPs said that they would access the tool in everyday practice. Integration of the tool at multiple sites within the EMR facilitated access to guidelines and assisted in ensuring that the tool fit the clinical workflow. Conclusion. Our CDS tool was acceptable to GPs. Key features required for the tool were easy navigation, clear and useful guideline content, ability to fit into the clinical workflow, and incorporation into the EMR. Piloting of the tool in general practices to assess the impact and feasibility of use in real-world consultations will now be undertaken.

Diabetes, Care Homes, and the Influence of Technology on Practice and Care Delivery in Care Homes: Systematic Review and Qualitative Synthesis

BACKGROUND

Diabetes is increasing in prevalence and complexity in the care home setting, affecting up to a quarter of care home residents. Health outcomes for these residents are impacted by management of the disease, health care professionals (HCPs)' decision-making skills within the care home setting, and access to specialist services. The use of technology has the potential to recognize opportunities for early intervention that enables efficient responsive care, taking a fundamental role in linking the care home community to wider multidisciplinary teams for support.

OBJECTIVE

The aim of this paper was to identify evidence that explores factors relevant to the use of technology in and around the care home setting to aid in the management of diabetes.

METHODS

Databases searched using a structured prespecified approach included: PubMed, CINAHL (Cumulative Index to Nursing and Allied Health Literature), OVID Nursing database, Scopus, MEDLINE, the Cochrane Library, and the King's Fund from 2012 to 2017: handsearching was undertaken additionally for any gray literature. Preferred Reporting Items for Systematic review and Meta-Analysis Protocol was used as protocol with Risk of Bias in Systematic reviews a tool to assess the risk of bias across studies. Studies had to include interventions that combined technology to or from the care home setting to support residents living with diabetes.

RESULTS

The combined search strategy identified a total of 493 electronic records. Of these, 171 papers were screened for eligibility, 66 full papers were accessed, and 13 have been included in this study. Qualitative synthesis has identified different strands of research evidence in what and how technology is currently being used in and around care homes to enhance diabetes management. New initiatives and implementations of technology and emerging models of care that included the use of technology have also been included.

CONCLUSIONS

By triangulating the perspectives of HCPs, practitioners, specialists, and members of the care home community, the authors anticipate that this review will represent an up-to-date, evidence-based overview of the potential for using technology within the care home setting for diabetes management as well as stimulate research in this area.

Acceptability of a decision-support electronic health record system and its impact on diabetes care goals in South Asia: a mixed-methods evaluation of the CARRS trial

AIMS

To describe physicians' acceptance of decision-support electronic health record system and its impact on diabetes care goals among people with Type 2 diabetes.

METHODS

We analysed data from participants in the Centre for Cardiometabolic Risk Reduction in South Asia (CARRS) trial, who received the study intervention (care coordinators and use of a decision-support electronic health record system; n=575) using generalized estimating equations to estimate the association between acceptance/rejection of decision-support system prompts and outcomes (mean changes in HbA_1c , blood pressure and LDL cholesterol) considering repeated measures across all time points available. We conducted in-depth interviews with physicians to understand the benefits, challenges and value of the decision-support electronic health record system and analysed physicians' interviews using Rogers' diffusion of innovation theory.

RESULTS

At end-of-trial, participants with diabetes for whom glycaemic, systolic blood pressure, diastolic blood pressure and LDL cholesterol decision-support electronic health record prompts were accepted vs rejected, experienced no reduction in HbA_1c [mean difference: -0.05 mmol/mol (95% CI -0.22, 0.13); P=0.599], but statistically significant improvements were observed for systolic blood pressure [mean difference: -11.6 mmHg (95% CI -13.9, -9.3); P ≤ 0.001], diastolic blood pressure [mean difference: -5.2 mmHg (95% CI -6.5, -3.8); P ≤ 0.001] and LDL cholesterol [mean difference: -0.7 mmol/l (95% CI -0.6, -0.8); P ≤0.001], respectively. The relative advantages and compatibility of the decision-support electronic health record system with existing clinic set-ups influenced physicians' acceptance of it. Software complexities and data entry challenges could be overcome by task-sharing.

CONCLUSION

Wider adherence to decision-support electronic health record prompts could potentially improve diabetes goal achievement, particularly when accompanied by assistance from a non-physician health worker.

Designing Colorectal Cancer Screening Decision Support: A Cognitive Engineering Enterprise

Adoption of clinical decision support has been limited. Important barriers include an emphasis on algorithmic approaches to decision support that do not align well with clinical work flow and human decision strategies, and the expense and challenge of developing, implementing, and refining decision support features in existing electronic health records (EHRs). We applied decision-centered design to create a modular software application to support physicians in managing and tracking colorectal cancer screening. Using decision-centered design facilitates a thorough understanding of cognitive support requirements from an end user perspective as a foundation for design. In this project, we used an iterative design process, including ethnographic observation and cognitive task analysis, to move from an initial design concept to a working modular software application called the Screening & Surveillance App. The beta version is tailored to work with the Veterans Health Administration's EHR Computerized Patient Record System (CPRS). Primary care providers using the beta version Screening & Surveillance App more accurately answered questions about patients and found relevant information more quickly compared to those using CPRS alone. Primary care providers also reported reduced mental effort and rated the Screening & Surveillance App positively for usability.

The challenges of real-world implementation of web-based shared care software: the HopSCOTCH Shared-Care Obesity Trial in Children

BACKGROUND

E-health initiatives hold promise to improve shared-care models of health care. In 2008-2011 we developed and trialled web-based software to facilitate a randomised trial of a shared-care approach for childhood obesity involving General Practitioners (GPs) working with tertiary specialists. We describe the software's development, implementation and evaluation, and make recommendations for future e-health initiatives. The web-based software was designed with the goals of allowing both GPs and specialists to communicate and review patient progress; integrating with existing GP software; and supporting GPs to deliver the structured intervention. Specifically, we aimed to highlight the challenges inherent in this process, and report on the extent to which the software ultimately met its implementation and user aims.

METHODS

The study was conducted at the Royal Children's Hospital and 22 general practices across Melbourne, Australia. Participants comprised 30 GPs delivering the shared-care intervention. Outcomes included the following. (1) GPs' pre-specified software requirements: transcribed from two focus groups and analysed for themes using content analysis. (2) Software implementation and performance based on the experience of the research team and GPs. (3) GP users' evaluation collected via questionnaire. (4) Software usage collected via GP questionnaire and qualified through visual inspection of the software meta-data.

RESULTS

Software implementation posed difficult and at times disabling technological barriers (e.g. out-dated hardware, poor internet connections). The software's speed and inability to seamlessly link with day-to-day software was a source of considerable frustration. Overall, GPs rated software usability as poor, although most (68%) felt that the structure and functionality of the software was useful. Recommendations for future e-health initiatives include thorough scoping of IT systems and server speed, testing across diverse environments, automated pre-requisite checks and upgrades of processors/memory where necessary, and user-created usernames and passwords.

CONCLUSIONS

GPs are willing to embrace novel technologies to support their practice. However, implementation remains challenging mainly for technical reasons, and this precludes further evaluation of potential user-specific barriers. These findings could inform future e-health ventures into shared-care, and highlight the need for an appropriate infrastructure.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry: ACTRN126080000553.

Guidelines for maternal and neonatal "point of care": needs of and attitudes towards a computerized clinical decision support system in rural Burkina Faso

BACKGROUND

In 2010, 245,000 women died due to pregnancy-related causes in sub-Saharan Africa and southern Asia. Our study is nested into the QUALMAT project and seeks to improve the quality of maternal care services through the introduction of a computerized clinical decision support system (CDSS) to help healthcare workers in rural areas. Healthcare information technology applications in low-income countries may improve healthcare provision but recent studies demonstrate unintended consequences with underuse or resistance to CDSS and that the fit between the system and the clinical needs does present challenges.

AIMS

To explore and describe perceived needs and attitudes among healthcare workers to access WHO guidelines using CDSS in maternal and neonatal care in rural Burkina Faso.

METHODS

Data were collected with semi-structured interviews in two rural districts in Burkina Faso with 45 informants. Descriptive statistics were used for the analysis of the quantitative part of the interview corresponding to informants' background. Qualitative data were analyzed using manifest content analysis.

RESULTS

Four main findings emerged: (a) an appreciable willingness among healthcare workers for and a great interest to adapt and use modern technologies like computers to learn more in the workplace, (b) a positive attitude to easy access of guidelines and implementation of decision-support using computers in the workplace, (c) a fear that the CDSS would require more working time and lead to double-work, and (d) that the CDSS is complicated and requires substantial computer training and extensive instructions to fully implement.

CONCLUSIONS

The findings can be divided into aspects of motivators and barriers in relation to how the CDSS is perceived and to be used. These aspects are closely connected to each other as the motivating aspects can easily be turned into barriers if not taken care of properly in the final design, during implementation and maintenance of the CDSS at point of care.