Factors influencing implementation success of guideline-based clinical decision support systems: A systematic review and gaps analysis

Implementation of a Clinical Decision Support System for Antimicrobial Prescribing in Sub-Saharan Africa: Multisectoral Qualitative Study

BACKGROUND

Suboptimal use of antimicrobials is a driver of antimicrobial resistance in West Africa. Clinical decision support systems (CDSSs) can facilitate access to updated and reliable recommendations.

OBJECTIVE

This study aimed to assess contextual factors that could facilitate the implementation of a CDSS for antimicrobial prescribing in West Africa and Central Africa and to identify tailored implementation strategies.

METHODS

This qualitative study was conducted through 21 semistructured individual interviews via videoconference with health care professionals between September and December 2020. Participants were recruited using purposive sampling in a transnational capacity-building network for hospital preparedness in West Africa. The interview guide included multiple constructs derived from the Consolidated Framework for Implementation Research. Interviews were transcribed, and data were analyzed using thematic analysis.

RESULTS

The panel of participants included health practitioners (12/21, 57%), health actors trained in engineering (2/21, 10%), project managers (3/21, 14%), antimicrobial resistance research experts (2/21, 10%), a clinical microbiologist (1/21, 5%), and an anthropologist (1/21, 5%). Contextual factors influencing the implementation of eHealth tools existed at the individual, health care system, and national levels. At the individual level, the main challenge was to design a user-centered CDSS adapted to the prescriber's clinical routine and structural constraints. Most of the participants stated that the CDSS should not only target physicians in academic hospitals who can use their network to disseminate the tool but also general practitioners, primary care nurses, midwives, and other health care workers who are the main prescribers of antimicrobials in rural areas of West Africa. The heterogeneity in antimicrobial prescribing training among prescribers was a significant challenge to the use of a common CDSS. At the country level, weak pharmaceutical regulations, the lack of official guidelines for antimicrobial prescribing, limited access to clinical microbiology laboratories, self-medication, and disparity in health care coverage lead to inappropriate antimicrobial use and could limit the implementation and diffusion of CDSS for antimicrobial prescribing. Participants emphasized the importance of building a solid eHealth ecosystem in their countries by establishing academic partnerships, developing physician networks, and involving diverse stakeholders to address challenges. Additional implementation strategies included conducting a local needs assessment, identifying early adopters, promoting network weaving, using implementation advisers, and creating a learning collaborative. Participants noted that a CDSS for antimicrobial prescribing could be a powerful tool for the development and dissemination of official guidelines for infectious diseases in West Africa.

CONCLUSIONS

These results suggest that a CDSS for antimicrobial prescribing adapted for nonspecialized prescribers could have a role in improving clinical decisions. They also confirm the relevance of adopting a cross-disciplinary approach with participants from different backgrounds to assess contextual factors, including social, political, and economic determinants.

Clinical decision support systems for maternity care: a systematic review and meta-analysis

Background

The use of Clinical Decision Support Systems (CDSS) is increasing throughout healthcare and may be able to improve safety and outcomes in maternity care, but maternity care has key differences to other disciplines that complicate the use of CDSS. We aimed to identify evaluated CDSS and synthesise evidence of their impact on maternity care.

Methods

We conducted a systematic review for articles published before 24th May 2024 that described i) CDSS that ii) investigated the impact of their use iii) in maternity settings. Medline, CINAHL, CENTRAL and HMIC were searched for articles relating to evaluations of CDSS in maternity settings, with forward- and backward-citation tracing conducted for included articles. Risk of bias was assessed using the Mixed Methods Assessment Tool, and CDSS were described according to the clinical problem, purpose, design, and technical environment. Quantitative results from articles reporting appropriate data were meta-analysed to estimate odds of a CDSS achieving its desired outcome using a multi-level random effects model, first by individual CDSS and then across all CDSS. PROSPERO ID: CRD42022348157.

Findings

We screened 12,039 papers and included 87 articles describing 47 unique CDSS. 24 articles (28%) described randomised controlled trials, 30 (34%) described non-randomised interventional studies, 10 (11%) described mixed methods studies, 10 (11%) described qualitative studies, 7 (8%) described quantitative descriptive studies, and 7 (8%) described economic evaluations. 49 (56%) were in High-Income Countries and 38 (44%) in Low- and Middle-Income countries, with no CDSS trialled in both income categories. Meta-analysis of 35 included studies found an odds ratio for improved outcomes of 1.69 (95% confidence interval 1.24-2.30). There was substantial variation in effects, aims, CDSS types, context, study designs, and outcomes.

Interpretation

Most CDSS evaluations showed improvements in outcomes, but there was heterogeneity in all aspects of design and evaluation of systems. CDSS are increasingly important in delivering healthcare, and Electronic Health Records and mHealth will increase their availability, but traditional epidemiological methods may be limited in guiding design and demonstrating effectiveness due to rapid CDSS development lifecycles and the complex systems in which they are embedded. Development methods that are attentive to context, such as Human Centred Design, will help to meet this need.

Funding

None.

Human factors methods in the design of digital decision support systems for population health: a scoping review

BACKGROUND

While Human Factors (HF) methods have been applied to the design of decision support systems (DSS) to aid clinical decision-making, the role of HF to improve decision-support for population health outcomes is less understood. We sought to comprehensively understand how HF methods have been used in designing digital population health DSS.

MATERIALS AND METHODS

We searched English documents published in health sciences and engineering databases (Medline, Embase, PsychINFO, Scopus, Comendex, Inspec, IEEE Xplore) between January 1990 and September 2023 describing the development, validation or application of HF principles to decision support tools in population health.

RESULTS

We identified 21,581 unique records and included 153 studies for data extraction and synthesis. We included research articles that had a target end-user in population health and that used HF. HF methods were applied throughout the design lifecycle. Users were engaged early in the design lifecycle in the needs assessment and requirements gathering phase and design and prototyping phase with qualitative methods such as interviews. In later stages in the lifecycle, during user testing and evaluation, and post deployment evaluation, quantitative methods were more frequently used. However, only three studies used an experimental framework or conducted A/B testing.

CONCLUSIONS

While HF have been applied in a variety of contexts in the design of data-driven DSSs for population health, few have used Human Factors to its full potential. We offer recommendations for how HF can be leveraged throughout the design lifecycle. Most crucially, system designers should engage with users early on and throughout the design process. Our findings can support stakeholders to further empower public health systems.

Development of the ADFICE_IT clinical decision support system to assist deprescribing of fall-risk increasing drugs: A user-centered design approach

INTRODUCTION

Deprescribing fall-risk increasing drugs (FRIDs) is promising for reducing the risk of falling in older adults. Applying appropriate deprescribing in practice can be difficult due to the outcome uncertainties associated with stopping FRIDs. The ADFICE_IT intervention addresses this complexity with a clinical decision support system (CDSS) that facilitates optimum deprescribing of FRIDs by using a fall-risk prediction model, aggregation of deprescribing guidelines, and joint medication management.

METHODS

The development process of the CDSS is described in this paper. Development followed a user-centered design approach in which users and experts were involved throughout each phase. In phase I, a prototype of the CDSS was developed which involved a literature and systematic review, European survey (n = 581), and semi-structured interviews with clinicians (n = 19), as well as the aggregation and testing of deprescribing guidelines and the development of the fall-risk prediction model. In phase II, the feasibility of the CDSS was tested by means of two usability testing rounds with users (n = 11).

RESULTS

The final CDSS consists of five web pages. A connection between the Electronic Health Record allows for the retrieval of patient data into the CDSS. Key design requirements for the CDSS include easy-to-use features for fast-paced clinical environments, actionable deprescribing recommendations, information transparency, and visualization of the patient's fall-risk estimation. Key elements for the software include a modular architecture, open source, and good security.

CONCLUSION

The ADFICE_IT CDSS supports physicians in deprescribing FRIDs optimally to prevent falls in older patients. Due to continuous user and expert involvement, each new feedback round led to an improved version of the system. Currently, a cluster-randomized controlled trial with process evaluation at hospitals in the Netherlands is being conducted to test the effect of the CDSS on falls. The trial is registered with ClinicalTrials.gov (date; 7-7-2022, identifier: NCT05449470).

Ontology of clinical practice guidelines for Integrated Traditional Chinese and Western Medicine

OBJECTIVE

Clinical practice guidelines (CPGs) for Integrated Traditional Chinese and Western Medicine (TCM and WM) are important medical documents used to assist medical decision-making and are of great significance for standardizing clinical pathways. However, due to the constraints of text format, it is difficult for Integrated TCM and WM CPGs to play a real role in medical practice. In addition, how to standardize the structure and semantic relationships between Integrated TCM and WM CPG knowledge, and realize the construction of computable, sharable and reliable CPGs, remains an urgent issue to be addressed. Therefore, we are proposing an ontology of CPGs for Integrated TCM and WM.

METHODS

We first initialized domain concepts and relationships to ensure the accuracy of the ontology knowledge structure. We then screened CPGs that meet the standards for Integrated TCM and WM, analyzed and classified the contents, and extracted the common structures. Based on the seven-step ontology construction method combined with inference-complement, referring to the representation methods and hierarchical relationships of terms and concepts in MeSH, ICD-10, SNOMED-CT, and other ontologies and terminology sets, we formed the concept structure and semantic relationship tables for the ontology. We also achieved the matching and mapping between the ontology and reference ontologies and term sets. Next, we defined the aspects and constraints of properties, selected multiple Integrated TCM and WM CPGs as instances to populate, and used ontology reasoning tools and formulated defined inference rules to reason and extend the ontology. Finally, we evaluated the performance of the ontology.

RESULTS

The content of the Integrated TCM and WM CPGs is divided into nine parts: basic information, background, development method, clinical question, recommendation, evidence, conclusion, result, and reason for recommendations. The Integrated TCM and WM CPG ontology has 152 classes and defines 90 object properties and 114 data properties, with a maximum classification depth of 4 layers. The terms of disease, drug and examination item names in the ontology have been standardized.

CONCLUSIONS

This study proposes an Integrated TCM and WM CPG ontology. The ontology adopts a modular design, which has both sharing and scaling ability, and can express rich guideline knowledge. It provides important support for the semantic processing and computational application of guideline documents.

Clinical Decision Support Tool to Promote Adoption of New Neonatal Hyperbilirubinemia Guidelines

OBJECTIVE

 This study aimed to increase the adoption of revised newborn hyperbilirubinemia guidelines by building a clinical decision support (CDS) tool into templated notes.

METHODS

 We created a rule-based CDS tool that correctly populates the phototherapy threshold from more than 2,700 possible values directly into the note and guides clinicians to an appropriate follow-up plan consistent with the new recommendations. We manually reviewed notes before and after CDS tool implementation to evaluate new guidelines adherence, and surveys were used to assess clinicians' perceptions.

RESULTS

 Postintervention documentation showed a decrease in old risk stratification methods (48 to 0.4%, p < 0.01) and an increase in new phototherapy threshold usage (39 to 95%, p < 0.01) and inclusion of follow-up guidance (28 to 79%, p < 0.01). Survey responses on workflow efficiency and satisfaction did not significantly change after CDS tool implementation.

CONCLUSION

 Our study details an innovative CDS tool that contributed to increased adoption of newly revised guidelines after the addition of this tool to templated notes.

Clinical decision support systems in addiction and concurrent disorders: A systematic review and meta-analysis

INTRODUCTION

This review aims to synthesise the literature on the efficacy, evolution, and challenges of implementing Clincian Decision Support Systems (CDSS) in the realm of mental health, addiction, and concurrent disorders.

METHODS

Following PRISMA guidelines, a systematic review and meta-analysis were performed. Searches conducted in databases such as MEDLINE, Embase, CINAHL, PsycINFO, and Web of Science through 25 May 2023, yielded 27,344 records. After necessary exclusions, 69 records were allocated for detailed synthesis. In the examination of patient outcomes with a focus on metrics such as therapeutic efficacy, patient satisfaction, and treatment acceptance, meta-analytic techniques were employed to synthesise data from randomised controlled trials.

RESULTS

A total of 69 studies were included, revealing a shift from knowledge-based models pre-2017 to a rise in data-driven models post-2017. The majority of models were found to be in Stage 2 or 4 of maturity. The meta-analysis showed an effect size of -0.11 for addiction-related outcomes and a stronger effect size of -0.50 for patient satisfaction and acceptance of CDSS.

DISCUSSION

The results indicate a shift from knowledge-based to data-driven CDSS approaches, aligned with advances in machine learning and big data. Although the immediate impact on addiction outcomes is modest, higher patient satisfaction suggests promise for wider CDSS use. Identified challenges include alert fatigue and opaque AI models.

CONCLUSION

CDSS shows promise in mental health and addiction treatment but requires a nuanced approach for effective and ethical implementation. The results emphasise the need for continued research to ensure optimised and equitable use in healthcare settings.

Clinical Practice Guidelines for the Management of Type 2 Diabetes in South Asia: A Systematic Review

BACKGROUND

Clinical practice guidelines (CPGs) are a helpful tool for the evidence-based management of Type 2 Diabetes Mellitus (T2D). The aim of this systematic review was to synthesize and appraise the scope and quality of South Asian T2D CPGs.

METHODS

This PROPSERO registered (CRD42023425150) systematic review adhered to the 2020 PRISMA guidelines. We searched the PubMed, Embase, Cochrane, and Google Scholar databases for relevant guidelines. Data synthesis was performed using a qualitative approach and methodological quality was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II tool.

RESULTS

We identified eleven unique CPGs (three each from Pakistan and Sri Lanka, two from India, and one each from Bangladesh, Nepal, and Bhutan) which were published or updated between 2017 and 2023. The CPGs included recommendations regarding screening, diagnosis, prevention, and management of T2D and its acute and chronic complications, comorbidities, and fasting with T2D. The AGREE II mean domain scores ranged from 37 % to 80 %; three CPGs were 'recommended for clinical use,' seven were 'recommended for use with modifications' and one was deemed unfit for implementation.

CONCLUSION

The present review summarized and appraised broadly CPGs from South Asia for T2D and can help direct improvements to future iterations.

Perceptions of infection control professionals toward electronic surveillance software supporting inpatient infections: A mixed methods study

BACKGROUND

Electronic surveillance software (ESS) collects multiple patient data from hospital software to assist infection control professionals in the prevention and control of hospital-associated infections. This study aimed to understand the perceptions of end users (i.e., infection control professionals) and the facilitators and barriers related to a commercial ESS named ZINC and to assess its usability.

METHODS

A mixed-method research approach was adopted among infection control professionals 10 months after the implementation of commercial ESS in the university hospital of Nancy, France. A qualitative analysis based on individual semistructured interviews was conducted to collect professionals' perceptions of ESS and to understand barriers and facilitators. Qualitative data were systematically coded and thematically analyzed. A quantitative analysis was performed using the System Usability Scale (SUS).

RESULTS

Thirteen infection control professionals were included. Qualitative analysis revealed technical, organizational and human barriers to the installation and use stages and five significant facilitators: the relevant design of the ESS, the improvement of infection prevention and control practices, the designation of a champion/superuser among professionals, training, and collaboration with the developer team. Quantitative analysis indicated that the evaluated ESS was a "good" system in terms of perceived ease of use, with an overall median SUS score of 85/100.

CONCLUSIONS

This study shows the value of ESS to support inpatient infections as perceived by infection control professionals. It reveals barriers and facilitators to the implementation and adoption of ESS. These barriers and facilitators should be considered to facilitate the installation of the software in other hospitals.

Electronic Immunization Registry in Rwanda: Qualitative Study of Health Worker Experiences

BACKGROUND

Monitoring childhood immunization programs is essential for health systems. Despite the introduction of an electronic immunization registry called e-Tracker in Rwanda, challenges such as lacking population denominators persist, leading to implausible reports of coverage rates of more than 100%.

OBJECTIVE

This study aimed to assess the extent to which the immunization e-Tracker responds to stakeholders' needs and identify key areas for improvement.

METHODS

In-depth interviews were conducted with all levels of e-Tracker users including immunization nurses, data managers, and supervisors from health facilities in 5 districts of Rwanda. We used an interview guide based on the constructs of the Human, Organization, and Technology-Fit (HOT-Fit) framework, and we analyzed and summarized our findings using the framework.

RESULTS

Immunization nurses reported using the e-Tracker as a secondary data entry tool in addition to paper-based forms, which resulted in considerable dissatisfaction among nurses. While users acknowledged the potential of a digital tool compared to paper-based systems, they also reported the need for improvement of functionalities to support their work, such as digital client appointment lists, lists of defaulters, search and register functions, automated monthly reports, and linkages to birth notifications and the national identity system.

CONCLUSIONS

Reducing dual documentation for users can improve e-Tracker use and user satisfaction. Our findings can help identify additional digital health interventions to support and strengthen the health information system for the immunization program.

Is the European Crohn's and Colitis organisation (ECCO) e-guide an acceptable and feasible tool for increasing gastroenterologists' guideline adherence? A mixed methods evaluation

BACKGROUND AND AIMS

Management of inflammatory bowel disease is constantly evolving, increasing the importance for gastroenterologists to keep up to date with guidelines. Traditional implementation strategies have had only small positive impacts on clinical practice. eHealth strategies such as the European Crohn's and Colitis Organisation e-guide may be beneficial for clinician decision making in keeping with guidelines. The aim of this study was to evaluate the feasibility and acceptability of the e-guide.

METHODS

A mixed methods approach was used to evaluate feasibility and acceptability. Cognitive (think-aloud) interviews were conducted with Australian gastroenterologists while using the e-guide. Two clinical scenarios were developed to allow evaluation of various aspects of the e-guide. Content analysis was applied to the qualitative interview data and descriptive analysis to the quantitative and observational data.

RESULTS

Seventeen participants completed the study. Data saturation were reached. The ECCO e-guide was largely feasible and acceptable, as demonstrated by most clinical questions answered correctly, 87% reaching the answer within 3 min, and most feeling it was useful, would be beneficial to their practice and would use it again. Issues raised included difficulties with website navigation, layout of the e-guide and difficulties with access (network firewalls, paid subscription required).

CONCLUSIONS

The ECCO e-guide is largely acceptable and feasible for gastroenterologists to use. Aspects of the e-guide could be modified to improve user experience. This study highlights the importance of engaging end-users in the development and evaluation of clinician educational tools.

Barriers to and Facilitators of Key Stakeholders Influencing Successful Digital Implementation of Remote Monitoring Solutions: Mixed Methods Analysis

BACKGROUND

Implementation of remote monitoring solutions and digital alerting tools in health care has historically been challenging, despite the impetus provided by the COVID-19 pandemic. To date, a health systems-based approach to systematically describe barriers and facilitators across multiple domains has not been undertaken.

OBJECTIVE

We aimed to undertake a comprehensive mixed methods analysis of barriers and facilitators for successful implementation of remote monitoring and digital alerting tools in complex health organizations.

METHODS

A mixed methods approach using a modified Technology Acceptance Model questionnaire and semistructured interviews mapped to the validated fit among humans, organizations, and technology (HOT-fit) framework was undertaken. Likert frequency responses and deductive thematic analyses were performed.

RESULTS

A total of 11 participants responded to the questionnaire and 18 participants to the interviews. Key barriers and facilitators could be mapped onto 6 dimensions, which incorporated aspects of digitization: system use (human), user satisfaction (human), environment (organization), structure (organization), information and service quality (technology), and system quality (technology).

CONCLUSIONS

The recommendations proposed can enhance the potential for future remote sensing solutions to be more successfully integrated in health care practice, resulting in more successful use of "virtual wards."

TRIAL REGISTRATION

ClinicalTrials.gov NCT05321004; https://www.clinicaltrials.gov/study/NCT05321004.

Imaging 3.0: A scoping review

We aim to provide a comprehensive summary of the current body of literature concerning the Imaging 3.0 initiative and its implications for patient care within the field of radiology. We offer a thorough analysis of the literature pertaining to the Imaging 3.0 initiative, emphasizing the practical application of the five pillars of the program, their cost-effectiveness, and their benefits in patient management. By doing so, we hope to illustrate the impact the Imaging 3.0 Initiative can have on the future of radiology and patient care.

Use of a digital tool to support the diagnostic process in memory clinics-a usability study

BACKGROUND

Both memory clinic professionals and patients see value in digital tools, yet these hardly find their way to clinical practice. We explored the usability of a digital tool to support the diagnostic work-up in daily memory clinic practice. We evaluated four modules that integrate multi-modal patient data (1.cognitive test; cCOG, and 2. MRI quantification; cMRI) into useful diagnostic information for clinicians (3. cDSI) and understandable and personalized information for patients (4. patient report).

METHODS

We conducted a mixed-methods study in five Dutch memory clinics. Fourteen clinicians (11 geriatric specialists/residents, two neurologists, one nurse practitioner) were invited to integrate the tool into routine care with 43 new memory clinic patients. We evaluated usability and user experiences through quantitative data from questionnaires (patients, care partners, clinicians), enriched with thematically analyzed qualitative data from interviews (clinicians).

RESULTS

We observed wide variation in tool use among clinicians. Our core findings were that clinicians: 1) were mainly positive about the patient report, since it contributes to patient-centered and personalized communication. This was endorsed by patients and care partners, who indicated that the patient report was useful and understandable and helped them to better understand their diagnosis, 2) considered the tool acceptable in addition to their own clinical competence, 3) indicated that the usefulness of the tool depended on the patient population and purpose of the diagnostic process, 4) addressed facilitators (ease of use, practice makes perfect) and barriers (high workload, lack of experience, data unavailability).

CONCLUSION

This multicenter usability study revealed a willingness to adopt a digital tool to support the diagnostic process in memory clinics. Clinicians, patients, and care partners appreciated the personalized diagnostic report. More attention to education and training of clinicians is needed to utilize the full functionality of the tool and foster implementation in actual daily practice. These findings provide an important step towards a lasting adoption of digital tools in memory clinic practice.

Implementation of an Electronic Clinical Decision Support System for the Early Recognition and Management of Dysglycemia in an Inpatient Mental Health Setting Using CogStack: Protocol for a Pilot Hybrid Type 3 Effectiveness-Implementation Randomized Controlled Cluster Trial

BACKGROUND

Severe mental illnesses (SMIs), including schizophrenia, bipolar affective disorder, and major depressive disorder, are associated with an increased risk of physical health comorbidities and premature mortality from conditions including cardiovascular disease and diabetes. Digital technologies such as electronic clinical decision support systems (eCDSSs) could play a crucial role in improving the clinician-led management of conditions such as dysglycemia (deranged blood sugar levels) and associated conditions such as diabetes in people with a diagnosis of SMI in mental health settings.

OBJECTIVE

We have developed a real-time eCDSS using CogStack, an information retrieval and extraction platform, to automatically alert clinicians with National Health Service Trust-approved, guideline-based recommendations for dysglycemia monitoring and management in secondary mental health care. This novel system aims to improve the management of dysglycemia and associated conditions, such as diabetes, in SMI. This protocol describes a pilot study to explore the acceptability, feasibility, and evaluation of its implementation in a mental health inpatient setting.

METHODS

This will be a pilot hybrid type 3 effectiveness-implementation randomized controlled cluster trial in inpatient mental health wards. A ward will be the unit of recruitment, where it will be randomly allocated to receive either access to the eCDSS plus usual care or usual care alone over a 4-month period. We will measure implementation outcomes, including the feasibility and acceptability of the eCDSS to clinicians, as primary outcomes, alongside secondary outcomes relating to the process of care measures such as dysglycemia screening rates. An evaluation of other implementation outcomes relating to the eCDSS will be conducted, identifying facilitators and barriers based on established implementation science frameworks.

RESULTS

Enrollment of wards began in April 2022, after which clinical staff were recruited to take part in surveys and interviews. The intervention period of the trial began in February 2023, and subsequent data collection was completed in August 2023. Data are currently being analyzed, and results are expected to be available in June 2024.

CONCLUSIONS

An eCDSS can have the potential to improve clinician-led management of dysglycemia in inpatient mental health settings. If found to be feasible and acceptable, then, in combination with the results of the implementation evaluation, the system can be refined and improved to support future successful implementation. A larger and more definitive effectiveness trial should then be conducted to assess its impact on clinical outcomes and to inform scalability and application to other conditions in wider mental health care settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04792268; https://clinicaltrials.gov/study/NCT04792268.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/49548.

Exploring the role of professional identity in the implementation of clinical decision support systems-a narrative review

BACKGROUND

Clinical decision support systems (CDSSs) have the potential to improve quality of care, patient safety, and efficiency because of their ability to perform medical tasks in a more data-driven, evidence-based, and semi-autonomous way. However, CDSSs may also affect the professional identity of health professionals. Some professionals might experience these systems as a threat to their professional identity, as CDSSs could partially substitute clinical competencies, autonomy, or control over the care process. Other professionals may experience an empowerment of the role in the medical system. The purpose of this study is to uncover the role of professional identity in CDSS implementation and to identify core human, technological, and organizational factors that may determine the effect of CDSSs on professional identity.

METHODS

We conducted a systematic literature review and included peer-reviewed empirical studies from two electronic databases (PubMed, Web of Science) that reported on key factors to CDSS implementation and were published between 2010 and 2023. Our explorative, inductive thematic analysis assessed the antecedents of professional identity-related mechanisms from the perspective of different health care professionals (i.e., physicians, residents, nurse practitioners, pharmacists).

RESULTS

One hundred thirty-one qualitative, quantitative, or mixed-method studies from over 60 journals were included in this review. The thematic analysis found three dimensions of professional identity-related mechanisms that influence CDSS implementation success: perceived threat or enhancement of professional control and autonomy, perceived threat or enhancement of professional skills and expertise, and perceived loss or gain of control over patient relationships. At the technological level, the most common issues were the system's ability to fit into existing clinical workflows and organizational structures, and its ability to meet user needs. At the organizational level, time pressure and tension, as well as internal communication and involvement of end users were most frequently reported. At the human level, individual attitudes and emotional responses, as well as familiarity with the system, most often influenced the CDSS implementation. Our results show that professional identity-related mechanisms are driven by these factors and influence CDSS implementation success. The perception of the change of professional identity is influenced by the user's professional status and expertise and is improved over the course of implementation.

CONCLUSION

This review highlights the need for health care managers to evaluate perceived professional identity threats to health care professionals across all implementation phases when introducing a CDSS and to consider their varying manifestations among different health care professionals. Moreover, it highlights the importance of innovation and change management approaches, such as involving health professionals in the design and implementation process to mitigate threat perceptions. We provide future areas of research for the evaluation of the professional identity construct within health care.

The Facilitators and Barriers of the Implementation of a Clinical Decision Support System for Breast Cancer Multidisciplinary Team Meetings-An Interview Study

BACKGROUND

AI-driven clinical decision support systems (CDSSs) hold promise for multidisciplinary team meetings (MDTMs). This study aimed to uncover the hurdles and aids in implementing CDSSs during breast cancer MDTMs.

METHODS

Twenty-four core team members from three hospitals engaged in semi-structured interviews, revealing a collective interest in experiencing CDSS workflows in clinical practice. All interviews were audio recorded, transcribed verbatim and analyzed anonymously. A standardized approach, 'the framework method', was used to create an analytical framework for data analysis, which was performed by two independent researchers.

RESULTS

Positive aspects included improved data visualization, time-saving features, automated trial matching, and enhanced documentation transparency. However, challenges emerged, primarily concerning data connectivity, guideline updates, the accuracy of AI-driven suggestions, and the risk of losing human involvement in decision making. Despite the complexities involved in CDSS development and integration, clinicians demonstrated enthusiasm to explore its potential benefits.

CONCLUSIONS

Acknowledging the multifaceted nature of this challenge, insights into the barriers and facilitators identified in this study offer a potential roadmap for smoother future implementations. Understanding these factors could pave the way for more effective utilization of CDSSs in breast cancer MDTMs, enhancing patient care through informed decision making.

A study protocol for implementing Canadian Practice Guidelines for Treating Children and Adolescents with Eating Disorders

BACKGROUND

Eating disorders have one of the highest mortality rates among psychiatric illnesses. Timely intervention is crucial for effective treatment, as eating disorders tend to be chronic and difficult to manage if left untreated. Clinical practice guidelines play a vital role in improving healthcare delivery, aiming to minimize variations in care and bridge the gap between research and practice. However, research indicates an active guideline implementation approach is crucial to effective uptake.

METHODS

Mixed methods will be used to inform and evaluate our guideline implementation approach. Semi-structured focus groups will be conducted in each of the eight provinces in Canada. Each focus group will comprise 8-10 key stakeholders, including clinicians, program administrators, and individuals with lived experience or caregivers. Qualitative data will be analyzed using conventional content analysis and the constant comparison technique and the results will be used to inform our implementation strategy. The study will then evaluate the effectiveness of our implementation approach through pre- and post-surveys, comparing changes in awareness, use, and impact of the guidelines in various stakeholder groups.

DISCUSSION

Through a multifaceted implementation strategy, involving the co-creation of educational materials, tailored training, and context-specific strategies, this study intends to enhance guideline uptake and promote adherence to evidence-based practices. Our study will also contribute valuable information on the impact of our implementation strategies.

Adolescent, parent, and provider attitudes toward a machine learning based clinical decision support system for selecting treatment for youth depression

BACKGROUND

Machine learning based clinical decision support systems (CDSSs) have been proposed as a means of advancing personalized treatment planning for disorders, such as depression, that have a multifaceted etiology, course, and symptom profile. However, machine learning based models for treatment selection are rare in the field of psychiatry. They have also not yet been translated for use in clinical practice. Understanding key stakeholder attitudes toward machine learning based CDSSs is critical for developing plans for their implementation that promote uptake by both providers and families.

METHODS

In Study 1, a prototype machine learning based Clinical Decision Support System for Youth Depression (CDSS-YD) was demonstrated to focus groups of adolescents with a diagnosis of depression (n = 9), parents (n = 11), and behavioral health providers (n = 8). Qualitative analysis was used to assess their attitudes towards the CDSS-YD. In Study 2, behavioral health providers were trained in the use of the CDSS-YD and they utilized the CDSS-YD in a clinical encounter with 6 adolescents and their parents as part of their treatment planning discussion. Following the appointment, providers, parents, and adolescents completed a survey about their attitudes regarding the use of the CDSS-YD.

RESULTS

All stakeholder groups viewed the CDSS-YD as an easy to understand and useful tool for making personalized treatment decisions, and families and providers were able to successfully use the CDSS-YD in clinical encounters. Parents and adolescents viewed their providers as having a critical role in the use the CDSS-YD, and this had implications for the perceived trustworthiness of the CDSS-YD. Providers reported that clinic productivity metrics would be the primary barrier to CDSS-YD implementation, with the creation of protected time for training, preparation, and use as a key facilitator.

CONCLUSIONS

Machine learning based CDSSs, if proven effective, have the potential to be widely accepted tools for personalized treatment planning. Successful implementation will require addressing the system-level barrier of having sufficient time and energy to integrate it into practice.

Clinical decision support system to optimise symptom management in palliative medicine: focus group study

OBJECTIVES

Suboptimal symptom control in patients with life-limiting illnesses is a major issue. A clinical decision support system (CDSS) that combines a patient-reported symptom assessment scale (SAS) and guideline-based individualised recommendations has the potential to improve symptom management. However, lacking end-user acceptance often prevents CDSS use in daily practice.We aimed to evaluate the acceptability and feasibility of a palliative care CDSS according to its targeted end-users.

METHODS

Six focus groups with different groups of stakeholders were conducted: (1) patient representatives; (2) community nurses; (3) hospital nurses; (4) general practitioners; (5) hospital physicians and (6) palliative care specialists. Audiotapes were transcribed verbatim and thematically analysed.

RESULTS

Fifty-one stakeholders (6-12 per focus group) participated. Six themes were discussed: effect, validity, continuity, practical usability, implementation and additional features. All participants expected a CDSS to improve symptom management, for example, by reminding clinicians of blind spots and prompting patient participation. They feared interference with professional autonomy of physicians, doubted the validity of using a patient-reported SAS as CDSS input and thought lacking care continuity would complicate CDSS use. Clinicians needed clear criteria for when to use the CDSS (eg, life-limiting illness, timing in illness trajectory). Participants preferred a patient-coordinated system but were simultaneously concerned patients may be unwilling or unable to fill out an SAS.

CONCLUSIONS

A palliative care CDSS was considered useful for improving symptom management. To develop a feasible system, barriers for successful implementation must be addressed including concerns about using a patient-reported SAS, lacking care continuity and unclear indications for use.

Unfamiliar personal protective equipment: The role of routine practice and other factors affecting healthcare personnel doffing strategies

BACKGROUND

Healthcare personnel (HCP) may encounter unfamiliar personal protective equipment (PPE) during clinical duties, yet we know little about their doffing strategies in such situations.

OBJECTIVE

To better understand how HCP navigate encounters with unfamiliar PPE and the factors that influence their doffing strategies.

SETTING

The study was conducted at 2 Midwestern academic hospitals.

PARTICIPANTS

The study included 70 HCP: 24 physicians and resident physicians, 31 nurses, 5 medical or nursing students, and 10 other staff. Among them, 20 had special isolation unit training.

METHODS

Participants completed 1 of 4 doffing simulation scenarios involving 3 mask designs, 2 gown designs, 2 glove designs, and a full PPE ensemble. Doffing simulations were video-recorded and reviewed with participants during think-aloud interviews. Interviews were audio-recorded and analyzed using thematic analysis.

RESULTS

Participants identified familiarity with PPE items and designs as an important factor in doffing. When encountering unfamiliar PPE, participants cited aspects of their routine practices such as designs typically used, donning and doffing frequency, and design cues, and their training as impacting their doffing strategies. Furthermore, they identified nonintuitive design and lack of training as barriers to doffing unfamiliar PPE appropriately.

CONCLUSION

PPE designs may not be interchangeable, and their use may not be intuitive. HCP drew on routine practices, experiences with familiar PPE, and training to adapt doffing strategies for unfamiliar PPE. In doing so, HCP sometimes deviated from best practices meant to prevent self-contamination. Hospital policies and procedures should include ongoing and/or just-in-time training to ensure HCP are equipped to doff different PPE designs encountered during clinical care.

Physicians' Perspectives on AI in Clinical Decision Support Systems: Interview Study of the CURATE.AI Personalized Dose Optimization Platform

BACKGROUND

Physicians play a key role in integrating new clinical technology into care practices through user feedback and growth propositions to developers of the technology. As physicians are stakeholders involved through the technology iteration process, understanding their roles as users can provide nuanced insights into the workings of these technologies that are being explored. Therefore, understanding physicians' perceptions can be critical toward clinical validation, implementation, and downstream adoption. Given the increasing prevalence of clinical decision support systems (CDSSs), there remains a need to gain an in-depth understanding of physicians' perceptions and expectations toward their downstream implementation. This paper explores physicians' perceptions of integrating CURATE.AI, a novel artificial intelligence (AI)-based and clinical stage personalized dosing CDSSs, into clinical practice.

OBJECTIVE

This study aims to understand physicians' perspectives of integrating CURATE.AI for clinical work and to gather insights on considerations of the implementation of AI-based CDSS tools.

METHODS

A total of 12 participants completed semistructured interviews examining their knowledge, experience, attitudes, risks, and future course of the personalized combination therapy dosing platform, CURATE.AI. Interviews were audio recorded, transcribed verbatim, and coded manually. The data were thematically analyzed.

RESULTS

Overall, 3 broad themes and 9 subthemes were identified through thematic analysis. The themes covered considerations that physicians perceived as significant across various stages of new technology development, including trial, clinical implementation, and mass adoption.

CONCLUSIONS

The study laid out the various ways physicians interpreted an AI-based personalized dosing CDSS, CURATE.AI, for their clinical practice. The research pointed out that physicians' expectations during the different stages of technology exploration can be nuanced and layered with expectations of implementation that are relevant for technology developers and researchers.

Opportunities for clinical decision support targeting medication safety in remote primary care management of chronic kidney disease: A qualitative study in Northern Australia

INTRODUCTION

This study aimed to identify opportunities for clinical decision support targeting medication safety in remote primary care, by investigating the relationship between clinical workflows, health system priorities, cognitive tasks, and reasoning processes in the context of medicines used in people with chronic kidney disease (CKD).

METHODS

This qualitative study involved one-on-one, semistructured interviews. The participants were healthcare professionals employed in a clinical or managerial capacity with clinical work experience in a remote health setting for at least 1 year.

RESULTS

Twenty-five clinicians were interviewed. Of these, four were rural medical practitioners, nine were remote area nurses, eight were Aboriginal health practitioners, and four were pharmacists. Four major themes were identified from the interviews: (1) the need for a clinical decision support system to support a sustainable remote health workforce, as clinicians were "constantly stretched" and problems may "fall through the cracks"; (2) reliance on digital health technologies, as medical staff are often not physically available and clinicians-on-duty usually "flick an email and give a call so that I can actually talk it through to our GP"; (3) knowledge gaps, as "it takes a lot of mental space" to know each patient's renal function and their medication history, and clinicians believe "mistakes can be made"; and (4) multiple risk factors impacting CKD management, including clinical, social and behavioural determinants.

CONCLUSIONS

The high prevalence of CKD and reliance on digital health systems in remote primary health settings can make a clinical decision support system valuable for supporting clinicians who may not have extensive experience in managing medicines for people with CKD.

Usability of an app-based clinical decision support system to monitor psychotropic drug prescribing appropriateness in dementia

BACKGROUND

Guidelines recommend reluctant psychotropic drug (PD) prescribing in nursing home residents with dementia and neuropsychiatric symptoms (NPS), as efficacy of PDs is limited, and side effects are common. Nevertheless, PDs are commonly prescribed to reduce NPS. A smartphone application that evaluates appropriateness of PD prescriptions and provides recommendations from the revised Dutch guideline on problem behaviour in dementia may promote guideline adherence and increase appropriate prescribing.

OBJECTIVE

This study aimed to assess user experiences, barriers and facilitators of the Dutch 'Psychotropic Drug Tool' smartphone application (PDT) in the context of appropriate prescribing of PDs to nursing home residents with dementia and NPS.

METHODS/DESIGN

The PDT was developed according to the recommendations of the Dutch guideline for treatment of NPS in people with dementia. Feedback provided during usability testing with two end-users was applied to improve the PDT before implementation in day-to-day practice. Sixty-three prescribers were asked to use the PDT at their own convenience for four months. User expectations and experiences were assessed at baseline and after four months with the System Usability Scale and the Assessment of Barriers and Facilitators for Implementation.

RESULTS

Expected usability (M = 72.59; SD = 11.84) was similar to experienced usability after four months (M = 69.13; SD = 16.48). Appreciation of the PDTs user-friendliness (on average 6.7 out of 10) and design (7.3) were moderately positive, in contrast to the global rating of the PDT (5.7). Perceived barriers for PDT use were time consumption and lack of integration with existing electronic systems. Perceived facilitators were ease of use and attractive lay out. For broader implementation, physicians suggested a change in direction of the PDT: start assessment of appropriateness based on the list of NPS instead of PD as primary input.

CONCLUSIONS

In this pragmatic prospective cohort study we found that the PDT was used by elderly care physicians, with mediocre user satisfaction. The PDT will be optimized based on user feedback regarding experienced usability, barriers and facilitators, after which broader implementation can be initialized. The Medical Ethics Review Board of the University Medical Center Groningen declared this is a non-WMO study (UMCG RR Number: 201800284).

An Evaluation Framework for a Novel Process to Codevelop Written and Computable Guidelines

Clinical practice guidelines (CPGs) support individual and population health by translating new, evidence-based knowledge into recommendations for health practice. CPGs can be provided as computable, machine-readable guidelines that support the translation of recommendations into shareable, interoperable clinical decision support and other digital tools (eg, quality measures, case reports, care plans). Interdisciplinary collaboration among guideline developers and health information technology experts can facilitate the translation of written guidelines into computable ones. The benefits of interdisciplinary work include a focus on the needs of end-users who apply guidelines in practice through clinic decision support systems as part of the Centers for Disease Control and Prevention's (CDC's) Adapting Clinical Guidelines for the Digital Age (ACG) initiative, a group of interdisciplinary experts proposed a process to facilitate the codevelopment of written and computable CPGs, referred to as the "integrated process (IP)."1 This paper presents a framework for evaluating the IP based on a combination of vetted evaluation models and expert opinions. This framework combines 3 types of evaluations: process, product, and outcomes. These evaluations assess the value of interdisciplinary expert collaboration in carrying out the IP, the quality, usefulness, timeliness, and acceptance of the guideline, and the guideline's health impact, respectively. A case study is presented that illustrates application of the framework.

Systematic User-centered Design of a Prototype Clinical Decision Support System for Glaucoma

Purpose

To rigorously develop a prototype clinical decision support (CDS) system to help clinicians determine the appropriate timing for follow-up visual field testing for patients with glaucoma and to identify themes regarding the context of use for glaucoma CDS systems, design requirements, and design solutions to meet these requirements.

Design

Semistructured qualitative interviews and iterative design cycles.

Participants

Clinicians who care for patients with glaucoma, purposefully sampled to ensure a representation of a range of clinical specialties (glaucoma specialist, general ophthalmologist, optometrist) and years in clinical practice.

Methods

Using the established User-Centered Design Process framework, we conducted semistructured interviews with 5 clinicians that addressed the context of use and design requirements for a glaucoma CDS system. We analyzed the interviews using inductive thematic analysis and grounded theory to generate themes regarding the context of use and design requirements. We created design solutions to address these requirements and used iterative design cycles with the clinicians to refine the CDS prototype.

Main Outcome Measures

Themes regarding decision support for determining the timing of visual field testing for patients with glaucoma, CDS design requirements, and CDS design features.

Results

We identified 9 themes that addressed the context of use for the CDS system, 9 design requirements for the prototype CDS system, and 9 design features intended to address these design requirements. Key design requirements included the preservation of clinician autonomy, incorporation of currently used heuristics, compilation of data, and increasing and communicating the level of certainty regarding the decision. After completing 3 iterative design cycles using this preliminary CDS system design solution, the design was satisfactory to the clinicians and was accepted as our prototype glaucoma CDS system.

Conclusions

We used a systematic design process based on the established User-Centered Design Process to rigorously develop a prototype glaucoma CDS system, which will be used as a starting point for a future, large-scale iterative refinement and implementation process. Clinicians who care for patients with glaucoma need CDS systems that preserve clinician autonomy, compile and present data, incorporate currently used heuristics, and increase and communicate the level of certainty regarding the decision.

Financial Disclosures

Proprietary or commercial disclosure may be found after the references.

User-Centered Design and Evaluation of Clinical Decision Support to Improve Early Peanut Introduction: Formative Study

BACKGROUND

Peanut allergy has recently become more prevalent. Peanut introduction recommendations have evolved from suggesting peanut avoidance until the age of 3 years to more recent guidelines encouraging early peanut introduction after the Learning Early about Peanut Allergy (LEAP) study in 2015. Guideline adherence is poor, leading to missed care opportunities.

OBJECTIVE

In this study, we aimed to develop a user-centered clinical decision support (CDS) tool to improve implementation of the most recent early peanut introduction guidelines in the primary care clinic setting.

METHODS

We edited the note template of the well-child check (WCC) visits at ages 4 and 6 months with CDS prompts and point-of-care education. Formative and summative usability testing were completed with pediatric residents in a simulated electronic health record (EHR). We estimated task completion rates and perceived usefulness of the CDS in summative testing, comparing a test EHR with and without the CDS.

RESULTS

Formative usability testing with the residents provided qualitative data that led to improvements in the build for both the 4-month and 6-month WCC note templates. During summative usability testing, the CDS tool significantly improved discussion of early peanut introduction at the 4-month WCC visit compared to scenarios without the CDS tool (9/15, 60% with CDS and 0/15, 0% without CDS). All providers except one at the 4-month WCC scenario gave at least an adequate score for the ease of use of the CDS tool for the history of present illness and assessment and plan sections. During the summative usability testing with the 6-month WCC new build note template, providers more commonly provided comprehensive care once obtaining a patient history concerning for an immunoglobulin E-mediated peanut reaction by placing a referral to allergy/immunology (P=.48), prescribing an epinephrine auto-injector (P=.07), instructing on how to avoid peanut products (P<.001), and providing an emergency treatment plan (P=.003) with CDS guidance. All providers gave at least an adequate score for ease of use of the CDS tool in the after-visit summary.

CONCLUSIONS

User-centered CDS improved application of early peanut introduction recommendations and comprehensive care for patients who have symptoms concerning for peanut allergy in a simulation.

Barriers to Adoption of Tailored Drug-Drug Interaction Clinical Decision Support

OBJECTIVE

Despite the benefits of the tailored drug-drug interaction (DDI) alerts and the broad dissemination strategy, the uptake of our tailored DDI alert algorithms that are enhanced with patient-specific and context-specific factors has been limited. The goal of the study was to examine barriers and health care system dynamics related to implementing tailored DDI alerts and identify the factors that would drive optimization and improvement of DDI alerts.

METHODS

We employed a qualitative research approach, conducting interviews with a participant interview guide framed based on Proctor's taxonomy of implementation outcomes and informed by the Theoretical Domains Framework. Participants included pharmacists with informatics roles within hospitals, chief medical informatics officers, and associate medical informatics directors/officers. Our data analysis was informed by the technique used in grounded theory analysis, and the reporting of open coding results was based on a modified version of the Safety-Related Electronic Health Record Research Reporting Framework.

RESULTS

Our analysis generated 15 barriers, and we mapped the interconnections of these barriers, which clustered around three entities (i.e., users, organizations, and technical stakeholders). Our findings revealed that misaligned interests regarding DDI alert performance and misaligned expectations regarding DDI alert optimizations among these entities within health care organizations could result in system inertia in implementing tailored DDI alerts.

CONCLUSION

Health care organizations primarily determine the implementation and optimization of DDI alerts, and it is essential to identify and demonstrate value metrics that health care organizations prioritize to enable tailored DDI alert implementation. This could be achieved via a multifaceted approach, such as partnering with health care organizations that have the capacity to adopt tailored DDI alerts and identifying specialists who know users' needs, liaise with organizations and vendors, and facilitate technical stakeholders' work. In the future, researchers can adopt the systematic approach to study tailored DDI implementation problems from other system perspectives (e.g., the vendors' system).

Decision-making for pediatric cervical spine imaging after blunt trauma: Investigating team dynamics in the emergency department

Objective

Cervical spine imaging decision-making for pediatric traumas is complex and multidisciplinary. Implementing a risk assessment tool has the potential to reduce variation in these decisions and unnecessary radiation exposure for pediatric patients. We sought to determine how emergency department-trauma team dynamics may affect implementation of such a tool.

Methods

We interviewed (pediatric and general emergency physicians, trauma surgeons, neurosurgeons, orthopedic surgeons and ED nurses at 21 hospitals to ascertain how team dynamics affect the pediatric cervical spine imaging decision-making process. Data were coded following a framework-driven deductive coding process and thematic analysis was used.

Results

Forty-eight physicians, advanced practice providers, and nurses from 21 hospitals (inclusive of three US regions, trauma levels I-III, and serving towns/cities of various population sizes) were interviewed. Overall, emergency physicians and trauma surgeons indicate being generally responsible for pediatric cervical spine imaging decisions. Conflict often occurs between these specialties due to differential weighting of concerns for missing an injury versus avoiding radiation exposure. Participants described a lack of trust and unclear roles regarding ownership for the final imaging decision. Nurses commonly described low psychological safety that prohibits them from participating in the decision-making process.

Conclusions

Implementation of a standardized risk assessment tool for cervical spine trauma imaging decisions must consider perspectives of both emergency medicine and trauma. Policies to define appropriate use of standardized tools within this team environment should be developed.

A Community-of-Practice-based Evaluation Methodology for Knowledge Intensive Computational Methods and its Application to Multimorbidity Decision Support

OBJECTIVE

The study has dual objectives. Our first objective (1) is to develop a community-of-practice-based evaluation methodology for knowledge-intensive computational methods. We target a whitebox analysis of the computational methods to gain insight on their functional features and inner workings. In more detail, we aim to answer evaluation questions on (i) support offered by computational methods for functional features within the application domain; and (ii) in-depth characterizations of the underlying computational processes, models, data and knowledge of the computational methods. Our second objective (2) involves applying the evaluation methodology to answer questions (i) and (ii) for knowledge-intensive clinical decision support (CDS) methods, which operationalize clinical knowledge as computer interpretable guidelines (CIG); we focus on multimorbidity CIG-based clinical decision support (MGCDS) methods that target multimorbidity treatment plans.

MATERIALS AND METHODS

Our methodology directly involves the research community of practice in (a) identifying functional features within the application domain; (b) defining exemplar case studies covering these features; and (c) solving the case studies using their developed computational methods-research groups detail their solutions and functional feature support in solution reports. Next, the study authors (d) perform a qualitative analysis of the solution reports, identifying and characterizing common themes (or dimensions) among the computational methods. This methodology is well suited to perform whitebox analysis, as it directly involves the respective developers in studying inner workings and feature support of computational methods. Moreover, the established evaluation parameters (e.g., features, case studies, themes) constitute a re-usable benchmark framework, which can be used to evaluate new computational methods as they are developed. We applied our community-of-practice-based evaluation methodology on MGCDS methods.

RESULTS

Six research groups submitted comprehensive solution reports for the exemplar case studies. Solutions for two of these case studies were reported by all groups. We identified four evaluation dimensions: detection of adverse interactions, management strategy representation, implementation paradigms, and human-in-the-loop support.Based on our whitebox analysis, we present answers to the evaluation questions (i) and (ii) for MGCDS methods.

DISCUSSION

The proposed evaluation methodology includes features of illuminative and comparison-based approaches; focusing on understanding rather than judging/scoring or identifying gaps in current methods. It involves answering evaluation questions with direct involvement of the research community of practice, who participate in setting up evaluation parameters and solving exemplar case studies. Our methodology was successfully applied to evaluate six MGCDS knowledge-intensive computational methods. We established that, while the evaluated methods provide a multifaceted set of solutions with different benefits and drawbacks, no single MGCDS method currently provides a comprehensive solution for MGCDS.

CONCLUSION

We posit that our evaluation methodology, applied here to gain new insights into MGCDS, can be used to assess other types of knowledge-intensive computational methods and answer other types of evaluation questions. Our case studies can be accessed at our GitHub repository (https://github.com/william-vw/MGCDS).

Multimodal predictions of treatment outcome in major depression: A comparison of data-driven predictors with importance ratings by clinicians

BACKGROUND

Reliable prediction models of treatment outcome in Major Depressive Disorder (MDD) are currently lacking in clinical practice. Data-driven outcome definitions, combining data from multiple modalities and incorporating clinician expertise might improve predictions.

METHODS

We used unsupervised machine learning to identify treatment outcome classes in 1060 MDD inpatients. Subsequently, classification models were created on clinical and biological baseline information to predict treatment outcome classes and compared to the performance of two widely used classical outcome definitions. We also related the findings to results from an online survey that assessed which information clinicians use for outcome prognosis.

RESULTS

Three and four outcome classes were identified by unsupervised learning. However, data-driven outcome classes did not result in more accurate prediction models. The best prediction model was targeting treatment response in its standard definition and reached accuracies of 63.9 % in the test sample, and 59.5 % and 56.9 % in the validation samples. Top predictors included sociodemographic and clinical characteristics, while biological parameters did not improve prediction accuracies. Treatment history, personality factors, prior course of the disorder, and patient attitude towards treatment were ranked as most important indicators by clinicians.

LIMITATIONS

Missing data limited the power to identify biological predictors of treatment outcome from certain modalities.

CONCLUSIONS

So far, the inclusion of available biological measures in addition to psychometric and clinical information did not improve predictive value of the models, which was overall low. Optimized biomarkers, stratified predictions and the inclusion of clinical expertise may improve future prediction models.

Developing and testing the usability, acceptability, and future implementation of the Whole Day Matters Tool and User Guide for primary care providers using think-aloud, near-live, and interview procedures

BACKGROUND

Canada's 24-Hour Movement Guidelines for Adults have shifted the focus from considering movement behaviours (i.e., physical activity, sedentary behaviour, and sleep) separately to a 24-h paradigm, which considers how they are integrated. Accordingly, primary care providers (PCPs) have the opportunity to improve their practice to promote all movement behaviours cohesively. However, PCPs have faced barriers to discussing physical activity alone (e.g., time, competing priorities, inadequate training), leading to low frequency of physical activity discussions. Consequently, discussing three movement behaviours may seem challenging. Tools to facilitate primary care discussions about physical activity have been developed and used; however, few have undergone usability testing and none have integrated all movement behaviours. Following a synthesis of physical activity, sedentary behaviour, and sleep tools for PCPs, we developed the Whole Day Matters Tool and User Guide that incorporate all movement behaviours. The present study aimed to explore PCPs' perceptions on the usability, acceptability, and future implementation of the Whole Day Matters Tool and User Guide to improve their relevancy among PCPs.

METHODS

Twenty-six PCPs were observed and audio-video recorded while using the Tool and User Guide in a think-aloud procedure, then in a near-live encounter with a mock service-user. A debriefing interview using a guide informed by Normalization Process Theory followed. Recordings were transcribed verbatim and analysed using content analysis and a critical friend to enhance rigour.

RESULTS

PCPs valued aspects of the Tool and User Guide including their structure, user-friendliness, visual appeal, and multi-behaviour focus and suggested modifications to improve usability and acceptability. Findings are further discussed in the context of Normalization Process Theory and previous literature.

CONCLUSIONS

The Tool and User Guide were revised, including adding plain language, reordering and renaming sections, reducing text, and clarifying instructions. Results also informed the addition of a Preamble and a Handout for adults accessing care (i.e., patients/clients/service-users) to explain the evidence underpinning the 24-Hour Movement Guidelines for Adults and support a person-centered approach. These four resources (i.e., Tool, User Guide, Preamble, Handout) have since undergone a consensus building process to arrive at their final versions before being disseminated into primary care practice.

Human-Centered Design of a Clinical Decision Support for Anemia Screening in Children with Inflammatory Bowel Disease

BACKGROUND

Inflammatory bowel disease (IBD) commonly leads to iron deficiency anemia (IDA). Rates of screening and treatment of IDA are often low. A clinical decision support system (CDSS) embedded in an electronic health record could improve adherence to evidence-based care. Rates of CDSS adoption are often low due to poor usability and fit with work processes. One solution is to use human-centered design (HCD), which designs CDSS based on identified user needs and context of use and evaluates prototypes for usefulness and usability.

OBJECTIVES

this study aimed to use HCD to design a CDSS tool called the IBD Anemia Diagnosis Tool, IADx.

METHODS

Interviews with IBD practitioners informed creation of a process map of anemia care that was used by an interdisciplinary team that used HCD principles to create a prototype CDSS. The prototype was iteratively tested with "Think Aloud" usability evaluation with clinicians as well as semi-structured interviews, a survey, and observations. Feedback was coded and informed redesign.

RESULTS

Process mapping showed that IADx should function at in-person encounters and asynchronous laboratory review. Clinicians desired full automation of clinical information acquisition such as laboratory trends and analysis such as calculation of iron deficit, less automation of clinical decision selection such as laboratory ordering, and no automation of action implementation such as signing medication orders. Providers preferred an interruptive alert over a noninterruptive reminder.

CONCLUSION

Providers preferred an interruptive alert, perhaps due to the low likelihood of noticing a noninterruptive advisory. High levels of desire for automation of information acquisition and analysis with less automation of decision selection and action may be generalizable to other CDSSs designed for chronic disease management. This underlines the ways in which CDSSs have the potential to augment rather than replace provider cognitive work.

The Reasons for Physicians and Pharmacists' Acceptance of Clinical Support Systems in Saudi Arabia

This research aims to identify the technological and non-technological factors influencing user acceptance of the CDSS in a group of healthcare facilities in Saudi Arabia. The study proposes an integrated model that indicates the factors to be considered when designing and evaluating CDSS. This model is developed by integrating factors from the "Fit between Individuals, Task, and Technology" (FITT) framework into the three domains of the human, organization, and technology-fit (HOT-fit) model. The resulting FITT-HOT-fit integrated model was tested using a quantitative approach to evaluate the currently implemented CDSS as a part of Hospital Information System BESTCare 2.0 in the Saudi Ministry of National Guard Health Affairs. For data collection, a survey questionnaire was conducted at all Ministry of National Guard Health Affairs hospitals. Then, the collected survey data were analyzed using Structural Equation Modeling (SEM). This analysis included measurement instrument reliability, discriminant validity, convergent validity, and hypothesis testing. Moreover, a CDSS usage data sample was extracted from the data warehouse to be analyzed as an additional data source. The results of the hypotheses test show that usability, availability, and medical history accessibility are critical factors influencing user acceptance of CDSS. This study provides prudence about healthcare facilities and their higher management to adopt CDSS.

Reinforcement learning strategies in cancer chemotherapy treatments: A review

BACKGROUND AND OBJECTIVE

Cancer is one of the major causes of death worldwide and chemotherapies are the most significant anti-cancer therapy, in spite of the emerging precision cancer medicines in the last 2 decades. The growing interest in developing the effective chemotherapy regimen with optimal drug dosing schedule to benefit the clinical cancer patients has spawned innovative solutions involving mathematical modeling since the chemotherapy regimens are administered cyclically until the futility or the occurrence of intolerable adverse events. Thus, in this present work, we reviewed the emerging trends involved in forming a computational solution from the aspect of reinforcement learning.

METHODS

Initially, this survey in-depth focused on the details of the dynamic treatment regimens from a broad perspective and then narrowed down to inspirations from reinforcement learning that were advantageous to chemotherapy dosing, including both offline reinforcement learning and supervised reinforcement learning.

RESULTS

The insights established in the chemotherapy-planning problem associated with the Reinforcement Learning (RL) has been discussed in this study. It showed that the researchers were able to widen their perspectives in comprehending the theoretical basis, dynamic treatment regimens (DTR), use of the adaptive control on DTR, and the associated RL techniques.

CONCLUSIONS

This study reviewed the recent researches relevant to the topic, and highlighted the challenges, open questions, possible solutions, and future steps in inventing a realistic solution for the aforementioned problem.

Health information communication technology evaluation frameworks for pharmacist prescribing: A systematic scoping review

BACKGROUND

Information communication technology (ICT) is instrumental in pharmacists' current practice and emerging roles. One such role is prescribing, which requires the use of clinical guidelines and documentation of decision-making, commonly via ICT. Development and refinement of ICT should be guided by evaluation frameworks that describe or measure features of ICT and its implementation. In the context of pharmacist prescribing, these evaluation frameworks should be specific to health stakeholders and the pharmacy setting.

OBJECTIVES

To identify ICT evaluation frameworks from health-related literature and review frameworks relevant to the development, implementation, and evaluation of pharmacist prescribing.

METHODS

A database search of CINAHL, Cochrane Library, EMBASE, Medline (Ovid), ProQuest, Scopus, Web of Science and grey literature was conducted, using combinations of keywords relating to 'ICT', 'utilization', 'usability', and 'evaluation framework'. Abstracts and titles were screened according to inclusion criteria. Identified evaluation frameworks were critiqued for relevance to pharmacy practice.

RESULTS

Twenty-two articles were identified, describing the development or application of 20 evaluation frameworks. None of the frameworks was developed specifically for pharmacy practice. The Technology Acceptance Model (TAM), describing use behavior, behavior intention, perceived usefulness, and perceived ease of use, was the most widely utilized framework. The Information System Success (ISS) and Human-Organization and Technology Fit (HOT-fit) are notable evaluation frameworks that address user and organizational influences in health ICT utility, and factors of both can address the limitation of TAM.

CONCLUSIONS

The findings call for development of an agile evaluation framework for the system under review; however, this can prove difficult due to the heterogenicity and complexity of the healthcare system, particularly contemporary pharmacy practice. While the TAM appears useful to evaluate user attitudes and intentions towards ICT, its relevance to ICT in contemporary community pharmacy practice requires exploration.

Australasian sonographers' knowledge, awareness, and attitudes towards the international evidence-based guidelines for the diagnosis of polycystic ovarian syndrome

Introduction/Purpose

Many guidelines have been utilised to diagnose polycystic ovarian syndrome (PCOS). The most recent are the International Evidence Based Guideline for the Assessment and Management of Polycystic Ovary Syndrome 2018 (2018 IEBG). This study aimed to assess the awareness, knowledge, and attitudes of Australasian sonographers' regarding these guidelines.

Methods

An online cross-sectional survey was disseminated to sonographers. Qualitative and quantitative questions were asked around awareness, knowledge, and attitudes towards the 2018 IEBG. Statistical and thematic analyses of the results were performed.

Results

Ninety responses were included in the final analysis. Fifty-two percent (52.2%) of participants were aware of the 2018 IEBG but only 31.1% used it in their workplaces. Fifty-eight percent (57.9%) of participants correctly identified the sonographic features that suggest PCOS, and 3.5% correctly identified all minimum recommended inclusions for reporting a gynaecological ultrasound for PCOS. Prior to being supplied the 2018 IEBG, 15.8% of participants correctly answered clinical scenario-based knowledge questions, which increased to 29.4% correctly after being supplied the guideline; however, this difference was not statistically significant. There were no statistically significant associations between demographics and knowledge of the 2018 IEBG.

Discussion

Several areas of confusion surrounding wording and interpretation of the 2018 IEBG were highlighted. Consideration should be given to barriers of implementation and strategies to overcome these.

Conclusion

More education surrounding the sonographic diagnosis of PCOS and the 2018 IEBG is needed. Scanning protocols used amongst sonographers varied, suggesting that inconsistency in sonographic diagnosis may exist. Future reviews of the 2018 IEBG should focus on reducing ambiguity in wording, which may be responsible for some of the varied interpretation of these guidelines.

External Validation of Prediction Models for Surgical Complications in People Considering Total Hip or Knee Arthroplasty Was Successful for Delirium but Not for Surgical Site Infection, Postoperative Bleeding, and Nerve Damage: A Retrospective Cohort Study

Although several models for the prediction of surgical complications after primary total hip or total knee replacement (THA and TKA, respectively) are available, only a few models have been externally validated. The aim of this study was to externally validate four previously developed models for the prediction of surgical complications in people considering primary THA or TKA. We included 2614 patients who underwent primary THA or TKA in secondary care between 2017 and 2020. Individual predicted probabilities of the risk for surgical complication per outcome (i.e., surgical site infection, postoperative bleeding, delirium, and nerve damage) were calculated for each model. The discriminative performance of patients with and without the outcome was assessed with the area under the receiver operating characteristic curve (AUC), and predictive performance was assessed with calibration plots. The predicted risk for all models varied between <0.01 and 33.5%. Good discriminative performance was found for the model for delirium with an AUC of 84% (95% CI of 0.82-0.87). For all other outcomes, poor discriminative performance was found; 55% (95% CI of 0.52-0.58) for the model for surgical site infection, 61% (95% CI of 0.59-0.64) for the model for postoperative bleeding, and 57% (95% CI of 0.53-0.61) for the model for nerve damage. Calibration of the model for delirium was moderate, resulting in an underestimation of the actual probability between 2 and 6%, and exceeding 8%. Calibration of all other models was poor. Our external validation of four internally validated prediction models for surgical complications after THA and TKA demonstrated a lack of predictive accuracy when applied in another Dutch hospital population, with the exception of the model for delirium. This model included age, the presence of a heart disease, and the presence of a disease of the central nervous system as predictor variables. We recommend that clinicians use this simple and straightforward delirium model during preoperative counselling, shared decision-making, and early delirium precautionary interventions.

Evaluating user experience with a chatbot designed as a public health response to the Covid-19 pandemic in Brazil: a mixed-methods study

BACKGROUND

The potential of chatbots for screening and monitoring COVID-19 was envisioned since the very outbreak of the disease. Chatbots can help disseminate up-to-date and trustworthy information, promote healthy social behavior and support the provision of healthcare services safely and at scale. In this scenario and in view of its far-reaching post-pandemic impact, it is critically important to evaluate user experience with this kind of application.

OBJECTIVE

To evaluate the quality of user experience with a chatbot designed in response to the COVID-19 pandemic by a large telehealth service in Brazil, focusing on an analysis of usability with real users and on an exploration of strengths and shortcomings of the chatbot as revealed in reports by participants in simulated scenarios.

METHODS

We examined a chatbot developed by a multidisciplinary team and used as a component within the workflow of a local public healthcare service. The chatbot had two core functionalities: assisting online screening of COVID-19 symptom severity and providing evidence-based information to the population. From October 2020 to January 2021, we conducted a mixed-methods approach and performed a twofold evaluation of user experience with our chatbot by two methods: (i) a post-task usability Likert-scale survey presented to all users upon concluding their interaction with the bot; and (ii) an interview with volunteer participants who engaged in a simulated interaction with the bot guided by the interviewer.

RESULTS

Usability assessment with 63 users revealed very good scores for chatbot usefulness (4.57), likelihood of being recommended (4.48), ease of use (4.44) and user satisfaction (4.38). Interviews with 15 volunteers provided insights into strengths and shortcomings in our bot. Comments on positive aspects and problems reported by users were analyzed in terms of recurrent themes. We identified six positive aspects and fifteen issues organized in two main categories: usability of the chatbot and health support offered by it, the former referring to usability of the chatbot and its interactive resources and the latter to the chatbot goal in supporting people during the pandemic through the screening process and education to users through informative content. We found six themes accounting for what people liked most about our chatbot and why they found it useful, three themes pertaining to the usability domain and three regarding health support. Besides positive feedback, our findings identified 15 types of problems producing a negative impact on users, ten of them related to the usability of the chatbot and five related to the health support it provides.

CONCLUSIONS

Our results indicate that users had an overall positive experience with the chatbot and found the health support relevant. Nonetheless, the qualitative evaluation of the chatbot indicated challenges and directions to be pursued in improving, not only our COVID chatbot, but health chatbots in general.

CLINICALTRIAL

Utilizing big data from electronic health records in pediatric clinical care

Big data has the capacity to transform both pediatric healthcare delivery and research, but its potential has yet to be fully realized. Curation of large multi-institutional datasets of high-quality data has allowed for significant advances in the timeliness of quality improvement efforts. Improved access to large datasets and computational power have also paved the way for the development of high-performing, data-driven decision support tools and precision medicine approaches. However, implementation of these approaches and tools into pediatric practice has been hindered by challenges in our ability to adequately capture the heterogeneity of the pediatric population as well as the nuanced complexities of pediatric diseases such as sepsis. Moreover, there are large gaps in knowledge and definitive evidence demonstrating the utility, usability, and effectiveness of these types of tools in pediatric practice, which presents significant challenges to provider willingness to leverage these solutions. The next wave of transformation for pediatric healthcare delivery and research through big data and sophisticated analytics will require focusing efforts on strategies to overcome cultural barriers to adoption and acceptance. IMPACT: Big data from EHRs can be used to drive improvement in pediatric clinical care. Clinical decision support, artificial intelligence, machine learning, and precision medicine can transform pediatric care using big data from the EHR. This article provides a review of barriers and enablers for the effective use of data analytics in pediatric clinical care using pediatric sepsis as a use case. The impact of this review is that it will inform influencers of pediatric care about the importance of current trends in data analytics and its use in improving outcomes of care through EHR-based strategies.

Predicting treatment outcome in depression: an introduction into current concepts and challenges

Improving response and remission rates in major depressive disorder (MDD) remains an important challenge. Matching patients to the treatment they will most likely respond to should be the ultimate goal. Even though numerous studies have investigated patient-specific indicators of treatment efficacy, no (bio)markers or empirical tests for use in clinical practice have resulted as of now. Therefore, clinical decisions regarding the treatment of MDD still have to be made on the basis of questionnaire- or interview-based assessments and general guidelines without the support of a (laboratory) test. We conducted a narrative review of current approaches to characterize and predict outcome to pharmacological treatments in MDD. We particularly focused on findings from newer computational studies using machine learning and on the resulting implementation into clinical decision support systems. The main issues seem to rest upon the unavailability of robust predictive variables and the lacking application of empirical findings and predictive models in clinical practice. We outline several challenges that need to be tackled on different stages of the translational process, from current concepts and definitions to generalizable prediction models and their successful implementation into digital support systems. By bridging the addressed gaps in translational psychiatric research, advances in data quantity and new technologies may enable the next steps toward precision psychiatry.

Nurses' and Physicians' Ideas on Initiatives for Effective Use of the Early Warning Score: A Participatory Study

Ineffective use of the early warning score (EWS) can compromise recognition and response to patients' deteriorating condition. This study explores nurses' and physicians' ideas on initiatives for supporting the effective use of the EWS in a hospital setting. Participatory workshops were conducted, and data were analyzed using content analysis. Ideas generated for integrating new functions into the EWS protocol to facilitate effective use are described. Also recommended was that all users receive training and an update on how to use the EWS score to support acceptance and confidence using the protocol and thereby increase adherence to the EWS. Further research is needed on the efficiency of incorporating nurses' clinical judgment in the EWS protocol within different specialties and the effect on adherence to the tool.

Health Care Professionals' Perspectives on Implementation, Adoption, and Maintenance of a Community Health Worker-Led Advance Care Planning and Cancer Symptom Screening Intervention: A Qualitative Study

PURPOSE

Advance care planning (ACP) and symptom screening are nationally recommended for all patients with advanced stages of cancer. Yet, routine delivery of such care remains challenging because of multilevel barriers. We hired and trained community health workers (CHWs) to assist with delivery of these services across the United States. The aim of this study was to explore health care professionals' perspectives on barriers and facilitators to these team-based approaches.

METHODS

We conducted semistructured interviews with 44 health care professionals in 21 cancer clinics in seven US cities using the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework. We recorded, transcribed, and analyzed interviews using the framework analysis approach.

RESULTS

Participants noted barriers and facilitators to implementation, adoption, and maintenance of CHW-led ACP and symptom management approaches. Participants were initially skeptical; however, they noted a positive shift in their views over time because of personal experiences and effectiveness in their clinics. There was significant variation in adoption with some using a prescriptive top-down approach and others a bottom-up approach. Most agreed that the combination of top-down and bottom-up approaches would be most efficient and effective for promoting team-based care. All participants discussed implementation and provided suggestions for maintenance including organizational support, leadership, and CHW retention.

CONCLUSION

CHW-led ACP and proactive symptom management interventions are effective and accepted by cancer care professionals at scale. Tailoring on the basis of organization and local contexts is required to ensure successful adoption, implementation, and maintenance of these effective team-based care delivery approaches.

Implementation of Machine Learning Pipelines for Clinical Practice: Development and Validation Study

BACKGROUND

Artificial intelligence (AI) technologies, such as machine learning and natural language processing, have the potential to provide new insights into complex health data. Although powerful, these algorithms rarely move from experimental studies to direct clinical care implementation.

OBJECTIVE

We aimed to describe the key components for successful development and integration of two AI technology-based research pipelines for clinical practice.

METHODS

We summarized the approach, results, and key learnings from the implementation of the following two systems implemented at a large, tertiary care children's hospital: (1) epilepsy surgical candidate identification (or epilepsy ID) in an ambulatory neurology clinic; and (2) an automated clinical trial eligibility screener (ACTES) for the real-time identification of patients for research studies in a pediatric emergency department.

RESULTS

The epilepsy ID system performed as well as board-certified neurologists in identifying surgical candidates (with a sensitivity of 71% and positive predictive value of 77%). The ACTES system decreased coordinator screening time by 12.9%. The success of each project was largely dependent upon the collaboration between machine learning experts, research and operational information technology professionals, longitudinal support from clinical providers, and institutional leadership.

CONCLUSIONS

These projects showcase novel interactions between machine learning recommendations and providers during clinical care. Our deployment provides seamless, real-time integration of AI technology to provide decision support and improve patient care.

Transition to a new nursing information system embedded with clinical decision support: a mixed-method study using the HOT-fit framework

BACKGROUND

Nursing information systems embedded with standardized nursing language and clinical decision support have been increasingly introduced in health care settings. User experience is key to the adoption of health information technologies. Despite extensive research into the user experience with nursing information systems, few studies have focused on the interaction between user, technology and organizational attributes during its implementation. Guided by the human, organization and technology-fit framework, this study aimed to investigate nurses' perceptions and experiences with transition to a new nursing information system (Care Direct) 2 years after its first introduction.

METHODS

This is a mixed-method study using an embedded design. An online survey was launched to collect nurses' self-reported use of the new system, perceived system effectiveness and experience of participation in system optimization. Twenty-two semi structured interviews were conducted with twenty nurses with clinical or administrative roles. The quantitative and qualitative data were merged using the Pillar Integration Process.

RESULTS

The average score of system use behavior was 3.76 ± 0.79. Regarding perceived system effectiveness, the score of each dimension ranged 3.07-3.34 out of 5. Despite large variations in approaches to participating in system optimization, nurses had generally positive experiences with management and technical support. Eight main categories emerged from the integrated findings, which were further condensed into three themes: perceptions on system content, structure, and functionality; perceptions on interdisciplinary and cross-level cooperation; and embracing and accepting the change.

CONCLUSIONS

Effective collaboration between clinicians, administrators and technical staff is required during system promotion to enhance system usability and user experience. Clear communication of organizational missions to staff and support from top management is needed to smooth the system implementation process and achieve broader system adoption.

Physician Compliance with Computerized Clinical Decision Support System is a Complete Intermediate Factor in the Anemia Management of Patients with End-Stage Kidney Disease on Hemodialysis: A Retrospective Electronic Health Record Observational Study (Preprint)

BACKGROUND

Previous studies on clinical decision support systems (CDSSs) for the management of renal anemia in patients with end-stage kidney disease undergoing hemodialysis have previously focused solely on the effects of the CDSS. However, the role of physician compliance in the efficacy of the CDSS remains ill-defined.

OBJECTIVE

We aimed to investigate whether physician compliance was an intermediate variable between the CDSS and the management outcomes of renal anemia.

METHODS

We extracted the electronic health records of patients with end-stage kidney disease on hemodialysis at the Far Eastern Memorial Hospital Hemodialysis Center (FEMHHC) from 2016 to 2020. FEMHHC implemented a rule-based CDSS for the management of renal anemia in 2019. We compared the clinical outcomes of renal anemia between the pre- and post-CDSS periods using random intercept models. Hemoglobin levels of 10 to 12 g/dL were defined as the on-target range. Physician compliance was defined as the concordance of adjustments of the erythropoietin-stimulating agent (ESA) between the CDSS recommendations and the actual physician prescriptions.

RESULTS

We included 717 eligible patients on hemodialysis (mean age 62.9, SD 11.6 years; male n=430, 59.9%) with a total of 36,091 hemoglobin measurements (average hemoglobin and on-target rate were 11.1, SD 1.4, g/dL and 59.9%, respectively). The on-target rate decreased from 61.3% (pre-CDSS) to 56.2% (post-CDSS) owing to a high hemoglobin percentage of >12 g/dL (pre: 21.5%; post: 29%). The failure rate (hemoglobin <10 g/dL) decreased from 17.2% (pre-CDSS) to 14.8% (post-CDSS). The average weekly ESA use of 5848 (SD 4211) units per week did not differ between phases. The overall concordance between CDSS recommendations and physician prescriptions was 62.3%. The CDSS concordance increased from 56.2% to 78.6%. In the adjusted random intercept model, the post-CDSS phase showed increased hemoglobin by 0.17 (95% CI 0.14-0.21) g/dL, weekly ESA by 264 (95% CI 158-371) units per week, and 3.4-fold (95% CI 3.1-3.6) increased concordance rate. However, the on-target rate (29%; odds ratio 0.71, 95% CI 0.66-0.75) and failure rate (16%; odds ratio 0.84, 95% CI 0.76-0.92) were reduced. After additional adjustments for concordance in the full models, increased hemoglobin and decreased on-target rate tended toward attenuation (from 0.17 to 0.13 g/dL and 0.71 to 0.73 g/dL, respectively). Increased ESA and decreased failure rate were completely mediated by physician compliance (from 264 to 50 units and 0.84 to 0.97, respectively).

CONCLUSIONS

Our results confirmed that physician compliance was a complete intermediate factor accounting for the efficacy of the CDSS. The CDSS reduced failure rates of anemia management through physician compliance. Our study highlights the importance of optimizing physician compliance in the design and implementation of CDSSs to improve patient outcomes.

Using Electronic Clinical Decision Support to Examine Vision Rehabilitation Referrals and Practice Guidelines in Ophthalmology

Purpose

To examine ophthalmologist use of an electronic health record (EHR)-based clinical decision support system (CDSS) to facilitate low vision rehabilitation (LVR) care referral.

Methods

The CDSS alert was designed to appear when best documented visual acuity was <20/40 or hemianopia or quadrantanopia diagnosis was identified during an ophthalmology encounter from November 6, 2017, to April 5, 2019. Fifteen ophthalmologists representing eight subspecialties from an academic medical center were required to respond to the referral recommendation (order, don't order). LVR referral rates and ophthalmologist user experience were assessed. Encounter characteristics associated with LVR referrals were explored using multilevel logistic regression analysis.

Results

The alert appeared for 3625 (8.9%) of 40,931 eligible encounters. The referral rate was 14.8% (535/3625). Of the 3413 encounters that met the visual acuity criterion only, patients who were worse than 20/60 were more likely to be referred, and 32.4% of referred patients were between 20/40 and 20/60. Primary reasons for deferring referrals included active medical or surgical treatment, refractive-related issues, and previous connection to LVR services. Eleven of the 13 ophthalmologists agreed that the alert was useful in identifying candidates for LVR services.

Conclusions

A CDSS for patient identification and referral offers an acceptable mechanism to apply practice guidelines and prompt ophthalmologists to facilitate LVR care. Further study is warranted to optimize ophthalmologist user experience while refining alert criteria beyond visual acuity.

Translational Relevance

The CDSS provides the framework for multi-center research to assess the development of pragmatic algorithms and standards for facilitating LVR care.

Effects of a Guideline-Informed Clinical Decision Support System Intervention to Improve Colony-Stimulating Factor Prescribing: A Cluster Randomized Clinical Trial

IMPORTANCE

Colony-stimulating factors are prescribed to patients undergoing chemotherapy to reduce the risk of febrile neutropenia. Research suggests that 55% to 95% of colony-stimulating factor prescribing is inconsistent with national guidelines.

OBJECTIVE

To examine whether a guideline-based standing order for primary prophylactic colony-stimulating factors improves use and reduces the incidence of febrile neutropenia.

DESIGN, SETTING, AND PARTICIPANTS

This cluster randomized clinical trial, the Trial Assessing CSF Prescribing Effectiveness and Risk (TrACER), involved 32 community oncology clinics in the US. Participants were adult patients with breast, colorectal, or non-small cell lung cancer initiating cancer therapy and enrolled between January 2016 and April 2020. Data analysis was performed from July to October 2021.

INTERVENTIONS

Sites were randomized 3:1 to implementation of a guideline-based primary prophylactic colony-stimulating factor standing order system or usual care. Automated orders were added for high-risk regimens, and an alert not to prescribe was included for low-risk regimens. Risk was based on National Comprehensive Cancer Network guidelines.

MAIN OUTCOMES AND MEASURES

The primary outcome was to find an increase in colony-stimulating factor use among high-risk patients from 40% to 75%, a reduction in use among low-risk patients from 17% to 7%, and a 50% reduction in febrile neutropenia rates in the intervention group. Mixed model logistic regression adjusted for correlation of outcomes within a clinic.

RESULTS

A total of 2946 patients (median [IQR] age, 59.0 [50.0-67.0] years; 2233 women [77.0%]; 2292 White [79.1%]) were enrolled; 2287 were randomized to the intervention, and 659 were randomized to usual care. Colony-stimulating factor use for patients receiving high-risk regimens was high and not significantly different between groups (847 of 950 patients [89.2%] in the intervention group vs 296 of 309 patients [95.8%] in the usual care group). Among high-risk patients, febrile neutropenia rates for the intervention (58 of 947 patients [6.1%]) and usual care (13 of 308 patients [4.2%]) groups were not significantly different. The febrile neutropenia rate for patients receiving high-risk regimens not receiving colony-stimulating factors was 14.9% (17 of 114 patients). Among the 585 patients receiving low-risk regimens, colony-stimulating factor use was low and did not differ between groups (29 of 457 patients [6.3%] in the intervention group vs 7 of 128 patients [5.5%] in the usual care group). Febrile neutropenia rates did not differ between usual care (1 of 127 patients [0.8%]) and the intervention (7 of 452 patients [1.5%]) groups.

CONCLUSIONS AND RELEVANCE

In this cluster randomized clinical trial, implementation of a guideline-informed standing order did not affect colony-stimulating factor use or febrile neutropenia rates in high-risk and low-risk patients. Overall, use was generally appropriate for the level of risk. Standing order interventions do not appear to be necessary or effective in the setting of prophylactic colony-stimulating factor prescribing.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02728596.

Strategies for developing evidence-based clinical practice guidelines to foster implementation into dental practice

BACKGROUND

Professional and other organizations, including oral health care organizations, have been developing evidence-based clinical practice guidelines (CPGs) to help providers incorporate the best available evidence into their clinical decision making. Although the rigor of guideline development has increased over time, ongoing challenges prevent the full adoption of CPGs into clinical practices that experience variability in provider expertise and opinion, patient flow pace, and use of electronic dental records. These challenges include lack of relevant evidence, failure to keep guidelines up to date, and failure to adopt strategies aimed at overcoming the barriers preventing implementation into clinical practice.

RESULTS

This article provides a brief overview of strategies that can be used to overcome common challenges to guideline adoption. Such strategies include creating evidence-based CPGs that use additional sources of evidence and methods to inform guideline development and accelerate the guideline updating and dissemination process (that is, evidence directly from clinical practice, big data, patients' values and preferences, and living guidelines) and applying implementation strategies that have been documented as improving translation of CPGs into routine clinical practice (that is, guideline implementability, implementation science, and computable guidelines).

PRACTICAL IMPLICATIONS

Adopting newer strategies for developing and translating evidence into practice could lead to improvements in patient care and population health.