Using formative evaluation to improve uptake of a web-based tool to support antimicrobial stewardship

Identifying barriers and facilitators to successful implementation of computerized clinical decision support systems in hospitals: a NASSS framework-informed scoping review

BACKGROUND

Successful implementation and utilization of Computerized Clinical Decision Support Systems (CDSS) in hospitals is complex and challenging. Implementation science, and in particular the Nonadoption, Abandonment, Scale-up, Spread and Sustainability (NASSS) framework, may offer a systematic approach for identifying and addressing these challenges. This review aimed to identify, categorize, and describe barriers and facilitators to CDSS implementation in hospital settings and map them to the NASSS framework. Exploring the applicability of the NASSS framework to CDSS implementation was a secondary aim.

METHODS

Electronic database searches were conducted (21 July 2020; updated 5 April 2022) in Ovid MEDLINE, Embase, Scopus, PyscInfo, and CINAHL. Original research studies reporting on measured or perceived barriers and/or facilitators to implementation and adoption of CDSS in hospital settings, or attitudes of healthcare professionals towards CDSS were included. Articles with a primary focus on CDSS development were excluded. No language or date restrictions were applied. We used qualitative content analysis to identify determinants and organize them into higher-order themes, which were then reflexively mapped to the NASSS framework.

RESULTS

Forty-four publications were included. These comprised a range of study designs, geographic locations, participants, technology types, CDSS functions, and clinical contexts of implementation. A total of 227 individual barriers and 130 individual facilitators were identified across the included studies. The most commonly reported influences on implementation were fit of CDSS with workflows (19 studies), the usefulness of the CDSS output in practice (17 studies), CDSS technical dependencies and design (16 studies), trust of users in the CDSS input data and evidence base (15 studies), and the contextual fit of the CDSS with the user's role or clinical setting (14 studies). Most determinants could be appropriately categorized into domains of the NASSS framework with barriers and facilitators in the "Technology," "Organization," and "Adopters" domains most frequently reported. No determinants were assigned to the "Embedding and Adaptation Over Time" domain.

CONCLUSIONS

This review identified the most common determinants which could be targeted for modification to either remove barriers or facilitate the adoption and use of CDSS within hospitals. Greater adoption of implementation theory should be encouraged to support CDSS implementation.

Factors influencing the implementation of decision support systems for antibiotic prescription in hospitals: a systematic review

BACKGROUND

Antibiotic resistance is a major health threat. Inappropriate antibiotic use has been shown to be an important determinant of the emergence of antibiotic resistance. Decision support systems for antimicrobial management can support clinicians to optimize antibiotic prescription.

OBJECTIVE

The aim of this systematic review is to identify factors influencing the implementation of decision support systems for antibiotic prescription in hospitals.

METHODS

A systematic search of factors impeding or facilitating successful implementation of decision support systems for antibiotic prescription was performed in January 2022 in the databases PubMed, Web of Science and The Cochrane Library. Only studies were included which comprised decision support systems in hospitals for prescribing antibiotic therapy, published in English with a qualitative, quantitative or mixed-methods study design and between 2011 and 2021. Factors influencing the implementation were identified through text analysis by two reviewers.

RESULTS

A total of 14 publications were identified matching the inclusion criteria. The majority of factors relate to technological and organizational aspects of decision support system implementation. Some factors include the integration of the decision support systems into existing systems, system design, consideration of potential end-users as well as training and support for end-users. In addition, user-related factors, like user attitude towards the system, computer literacy and prior experience with the system seem to be important for successful implementation of decision support systems for antibiotic prescription in hospitals.

CONCLUSION

The results indicate a broad spectrum of factors of decision support system implementation for antibiotic prescription and contributes to the literature by identifying important organizational as well as user-related factors. Wider organizational dimensions as well as the interaction between user and technology appear important for supporting implementation.

Digital interventions for antimicrobial prescribing and monitoring: a qualitative meta-synthesis of factors influencing user acceptance

OBJECTIVE

To understand and synthesize factors influencing user acceptance of digital interventions used for antimicrobial prescribing and monitoring in hospitals.

MATERIALS AND METHODS

A meta-synthesis was conducted to identify qualitative studies that explored user acceptance of digital interventions for antimicrobial prescribing and/or monitoring in hospitals. Databases were searched and qualitative data were extracted and systematically classified using the unified theory of acceptance and use of technology (UTAUT) model.

RESULTS

Fifteen qualitative studies met the inclusion criteria. Eleven papers used interviews and four used focus groups. Most digital interventions evaluated in studies were decision support for prescribing (n = 13). Majority of perceptions were classified in the UTAUT performance expectancy domain in perceived usefulness and relative advantage constructs. Key facilitators in this domain included systems being trusted and credible sources of information, improving performance of tasks and increasing efficiency. Reported barriers were that interventions were not considered useful for all settings or patient conditions. Facilitating conditions was the second largest domain, which highlights the importance of users having infrastructure to support system use. Digital interventions were viewed positively if they were compatible with values, needs, and experiences of users.

CONCLUSIONS

User perceptions that drive users to accept and utilize digital interventions for antimicrobial prescribing and monitoring were predominantly related to performance expectations and facilitating conditions. To ensure digital interventions for antimicrobial prescribing are accepted and used, we recommend organizations ensure systems are evaluated and benefits are conveyed to users, that utility meets expectations, and that appropriate infrastructure is in place to support use.

The effectiveness of Check of Medication Appropriateness for antimicrobial stewardship: an interrupted time series analysis

OBJECTIVES

Inappropriate prescribing of antimicrobials in hospitals contributes to the emergence of resistance and adverse drug events. To support antimicrobial stewardship (AMS), clinical decision rules focusing on antimicrobial therapy were implemented in the 'Check of Medication Appropriateness' (CMA). The CMA is a hospital-wide pharmacist-led medication review service consisting of a clinical rule-based screening for potentially inappropriate prescriptions (PIPs). We aimed to investigate the impact of the CMA on antimicrobial prescribing.

METHODS

An interrupted time series study was performed at the University Hospitals Leuven. The pre-implementation cohort was exposed to standard-of-care AMS. Afterwards, an AMS-focused CMA comprising 41 specific clinical rules, targeting six AMS objectives, was implemented in the post-implementation period. A regression model was used to assess the impact of the intervention on the number of AMS-related residual PIPs between both periods. The total number of recommendations and acceptance rate was recorded for the 2 year post-implementation period.

RESULTS

Pre-implementation, a median proportion of 75% (range: 33%-100%) residual PIPs per day was observed. After the CMA intervention, the proportion was reduced to 8% (range: 0%-33%) per day. Use of clinical rules resulted in an immediate relative reduction of 86.70% (P < 0.0001) in AMS-related residual PIPs. No significant underlying time trends were observed during the study period. Post-implementation, 2790 recommendations were provided of which 81.32% were accepted.

CONCLUSIONS

We proved that the CMA approach reduced the number of AMS-related residual PIPs in a highly significant and sustained manner, with the potential to further expand the service to other AMS objectives.

Barriers and facilitators influencing medication-related CDSS acceptance according to clinicians: A systematic review

BACKGROUND

A medication-related Clinical Decision Support System (CDSS) is an application that analyzes patient data to provide assistance in medication-related care processes. Despite its potential to improve the clinical decision-making process, evidence shows that clinicians do not always use CDSSs in such a way that their potential can be fully realized. This systematic literature review provides an overview of frequently-reported barriers and facilitators for acceptance of medication-related CDSS.

MATERIALS AND METHODS

Search terms and MeSH headings were developed in collaboration with a librarian, and database searches were conducted in Medline, Scopus, Embase and Web of Science Conference Proceedings. After screening 5404 records and 140 full papers, 63 articles were included in this review. Quality assessment was performed for all 63 included articles. The identified barriers and facilitators are categorized within the Human, Organization, Technology fit (HOT-fit) model.

RESULTS

A total of 327 barriers and 291 facilitators were identified. Results show that factors most often reported were related to (a lack of) usefulness and relevance of information, and ease of use and efficiency of the system.

DISCUSSION

This review provides a valuable insight into a broad range of barriers and facilitators for using a medication-related CDSS as perceived by clinicians. The results can be used as a stepping stone in future studies developing medication-related CDSSs.

Interventions to improve antimicrobial prescribing of doctors in training (IMPACT): a realist review

Background

Interventions to improve the antimicrobial prescribing practices of doctors have been implemented widely to curtail the emergence and spread of antimicrobial resistance, but have been met with varying levels of success.

Objectives

This study aimed to generate an in-depth understanding of how antimicrobial prescribing interventions ‘work’ (or do not work) for doctors in training by taking into account the wider context in which prescribing decisions are enacted.

Design

The review followed a realist approach to evidence synthesis, which uses an interpretive, theory-driven analysis of qualitative, quantitative and mixed-methods data from relevant studies.

Setting

Primary and secondary care.

Participants

Not applicable.

Interventions

Studies related to antimicrobial prescribing for doctors in training.

Main outcome measures

Not applicable.

Data sources

EMBASE (via Ovid), MEDLINE (via Ovid), MEDLINE In-Process & Other Non-Indexed Citations (via Ovid), PsycINFO (via Ovid), Web of Science core collection limited to Science Citation Index Expanded (SCIE) and Conference Proceedings Citation Index – Science (CPCI-S) (via Thomson Reuters), Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Database of Systematic Reviews, the Health Technology Assessment (HTA) database (all via The Cochrane Library), Applied Social Sciences Index and Abstracts (ASSIA) (via ProQuest), Google Scholar (Google Inc., Mountain View, CA, USA) and expert recommendations.

Review methods

Clearly bounded searches of electronic databases were supplemented by citation tracking and grey literature. Following quality standards for realist reviews, the retrieved articles were systematically screened and iteratively analysed to develop theoretically driven explanations. A programme theory was produced with input from a stakeholder group consisting of practitioners and patient representatives.

Results

A total of 131 articles were included. The overarching programme theory developed from the analysis of these articles explains how and why doctors in training decide to passively comply with or actively follow (1) seniors’ prescribing habits, (2) the way seniors take into account prescribing aids and seek the views of other health professionals and (3) the way seniors negotiate patient expectations. The programme theory also explains what drives willingness or reluctance to ask questions about antimicrobial prescribing or to challenge the decisions made by seniors. The review outlines how these outcomes result from complex inter-relationships between the contexts of practice doctors in training are embedded in (hierarchical relationships, powerful prescribing norms, unclear roles and responsibilities, implicit expectations about knowledge levels and application in practice) and the mechanisms triggered in these contexts (fear of criticism and individual responsibility, reputation management, position in the clinical team and appearing competent). Drawing on these findings, we set out explicit recommendations for optimal tailoring, design and implementation of antimicrobial prescribing interventions targeted at doctors in training.

Limitations

Most articles included in the review discussed hospital-based, rather than primary, care. In cases when few data were available to fully capture all the nuances between context, mechanisms and outcomes, we have been explicit about the strength of our arguments.

Conclusions

This review contributes to our understanding of how antimicrobial prescribing interventions for doctors in training can be better embedded in the hierarchical and interprofessional dynamics of different health-care settings.

Future work

More work is required to understand how interprofessional support for doctors in training can contribute to appropriate prescribing in the context of hierarchical dynamics.

Study registration

This study is registered as PROSPERO CRD42015017802.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Social and professional influences on antimicrobial prescribing for doctors-in-training: a realist review

Background

Antimicrobial resistance has led to widespread implementation of interventions for appropriate prescribing. However, such interventions are often adopted without an adequate understanding of the challenges facing doctors-in-training as key prescribers.

Methods

The review followed a realist, theory-driven approach to synthesizing qualitative, quantitative and mixed-methods literature. Consistent with realist review quality standards, articles retrieved from electronic databases were systematically screened and analysed to elicit explanations of antimicrobial prescribing behaviours. These explanations were consolidated into a programme theory drawing on social science and learning theory, and shaped though input from patients and practitioners.

Results

By synthesizing data from 131 articles, the review highlights the complex social and professional dynamics underlying antimicrobial prescribing decisions of doctors-in-training. The analysis shows how doctors-in-training often operate within challenging contexts (hierarchical relationships, powerful prescribing norms, unclear roles and responsibilities, implicit expectations about knowledge levels, uncertainty about application of knowledge in practice) where they prioritize particular responses (fear of criticism and individual responsibility, managing one's reputation and position in the team, appearing competent). These complex dynamics explain how and why doctors-in-training decide to: (i) follow senior clinicians' prescribing habits; (ii) take (or not) into account prescribing aids, advice from other health professionals or patient expectations; and (iii) ask questions or challenge decisions. This increased understanding allows for targeted tailoring, design and implementation of antimicrobial prescribing interventions.

Conclusions

This review contributes to a better understanding of how antimicrobial prescribing interventions for doctors-in-training can be embedded more successfully in the hierarchical and inter-professional dynamics of different healthcare settings.

A systematic review of clinical decision support systems for antimicrobial management: are we failing to investigate these interventions appropriately?

OBJECTIVES

Clinical decision support systems (CDSS) for antimicrobial management can support clinicians to optimize antimicrobial therapy. We reviewed all original literature (qualitative and quantitative) to understand the current scope of CDSS for antimicrobial management and analyse existing methods used to evaluate and report such systems.

METHOD

PRISMA guidelines were followed. Medline, EMBASE, HMIC Health and Management and Global Health databases were searched from 1 January 1980 to 31 October 2015. All primary research studies describing CDSS for antimicrobial management in adults in primary or secondary care were included. For qualitative studies, thematic synthesis was performed. Quality was assessed using Integrated quality Criteria for the Review Of Multiple Study designs (ICROMS) criteria. CDSS reporting was assessed against a reporting framework for behaviour change intervention implementation.

RESULTS

Fifty-eight original articles were included describing 38 independent CDSS. The majority of systems target antimicrobial prescribing (29/38;76%), are platforms integrated with electronic medical records (28/38;74%), and have a rules-based infrastructure providing decision support (29/38;76%). On evaluation against the intervention reporting framework, CDSS studies fail to report consideration of the non-expert, end-user workflow. They have narrow focus, such as antimicrobial selection, and use proxy outcome measures. Engagement with CDSS by clinicians was poor.

CONCLUSION

Greater consideration of the factors that drive non-expert decision making must be considered when designing CDSS interventions. Future work must aim to expand CDSS beyond simply selecting appropriate antimicrobials with clear and systematic reporting frameworks for CDSS interventions developed to address current gaps identified in the reporting of evidence.