Study protocol for a multicentre, cluster randomised, superiority trial evaluating the impact of computerised decision support, audit and feedback on antibiotic use: the COMPuterized Antibiotic Stewardship Study (COMPASS)

Impact of interactive computerised decision support for hospital antibiotic use (COMPASS): an open-label, cluster-randomised trial in three Swiss hospitals

Background

Computerised decision-support systems (CDSSs) for antibiotic stewardship could help to assist physicians in the appropriate prescribing of antibiotics. However, high-quality evidence for their effect on the quantity and quality of antibiotic use remains scarce. The aim of our study was to assess whether a computerised decision support for antimicrobial stewardship combined with feedback on prescribing indicators can reduce antimicrobial prescriptions for adults admitted to hospital.

Methods

The Computerised Antibiotic Stewardship Study (COMPASS) was a multicentre, cluster-randomised, parallel-group, open-label superiority trial that aimed to assess whether a multimodal computerised antibiotic-stewardship intervention is effective in reducing antibiotic use for adults admitted to hospital. After pairwise matching, 24 wards in three Swiss tertiary-care and secondary-care hospitals were randomised (1:1) to the CDSS intervention or to standard antibiotic stewardship measures using an online random sequence generator. The multimodal intervention consisted of a CDSS providing support for choice, duration, and re-evaluation of antimicrobial therapy, and feedback on antimicrobial prescribing quality. The primary outcome was overall systemic antibiotic use measured in days of therapy per admission, using adjusted-hurdle negative-binomial mixed-effects models. The analysis was done by intention to treat and per protocol. The study was registered with ClinicalTrials.gov (identifier NCT03120975).

Findings

24 clusters (16 at Geneva University Hospitals and eight at Ticino Regional Hospitals) were eligible and randomly assigned to control or intervention between Oct 1, 2018, and Dec 31, 2019. Overall, 4578 (40·2%) of 11 384 admissions received antibiotic therapy in the intervention group and 4142 (42·8%) of 9673 in the control group. The unadjusted overall mean days of therapy per admission was slightly lower in the intervention group than in the control group (3·2 days of therapy per admission, SD 6·2, vs 3·5 days of therapy per admission, SD 6·8; p<0·0001), and was similar among patients receiving antibiotics (7·9 days of therapy per admission, SD 7·6, vs 8·1 days of therapy per admission, SD 8·4; p=0·50). After adjusting for confounders, there was no statistically significant difference between groups for the odds of an admission receiving antibiotics (odds ratio [OR] for intervention vs control 1·12, 95% CI 0·94–1·33). For admissions with antibiotic exposure, days of therapy per admission were also similar (incidence rate ratio 0·98, 95% CI 0·90–1·07). Overall, the CDSS was used at least once in 3466 (75·7%) of 4578 admissions with any antibiotic prescription, but from the first day of antibiotic treatment for only 1602 (58·9%) of 2721 admissions in Geneva. For those for whom the CDSS was not used from the first day, mean time to use of CDSS was 8·9 days. Based on the manual review of 1195 randomly selected charts, transition from intravenous to oral therapy was significantly more frequent in the intervention group after adjusting for confounders (154 [76·6%] of 201 vs 187 [87%] of 215, +10·4%; OR 1·9, 95% CI 1·1–3·3). Consultations by infectious disease specialists were less frequent in the intervention group (388 [13·4%] of 2889) versus the control group (405 [16·9%] of 2390; OR 0·84, 95% CI 0·59–1·25).

Interpretation

An integrated multimodal computerised antibiotic stewardship intervention did not significantly reduce overall antibiotic use, the primary outcome of the study. Contributing factors were probably insufficient uptake, a setting with relatively low antibiotic use at baseline, and delays between ward admission and first CDSS use.

Funding

Swiss National Science Foundation.

Translations

For the French and Italian translations of the abstract see Supplementary Materials section.

Machine Learning and Antibiotic Management

Machine learning and cluster analysis applied to the clinical setting of an intensive care unit can be a valuable aid for clinical management, especially with the increasing complexity of clinical monitoring. Providing a method to measure clinical experience, a proxy for that automatic gestalt evaluation that an experienced clinician sometimes effortlessly, but often only after long, hard consideration and consultation with colleagues, relies upon for decision making, is what we wanted to achieve with the application of machine learning to antibiotic therapy and clinical monitoring in the present work. This is a single-center retrospective analysis proposing methods for evaluation of vitals and antimicrobial therapy in intensive care patients. For each patient included in the present study, duration of antibiotic therapy, consecutive days of treatment and type and combination of antimicrobial agents have been assessed and considered as single unique daily record for analysis. Each parameter, composing a record was normalized using a fuzzy logic approach and assigned to five descriptive categories (fuzzy domain sub-sets ranging from “very low” to “very high”). Clustering of these normalized therapy records was performed, and each patient/day was considered to be a pertaining cluster. The same methodology was used for hourly bed-side monitoring. Changes in patient conditions (monitoring) can lead to a shift of clusters. This can provide an additional tool for assessing progress of complex patients. We used Fuzzy logic normalization to descriptive categories of parameters as a form nearer to human language than raw numbers.

In search of lost time: A timing evaluation of antimicrobial prescribing with and without a computerized decision support system using clinical vignettes

BACKGROUND

We implemented a computerized decision support system (CDSS) integrated in the in-house computerized physician order entry (CPOE) system to assist physicians with antimicrobial prescribing decisions in the context of the multicenter cluster-randomized COMPASS trial (NCT03120975). Some physicians in the intervention wards complained about the perceived extra-time associated with the use of the CDSS compared with routine prescribing through CPOE. The aim of this study was to compare the time needed to prescribe antimicrobials with and without the CDSS.

METHODS

Physicians with and without previous experience with the COMPASS CDSS working at our hospital in Geneva, Switzerland, were recruited to prescribe antimicrobials using clinical vignettes. Physicians without experience received a brief explanation of the CDSS. Each physician received 2 groups of 5-7 clinical vignettes randomly selected from a pool of 28. Each group of vignettes included prescriptions with different levels of complexity (empiric versus targeted or pre-defined treatment, dose adjustment for renal function, oral switch, treatments for which COMPASS does not provide recommendations or where a deviation was necessary). Prescriptions were completed using the standard CPOE (first set), then using COMPASS (second set). A print version of the local antimicrobial guidelines was available for consultation. Time to complete each prescription was recorded (including time needed to consult paper guidelines). The Mann-Whitney test was used for comparisons. Consultation of guidelines booklet and concordance with local guidelines were assessed.

RESULTS

Twenty-five physicians were recruited. Thirteen (52%) had previously used COMPASS. Among them, 11 (85%) estimated the extra-time being above 1 min. We evaluated a total of 296 vignettes. Overall, the median time to complete a prescription was 55.5 s (IQR 38-86) using COMPASS and 50 s (IQR 31-88) using the standard CPOE (p = 0.24). Concordance of prescriptions with local guidelines was similar with the 2 systems (127/148, 85.8% for both), but consultation of paper guidelines was more frequent when prescribing without the CDSS (49.3% (73/148) vs 22.3% (33/148)).

CONCLUSIONS

The increased time required for prescribing using COMPASS is overestimated by end-users. Information collected in the study will be used to streamline the prescribing process via COMPASS and increase acceptance.

How to Develop and Implement a Computerized Decision Support System Integrated for Antimicrobial Stewardship? Experiences From Two Swiss Hospital Systems

Background: Computerized decision support systems (CDSS) provide new opportunities for automating antimicrobial stewardship (AMS) interventions and integrating them in routine healthcare. CDSS are recommended as part of AMS programs by international guidelines but few have been implemented so far. In the context of the publicly funded COMPuterized Antibiotic Stewardship Study (COMPASS), we developed and implemented two CDSSs for antimicrobial prescriptions integrated into the in-house electronic health records of two public hospitals in Switzerland. Developing and implementing such systems was a unique opportunity for learning during which we faced several challenges. In this narrative review we describe key lessons learned.

Recommendations: (1) During the initial planning and development stage, start by drafting the CDSS as an algorithm and use a standardized format to communicate clearly the desired functionalities of the tool to all stakeholders. (2) Set up a multidisciplinary team bringing together Information Technologies (IT) specialists with development expertise, clinicians familiar with “real-life” processes in the wards and if possible, involve collaborators having knowledge in both areas. (3) When designing the CDSS, make the underlying decision-making process transparent for physicians and start simple and make sure to find the right balance between force and persuasion to ensure adoption by end-users. (4) Correctly assess the clinical and economic impact of your tool, therefore try to use standardized terminologies and limit the use of free text for analysis purpose. (5) At the implementation stage, plan usability testing early, develop an appropriate training plan suitable to end users' skills and time-constraints and think ahead of additional challenges related to the study design that may occur (such as a cluster randomized trial). Stay also tuned to react quickly during the intervention phase. (6) Finally, during the assessment stage plan ahead maintenance, adaptation and related financial challenges and stay connected with institutional partners to leverage potential synergies with other informatics projects.

Effective Antimicrobial StewaRdship StrategIES (ARIES): Cluster Randomized Trial of Computerized Decision Support System and Prospective Review and Feedback

Background

Prospective review and feedback (PRF) of antibiotic prescriptions and compulsory computerized decision support system (CDSS) are 2 strategies of antimicrobial stewardship. There are limited studies investigating their combined effects. We hypothesized that the use of on-demand (voluntary) CDSS would achieve similar patient outcomes compared with automatically triggered (compulsory) CDSS whenever broad-spectrum antibiotics are ordered.

Methods

A parallel-group, 1:1 block cluster randomized crossover study was conducted in 32 medical and surgical wards from March to August 2017. CDSS use for piperacillin-tazobactam or carbapenem in the intervention clusters was at the demand of the doctor, while in the control clusters CDSS use was compulsory. PRF was continued for both arms. The primary outcome was 30-day mortality.

Results

Six hundred forty-one and 616 patients were randomized to voluntary and compulsory CDSS, respectively. There were no differences in 30-day mortality (hazard ratio [HR], 0.87; 95% CI, 0.67–1.12), re-infection and re-admission rates, antibiotic duration, length of stay, or hospitalization cost. The proportion of patients receiving PRF recommendations was not significantly lower in the voluntary CDSS arm (62 [10%] vs 81 [13%]; P = .05). Appropriate indication of antibiotics was high in both arms (351/448 [78%] vs 330/433 [74%]; P = .18). However, in geriatric medicine patients where antibiotic appropriateness was <50%, prescription via compulsory CDSS resulted in a shorter length of stay and lower hospitalization cost.

Conclusions

Voluntary broad-spectrum antibiotics with PRF via CDSS did not result in differing clinical outcomes, antibiotic duration, or length of stay. However, in the setting of low antibiotic appropriateness, compulsory CDSS may be beneficial.

Machine learning for the prediction of antimicrobial stewardship intervention in hospitalized patients receiving broad-spectrum agents

OBJECTIVE

A significant proportion of inpatient antimicrobial prescriptions are inappropriate. Post-prescription review with feedback has been shown to be an effective means of reducing inappropriate antimicrobial use. However, implementation is resource intensive. Our aim was to evaluate the performance of traditional statistical models and machine-learning models designed to predict which patients receiving broad-spectrum antibiotics require a stewardship intervention.

METHODS

We performed a single-center retrospective cohort study of inpatients who received an antimicrobial tracked by the antimicrobial stewardship program. Data were extracted from the electronic medical record and were used to develop logistic regression and boosted-tree models to predict whether antibiotic therapy required stewardship intervention on any given day as compared to the criterion standard of note left by the antimicrobial stewardship team in the patient's chart. We measured the performance of these models using area under the receiver operating characteristic curves (AUROC), and we evaluated it using a hold-out validation cohort.

RESULTS

Both the logistic regression and boosted-tree models demonstrated fair discriminatory power with AUROCs of 0.73 (95% confidence interval [CI], 0.69-0.77) and 0.75 (95% CI, 0.72-0.79), respectively (P = .07). Both models demonstrated good calibration. The number of patients that would need to be reviewed to identify 1 patient who required stewardship intervention was high for both models (41.7-45.5 for models tuned to a sensitivity of 85%).

CONCLUSIONS

Complex models can be developed to predict which patients require a stewardship intervention. However, further work is required to develop models with adequate discriminatory power to be applicable to real-world antimicrobial stewardship practice.

Principles and practice of antibiotic stewardship in the management of diabetic foot infections

PURPOSE OF REVIEW

Systemic antibiotic therapy in persons with a diabetic foot infection (DFI) is frequent, increasing the risk of promoting resistance to common pathogens. Applying principles of antibiotic stewardship may help avoid this problem.

RECENT FINDINGS

We performed a systematic review of the literature, especially seeking recently published studies, for data on the role and value of antibiotic stewardship (especially reducing the spectrum and duration of antibiotic therapy) in community and hospital populations of persons with a DFI.

SUMMARY

We found very few publications specifically concerning antibiotic stewardship in persons with a DFI. The case-mix of these patients is substantial and infection plays only one part among several chronic problems. As with other types of infections, attempting to prevent infections and avoiding or reducing the spectrum and duration of antibiotic therapy are perhaps the best ways to reduce antibiotic prescribing in the DFI population. The field is complex and necessitates knowledge over the current scientific literature and clinical experience. On a larger scale, clinical pathways, guidelines, and recommendations are additionally supportive.

Management of diabetic foot infections in the light of recent literature and new international guidelines

Introduction: In May 2019 the International Working Group on the Diabetic Foot (IWGDF) launched their quadrennially updated guidelines on the management of diabetic foot infections (DFIs). Concomitantly, the number of new publications regarding DFI increased.Areas covered: The IWGDF committee developed and addressed key questions and produced evidence-based recommendations related to diagnosing and treating DFIs. This narrative review provides an overview of this new guideline and also of other recently published literature in the field of DFIs.Expert opinion: The 2019 IWGDF guidelines provide an authoritative, international, evidence-based approach to diagnosing and treating DFIs. The 27 recommendations are supported by systematic reviews of both diagnosis and interventions. Our review of this guideline, along with other recent publications in the field, allows us to offer state-of-the-art guidance for caring for these difficult infections. As the evidence base for management of DFIs remains suboptimal, we need further research to improve the management of DFIs.