Combination of Antistaphylococcal β-Lactam With Standard Therapy Compared to Standard Therapy Alone for the Treatment of Methicillin-Resistant Staphylococcus aureus Bacteremia: A Post Hoc Analysis of the CAMERA2 Trial Using a Desirability of Outcome Ranking Approach

Background

Desirability of outcome ranking (DOOR) is an emerging approach to clinical trial outcome measurement using an ordinal scale to incorporate efficacy and safety endpoints.

Methods

We applied a previously validated DOOR endpoint to a cohort of CAMERA2 trial participants with methicillin-resistant Staphylococcus aureus bacteremia (MRSAB). Participants were randomly assigned to standard therapy, or to standard therapy plus an antistaphylococcal β-lactam (combination therapy). Each participant was assigned a DOOR category, within which they were further ranked according to their hospital length of stay (LOS) and duration of intravenous antibiotic treatment. We calculated the probability and the generalized odds ratio of participants receiving combination therapy having worse outcomes than those receiving standard therapy.

Results

Participants assigned combination therapy had a 54.5% (95% confidence interval [CI], 48.9%-60.1%; P = .11) probability and a 1.2-fold odds (95% CI, .95-1.50; P = .12) of having a worse outcome than participants on standard therapy. When further ranked according to LOS and duration of antibiotic treatment, participants in the combination group had a 55.6% (95% CI, 49.5%-61.7%) and 55.3% (95% CI, 49.2%-61.4%) probability of having a worse outcome than participants in the standard treatment group, respectively.

Conclusions

When considering both efficacy and safety, treatment of MRSAB with a combination of standard therapy and a β-lactam likely results in a worse clinical outcome than standard therapy. However, a small benefit of combination therapy cannot be excluded. Most likely the toxicity of combination therapy outweighed any benefit from faster clearance of bacteremia.

Desirability of Outcome Ranking and Response Adjusted for Antibiotic Risk (DOOR/RADAR) Post Hoc Analysis Supports Equipoise for Antibiotic Initiation Strategies in Intensive Care Unit-Acquired Pneumonia

Background: Pneumonia is the most common intensive care unit (ICU)-acquired infection and source of potential sepsis in ICU populations but can be difficult to diagnose in real-time. Despite limited data, rapid initiation of antibiotic agents is endorsed by society guidelines. We hypothesized that a post hoc analysis of a recent randomized pilot study would show no difference between two antibiotic initiation strategies. Patients and Methods: The recent Trial of Antibiotic Restraint in Presumed Pneumonia (TARPP) was a pragmatic cluster-randomized pilot of antibiotic initiation strategies for patients with suspected ICU-acquired pneumonia. Participating ICUs were cluster-randomized to either an immediate initiation protocol or a specimen-initiated protocol where a gram stain was required for initiation of antibiotics. Patients in the study were divided into one of seven mutually exclusive outcome rankings (desirability of outcome ranking; DOOR): (1) Survival, No Pneumonia, No adverse events; (2) Survival, Pneumonia, No adverse events; (3) Survival, No Pneumonia, ventilator-free-alive days ≤14; (4) Survival, Pneumonia, ventilator-free-alive days ≤14; (5) Survival, No Pneumonia, Subsequent episode of suspected pneumonia; (6) Survival, Pneumonia, Subsequent episode of suspected pneumonia; and (7) Death. These rankings were further refined using the duration of antibiotics prescribed for pneumonia (response adjusted for antibiotic risk; RADAR). Results: There were 186 patients enrolled in the study. After applying the DOOR analysis, a randomly selected patient was equally likely to have a better outcome in specimen-initiated arm as in the immediate initiation arm (DOOR probability: 50.8%; 95% confidence interval [CI], 42.7%-58.9%). Outcome probabilities were similar after applying the RADAR analysis (52.5%; 95% CI, 44.2%-60.6%; p = 0.31). Conclusions: We found that patients for whom antibiotic agents were withheld until there was objective evidence (specimen-initiated group) had similar outcome rankings to patients for whom antibiotic agents were started immediately. This supports the findings of the TARPP pilot trial and provides further evidence for equipoise between these two treatment strategies.

Exploration of a Potential DOOR Endpoint for Hospital-Acquired Bacterial Pneumonia and Ventilator-Associated Bacterial Pneumonia Using Six Registrational Trials for Antibacterial Drugs

BACKGROUND

Desirability of outcome ranking (DOOR) is an innovative approach to clinical trial design and analysis that uses an ordinal ranking system to incorporate the overall risks and benefits of a therapeutic intervention into a single measurement. Here, we derived and evaluated a disease-specific DOOR endpoint for registrational trials for hospital-acquired bacterial pneumonia and ventilator-associated bacterial pneumonia (HABP/VABP).

METHODS

Through comprehensive examination of data from nearly 4,000 participants enrolled in six registrational trials for HABP/VABP submitted to the FDA between 2005-2022, we derived and applied a HABP/VABP specific endpoint. We estimated the probability that a participant assigned to the study treatment arm would have a more favorable overall DOOR or component outcome than a participant assigned to comparator.

RESULTS

DOOR distributions between treatment arms were similar in all trials. DOOR probability estimates ranged from 48.3% to 52.9% and were not statistically different. There were no significant differences between treatment arms in the component analyses. Though infectious complications and serious adverse events occurred more frequently in ventilated participants compared to non-ventilated participants, the types of events were similar.

CONCLUSIONS

Through a data-driven approach, we constructed and applied a potential DOOR endpoint for HABP/VABP trials. The inclusion of syndrome-specific events may help to better delineate and evaluate participant experiences and outcomes in future HABP/VABP trials and could help inform data collection and trial design.

The Future Ain't What It Used to Be…Out With the Old…In With the Better: Antibacterial Resistance Leadership Group Innovations

Clinical research networks conduct important studies that would not otherwise be performed by other entities. In the case of the Antibacterial Resistance Leadership Group (ARLG), such studies include diagnostic studies using master protocols, controlled phage intervention trials, and studies that evaluate treatment strategies or dynamic interventions, such as sequences of empiric and definitive therapies. However, the value of a clinical research network lies not only in the results from these important studies but in the creation of new approaches derived from collaborative thinking, carefully examining and defining the most important research questions for clinical practice, recognizing and addressing common but suboptimal approaches, and anticipating that the standard approaches of today may be insufficient for tomorrow. This results in the development and implementation of new methodologies and tools for the design, conduct, analyses, and reporting of research studies. These new methodologies directly impact the studies conducted within the network and have a broad and long-lasting impact on the field, enhancing the scientific value and efficiency of generations of research studies. This article describes innovations from the ARLG in diagnostic studies, observational studies, and clinical trials evaluating interventions for the prevention and treatment of antibiotic-resistant bacterial infections.

Exploration of a Potential Desirability of Outcome Ranking Endpoint for Complicated Intra-Abdominal Infections Using 9 Registrational Trials for Antibacterial Drugs

BACKGROUND

Desirability of outcome ranking (DOOR) is a novel approach to clinical trial design that incorporates safety and efficacy assessments into an ordinal ranking system to evaluate overall outcomes of clinical trial participants. Here, we derived and applied a disease-specific DOOR endpoint to registrational trials for complicated intra-abdominal infection (cIAI).

METHODS

Initially, we applied an a priori DOOR prototype to electronic patient-level data from 9 phase 3 noninferiority trials for cIAI submitted to the US Food and Drug Administration between 2005 and 2019. We derived a cIAI-specific DOOR endpoint based on clinically meaningful events that trial participants experienced. Next, we applied the cIAI-specific DOOR endpoint to the same datasets and, for each trial, estimated the probability that a participant assigned to the study treatment would have a more desirable DOOR or component outcome than if assigned to the comparator.

RESULTS

Three key findings informed the cIAI-specific DOOR endpoint: (1) a significant proportion of participants underwent additional surgical procedures related to their baseline infection; (2) infectious complications of cIAI were diverse; and (3) participants with worse outcomes experienced more infectious complications, more serious adverse events, and underwent more procedures. DOOR distributions between treatment arms were similar in all trials. DOOR probability estimates ranged from 47.4% to 50.3% and were not significantly different. Component analyses depicted risk-benefit assessments of study treatment versus comparator.

CONCLUSIONS

We designed and evaluated a potential DOOR endpoint for cIAI trials to further characterize overall clinical experiences of participants. Similar data-driven approaches can be utilized to create other infectious disease-specific DOOR endpoints.

The DOOR to Antibiotic Stewardship: Refining Assessments of Interventions With Desirability of Outcome Ranking

BACKGROUND

Limited sample size and disparate outcome measures can hinder the ability of antimicrobial stewardship programs to assess the utility of their quality improvement interventions. Desirability of outcome ranking (DOOR) is a novel methodology that incorporates multiple outcomes into a single value to more comprehensively compare therapeutic strategies. The objective of this study was to apply DOOR to a single center antibiotic stewardship intervention.

METHODS

A pre- and post-interventional study was conducted evaluating the impact of prospective pharmacist review of rapid molecular diagnostic testing (RDT) of blood cultures on antibiotic optimization. Outcomes included the percentage of patients who were switched to appropriate therapy, the time to appropriate therapy, and the percentage of patients who had missed de-escalation opportunities.

RESULTS

A total of 19 and 29 patients were included in the final analysis. The percentage of patients reaching appropriate therapy was 84% (16/19) and 97% ([28/29], p = 0.16) in the pre-intervention and post-intervention groups respectively. Median time to appropriate therapy was 26 hours and 36 minutes (IQR 13:05-50:45) and 22:40 (IQR 3:42-48:23, p = 0.32), respectively. One missed de-escalation opportunity was identified in the post-intervention group (0% vs 3%, p = 1.00). DOOR analysis indicated that the probability of a better outcome for the post-intervention group than the pre-intervention group was 58% (95% CI 54-62).

CONCLUSION

In this analysis, DOOR revealed a benefit that would not have been apparent with traditional outcomes assessments. Antimicrobial stewardship programs conducting quality improvement studies should consider incorporating DOOR into their methodology.

Antibiotic Usage After Procalcitonin-Guided Therapy Algorithm Implementation In A Burn Intensive Care Unit

The study investigated antibiotic utilization after the implementation of a procalcitonin (PCT)-guided antibiotic algorithm in the burn intensive care unit (BICU) to minimize antibiotic exposure appropriately. An algorithm established the ordering of an initial procalcitonin level, an additional level following 48 hours post-admission, and upon suspicion of sepsis. The primary endpoint was the percent of days on antibiotics in the BICU. Secondary endpoints were the percent of patients reinitiated on antibiotics, length of BICU and hospital stay, and 30-day mortality. Desirability of Outcome Ranking (DOOR) and Response Adjusted for Duration of Antibiotic Risk (RADAR) methodology aided in antibiotic usage evaluation. The retrospective and prospective phases involved five and seven patients in the final analysis, respectively. The median percent of days on antibiotics in the BICU was 33.3% versus 14.3% in the retrospective and prospective phases, respectively (p=0.222). Secondary outcomes evaluated were percent of patients reinitiated on antibiotics at 80.0% versus 28.6% (p=0.242), the median length of BICU stay at 38 days versus 31 days (p=0.465), the median duration of hospital stay at 39 days versus 37 days (p=0.624) and 30-day mortality of one versus zero cases (p=0.417) in the retrospective and prospective group, respectively. The probability of better DOOR with a PCT-guided antibiotic algorithm versus the control group was 95.7% (95% CI, 81.4-99.5%). The benefit of a PCT-guided antibiotic algorithm implementation cannot be determined based on the small sample size producing a lack of internal validity. Future studies warrant utilizing DOOR/RADAR to evaluate antibiotic stewardship strategies in the burn patient population.

DOOR: a prokaryotic operon database for genome analyses and functional inference

The rapid accumulation of fully sequenced prokaryotic genomes provides unprecedented information for biological studies of bacterial and archaeal organisms in a systematic manner. Operons are the basic functional units for conducting such studies. Here, we review an operon database DOOR (the Database of prOkaryotic OpeRons) that we have previously developed and continue to update. Currently, the database contains 6 975 454 computationally predicted operons in 2072 complete genomes. In addition, the database also contains the following information: (i) transcriptional units for 24 genomes derived using publicly available transcriptomic data; (ii) orthologous gene mapping across genomes; (iii) 6408 cis-regulatory motifs for transcriptional factors of some operons for 203 genomes; (iv) 3 456 718 Rho-independent terminators for 2072 genomes; as well as (v) a suite of tools in support of applications of the predicted operons. In this review, we will explain how such data are computationally derived and demonstrate how they can be used to derive a wide range of higher-level information needed for systems biology studies to tackle complex and fundamental biology questions.

Analysis of ordered composite endpoints

Composite endpoints are frequently used in clinical trials, but simple approaches, such as the time to first event, do not reflect any ordering among the endpoints. However, some endpoints, such as mortality, are worse than others. A variety of procedures have been proposed to reflect the severity of the individual endpoints such as pairwise ranking approaches, the win ratio, and the desirability of outcome ranking. When patients have different lengths of follow-up, however, ranking can be difficult and proposed methods do not naturally lead to regression approaches and require specialized software. This paper defines an ordering score O to operationalize the patient ranking implied by hierarchical endpoints. We show how differential right censoring of follow-up corresponds to multiple interval censoring of the ordering score allowing standard software for survival models to be used to calculate the nonparametric maximum likelihood estimators (NPMLEs) of different measures. Additionally, if one assumes that the ordering score is transformable to an exponential random variable, a semiparametric regression is obtained, which is equivalent to the proportional hazards model subject to multiple interval censoring. Standard software can be used for estimation. We show that the NPMLE can be poorly behaved compared to the simple estimators in staggered entry trials. We also show that the semiparametric estimator can be more efficient than simple estimators and explore how standard Cox regression maneuvers can be used to assess model fit, allow for flexible generalizations, and assess interactions of covariates with treatment. We analyze a trial of short versus long-term antiplatelet therapy using our methods.

Desirability of outcome ranking (DOOR) for comparing diagnostic tools and early therapeutic choices in patients with suspected candidemia

Desirability of outcome ranking (DOOR) has been developed for assessing desirability of outcome in interventional studies. However, its possible use in observational studies of the diagnosis and early treatment of infectious diseases has not been explored so far, and it might introduce interesting features in specific scenarios. This was a post hoc analysis of a prospective observational study in intensive care unit patients with sepsis and at risk of candidemia. The probabilities that a randomly selected patient would have a more, less, and equally cost-effective early therapeutic choice following a BDG-based diagnostic strategy rather than the empirical administration of antifungals to all patients were calculated using DOOR methods. The probability of a more cost-effective therapeutic choice following the BDG-based rather than the empirical strategy was 67.81% (95% CI 67.32-68.30), whereas the probabilities of a less and equally cost-effective early therapeutic choice were 19.68% (95% CI 19.27-20.10) and 12.50% (95% CI 12.16-12.85), respectively. The application of DOOR methods to observational studies focused on diagnosis and early treatment is a novel field that could merit further investigation.

Using Outcomes to Analyze Patients Rather than Patients to Analyze Outcomes: A Step toward Pragmatism in Benefit:risk Evaluation

In the future, clinical trials will have an increased emphasis on pragmatism, providing a practical description of the effects of new treatments in realistic clinical settings. Accomplishing pragmatism requires better summaries of the totality of the evidence in ways that clinical trials consumers---patients, physicians, insurers---find transparent and allow for informed benefit:risk decision-making. The current approach to the analysis of clinical trials is to analyze efficacy and safety separately and then combine these analyses into a benefit:risk assessment. Many assume that this will effectively describe the impact on patients. But this approach is suboptimal for evaluating the totality of effects on patients. We discuss methods for benefit:risk assessment that have greater pragmatism than methods that separately analyze efficacy and safety. These include the concepts of within-patient analyses and composite benefit:risk endpoints with a goal of understanding how to analyze one patient before trying to figure out how to analyze many. We discuss the desirability of outcome ranking (DOOR) and introduce the partial credit strategy using an example in a clinical trial evaluating the effects of a new antibiotic. As part of the example we introduce a strategy to engage patients as a resource to inform benefit:risk analyses consistent with the goal of measuring and weighing outcomes that are most important from the patient's perspective. We describe a broad vision for the future of clinical trials consistent with increased pragmatism. Greater focus on using endpoints to analyze patients rather than patients to analyze endpoints particularly in late-phase/stage clinical trials is an important part of this vision.

Desirability of Outcome Ranking (DOOR) and Response Adjusted for Duration of Antibiotic Risk (RADAR)

Clinical trials that compare strategies to optimize antibiotic use are of critical importance but are limited by competing risks that distort outcome interpretation, complexities of noninferiority trials, large sample sizes, and inadequate evaluation of benefits and harms at the patient level. The Antibacterial Resistance Leadership Group strives to overcome these challenges through innovative trial design. Response adjusted for duration of antibiotic risk (RADAR) is a novel methodology utilizing a superiority design and a 2-step process: (1) categorizing patients into an overall clinical outcome (based on benefits and harms), and (2) ranking patients with respect to a desirability of outcome ranking (DOOR). DOORs are constructed by assigning higher ranks to patients with (1) better overall clinical outcomes and (2) shorter durations of antibiotic use for similar overall clinical outcomes. DOOR distributions are compared between antibiotic use strategies. The probability that a randomly selected patient will have a better DOOR if assigned to the new strategy is estimated. DOOR/RADAR represents a new paradigm in assessing the risks and benefits of new strategies to optimize antibiotic use.