Uncertainty and equipoise: at interplay between epistemology, decision making and ethics

Evidence and Decision-Making

Today, every country struggles to provide adequate health care to its citizens. Globally, an average of $8.3 trillion or 10% of gross domestic product (GDP) is annually spent on health services. In 2019, the USA spent nearly 18% ($3.2 trillion) of its GDP on health care, projected to reach $6.2 trillion by 2028.

Guiding Efficient, Effective, and Patient-Oriented Electrolyte Replacement in Critical Care: An Artificial Intelligence Reinforcement Learning Approach

Both provider- and protocol-driven electrolyte replacement have been linked to the over-prescription of ubiquitous electrolytes. Here, we describe the development and retrospective validation of a data-driven clinical decision support tool that uses reinforcement learning (RL) algorithms to recommend patient-tailored electrolyte replacement policies for ICU patients. We used electronic health records (EHR) data that originated from two institutions (UPHS; MIMIC-IV). The tool uses a set of patient characteristics, such as their physiological and pharmacological state, a pre-defined set of possible repletion actions, and a set of clinical goals to present clinicians with a recommendation for the route and dose of an electrolyte. RL-driven electrolyte repletion substantially reduces the frequency of magnesium and potassium replacements (up to 60%), adjusts the timing of interventions in all three electrolytes considered (potassium, magnesium, and phosphate), and shifts them towards orally administered repletion over intravenous replacement. This shift in recommended treatment limits risk of the potentially harmful effects of over-repletion and implies monetary savings. Overall, the RL-driven electrolyte repletion recommendations reduce excess electrolyte replacements and improve the safety, precision, efficacy, and cost of each electrolyte repletion event, while showing robust performance across patient cohorts and hospital systems.

IDENTIFICATION OF THRESHOLD FOR LARGE (DRAMATIC) EFFECTS THAT WOULD OBVIATE RANDOMIZED TRIALS IS NOT POSSIBLE

OBJECTIVE

To analyze distribution of "dramatic", large treatment effects.

STUDY DESIGN & SETTING

Pareto distribution modeling of previously reported cohorts of 3,486 randomized trials (RCTs) that enrolled 1,532,459 patients and 730 non-randomized studies (NRS) enrolling 1,650,658 patients.

RESULTS

We calculated the Pareto α parameter, which determines the tail of the distribution for various starting points of distribution [odds ratio_min (OR_min)]. In default analysis using all data at OR_min ≥1, Pareto distribution fit well to the treatment effects of RCTs favoring the new treatments (P = 0.21, Kolmogorov-Smirnov test) with best α = 2.32. For NRS, Pareto fit for OR_min ≥2 with best α = 1.91. For RCTs, theoretical 99th percentile OR was 32.7. The actual 99th percentile OR was 25; which converted into relative risk (RR) = 7.1. The maximum observed effect size was OR = 121 (RR = 11.45). For NRS, theoretical 99th percentile was OR = 315. The actual 99th percentile OR was 294 (RR = 13). The maximum observed effect size was OR = 1473 (RR = 66).

CONCLUSIONS

The effects sizes observed in RCTs and NRS considerably overlap. Large effects are rare and there is no clear threshold for dramatic effects that would obviate future RCTs.

Ethics of uncertainty

Uncertainty is inherent in clinical medicine. However, just because absolute certainty is unachievable does not mean that rational and optimal decisions cannot be made. It is argued that we need to distinguish legitimate from illegitimate scientific uncertainties that are generated by manufacturing doubts aiming to create mis- and disinformation. The attempt to create doubts implies that actions under uncertainties are impossible. Such a belief ultimately harms public, which requires reasoned actions within a context of genuine scientific and medical uncertainties. The latter indicates that rational decisions, even in the absence of guaranteed absolute certainty, are not only possible but, on average, beneficial both for society and individuals.

Ethics of HIV cure research: an unfinished agenda

BACKGROUND

The pursuit of a cure for HIV is a high priority for researchers, funding agencies, governments and people living with HIV (PLWH). To date, over 250 biomedical studies worldwide are or have been related to discovering a safe, effective, and scalable HIV cure, most of which are early translational research and experimental medicine. As HIV cure research increases, it is critical to identify and address the ethical challenges posed by this research.

METHODS

We conducted a scoping review of the growing HIV cure research ethics literature, focusing on articles published in English peer-reviewed journals from 2013 to 2021. We extracted and summarized key developments in the ethics of HIV cure research. Twelve community advocates actively engaged in HIV cure research provided input on this summary and suggested areas warranting further ethical inquiry and foresight via email exchange and video conferencing.

DISCUSSION

Despite substantial scholarship related to the ethics of HIV cure research, additional attention should focus on emerging issues in six categories of ethical issues: (1) social value (ongoing and emerging biomedical research and scalability considerations); (2) scientific validity (study design issues, such as the use of analytical treatment interruptions and placebos); (3) fair selection of participants (equity and justice considerations); (4) favorable benefit/risk balance (early phase research, benefit-risk balance, risk perception, psychological risks, and pediatric research); (5) informed consent (attention to language, decision-making, informed consent processes and scientific uncertainty); and (6) respect for enrolled participants and community (perspectives of people living with HIV and affected communities and representation).

CONCLUSION

HIV cure research ethics has an unfinished agenda. Scientific research and bioethics should work in tandem to advance ethical HIV cure research. Because the science of HIV cure research will continue to rapidly advance, ethical considerations of the major themes we identified will need to be revisited and refined over time.

Microbiota changes and intestinal microbiota transplantation in liver diseases and cirrhosis

Patients with chronic liver disease and cirrhosis demonstrate a global mucosal immune impairment, which is associated with altered gut microbiota composition and functionality. These changes progress along with the advancing degree of cirrhosis and can be linked with hepatic encephalopathy, infections and even prognostication independent of clinical biomarkers. Along with compositional changes, functional alterations to the microbiota, related to short-chain fatty acids, bioenergetics and bile acid metabolism, are also associated with cirrhosis progression and outcomes. Altering the functional and structural profile of the microbiota is partly achieved by medications used in patients with cirrhosis such as rifaximin, lactulose, proton pump inhibitors and other antibiotics. However, the role of faecal or intestinal microbiota transplantation is increasingly being recognised. Herein, we review the challenges, opportunities and road ahead for the appropriate and safe use of intestinal microbiota transplantation in liver disease.

Veterans Healthcare Administration providers' attitudes and perceptions regarding pragmatic trials embedded at the point of care

Background The Veterans Healthcare Administration (VA) is implementing an adaptation of a pragmatic trial program, Point of Care Research (POC-R). The goal of POC-R is to embed research into clinical practice, contributing to a Learning Healthcare System. Provider acceptance and participation in POC-R is essential to its successful implementation. The purpose of this study is to evaluate provider's perceptions and beliefs regarding the POC-R program. Methods Provider focus groups and interviews were conducted at seven VA medical facilities involving 62 providers. A semi-structured script was used that included descriptions of four use cases and targeted questions regarding perceptions, concerns, and attitudes about the POC-R program. Sessions were audio-taped, de-identified, transcribed, and analyzed using systematic qualitative techniques to create response categories and overarching themes. Results The emergent themes were as follows: (1) POC-R is a valuable component of evidence-based practice, providing an opportunity to base clinical practice on more generalizable evidence as well as providing tools to improve local practice; (2) POC-R highlights the tension between the need for autonomy of practice and compliance with protocols; (3) POC-R may create increased time and burden resulting from added research responsibilities; (4) concern about the scientific validity and reliability of results; (5) potential for a negative impact on the provider-patient relationship; and (6) uncertainty regarding what constitutes equipoise, given differences in provider knowledge and preferences. Despite substantive concerns, barriers were generally felt to be solvable. Implementation should include provider education, careful attention to workflow for all arms of the study, inclusion of the entire team, and adequate oversight. Limitations The study design is qualitative with limited implications for causal inference. Participants are from the VA and may not be representative of other clinicians. Conclusion VA providers are supportive of the importance and value of pragmatic trials in general and of POC-R in particular. However, providers have significant concerns regarding the burden, ethics, and evidence regarding equipoise. Results are discussed in terms of implementation recommendations.

When are clinical trials beneficial for study patients and future patients? A factorial vignette-based survey of institutional review board members

OBJECTIVE

The ethicists believe that the goal of clinical research is to benefit future and not current (trial) patients. Many clinicians believe that the clinical trial enrolment offers best management for their patients. The objective of our study was to identify the situations when a clinical trial is beneficial for the patients enrolled in the trial and future patients.

DESIGN

Factorial vignette-based cross-sectional survey via the internet.

PARTICIPANTS

Institutional review board (IRB) members of the US Medical Schools.

MAIN OUTCOME MEASURES

Each participant was invited to review 9 clinical vignettes related to (1) study approval and (2) the assessment if the study is designed to help future or current patients more.

RESULTS

A total of 232 IRB members from 42 institutions participated. When considering approval of the trial, we found that uncertainty about treatment effects (OR=1.13; 95% CI 1.08 to 1.19) and requirement for continuation of standard therapy (OR=3.84; 95% CI 2.7 to 5.55) were the only statistically significant factors affecting IRB members' decisions to approve the study. The odds of IRB members approving a trial increased when a trial proposed to enrol patients with life-threatening versus chronic debilitating disease (OR=2.04; 95% CI 1.47 to 2.86). We also found that similar factors affected judgements related to the assessment whether the trial will benefit future or current patients more-(1) future patients: uncertainty (OR=1.27; 95% CI 1.18 to 1.37); continuation of standard treatment (OR=1.66; 95% CI 1.07 to 2.56); seriousness of condition (OR=1.78; 95% CI 1.15 to 2.28); (2) current patients: uncertainty (OR=1.54; 95% CI 1.4 to 1.7); continuation of standard therapy (OR=2.17; 95% CI 1.39 to 3.44).

CONCLUSIONS

IRB members view the proposed studies to be beneficial for current patients and future patients if there is uncertainty regarding treatment effects and if studies do not require stopping the current treatment. This finding supports the design and use of pragmatic trials which may reduce therapeutic misconception and improve trial enrolment, speeding up therapeutic discoveries.

Personalized prescription feedback to reduce antibiotic overuse in primary care: rationale and design of a nationwide pragmatic randomized trial

Background

Antimicrobial resistance has become a serious worldwide public health problem and is associated with antibiotic overuses. Whether personalized prescription feedback to high antibiotic prescribers using routinely collected data can lower antibiotic use in the long run is unknown.

Methods

We describe the design and rationale of a nationwide pragmatic randomized controlled trial enrolling 2900 primary care physicians in Switzerland with high antibiotic prescription rates based on national reimbursement claims data. About 1450 physicians receive quarterly postal and online antibiotic prescription feedback over 24 months allowing a comparison of the individual prescription rates with peers. Initially, they also receive evidence based treatment guidelines. The 1450 physicians in the control group receive no information. The primary outcome is the amount of antibiotics prescribed over a one year-period, measured as defined daily doses per 100 consultations. Other outcomes include the amount of antibiotics prescribed to specific age groups (<6, 6 to 18, 19 to 65, >65 years), to male and female patients, in addition to prescriptions of specific antibiotic groups. Further analyses address disease-specific quality indicators for outpatient antibiotic prescriptions, the acceptance of the intervention, and the impact on costs.

Discussion

This trial investigates whether continuous personalized prescription feedback on a health system level using routinely collected health data reduces antibiotic overuse. The feasibility and applicability of a web-based interface for communication with primary care physicians is further assessed.

Trial registration

ClinTrials.gov NCT01773824 (Date registered: August 24, 2012).

Uncertainty about effects is a key factor influencing institutional review boards' approval of clinical studies

PURPOSE

To investigate factors, which influence institutional review boards' (IRBs') decision to approve or not approve clinical studies, a nationwide vignette-based online survey of IRB members was conducted.

METHODS

A factorial design was used, whereby seven aspects of each hypothetical study were randomly varied in 15 phrases in each vignette to produce unique vignettes. Participants indicated the degree of study approval and described factors influencing approval decision. Qualitative responses were thematically content analyzed.

RESULTS

Sixteen themes were obtained from 208 participants from 42 institutions. Uncertainty, adherence, study design, and harms were frequently and intensely cited to influence study approval. Analysis of two extreme subgroups (approvers vs. nonapprovers) showed that uncertainty influenced approval decisions, odds ratios (OR) = 3.5 (95% confidence interval [CI], 1.3-9.8) and OR = 3.2 (95% CI, 1.1-8.9), respectively, based on theme frequency and theme intensity, ignoring multiple observations per person. Taking into consideration multiple observations per person, similar results were obtained for uncertainty: OR = 8.9 (95% CI, 0.93-85.4).

CONCLUSIONS

Perceived uncertainty about benefits and harms of a proposed intervention is a key driver in IRB members' approval of clinical trials. This, in turn, calls for improved standardization in the communications of information on benefits and harms in the research protocols considered by the IRBs.

At what level of collective equipoise does a randomized clinical trial become ethical for the members of institutional review board/ethical committees?

Background:

The conduct of a randomized controlled trial (RCT) is deemed ethical only if we are in state of “equipoise” as to which treatment would be most beneficial for the patients. Individual equipoise applies to an individual clinician or a member of ethical, institutional review board (IRB), whilst collective equipoise refers to the profession as a whole. It is argued that physicians are not bound by the equipoise but their actions are directed by the confines of the expert opinion. Experts can agree or disagree in various proportions on the merit of a given treatment. Hence, the collective equipoise will be often incomplete. In turn, the opinions of content expert in the field of the proposed trial influence the IRB members’ decision regarding trial approval.

Methods:

We conducted a survey of IRB members at University of South Florida and the IRB members attending the bioethics conference organized in Clearwater, Florida, USA. The survey was made available as hard copy (paper based) and included six hypothetical scenarios outlining clinical trials targeted at measuring the collective equipoise. We defined the collective equipoise as the situation when survey participants were equally split (50:50) in their decision regarding whether a proposed clinical trial would be ethical to conduct. The opinion of 100 experts in the field expressed as proportion of experts favoring treatment A vs. B in each of the five scenarios was made available to the participants.

Results:

The response rate of our survey was 33% (71/218). Fifty percent of the IRB members would approve an RCT addressing the efficacy of two drugs for the management of headache even if 80% of experts favor one treatment over another (median: 80%; third quartile: 80%). Similarly, half of participating IRB members would approve the study when the median distribution of equipoise among experts was 70% (70 in favor of treatment A vs. 30 in favor of treatment B) for treatment of leukemia, 60% for treatment of geriatric patients and 70% for treatment of newborns. Half of IRB members would approve the study when the median distribution of equipoise among experts was 70% for treatment for leukemia in dogs and 85% for leukemia in rats (and 25% of IRB members would approve such a study even if 100% of experts favors one treatment over another). None of the demographic features of respondents affected collective equipoise.

Conclusions:

This is the first study assessing collective equipoise among ethical committee/IRB members. Our study findings show that IRB members perceived that conduct of a trial enrolling humans is unethical when the equipoise level is beyond 80% (80:20 distribution of uncertainty). IRB members require a higher level of equipoise when it comes to testing a new drug in humans than in animals. A relatively high level of equipoise is needed for IRB members to be comfortable to approve trials involving life-threatening situations, children and elderly patients.

Treatment success in cancer: industry compared to publicly sponsored randomized controlled trials

OBJECTIVE

To assess if commercially sponsored trials are associated with higher success rates than publicly-sponsored trials.

STUDY DESIGN AND SETTINGS

We undertook a systematic review of all consecutive, published and unpublished phase III cancer randomized controlled trials (RCTs) conducted by GlaxoSmithKline (GSK) and the NCIC Clinical Trials Group (CTG). We included all phase III cancer RCTs assessing treatment superiority from 1980 to 2010. Three metrics were assessed to determine treatment successes: (1) the proportion of statistically significant trials favouring the experimental treatment, (2) the proportion of the trials in which new treatments were considered superior according to the investigators, and (3) quantitative synthesis of data for primary outcomes as defined in each trial.

RESULTS

GSK conducted 40 cancer RCTs accruing 19,889 patients and CTG conducted 77 trials enrolling 33,260 patients. 42% (99%CI 24 to 60) of the results were statistically significant favouring experimental treatments in GSK compared to 25% (99%CI 13 to 37) in the CTG cohort (RR = 1.68; p = 0.04). Investigators concluded that new treatments were superior to standard treatments in 80% of GSK compared to 44% of CTG trials (RR = 1.81; p<0.001). Meta-analysis of the primary outcome indicated larger effects in GSK trials (odds ratio = 0.61 [99%CI 0.47-0.78] compared to 0.86 [0.74-1.00]; p = 0.003). However, testing for the effect of treatment over time indicated that treatment success has become comparable in the last decade.

CONCLUSIONS

While overall industry sponsorship is associated with higher success rates than publicly-sponsored trials, the difference seems to have disappeared over time.

Paradoxical interpretations of noninferiority studies: violating the excluded middle

Background: The noninferiority of a novel therapy compared with a standard of care is customarily defined by a noninferiority margin derived from an assessment of what would constitute a clinically relevant decrement in efficacy while preserving some of the treatment effect over placebo. Conundrum: If the one-sided 97.5% CI around the difference in the point estimates of efficacy between the two treatments (investigational drug minus comparator drug) does not extend below the prespecified threshold, noninferiority of the new agent to the comparator is typically concluded. In some cases, the corresponding two-sided 95% CI will fall entirely between zero and the noninferiority delta, technically implying inferiority and noninferiority concurrently. Solution: Stipulating that the upper bound of the two-sided confidence interval reach or exceed zero (as well as fall entirely above the noninferiority limit) to establish statistical noninferiority versus the comparator would avoid paradoxical interpretations.

New treatments compared to established treatments in randomized trials

BACKGROUND

The proportion of proposed new treatments that are 'successful' is of ethical, scientific, and public importance. We investigated how often new, experimental treatments evaluated in randomized controlled trials (RCTs) are superior to established treatments.

OBJECTIVES

Our main question was: "On average how often are new treatments more effective, equally effective or less effective than established treatments?" Additionally, we wanted to explain the observed results, i.e. whether the observed distribution of outcomes is consistent with the 'uncertainty requirement' for enrollment in RCTs. We also investigated the effect of choice of comparator (active versus no treatment/placebo) on the observed results.

SEARCH METHODS

We searched the Cochrane Methodology Register (CMR) 2010, Issue 1 in The Cochrane Library (searched 31 March 2010); MEDLINE Ovid 1950 to March Week 2 2010 (searched 24 March 2010); and EMBASE Ovid 1980 to 2010 Week 11 (searched 24 March 2010).

SELECTION CRITERIA

Cohorts of studies were eligible for the analysis if they met all of the following criteria: (i) consecutive series of RCTs, (ii) registered at or before study onset, and (iii) compared new against established treatments in humans.

DATA COLLECTION AND ANALYSIS

RCTs from four cohorts of RCTs met all inclusion criteria and provided data from 743 RCTs involving 297,744 patients. All four cohorts consisted of publicly funded trials. Two cohorts involved evaluations of new treatments in cancer, one in neurological disorders, and one for mixed types of diseases. We employed kernel density estimation, meta-analysis and meta-regression to assess the probability of new treatments being superior to established treatments in their effect on primary outcomes and overall survival.

MAIN RESULTS

The distribution of effects seen was generally symmetrical in the size of difference between new versus established treatments. Meta-analytic pooling indicated that, on average, new treatments were slightly more favorable both in terms of their effect on reducing the primary outcomes (hazard ratio (HR)/odds ratio (OR) 0.91, 99% confidence interval (CI) 0.88 to 0.95) and improving overall survival (HR 0.95, 99% CI 0.92 to 0.98). No heterogeneity was observed in the analysis based on primary outcomes or overall survival (I(2) = 0%). Kernel density analysis was consistent with the meta-analysis, but showed a fairly symmetrical distribution of new versus established treatments indicating unpredictability in the results. This was consistent with the interpretation that new treatments are only slightly superior to established treatments when tested in RCTs. Additionally, meta-regression demonstrated that results have remained stable over time and that the success rate of new treatments has not changed over the last half century of clinical trials. The results were not significantly affected by the choice of comparator (active versus placebo/no therapy).

AUTHORS' CONCLUSIONS

Society can expect that slightly more than half of new experimental treatments will prove to be better than established treatments when tested in RCTs, but few will be substantially better. This is an important finding for patients (as they contemplate participation in RCTs), researchers (as they plan design of the new trials), and funders (as they assess the 'return on investment'). Although we provide the current best evidence on the question of expected 'success rate' of new versus established treatments consistent with a priori theoretical predictions reflective of 'uncertainty or equipoise hypothesis', it should be noted that our sample represents less than 1% of all available randomized trials; therefore, one should exercise the appropriate caution in interpretation of our findings. In addition, our conclusion applies to publicly funded trials only, as we did not include studies funded by commercial sponsors in our analysis.

When is it rational to participate in a clinical trial? A game theory approach incorporating trust, regret and guilt

BACKGROUND

Randomized controlled trials (RCTs) remain an indispensable form of human experimentation as a vehicle for discovery of new treatments. However, since their inception RCTs have raised ethical concerns. The ethical tension has revolved around "duties to individuals" vs. "societal value" of RCTs. By asking current patients "to sacrifice for the benefit of future patients" we risk subjugating our duties to patients' best interest to the utilitarian goal for the good of others. This tension creates a key dilemma: when is it rational, from the perspective of the trial patients and researchers (as societal representatives of future patients), to enroll in RCTs?

METHODS

We employed the trust version of the prisoner's dilemma since interaction between the patient and researcher in the setting of a clinical trial is inherently based on trust. We also took into account that the patient may have regretted his/her decision to participate in the trial, while a researcher may feel guilty because he/she abused the patient's trust.

RESULTS

We found that under typical circumstances of clinical research, most patients can be expected not to trust researchers, and most researchers can be expected to abuse the patients' trust. The most significant factor determining trust was the success of experimental or standard treatments, respectively. The more that a researcher believes the experimental treatment will be successful, the more incentive the researcher has to abuse trust. The analysis was sensitive to the assumptions about the utilities related to success and failure of therapies that are tested in RCTs. By varying all variables in the Monte Carlo analysis we found that, on average, the researcher can be expected to honor a patient's trust 41% of the time, while the patient is inclined to trust the researcher 69% of the time. Under assumptions of our model, enrollment into RCTs represents a rational strategy that can meet both patients' and researchers' interests simultaneously 19% of the time.

CONCLUSIONS

There is an inherent ethical dilemma in the conduct of RCTs. The factors that hamper full co-operation between patients and researchers in the conduct of RCTs can be best addressed by: a) having more reliable estimates on the probabilities that new vs. established treatments will be successful, b) improving transparency in the clinical trial system to ensure fulfillment of "the social contract" between patients and researchers.