Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials

A unified Bayesian framework for bias adjustment in multiple comparisons from clinical trials

In clinical trials, multiple comparisons arising from various treatments/doses, subgroups, or endpoints are common. Typically, trial teams focus on the comparison showing the largest observed treatment effect, often involving a specific treatment pair and endpoint within a subgroup. These findings frequently lead to follow-up pivotal studies, many of which do not confirm the initial positive results. Selection bias occurs when the most promising treatment, subgroup, or endpoint is chosen for further development, potentially skewing subsequent investigations. Such bias can be defined as the deviation in the observed treatment effects from the underlying truth. In this article, we propose a general and unified Bayesian framework to address selection bias in clinical trials with multiple comparisons. Our approach does not require a priori specification of a parametric distribution for the prior, offering a more flexible and generalized solution. The proposed method facilitates a more accurate interpretation of clinical trial results by adjusting for such selection bias. Through simulation studies, we compared several methods and demonstrated their superior performance over the normal shrinkage estimator. We recommended the use of Bayesian Model Averaging estimator averaging over Gaussian Mixture Models as the prior distribution based on its performance and flexibility. We applied the method to a multicenter, randomized, double-blind, placebo-controlled study investigating the cardiovascular effects of dulaglutide.

Clinical data mining: challenges, opportunities, and recommendations for translational applications

Clinical data mining of predictive models offers significant advantages for re-evaluating and leveraging large amounts of complex clinical real-world data and experimental comparison data for tasks such as risk stratification, diagnosis, classification, and survival prediction. However, its translational application is still limited. One challenge is that the proposed clinical requirements and data mining are not synchronized. Additionally, the exotic predictions of data mining are difficult to apply directly in local medical institutions. Hence, it is necessary to incisively review the translational application of clinical data mining, providing an analytical workflow for developing and validating prediction models to ensure the scientific validity of analytic workflows in response to clinical questions. This review systematically revisits the purpose, process, and principles of clinical data mining and discusses the key causes contributing to the detachment from practice and the misuse of model verification in developing predictive models for research. Based on this, we propose a niche-targeting framework of four principles: Clinical Contextual, Subgroup-Oriented, Confounder- and False Positive-Controlled (CSCF), to provide guidance for clinical data mining prior to the model's development in clinical settings. Eventually, it is hoped that this review can help guide future research and develop personalized predictive models to achieve the goal of discovering subgroups with varied remedial benefits or risks and ensuring that precision medicine can deliver its full potential.

Evidence of subgroup differences in meta-analyses evaluating medications for alcohol use disorder: An umbrella review

Randomized controlled trials (RCTs) evaluating medications for alcohol use disorder (AUD) often examine heterogeneity of treatment effects through subgroup analyses that contrast effect estimates in groups of patients across individual demographic, clinical, and study design-related characteristics. However, these analyses are often not prespecified or adequately powered, highlighting the potential role of subgroup analyses in meta-analysis. Here, we conducted an umbrella review (i.e., a systematic review of meta-analyses) to determine the range and characteristics of reported subgroup analyses in meta-analyses of AUD medications. We searched PubMed to identify meta-analyses of RCTs evaluating medications for the management of AUD, alcohol abuse, or alcohol dependence in adults. We sought studies that measured drinking-related outcomes; quality of life, function, and rates of mortality; adverse events; and dropout. We considered meta-analyses that reported the results from formal subgroup analyses (comparing the summary effects across subgroup levels); summary effect estimates stratified across subgroup levels; and meta-regression, regression, or correlation-based subgroup analyses. We analyzed nine meta-analyses that included 61 formal subgroup analyses (median = 6 per meta-analysis), of which 33 (54%) were based on baseline participant-level and 28 (46%) were based on trial-level characteristics. Of the 58 subgroup analyses with either a p-value from a subgroup test or a statement by the authors that the subgroup analyses were not statistically significant, eight (14%) were statistically significant at the p < 0.05 level. Twelve meta-analyses reported the results of 102 meta-regression analyses, of which 25 (25%) identified statistically significant predictors of the relevant outcome of interest; nine (9%) were based on baseline participant-level and 93 (91%) were based on trial characteristics. Subgroup analyses across meta-analyses of AUD medications often focus on study-level characteristics, which may not be as clinically informative as subgroup analyses based on participant-level characteristics. Opportunities exist for future meta-analyses to standardize their subgroup methodology, focus on more clinically informative participant-level characteristics, and use predictive approaches to account for multiple relevant variables.

Cell-of-origin effect of polatuzumab vedotin in diffuse large B-cell lymphoma: no ordinary subgroup analysis

ABSTRACT

Subgroup analysis from the POLARIX trial of polatuzumab vedotin plus chemotherapy for untreated large B-cell lymphoma suggests greater efficacy among patients with activated B-cell subtype disease. Both preclinical and additional clinical evidence support this interaction between cell-of-origin and polatuzumab efficacy.

Angiotensin II treatment is associated with improved oxygenation in ARDS patients with refractory vasodilatory shock

BACKGROUND

The physiological effects of renin-angiotensin system modulation in acute respiratory distress syndrome (ARDS) remain controversial and have not been investigated in randomized trials. We sought to determine whether angiotensin-II treatment is associated with improved oxygenation in shock-associated ARDS.

METHODS

Post-hoc subgroup analysis of the Angiotensin Therapy for High Output Shock (ATHOS-3) trial. We studied patients who met modified Berlin ARDS criteria at enrollment. The primary outcome was PaO2/FiO2-ratio (P:F) at 48-h adjusted for baseline P:F. Secondary outcomes included oxygenation index, ventilatory ratio, PEEP, minute-ventilation, hemodynamic measures, patients alive and ventilator-free by day-7, and mortality.

RESULTS

Of 81 ARDS patients, 34 (42%) and 47 (58%) were randomized to angiotensin-II or placebo, respectively. In angiotensin-II patients, mean P:F increased from 155 mmHg (SD: 69) at baseline to 265 mmHg (SD: 160) at hour-48 compared with no change with placebo (148 mmHg (SD: 63) at baseline versus 164 mmHg (SD: 74) at hour-48)(baseline-adjusted difference: + 98.4 mmHg [95%CI 35.2-161.5], p = 0.0028). Similarly, oxygenation index decreased by - 6.0 cmH2O/mmHg at hour-48 with angiotensin-II versus - 0.4 cmH2O/mmHg with placebo (baseline-adjusted difference: -4.8 cmH2O/mmHg, [95%CI - 8.6 to - 1.1], p = 0.0273). There was no difference in PEEP, minute ventilation, or ventilatory ratio. Twenty-two (64.7%) angiotensin-II patients had sustained hemodynamic response to treatment at hour-3 versus 17 (36.2%) placebo patients (absolute risk-difference: 28.5% [95%CI 6.5-47.0%], p = 0.0120). At day-7, 7/34 (20.6%) angiotensin-II patients were alive and ventilator-free versus 5/47(10.6%) placebo patients. Day-28 mortality was 55.9% in the angiotensin-II group versus 68.1% in the placebo group.

CONCLUSIONS

In post-hoc analysis of the ATHOS-3 trial, angiotensin-II was associated with improved oxygenation versus placebo among patients with ARDS and catecholamine-refractory vasodilatory shock. These findings provide a physiologic rationale for trials of angiotensin-II as treatment for ARDS with vasodilatory shock.

TRIAL REGISTRATION

ClinicalTrials.Gov Identifier: NCT02338843 (Registered January 14th 2015).

The credibility of subgroup analyses reported in stroke trials is low: A systematic review

BACKGROUND

Subgroup analyses are widely used to evaluate the heterogeneity of treatment effects in randomized clinical trials. However, there is a limited investigation of the quality of prespecified and reported subgroup analyses in stroke trials. This study evaluated the credibility of subgroup analyses in stroke trials.

METHODS AND ANALYSIS

We searched Medline/PubMed, Embase, the Cochrane Central Register of Controlled Trials, and the Web of Science from inception to 24 March 2021. Three reviewers screened, extracted, and analyzed the data from the publications. Primary publications of stroke trials that reported at least one subgroup effect and had published corresponding study protocols were included. The Instrument for Assessing the Credibility of Effect Modification Analyses (ICEMAN) was used to examine the quality of the subgroup effects reported, with each subgroup effect assigned a credibility rating ranging from very low to high. Subgroup effects with two or more "definitely no" responses received a low credibility rating. The risk of bias was assessed using the Cochrane Risk-of-Bias tool for randomized trials version 2.

RESULTS

Seventy-four articles met the inclusion criteria and reported a combined total of 647 subgroup effects. The median sample size was 1264 (interquartile range (IQR): 380-3876), and the median number of subgroups prespecified in the protocol was 6 (IQR: 2-10). Sixty-one (82%) studies used the univariate test of interaction. Of the total 647 subgroup effects reported in these studies, 319 (49%) were reported in acute stroke trials, while 423 (65%) had low credibility.

CONCLUSION

The quality of subgroup analysis reporting in stroke trials remains poor. More effort is needed to train trialists on the best methods for designing and performing subgroup analyses, and how to report the results.

TRIAL REGISTRATION NUMBER

We prospectively registered the review with International Prospective Register for Systematic Reviews (registration number: CRD42020223133).

Assessing heterogeneity of treatment effect in multiple sclerosis trials

Multiple sclerosis (MS) is heterogeneous with respect to outcomes, and evaluating possible heterogeneity of treatment effect (HTE) is of high interest. HTE is non-random variation in the magnitude of a treatment effect on a clinical outcome across levels of a covariate (i.e. a patient attribute or set of attributes). Multiple statistical techniques can evaluate HTE. The simplest but most bias-prone is conventional one variable-at-a-time subgroup analysis. Recently, multivariable predictive approaches have been promoted to provide more patient-centered results, by accounting for multiple relevant attributes simultaneously. We review approaches used to estimate HTE in clinical trials of MS.

Overall average treatment effects from clinical trials, one-variable-at-a-time subgroup analyses and predictive approaches to heterogeneous treatment effects: Toward a more patient-centered evidence-based medicine

Despite the predominance of the evidence-based medicine paradigm, a fundamental incongruity remains: Evidence is derived from groups of people, yet medical decisions are made by and for individuals. Randomization ensures the comparability of treatment groups within a clinical trial, which allows for unbiased estimation of average treatment effects. If we treated groups of patients instead of individuals, or if patients with the same disease were identical to one another in all factors that determined the harms and the benefits of therapy, then these group-level averages would make a perfectly sound foundation for medical decision-making. But patients differ from one another in many ways that determine the likelihood of an outcome, both with and without a treatment. Nevertheless, popular approaches to evidence-based medicine have encouraged a reliance on the average treatment effects estimated from clinical trials and meta-analysis as guides to decision-making for individuals. Here, we discuss the limitations of this approach as well as limitations of conventional, one-variable-at-a-time subgroup analysis; finally, we discuss the rationale for "predictive" approaches to heterogeneous treatment effects. Predictive approaches to heterogeneous treatment effects combine methods for causal inference (e.g. randomization) with methods for prediction that permit inferences about which patients are likely to benefit and which are not, taking into account multiple relevant variables simultaneously to yield "personalized" estimates of benefit-harm trade-offs. We focus on risk modeling approaches, which rely on the mathematical dependence of the absolute treatment effect with the baseline risk, which varies substantially "across patients" in most trials. While there are a number of examples of risk modeling approaches that have been practice-changing, risk modeling does not provide ideal estimates of individual treatment effects, since risk modeling does not account for how individual variables might modify the effects of therapy. In "effect modeling," prediction models are developed directly on clinical trial data, including terms for treatment and treatment effect interactions. These more flexible approaches may better uncover individualized treatment effects, but are also prone to overfitting when dimensionality is high, power is low, and there is limited prior knowledge about effect modifiers.

Intensive versus standard blood pressure control in older persons with or without diabetes: a systematic review and meta-analysis of randomised controlled trials

OBJECTIVES

To assess and compare the benefits and harms of intensive versus standard blood pressure (BP) control in older people with or without diabetes mellitus (DM).

DESIGN

Systematic review and meta-analysis.

SETTING

Randomised controlled trials comparing intensive versus standard BP control, identified from MEDLINE, Embase, The Cochrane library, Web of Science and a search of bibliographies from inception till August 2022.

PARTICIPANTS

Older people (≥65 years) with or without DM.

MAIN OUTCOME MEASURES

Study-specific risk ratios (RRs) with 95% confidence intervals (CIs) were pooled for adverse vascular and safety outcomes.

RESULTS

We included six randomised controlled trials (RCTs) comprising 20,985 patients (intensive BP = 10,474 and standard BP = 10,511) with a weighted mean follow-up of 3.1 years. In the general population, the RRs (95% CIs) of intensive versus standard BP control for composite cardiovascular events or major adverse cardiovascular events (CVD/MACE), CVD mortality, coronary heart disease, stroke and heart failure were 0.71 (0.62-0.82), 0.65 (0.49-0.86), 0.75 (0.60-0.95), 0.75 (0.61-0.92) and 0.58 (0.41-0.82), respectively. Intensive BP control did not increase the risk of renal failure or serious adverse events in the general population. Two RCTs reported results for composite CVD/MACE in patients with DM with a pooled estimate of 0.85 (0.67-1.07).

CONCLUSIONS

Aggregate trial evidence shows that intensive BP control (<120 to <140 mmHg) reduces the risk of adverse cardiovascular outcomes in older hypertensive patients in the general population with no increase in adverse events. Intensive BP control may confer similar benefits for older patients with DM with no evidence for harm, but this is based on limited data.PROSPERO Registration: CRD42022349791.

Reducing bias, increasing transparency and calibrating confidence with preregistration

Flexibility in the design, analysis and interpretation of scientific studies creates a multiplicity of possible research outcomes. Scientists are granted considerable latitude to selectively use and report the hypotheses, variables and analyses that create the most positive, coherent and attractive story while suppressing those that are negative or inconvenient. This creates a risk of bias that can lead to scientists fooling themselves and fooling others. Preregistration involves declaring a research plan (for example, hypotheses, design and statistical analyses) in a public registry before the research outcomes are known. Preregistration (1) reduces the risk of bias by encouraging outcome-independent decision-making and (2) increases transparency, enabling others to assess the risk of bias and calibrate their confidence in research outcomes. In this Perspective, we briefly review the historical evolution of preregistration in medicine, psychology and other domains, clarify its pragmatic functions, discuss relevant meta-research, and provide recommendations for scientists and journal editors.

Diabetes precision medicine: plenty of potential, pitfalls and perils but not yet ready for prime time

Rapid advances in technology and data science have the potential to improve the precision of preventive and therapeutic interventions, and enable the right treatment to be recommended, at the right time, to the right person. There are well-described examples of successful precision medicine approaches for monogenic conditions such as specific diets for phenylketonuria, and sulfonylurea treatments for certain types of MODY. However, the majority of chronic diseases are polygenic, and it is unlikely that the research strategies used for monogenic diseases will deliver similar changes to practice for polygenic traits. Type 2 diabetes, for example, is a multifactorial, heterogeneous, polygenic palette of metabolic disorders. In this non-systematic review I highlight limitations of the evidence, and the challenges that need to be overcome prior to implementation of precision medicine in the prevention and management of type 2 diabetes. Most precision medicine approaches are spuriously precise, overly complex and too narrowly focused on predicting blood glucose levels with a limited set of characteristics of individuals rather than the whole person and their context. Overall, the evidence to date is insufficient to justify widespread implementation of precision medicine approaches into routine clinical practice for type 2 diabetes. We need to retain a degree of humility and healthy scepticism when evaluating novel strategies, and to demand that existing evidence thresholds are exceeded prior to implementation.

Subgroup Analysis in Pulmonary Hypertension-Specific Therapy Clinical Trials: A Systematic Review

Pulmonary hypertension (PH) treatment decisions are driven by the results of randomized controlled trials (RCTs). Subgroup analyses are often performed to assess whether the intervention effect will change due to the patient’s characteristics, thus allowing for individualized decisions. This review aimed to evaluate the appropriateness and interpretation of subgroup analyses performed in PH-specific therapy RCTs published between 2000 and 2020. Claims of subgroup effects were evaluated with prespecified criteria. Overall, 30 RCTs were included. Subgroup analyses presented: a high number of subgroup analyses reported, lack of prespecification, and lack of interaction tests. The trial protocol was not available for most RCTs; significant differences were found in those articles that published the protocol. Authors reported 13 claims of subgroup effect, with 12 claims meeting four or fewer of Sun’s criteria. Even when most RCTs were generally at low risk of bias and were published in high-impact journals, the credibility and general quality of subgroup analyses and subgroup claims were low due to methodological flaws. Clinicians should be skeptical of claims of subgroup effects and interpret subgroup analyses with caution, as due to their poor quality, these analyses may not serve as guidance for personalized care.

A permutation procedure to detect heterogeneous treatments effects in randomized clinical trials while controlling the type I error rate

BACKGROUND/AIMS

Secondary analyses of randomized clinical trials often seek to identify subgroups with differential treatment effects. These discoveries can help guide individual treatment decisions based on patient characteristics and identify populations for which additional treatments are needed. Traditional analyses require researchers to pre-specify potential subgroups to reduce the risk of reporting spurious results. There is a need for methods that can detect such subgroups without a priori specification while allowing researchers to control the probability of falsely detecting heterogeneous subgroups when treatment effects are uniform across the study population.

METHODS

We propose a permutation procedure for tuning parameter selection that allows for type I error control when testing for heterogeneous treatment effects framed within the Virtual Twins procedure for subgroup identification. We verify that the type I error rate can be controlled at the nominal rate and investigate the power for detecting heterogeneous effects when present through extensive simulation studies. We apply our method to a secondary analysis of data from a randomized trial of very low nicotine content cigarettes.

RESULTS

In the absence of type I error control, the observed type I error rate for Virtual Twins was between 99% and 100%. In contrast, models tuned via the proposed permutation were able to control the type I error rate and detect heterogeneous effects when present. An application of our approach to a recently completed trial of very low nicotine content cigarettes identified several variables with potentially heterogeneous treatment effects.

CONCLUSIONS

The proposed permutation procedure allows researchers to engage in secondary analyses of clinical trials for treatment effect heterogeneity while maintaining the type I error rate without pre-specifying subgroups.

Adequate enrollment of women in cardiovascular drug trials and the need for sex-specific assessment and reporting

Cardiovascular disease (CVD) is the leading cause of death for women in the United States and globally. There is an abundance of evidence-based trials evaluating the efficacy of drug therapies to reduce morbidity and mortality in CVD. Additionally, there are well-established influences of sex, through a variety of mechanisms, on pharmacologic treatments in CVD. Despite this, the majority of drug trials are not powered to evaluate sex-specific outcomes, and much of the data that exists is gathered post hoc and through meta-analysis. The FDA established a committee in 1993 to increase the enrollment of women in clinical trials to improve this situation. Several authors, reviewing committees, and professional societies have highlighted the importance of sex-specific analysis and reporting. Despite these statements, there has not been a major improvement in representation or reporting. There are ongoing efforts to assess trial design, female representation on steering committees, and clinical trial processes to improve the representation of women. This review will describe the pharmacologic basis for the need for sex-specific assessment of cardiovascular drug therapies. It will also review the sex-specific reporting of landmark drug trials in hypertension, coronary artery disease (CAD), hyperlipidemia, and heart failure (HF). In reporting enrollment of women, several therapeutic areas like antihypertensives and newer anticoagulation trials fare better than therapeutics for HF and acute coronary syndromes. Further, drug trials and cardiometabolic or lifestyle intervention trials had a higher percentage of female participants than the device or procedural trials.

Closing the Gaps in Racial Disparities in Critical Limb Ischemia Outcome and Amputation Rates: Proceedings from a Society of Interventional Radiology Foundation Research Consensus Panel

Minority patients such as Blacks, Hispanics, and Native Americans are disproportionately impacted by critical limb ischemia and amputation due to multiple factors such as socioeconomic status, type or lack of insurance, lack of access to health care, capacity and expertise of local hospitals, prevalence of diabetes, and unconscious bias. The Society of Interventional Radiology Foundation recognizes that it is imperative to close the disparity gaps and funded a Research Consensus Panel to prioritize a research agenda. The following research priorities were ultimately prioritized: (a) randomized controlled trial with peripheral arterial disease screening of at-risk patients with oversampling of high-risk racial groups, (b) prospective trial with the introduction of an intervention to alter a social determinant of health, and (c) a prospective trial with the implementation of an algorithm that requires criteria be met prior to an amputation. This article presents the proceedings and recommendations from the panel.

Do recommended interventions widen or narrow inequalities in musculoskeletal health? An equity-focussed systematic review of differential effectiveness

Background

It is unclear whether seven interventions recommended by Public Health England for preventing and managing common musculoskeletal conditions reduce or widen health inequalities in adults with musculoskeletal conditions.

Methods

We used citation searches of Web of Science (date of ‘parent publication’ for each intervention to April 2021) to identify original research articles reporting subgroup or moderator analyses of intervention effects by social stratifiers defined using the PROGRESS-Plus frameworks. Randomized controlled trials, controlled before-after studies, interrupted time series, systematic reviews presenting subgroup/stratified analyses or meta-regressions, individual participant data meta-analyses and modelling studies were eligible. Two reviewers independently assessed the credibility of effect moderation claims using Instrument to assess the Credibility of Effect Moderation Analyses. A narrative approach to synthesis was used (PROSPERO registration number: CRD42019140018).

Results

Of 1480 potentially relevant studies, seven eligible analyses of single trials and five meta-analyses were included. Among these, we found eight claims of potential differential effectiveness according to social characteristics, but none that were judged to have high credibility.

Conclusions

In the absence of highly credible evidence of differential effectiveness in different social groups, and given ongoing national implementation, equity concerns may be best served by investing in monitoring and action aimed at ensuring fair access to these interventions.

How effects on health equity are assessed in systematic reviews of interventions

BACKGROUND

Enhancing health equity is endorsed in the Sustainable Development Goals. The failure of systematic reviews to consider potential differences in effects across equity factors is cited by decision-makers as a limitation to their ability to inform policy and program decisions.  OBJECTIVES: To explore what methods systematic reviewers use to consider health equity in systematic reviews of effectiveness.

SEARCH METHODS

We searched the following databases up to 26 February 2021: MEDLINE, PsycINFO, the Cochrane Methodology Register, CINAHL, Education Resources Information Center, Education Abstracts, Criminal Justice Abstracts, Hein Index to Foreign Legal Periodicals, PAIS International, Social Services Abstracts, Sociological Abstracts, Digital Dissertations and the Health Technology Assessment Database. We searched SCOPUS to identify articles that cited any of the included studies on 10 June 10 2021. We contacted authors and searched the reference lists of included studies to identify additional potentially relevant studies.

SELECTION CRITERIA

We included empirical studies of cohorts of systematic reviews that assessed methods for measuring effects on health inequalities. We define health inequalities as unfair and avoidable differences across socially stratifying factors that limit opportunities for health. We operationalised this by assessing studies which evaluated differences in health across any component of the PROGRESS-Plus acronym, which stands for Place of residence, Race/ethnicity/culture/language, Occupation, Gender or sex, Religion, Education, Socioeconomic status, Social capital. "Plus" stands for other factors associated with discrimination, exclusion, marginalisation or vulnerability such as personal characteristics (e.g. age, disability), relationships that limit opportunities for health (e.g. children in a household with parents who smoke) or environmental situations which provide limited control of opportunities for health (e.g. school food environment).

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data using a pre-tested form. Risk of bias was appraised for included studies according to the potential for bias in selection and detection of systematic reviews.  MAIN RESULTS: In total, 48,814 studies were identified and the titles and abstracts were screened in duplicate. In this updated review, we identified an additional 124 methodological studies published in the 10 years since the first version of this review, which included 34 studies. Thus, 158 methodological studies met our criteria for inclusion. The methods used by these studies focused on evidence relevant to populations experiencing health inequity (108 out of 158 studies), assess subgroup analysis across PROGRESS-Plus (26 out of 158 studies), assess analysis of a gradient in effect across PROGRESS-Plus (2 out of 158 studies) or use a combination of subgroup analysis and focused approaches (20 out of 158 studies). The most common PROGRESS-Plus factors assessed were age (43 studies), socioeconomic status in 35 studies, low- and middle-income countries in 24 studies, gender or sex in 22 studies, race or ethnicity in 17 studies, and four studies assessed multiple factors across which health inequity may exist. Only 16 studies provided a definition of health inequity. Five methodological approaches to consider health equity in systematic reviews of effectiveness were identified: 1) descriptive assessment of reporting and analysis in systematic reviews (140 of 158 studies used a type of descriptive method); 2) descriptive assessment of reporting and analysis in original trials (50 studies); 3) analytic approaches which assessed differential effects across one or more PROGRESS-Plus factors (16 studies); 4) applicability assessment (25 studies) and 5) stakeholder engagement (28 studies), which is a new finding in this update and examines the appraisal of whether relevant stakeholders with lived experience of health inequity were included in the design of systematic reviews or design and delivery of interventions. Reporting for both approaches (analytic and applicability) lacked transparency and was insufficiently detailed to enable the assessment of credibility.

AUTHORS' CONCLUSIONS

There is a need for improvement in conceptual clarity about the definition of health equity, describing sufficient detail about analytic approaches (including subgroup analyses) and transparent reporting of judgments required for applicability assessments in order to consider health equity in systematic reviews of effectiveness.

Clinical and pharmacoeconomic impact of subgroup analysis in onco-hematological patients

Subgroup analysis evaluates a health intervention in subpopulations according to a characteristic or factor. It can be useful for generating new hypotheses or conducting new studies. However, subgroup analysis presents several limitations and it should be considered cautiously. The development of new onco-hematological drugs is accelerating in recent years and the impact of subgroup analysis on clinical decision-making is increasing. The interpretation of subgroup analyses can be controversial in some cases, negatively affecting patients and healthcare systems. This work is a review of the clinical and pharmacoeconomic impact of subgroup analysis in onco-hematological patients. The study describes some illustrative examples of inadequate interpretations about subset analysis: combination of pembrolizumab plus chemotherapy in lung cancer, inhibitors of cyclin-dependent kinases in breast cancer, daratumumab-based regimens in newly diagnosed multiple myeloma, combination of nivolumab with ipilimumab in melanoma and docetaxel in prostate cancer. Subgroup analysis can have a significant impact on the data selection for the development of studies; efficacy, safety, and convenience of treatments in onco-hematological patients; efficiency of therapies in health systems; and therapeutic positioning of antineoplastic drugs. There is a strong need to establish homogeneous criteria for the assessment of subgroup analysis and to develop new tools for its consideration.

Generalizability of heterogeneous treatment effects based on causal forests applied to two randomized clinical trials of intensive glycemic control

Purpose Machine learning is an attractive tool for identifying heterogeneous treatment effects (HTE) of interventions but generalizability of machine learning derived HTE remains unclear. We examined generalizability of HTE detected using causal forests in two similarly designed randomized trials in type II diabetes patients. Methods We evaluated published HTE of intensive versus standard glycemic control on all-cause mortality from the Action to Control Cardiovascular Risk in Diabetes study (ACCORD) in a second trial, the Veterans Affairs Diabetes Trial (VADT). We then applied causal forests to VADT, ACCORD, and pooled data from both studies and compared variable importance and subgroup effects across samples. Results HTE in ACCORD did not replicate in similar subgroups in VADT, but variable importance was correlated between VADT and ACCORD (Kendall's tau-b 0.75). Applying causal forests to pooled individual-level data yielded seven subgroups with similar HTE across both studies, ranging from risk difference of all-cause mortality of -3.9% (95% CI -7.0, -0.8) to 4.7% (95% CI 1.8, 7.5). Conclusions Machine learning detection of HTE subgroups from randomized trials may not generalize across study samples even when variable importance is correlated. Pooling individual-level data may overcome differences in study populations and/or differences in interventions that limit HTE generalizability.

Comparison of machine learning clustering algorithms for detecting heterogeneity of treatment effect in acute respiratory distress syndrome: A secondary analysis of three randomised controlled trials

Background

Heterogeneity in Acute Respiratory Distress Syndrome (ARDS), as a consequence of its non-specific definition, has led to a multitude of negative randomised controlled trials (RCTs). Investigators have sought to identify heterogeneity of treatment effect (HTE) in RCTs using clustering algorithms. We evaluated the proficiency of several commonly-used machine-learning algorithms to identify clusters where HTE may be detected.

Methods

Five unsupervised: Latent class analysis (LCA), K-means, partition around medoids, hierarchical, and spectral clustering; and four supervised algorithms: model-based recursive partitioning, Causal Forest (CF), and X-learner with Random Forest (XL-RF) and Bayesian Additive Regression Trees were individually applied to three prior ARDS RCTs. Clinical data and research protein biomarkers were used as partitioning variables, with the latter excluded for secondary analyses. For a clustering schema, HTE was evaluated based on the interaction term of treatment group and cluster with day-90 mortality as the dependent variable.

Findings

No single algorithm identified clusters with significant HTE in all three trials. LCA, XL-RF, and CF identified HTE most frequently (2/3 RCTs). Important partitioning variables in the unsupervised approaches were consistent across algorithms and RCTs. In supervised models, important partitioning variables varied between algorithms and across RCTs. In algorithms where clusters demonstrated HTE in the same trial, patients frequently interchanged clusters from treatment-benefit to treatment-harm clusters across algorithms. LCA aside, results from all other algorithms were subject to significant alteration in cluster composition and HTE with random seed change. Removing research biomarkers as partitioning variables greatly reduced the chances of detecting HTE across all algorithms.

Interpretation

Machine-learning algorithms were inconsistent in their abilities to identify clusters with significant HTE. Protein biomarkers were essential in identifying clusters with HTE. Investigations using machine-learning approaches to identify clusters to seek HTE require cautious interpretation.

Funding

NIGMS R35 GM142992 (PS), NHLBI R35 HL140026 (CSC); NIGMS R01 GM123193, Department of Defense W81XWH-21-1-0009, NIA R21 AG068720, NIDA R01 DA051464 (MMC)

An urgent call to raise the bar in oncology

Important breakthroughs in medical treatments have improved outcomes for patients suffering from several types of cancer. However, many oncological treatments approved by regulatory agencies are of low value and do not contribute significantly to cancer mortality reduction, but lead to unrealistic patient expectations and push even affluent societies to unsustainable health care costs. Several factors that contribute to approvals of low-value oncology treatments are addressed, including issues with clinical trials, bias in reporting, regulatory agency shortcomings and drug pricing. With the COVID-19 pandemic enforcing the elimination of low-value interventions in all fields of medicine, efforts should urgently be made by all involved in cancer care to select only high-value and sustainable interventions. Transformation of medical education, improvement in clinical trial design, quality, conduct and reporting, strict adherence to scientific norms by regulatory agencies and use of value-based scales can all contribute to raising the bar for oncology drug approvals and influence drug pricing and availability.

Evidence-based Urology: Subgroup Analysis in Randomized Controlled Trials

In randomized controlled trials, investigators often explore the possibility that the treatment effects differ between subgroups (eg, women vs men, old vs young, more versus less severe disease). Investigators often inappropriately claim subgroup effects (also called "effect modification" or "interaction") when the likelihood of a true effect modification is low. Criteria for assessing the credibility of subgroup analyses, nicely summarized in a formal Instrument for Assessing the Credibility of Effect Modification Analyses (ICEMAN), include investigator postulation of a priori hypotheses with a specified direction; support from prior evidence; a low likelihood that chance explains the apparent subgroup effect; and only testing a small number of subgroup hypotheses. PATIENT SUMMARY: Randomized clinical trials often use subgroup analyses to explore whether a treatment is more or less effective in a particular patient subgroup (eg, women vs men, old vs young). In this mini-review, we explore the common pitfalls of subgroup analyses.

Subgroup discovery in non-inferiority trials

Approaches and guidelines for performing subgroup analysis to assess heterogeneity of treatment effect in clinical trials have been the topic of numerous papers in the statistical and clinical literature, but have been discussed predominantly in the context of conventional superiority trials. Concerns about treatment heterogeneity are the same if not greater in non-inferiority (NI) trials, especially since overall similarity between two treatment arms in a successful NI trial could be due to the existence of qualitative interactions that are more likely when comparing two active therapies. Even in unsuccessful NI trials, subgroup analyses can yield important insights about the potential reasons for failure to demonstrate non-inferiority of the experimental therapy. Recent NI trials have performed a priori subgroup analyses using standard statistical tests for interaction, but there is increasing interest in more flexible machine learning approaches for post-hoc subgroup discovery. The performance and practical application of such methods in NI trials have not been systematically explored, however. We considered the Virtual Twin method for the NI setting, an algorithm for subgroup identification that combines random forest with classification and regression trees, and conducted extensive simulation studies to examine its performance under different NI trial conditions and to devise decision rules for selecting the final subgroups. We illustrate the utility of the method with data from a NI trial that was conducted to compare two acupuncture treatments for chronic musculoskeletal pain.

Preventive psychiatry: a blueprint for improving the mental health of young people

Preventive approaches have latterly gained traction for improving mental health in young people. In this paper, we first appraise the conceptual foundations of preventive psychiatry, encompassing the public health, Gordon's, US Institute of Medicine, World Health Organization, and good mental health frameworks, and neurodevelopmentally-sensitive clinical staging models. We then review the evidence supporting primary prevention of psychotic, bipolar and common mental disorders and promotion of good mental health as potential transformative strategies to reduce the incidence of these disorders in young people. Within indicated approaches, the clinical high-risk for psychosis paradigm has received the most empirical validation, while clinical high-risk states for bipolar and common mental disorders are increasingly becoming a focus of attention. Selective approaches have mostly targeted familial vulnerability and non-genetic risk exposures. Selective screening and psychological/psychoeducational interventions in vulnerable subgroups may improve anxiety/depressive symptoms, but their efficacy in reducing the incidence of psychotic/bipolar/common mental disorders is unproven. Selective physical exercise may reduce the incidence of anxiety disorders. Universal psychological/psychoeducational interventions may improve anxiety symptoms but not prevent depressive/anxiety disorders, while universal physical exercise may reduce the incidence of anxiety disorders. Universal public health approaches targeting school climate or social determinants (demographic, economic, neighbourhood, environmental, social/cultural) of mental disorders hold the greatest potential for reducing the risk profile of the population as a whole. The approach to promotion of good mental health is currently fragmented. We leverage the knowledge gained from the review to develop a blueprint for future research and practice of preventive psychiatry in young people: integrating universal and targeted frameworks; advancing multivariable, transdiagnostic, multi-endpoint epidemiological knowledge; synergically preventing common and infrequent mental disorders; preventing physical and mental health burden together; implementing stratified/personalized prognosis; establishing evidence-based preventive interventions; developing an ethical framework, improving prevention through education/training; consolidating the cost-effectiveness of preventive psychiatry; and decreasing inequalities. These goals can only be achieved through an urgent individual, societal, and global level response, which promotes a vigorous collaboration across scientific, health care, societal and governmental sectors for implementing preventive psychiatry, as much is at stake for young people with or at risk for emerging mental disorders.

Development of the Instrument to assess the Credibility of Effect Modification Analyses (ICEMAN) in randomized controlled trials and meta-analyses

BACKGROUND

Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN).

METHODS

To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification.

RESULTS

The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument.

INTERPRETATION

The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others.

Sex-related differences in the pharmacological treatment of heart failure

Heart failure (HF) represents a leading cause of morbidity and mortality. However, HF trials highlighted many differences between men and women with HF. Thus, women represent approximately a quarter of people with HF with reduced ejection fraction (HFrEF), while they account for over half of those with HF with preserved EF (HFpEF). There are also sex-related differences (SRDs) in the pharmacokinetics, pharmacodynamics and safety profile of some guideline-recommended drugs for the treatment of HF. As compared with men, women with HFrEF are less often treated with guideline-recommended HF drugs, experience more frequent and severe adverse reactions when these drugs are prescribed at the same doses in both sexes, and recent evidence suggests that women might need lower doses than men, bringing into question which are the optimal doses of HF drugs in women and men separately. However, information on SRDs in drug efficacy and safety in patients with HFrEF is very limited due to the underrepresentation of women and the lack of sex-specific evaluations of drug efficacy and safety in HF clinical trials. As a consequence, current clinical guidelines do not provide sex-specific recommendations, even when significant differences exist, at least, in drug safety. The aim of this article is to review the SRDs in the pharmacokinetics, efficacy and safety of guideline-recommended HF drugs and to identify emerging areas of research to improve our understanding of the SRDs, because a better understanding of these differences is the first step to achieve a personalized treatment of HF in women and men.

A chronicle of the 17-alpha hydroxyprogesterone caproate story to prevent recurrent preterm birth

Preterm birth is a substantial public health concern. In 2019, the US preterm birth rate was 10.23%, which is the fifth straight year of increase in this rate. Moreover, preterm birth accounts for approximately 1 in 6 infant deaths, and surviving children often suffer developmental delay or long-term neurologic impairment. Although the burden of preterm birth is clear, identifying strategies to reduce preterm birth has been challenging. On October 29, 2019, a US Food and Drug Administration advisory committee voted 9 vs 7 to withdraw interim accelerated approval of 17-alpha hydroxyprogesterone caproate for preventing recurrent preterm birth because the called for a confirmatory trial, known as the Prevention of Preterm Birth in Women With a Previous Singleton Spontaneous Preterm Delivery trial, was not confirmatory. The Prevention of Preterm Birth in Women With a Previous Singleton Spontaneous Preterm Delivery trial included subjects enrolled in the United States and Canada to ensure that at least 10% of patients would be from North America; however, this trial took 9 years to complete and did not demonstrate significant treatment effects in the 2 primary outcomes of interest. Delivery before 35 weeks' gestation occurred in 122 of 1130 women (11%) given 17-alpha hydroxyprogesterone caproate compared with 66 of 578 women (11.5%) given placebo (relative risk, 0.95; 95% confidence interval, 0.71-1.26; P=.72). Similarly, the coprimary outcome neonatal composite index occurred in 61 of 1093 women (5.6%) given 17-alpha hydroxyprogesterone caproate compared with 28 of 559 women (5.0%) given placebo (relative risk, 1.12; 95% confidence interval, 0.68-1.61; P=.73). There was also a lack of efficacy for 17-alpha hydroxyprogesterone caproate treatment in the analysis of a variety of secondary outcomes. Like the Maternal-Fetal Medicine Units Network trial, the Prevention of Preterm Birth in Women With a Previous Singleton Spontaneous Preterm Delivery trial was also flawed. Importantly, the Maternal-Fetal Medicine Unit Network trial was the sole justification for treating women in the United States with 17-alpha hydroxyprogesterone caproate for nearly 2 decades. Currently, despite more than half a century, 17-alpha hydroxyprogesterone caproate still has not been found to be clearly effective. In this context, how does the advising physician dependent on scientific evidence advise a patient that 17-alpha hydroxyprogesterone caproate is effective when the evidence to support this advice has repeatedly been found to be inadequate? This clinical opinion is a critical appraisal of the 2 randomized trials examining the efficacy of 17-alpha hydroxyprogesterone caproate to prevent recurrent preterm birth and a chronicle of events in the regulatory process of drug approval to help answer this question. With this examination, these events illustrate the complexity of pharmaceutical regulations in the era of accelerated Food and Drug Administration approval and characterize the financial impact and influence in medicine. In this report, we also emphasize the value of observational studies in contemporary practice and identify other examples in medicine where accelerated Food and Drug Administration approval has been withdrawn. Importantly, the themes of the 17-alpha hydroxyprogesterone caproate story are not limited to obstetrics. It can also serve as a microcosm of issues within the US healthcare system, which ultimately contributes to the high cost of healthcare. In our opinion, the answer to the question is clear-the facts speak for themselves-and we believe 17-alpha hydroxyprogesterone caproate should not be endorsed for use to prevent recurrent preterm birth in the United States.

A systematic review of subgroup analyses in randomised clinical trials in cardiovascular disease

Background:

Subgroup analyses are frequently used to assess heterogeneity of treatment effects in randomised clinical trials. Inconsistent, improper and incomplete implementation, reporting and interpretation have been identified as ongoing challenges. Further, subgroup analyses were frequently criticised because of unreliable or potentially misleading results. More recently, recommendations and guidelines have been provided to improve the reporting of data in this regard.

Methods:

This systematic review was based on a literature search within the digital archives of three selected medical journals, The New England Journal of Medicine, The Lancet and Circulation. We reviewed articles of randomised clinical trials in the domain of cardiovascular disease which were published in 2015 and 2016. We screened and evaluated the selected articles for the mode of implementation and reporting of subgroup analyses.

Results:

We were able to identify a total of 130 eligible publications of randomised clinical trials. In 89/130 (68%) articles, results of at least one subgroup analysis were presented. This was dependent on the considered journal (p < 0.001), the number of included patients (p < 0.001) and the lack of statistical significance of a trial’s primary analysis (p < 0.001). The number of reported subgroup analyses ranged from 1 to 101 (median = 13). We were able to comprehend the specification time of reported subgroup analyses for 71/89 (80%) articles, with 55/89 (62%) articles presenting exclusively pre-specified analyses. This information was not always traceable on the basis of provided trial protocols and often did not include the pre-definition of cut-off values for the categorization of subgroups. The use of interaction tests was reported in 84/89 (94%) articles, with 36/89 (40%) articles reporting heterogeneity of the treatment effect for at least one primary or secondary trial outcome. Subgroup analyses were reported more frequently for larger randomised clinical trials, and if primary analyses did not reach statistical significance. Information about the implementation of subgroup analyses was reported most consistently for articles from The New England Journal of Medicine, since it was also traceable on the basis of provided trial protocols. We were able to comprehend whether subgroup analyses were pre-specified in a majority of the reviewed publications. Even though results of multiple subgroup analyses were reported for most published trials, a corresponding adjustment for multiple testing was rarely considered.

Conclusion:

Compared to previous reviews in this context, we observed improvements in the reporting of subgroup analyses of cardiovascular randomised clinical trials. Nonetheless, critical shortcomings, such as inconsistent reporting of the implementation and insufficient pre-specification, persist.

Pragmatic Recommendations for Therapeutics of Hospitalized COVID-19 Patients in Low- and Middle-Income Countries

The therapeutic options for COVID-19 patients are currently limited, but numerous randomized controlled trials are being completed, and many are on the way. For COVID-19 patients in low- and middle-income countries (LMICs), we recommend against using remdesivir outside of a clinical trial. We recommend against using hydroxychloroquine ± azithromycin or lopinavir-ritonavir. We suggest empiric antimicrobial treatment for likely coinfecting pathogens if an alternative infectious cause is likely. We suggest close monitoring without additional empiric antimicrobials if there are no clinical or laboratory signs of other infections. We recommend using oral or intravenous low-dose dexamethasone in adults with COVID-19 disease who require oxygen or mechanical ventilation. We recommend against using dexamethasone in patients with COVID-19 who do not require supplemental oxygen. We recommend using alternate equivalent doses of steroids in the event that dexamethasone is unavailable. We also recommend using low-dose corticosteroids in patients with refractory shock requiring vasopressor support. We recommend against the use of convalescent plasma and interleukin-6 inhibitors, such as tocilizumab, for the treatment of COVID-19 in LMICs outside of clinical trials.

An Evolutionary Medicine Perspective on Treatment of Pediatric Functional Abdominal Pain

In a recent issue, Kovacic et al. analyze data from a randomized sham-controlled trial and show that pretreatment vagal efficiency, an index related to respiratory sinus arrhythmia, is a predictor of pain improvement in adolescents with functional abdominal pain when treated with auricular percutaneous electrical nerve field stimulation. The underlying premise is the polyvagal hypothesis, an explanatory framework for the evolution of the mammalian autonomic nervous system, which proposes that functional gastrointestinal disorders can result from a chronic maladaptive state of autonomic neural control mechanisms after traumatic stress. This is an opportunity for us to stimulate physicians' interest in evolutionary medicine.

The ethical and legal status of neurosurgical guidelines: the neurosurgeon's golden fleece or Achilles' heel?

Neurosurgical guidelines are fundamental for evidence-based practice and have considerably increased both in number and content over the last decades. Yet, guidelines in neurosurgery are not without limitations, as they are overwhelmingly based on low-level evidence. Such recommendations have in the past been occasionally overturned by well-designed randomized controlled trials (RCTs), demonstrating the volatility of poorly underpinned evidence. Furthermore, even RCTs in surgery come with several limitations; most notably, interventions are often insufficiently standardized and assume a homogeneous patient population, which is not always applicable to neurosurgery. Lastly, guidelines are often outdated by the time they are published and smaller fields such as neurosurgery may lack a sufficient workforce to provide regular updates. These limitations raise the question of whether it is ethical to use low-level evidence for guideline recommendations, and if so, how strictly guidelines should be adhered to from an ethical and legal perspective. This article aims to offer a critical approach to the ethical and legal status of guidelines in neurosurgery. To this aim, the authors discuss: 1) the current state of neurosurgical guidelines and the evidence they are based on; 2) the degree of implementation of these guidelines; 3) the legal status of guidelines in medical disciplinary cases; and 4) the ethical balance between confident and critical use of guidelines. Ultimately, guidelines are neither laws that should always be followed nor purely academic efforts with little practical use. Every patient is unique, and tailored treatment defined by the surgeon will ensure optimal care; guidelines play an important role in creating a solid base that can be adhered to or deviated from, depending on the situation. From a research perspective, it is inevitable to rely on weaker evidence initially in order to generate more robust evidence later, and clinician-researchers have an ethical duty to contribute to generating and improving neurosurgical guidelines.

The Effect of Soluble Fiber Dextrin on Subjective and Physiological Markers of Appetite: A Randomized Trial

Obesity is a leading public health problem throughout the world. The development of foods that increase satiety and reduce food may aid weight management. This study determined the effect of consuming soluble fiber dextrin (SFD) on appetite, appetitive hormones, breath hydrogen and food intake in adults. Forty-three participants completed this study. For each treatment, 50% of the SFD was provided in liquid form as part of breakfast and 50% in solid form as a morning snack. Appetite questionnaires, blood and breath samples were collected immediately before breakfast and at regular intervals during the test session. The participants consumed an ad libitum lunch meal, afternoon snack and dinner meal, and the amount eaten was recorded. Following dinner, participants left the laboratory but were required to keep a diet diary for the remainder of the day. Breath hydrogen concentration was significantly higher following the consumption of SFD compared to control (p < 0.05). There was no observed overall treatment effect of consuming SFD on GLP-1 (Glucagon-Like-Peptide-1), ghrelin, CCK-8 (Cholecystokinin) or PYY_3-36 (Petptide YY) (p > 0.05). Moreover, consuming foods containing SFD had no effect on subjective appetite or food intake (p > 0.05). Consuming foods containing SFD increased breath hydrogen but did not influence food intake, appetite or appetitive hormones. However, the limitations of this study may have individually or collectively masked an effect of SFD on food intake and appetite.

Precision Medicine in Type 2 Diabetes: Using Individualized Prediction Models to Optimize Selection of Treatment

Despite the known heterogeneity of type 2 diabetes and variable response to glucose lowering medications, current evidence on optimal treatment is predominantly based on average effects in clinical trials rather than individual-level characteristics. A precision medicine approach based on treatment response would aim to improve on this by identifying predictors of differential drug response for people based on their characteristics and then using this information to select optimal treatment. Recent research has demonstrated robust and clinically relevant differential drug response with all noninsulin treatments after metformin (sulfonylureas, thiazolidinediones, dipeptidyl peptidase 4 [DPP-4] inhibitors, glucagon-like peptide 1 [GLP-1] receptor agonists, and sodium-glucose cotransporter 2 [SGLT2] inhibitors) using routinely available clinical features. This Perspective reviews this current evidence and discusses how differences in drug response could inform selection of optimal type 2 diabetes treatment in the near future. It presents a novel framework for developing and testing precision medicine-based strategies to optimize treatment, harnessing existing routine clinical and trial data sources. This framework was recently applied to demonstrate that "subtype" approaches, in which people are classified into subgroups based on features reflecting underlying pathophysiology, are likely to have less clinical utility compared with approaches that combine the same features as continuous measures in probabilistic "individualized prediction" models.

Safety and efficacy of probiotic administration to preterm infants: ten common questions

In spite of a large number of randomized placebo-controlled clinical trials and observational cohort studies including >50,000 preterm infants from 29 countries that have demonstrated a decrease in the risk of necrotizing enterocolitis, death, and sepsis, routine prophylactic probiotic administration to preterm infants remains uncommon in much of the world. This manuscript reflects talks given at NEC Society Symposium in 2019 and is not intended to be a state-of-the-art review or systematic review, but a summary of the probiotic-specific aspects of the symposium with limited additions including a recent strain-specific network analysis and position statement from the European Society for Paediatric Gastroenterology Hepatology and Nutrition (ESPGHAN). We address ten common questions related to the intestinal microbiome and probiotic administration to the preterm infant.

Checklist for clinical applicability of subgroup analysis

WHAT IS KNOWN AND OBJECTIVE

Subgroup analysis plays an important role in clinical decision-making. Correct management of subgroup analysis is necessary to optimize effectiveness, safety and efficiency of treatments. No homogeneous criteria have been developed for interpretation of subgroup analysis. In this study, the researcher develops a checklist to evaluate the reliability and applicability of the results of subset analyses.

METHODS

With a review of previous literature, three main criteria were included in the checklist: statistical association, biological plausibility and consistency among studies. Statistical association considered interaction probability, prespecification of analysis, number of subgroups analysed, sample size and positive/negative result in global analysis. Each item was given an indicative score. Total score was related to a level of applicability for the results in clinical practice. Checklist validation included interinvestigator concordance and assessment about utility. Three drug examples were used to validate the tool.

RESULTS AND DISCUSSION

Twenty-six evaluators showed adequate interinvestigator concordance (kappa 0.79, 1 and 0.83 for each drug example regarding applicability). Kappa values increased to 0.94, 1 and 1 after group discussion. Checklist utility score was greater than 4.7/5 in three drug examples. In pre-analysis, inter-researcher agreement on global applicability recommendation of subgroup results to practice was 92.3% (ramucirumab), 96% (nivolumab) and 100% (mepolizumab). In post-analysis, inter-researcher agreement on applicability recommendation of subgroup results was 100%, 94.45% and 100%, respectively. The checklist validation shows a high interindividual agreement of the results, both with respect to the evaluation of the applicability of subgroup analysis and concerning clinical decision-making.

WHAT IS NEW AND CONCLUSION

We have developed the first validated tool for interpretation of subgroup analyses. The checklist contributes to the adoption of homogeneous criteria for subgroup analyses, thereby allowing discussion and evaluation of the effects of a health intervention.

Characteristics and interpretation of subgroup analyses based on tumour characteristics in randomised trials testing target-specific anticancer drugs: design of a systematic survey

BACKGROUND

Target-specific anticancer drugs are under rapid development. Little is known, however, about the risk of administering target-specific drugs to patients who have tumours with molecular alterations or other characteristics that can make the drug ineffective or even harmful. An increasing number of randomised clinical trials (RCTs) investigating target-specific anticancer drugs include subgroup analyses based on tumour characteristics. Such subgroup analyses have the potential to be more credible and influential than subgroup analyses based on traditional factors such as sex or tumour stage. In addition, they may more frequently lead to qualitative subgroup effects, that is, show benefit in one but harm in another subgroup of patients (eg, if the tumour characteristic makes the drug ineffective or even enhance tumour growth). If so, subgroup analyses based on tumour characteristics would be highly relevant for patient safety. The aim of this study is to systematically assess the frequency and characteristics of subgroup analyses based on tumour characteristics, the frequency of qualitative subgroup effects, their credibility, and the interpretations that investigators and guidelines developers report.

METHODS AND ANALYSIS

We will perform a systematic survey of 433 RCTs testing the effect of target-specific anticancer drugs. Teams of methodologically trained investigators and oncologists will identify eligible studies, extract relevant data and assess the credibility of putative subgroup effects using a recently developed formal instrument. We will systematically assess how trial investigators interpret apparent subgroup effects based on tumour characteristics and the extent to which they influence subsequent practice guidelines. Our results will provide empirical data characterising an increasingly used type of subgroup analysis in cancer trials and its potential impact on precision medicine to predict benefit or harm.

ETHICS AND DISSEMINATION

Formal ethical approval is not required for this study. We will disseminate the findings in a peer-reviewed and open-access journal publication.

Individual participant data meta-analysis to examine interactions between treatment effect and participant-level covariates: Statistical recommendations for conduct and planning

Precision medicine research often searches for treatment‐covariate interactions, which refers to when a treatment effect (eg, measured as a mean difference, odds ratio, hazard ratio) changes across values of a participant‐level covariate (eg, age, gender, biomarker). Single trials do not usually have sufficient power to detect genuine treatment‐covariate interactions, which motivate the sharing of individual participant data (IPD) from multiple trials for meta‐analysis. Here, we provide statistical recommendations for conducting and planning an IPD meta‐analysis of randomized trials to examine treatment‐covariate interactions. For conduct, two‐stage and one‐stage statistical models are described, and we recommend: (i) interactions should be estimated directly, and not by calculating differences in meta‐analysis results for subgroups; (ii) interaction estimates should be based solely on within‐study information; (iii) continuous covariates and outcomes should be analyzed on their continuous scale; (iv) nonlinear relationships should be examined for continuous covariates, using a multivariate meta‐analysis of the trend (eg, using restricted cubic spline functions); and (v) translation of interactions into clinical practice is nontrivial, requiring individualized treatment effect prediction. For planning, we describe first why the decision to initiate an IPD meta‐analysis project should not be based on between‐study heterogeneity in the overall treatment effect; and second, how to calculate the power of a potential IPD meta‐analysis project in advance of IPD collection, conditional on characteristics (eg, number of participants, standard deviation of covariates) of the trials (potentially) promising their IPD. Real IPD meta‐analysis projects are used for illustration throughout.

The evidence base for psychotropic drugs approved by the European Medicines Agency: a meta-assessment of all European Public Assessment Reports

AIMS

To systematically assess the level of evidence for psychotropic drugs approved by the European Medicines Agency (EMA).

METHODS

Cross-sectional analysis of all European Public Assessment Reports (EPARs) and meta-analyses of the many studies reported in these EPARs. Eligible EPARs were identified from the EMA's website and individual study reports were requested from the Agency when necessary. All marketing authorisation applications (defined by the drug, the route of administration and given indications) for psychotropic medications for adults (including drugs used in psychiatry and addictology) were considered. EPARs solely based on bioequivalence studies were excluded. Our primary outcome measure was the presence of robust evidence of comparative effectiveness, defined as at least two 'positive' superiority studies against an active comparator. Various other features of the approvals were assessed, such as evidence of non-inferiority v. active comparator and superiority v. placebo. For studies with available data, effect sizes were computed and pooled using a random effect meta-analysis for each dose of each drug in each indication.

RESULTS

Twenty-seven marketing authorisations were identified. For one, comparative effectiveness was explicitly considered as not needed in the EPAR. Of those remaining, 21/26 (81%) did not provide any evidence of superiority against an active comparator, 2/26 (8%) were based on at least two trials showing superiority against active comparator and three (11%) were based on one positive trial; 1/26 provided evidence for two positive non-inferiority analyses v. active comparator and seven (26%) provided evidence for one. In total, 20/27 (74%) evaluations reported evidence of superiority v. placebo with two or more trials. Among the meta-analyses of initiation studies against active comparator (57 available comparisons), the median effect size was 0.051 (range -0.503; 0.318). Twenty approved evaluations (74%) reported evidence of superiority v. placebo on the basis of two or more initiation trials and seven based on a single trial. Among meta-analyses of initiation studies against placebo (125 available comparisons), the median effect size was -0.283 (range -0.820; 0.091). Importantly, among the 89 study reports requested on the EMA website, only 19 were made available 1 year after our requests.

CONCLUSIONS

The evidence for psychiatric drug approved by the EMA was in general poor. Small to modest effects v. placebo were considered sufficient in indications where an earlier drug exists. Data retrieval was incomplete after 1 year despite EMA's commitment to transparency. Improvements are needed.

Preterm neonates benefit from low prophylactic platelet transfusion threshold despite varying risk of bleeding or death

The Platelets for Neonatal Thrombocytopenia (PlaNeT-2) trial reported an unexpected overall benefit of a prophylactic platelet transfusion threshold of 25 × 109/L compared with 50 × 109/L for major bleeding and/or mortality in preterm neonates (7% absolute-risk reduction). However, some neonates in the trial may have experienced little benefit or even harm from the 25 × 109/L threshold. We wanted to assess this heterogeneity of treatment effect in the PlaNet-2 trial, to investigate whether all preterm neonates benefit from the low threshold. We developed a multivariate logistic regression model in the PlaNet-2 data to predict baseline risk of major bleeding and/or mortality for all 653 neonates. We then ranked the neonates based on their predicted baseline risk and categorized them into 4 risk quartiles. Within these quartiles, we assessed absolute-risk difference between the 50 × 109/L- and 25 × 109/L-threshold groups. A total of 146 neonates died or developed major bleeding. The internally validated C-statistic of the model was 0.63 (95% confidence interval, 0.58-0.68). The 25 × 109/L threshold was associated with absolute-risk reduction in all risk groups, varying from 4.9% in the lowest risk group to 12.3% in the highest risk group. These results suggest that a 25 × 109/L prophylactic platelet count threshold can be adopted in all preterm neonates, irrespective of predicted baseline outcome risk. Future studies are needed to improve the predictive accuracy of the baseline risk model. This trial was registered at www.isrctn.com as #ISRCTN87736839.

How often can meta-analyses of individual-level data individualize treatment? A meta-epidemiologic study

BACKGROUND

One of the claimed main advantages of individual participant data meta-analysis (IPDMA) is that it allows assessment of subgroup effects based on individual-level participant characteristics, and eventually stratified medicine. In this study, we evaluated the conduct and results of subgroup analyses in IPDMA.

METHODS

We searched PubMed, EMBASE and the Cochrane Library from inception to 31 December 2014. We included papers if they described an IPDMA based on randomized clinical trials that investigated a therapeutic intervention on human subjects and in which the meta-analysis was preceded by a systematic literature search. We extracted data items related to subgroup analysis and subgroup differences (subgroup-treatment interaction p < 0.05).

RESULTS

Overall, 327 IPDMAs were eligible. A statistically significant subgroup-treatment interaction for the primary outcome was reported in 102 (36.6%) of 279 IPDMAs that reported at least one subgroup analysis. This corresponded to 187 different statistically significant subgroup-treatment interactions: 124 for an individual-level subgrouping variable (in 76 IPDMAs) and 63 for a group-level subgrouping variable (in 36 IPDMAs). Of the 187, only 7 (3.7%; 6 individual and 1 group-level subgrouping variables) had a large difference between strata (standardized effect difference d  ≥  0.8). Among the 124 individual-level statistically significant subgroup differences, the IPDMA authors claimed that 42 (in 21 IPDMAs) should lead to treating the subgroups differently. None of these 42 had d  ≥  0.8.

CONCLUSIONS

Availability of individual-level data provides statistically significant interactions for relative treatment effects in about a third of IPDMAs. A modest number of these interactions may offer opportunities for stratified medicine decisions.

Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review

Background

A relatively novel method of appraisal, methodological reviews (MRs) are used to synthesize information on the methods used in health research. There are currently no guidelines available to inform the reporting of MRs.

Objectives

This pilot review aimed to determine the feasibility of a full review and the need for reporting guidance for methodological reviews.

Methods

Search strategy: We conducted a search of PubMed, restricted to 2017 to include the most recently published studies, using different search terms often used to describe methodological reviews: "literature survey" OR "meta-epidemiologic* review" OR "meta-epidemiologic* survey" OR "methodologic* review" OR "methodologic* survey" OR "systematic survey."Data extraction: Study characteristics including country, nomenclature, number of included studies, search strategy, a priori protocol use, and sampling methods were extracted in duplicate and summarized.Outcomes: Primary feasibility outcomes were the sensitivity and specificity of the search terms (criteria for success of feasibility set at sensitivity and specificity of ≥ 70%).Analysis: The estimates are reported as a point estimate (95% confidence interval).

Results

Two hundred thirty-six articles were retrieved and 31 were included in the final analysis. The most accurate search term was "meta-epidemiological" (sensitivity [Sn] 48.39; 95% CI 31.97-65.16; specificity [Sp] 97.56; 94.42-98.95). The majority of studies were published by authors from Canada (n = 12, 38.7%), and Japan and USA (n = 4, 12.9% each). The median (interquartile range [IQR]) number of included studies in the MRs was 77 (13-1127). Reporting of a search strategy was done in most studies (n = 23, 74.2%). The use of a pre-published protocol (n = 7, 22.6%) or a justifiable sampling method (n = 5, 16.1%) occurred rarely.

Conclusions

Using the MR nomenclature identified, it is feasible to build a comprehensive search strategy and conduct a full review. Given the variation in reporting practices and nomenclature attributed to MRs, there is a need for guidance on standardized and transparent reporting of MRs. Future guideline development would likely include stakeholders from Canada, USA, and Japan.

Re-Evaluating Evidence for Best Practice in Paediatric Speech-Language Pathology

BACKGROUND

Systematic reviews of treatment trials for children with speech and language difficulties often exemplify the limited clinical usefulness of the evidence base, reflecting recent literature in evidence-based medicine. Other studies report that clinicians often fail to seek information about best practice, across the health professions. Consequently, clinical researchers, including those in speech-language pathology, have sought alternative methodologies for determining best practice.

SUMMARY

Some approaches focus on "pragmatic trials," usually as part of existing health services. Others place case management of individuals at the centre of intervention presenting studies of one or more cases, including N-of-1 randomized controlled trials and cross-over group designs. Clinical case studies can provide important theoretical data contributing to our understanding of the development of typical and atypical communication. Precision medicine (also known as personalized medicine) is an emerging approach to building the clinical evidence base that acknowledges the importance of individual genetic and environmental differences between people. With increasing knowledge of aetiological heterogeneity, even within children presenting with the same diagnosis (e.g., childhood apraxia of speech), data reinforce the edict that children are not all born equal. Key Message: This review argues that to understand response to treatment, it is critical to examine child-related factors as well as the variables of the intervention itself.

The Predictive Approaches to Treatment effect Heterogeneity (PATH) Statement: Explanation and Elaboration

The PATH (Predictive Approaches to Treatment effect Heterogeneity) Statement was developed to promote the conduct of, and provide guidance for, predictive analyses of heterogeneity of treatment effects (HTE) in clinical trials. The goal of predictive HTE analysis is to provide patient-centered estimates of outcome risk with versus without the intervention, taking into account all relevant patient attributes simultaneously, to support more personalized clinical decision making than can be made on the basis of only an overall average treatment effect. The authors distinguished 2 categories of predictive HTE approaches (a "risk-modeling" and an "effect-modeling" approach) and developed 4 sets of guidance statements: criteria to determine when risk-modeling approaches are likely to identify clinically meaningful HTE, methodological aspects of risk-modeling methods, considerations for translation to clinical practice, and considerations and caveats in the use of effect-modeling approaches. They discuss limitations of these methods and enumerate research priorities for advancing methods designed to generate more personalized evidence. This explanation and elaboration document describes the intent and rationale of each recommendation and discusses related analytic considerations, caveats, and reservations.

Exploring app features with outcomes in mHealth studies involving chronic respiratory diseases, diabetes, and hypertension: a targeted exploration of the literature

Objectives

Limited data are available on the correlation of mHealth features and statistically significant outcomes. We sought to identify and analyze: types and categories of features; frequency and number of features; and relationship of statistically significant outcomes by type, frequency, and number of features.

Materials and Methods

This search included primary articles focused on app-based interventions in managing chronic respiratory diseases, diabetes, and hypertension. The initial search yielded 3622 studies with 70 studies meeting the inclusion criteria. We used thematic analysis to identify 9 features within the studies.

Results

Employing existing terminology, we classified the 9 features as passive or interactive. Passive features included: 1) one-way communication; 2) mobile diary; 3) Bluetooth technology; and 4) reminders. Interactive features included: 1) interactive prompts; 2) upload of biometric measurements; 3) action treatment plan/personalized health goals; 4) 2-way communication; and 5) clinical decision support system.

Discussion

Each feature was included in only one-third of the studies with a mean of 2.6 mHealth features per study. Studies with statistically significant outcomes used a higher combination of passive and interactive features (69%). In contrast, studies without statistically significant outcomes exclusively used a higher frequency of passive features (46%). Inclusion of behavior change features (ie, plan/goals and mobile diary) were correlated with a higher incident of statistically significant outcomes (100%, 77%).

Conclusion

This exploration is the first step in identifying how types and categories of features impact outcomes. While the findings are inconclusive due to lack of homogeneity, this provides a foundation for future feature analysis.