Does evidence support the high expectations placed in precision medicine? A bibliographic review

Design, analysis, and interpretation of treatment response heterogeneity in personalized nutrition and obesity treatment research

It is increasingly assumed that there is no one-size-fits-all approach to dietary recommendations for the management and treatment of chronic diseases such as obesity. This phenomenon that not all individuals respond uniformly to a given treatment has become an area of research interest given the rise of personalized and precision medicine. To conduct, interpret, and disseminate this research rigorously and with scientific accuracy, however, requires an understanding of treatment response heterogeneity. Here, we define treatment response heterogeneity as it relates to clinical trials, provide statistical guidance for measuring treatment response heterogeneity, and highlight study designs that can quantify treatment response heterogeneity in nutrition and obesity research. Our goal is to educate nutrition and obesity researchers in how to correctly identify and consider treatment response heterogeneity when analyzing data and interpreting results, leading to rigorous and accurate advancements in the field of personalized medicine.

Is it worth it to personalize the treatment of PTSD? - A variance-ratio meta-analysis and estimation of treatment effect heterogeneity in RCTs of PTSD

Several evidence-based treatments for posttraumatic stress disorder (PTSD) are recommended by international guidelines (e.g., APA, NICE). While their average effects are in general high, non-response rates indicate differential treatment effects. Here, we used a large database of RCTs on psychotherapy for PTSD to determine a reliable estimate of this heterogeneity in treatment effects (HTE) by applying Bayesian variance ratio meta-analysis. In total, 66 studies with a total of 8803 patients were included in our study. HTE was found for all psychological treatments, with varying degrees of certainty, only slight differences between psychological treatments, and active control groups yielding a smaller variance ratio compared to waiting list control groups. Across all psychological treatment and control group types, the estimate for the intercept was 0.12, indicating a 12% higher variance of posttreatment values in the intervention groups after controlling for differences in treatment outcomes. This study is the first to determine the maximum increase in treatment effects of psychological treatments for PTSD by personalization. The results indicate that there is comparatively high heterogeneity in treatment effects across all psychological treatment and control groups, which in turn allow personalizing psychological treatments by using treatment selection approaches.

Methods for Stratification and Validation Cohorts: A Scoping Review

Personalized medicine requires large cohorts for patient stratification and validation of patient clustering. However, standards and harmonized practices on the methods and tools to be used for the design and management of cohorts in personalized medicine remain to be defined. This study aims to describe the current state-of-the-art in this area. A scoping review was conducted searching in PubMed, EMBASE, Web of Science, Psycinfo and Cochrane Library for reviews about tools and methods related to cohorts used in personalized medicine. The search focused on cancer, stroke and Alzheimer’s disease and was limited to reports in English, French, German, Italian and Spanish published from 2005 to April 2020. The screening process was reported through a PRISMA flowchart. Fifty reviews were included, mostly including information about how data were generated (25/50) and about tools used for data management and analysis (24/50). No direct information was found about the quality of data and the requirements to monitor associated clinical data. A scarcity of information and standards was found in specific areas such as sample size calculation. With this information, comprehensive guidelines could be developed in the future to improve the reproducibility and robustness in the design and management of cohorts in personalized medicine studies.

Detecting Heterogeneity of Intervention Effects Using Analysis and Meta-analysis of Differences in Variance Between Trial Arms

BACKGROUND

Randomized controlled trials (RCTs) with continuous outcomes usually only examine mean differences in response between trial arms. If the intervention has heterogeneous effects, then outcome variances will also differ between arms. Power of an individual trial to assess heterogeneity is lower than the power to detect the same size of main effect.

METHODS

We describe several methods for assessing differences in variance in trial arms and apply them to a single trial with individual patient data and to meta-analyses using summary data. Where individual data are available, we use regression-based methods to examine the effects of covariates on variation. We present an additional method to meta-analyze differences in variances with summary data.

RESULTS

In the single trial, there was agreement between methods, and the difference in variance was largely due to differences in prevalence of depression at baseline. In two meta-analyses, most individual trials did not show strong evidence of a difference in variance between arms, with wide confidence intervals. However, both meta-analyses showed evidence of greater variance in the control arm, and in one example, this was perhaps because mean outcome in the control arm was higher.

CONCLUSIONS

Using meta-analysis, we overcame low power of individual trials to examine differences in variance using meta-analysis. Evidence of differences in variance should be followed up to identify potential effect modifiers and explore other possible causes such as varying compliance.

Treatment effect variability in brain stimulation across psychiatric disorders: A meta-analysis of variance

Noninvasive brain stimulation methods such as transcranial magnetic stimulation (TMS) and transcranial direct current stimulation (tDCS) are promising add-on treatments for a number of psychiatric conditions. Yet, some of the initial excitement is wearing off. Randomized controlled trials (RCT) have found inconsistent results. This inconsistency is suspected to be the consequence of variation in treatment effects and solvable by identifying responders in RCTs and individualizing treatment. However, is there enough evidence from RCTs that patients respond differently to treatment? This question can be addressed by comparing the variability in the active stimulation group with the variability in the sham group. We searched MEDLINE/PubMed and included all double-blinded, sham-controlled RCTs and crossover trials that used TMS or tDCS in adults with a unipolar or bipolar depression, bipolar disorder, schizophrenia spectrum disorder, or obsessive compulsive disorder. In accordance with the PRISMA guidelines to ensure data quality and validity, we extracted a measure of variability of the primary outcome. A total of 130 studies with 5748 patients were considered in the analysis. We calculated variance-weighted variability ratios for each comparison of active stimulation vs sham and entered them into a random-effects model. We hypothesized that treatment effect variability in TMS or tDCS would be reflected by increased variability after active compared with sham stimulation, or in other words, a variability ratio greater than one. Across diagnoses, we found only a minimal increase in variability after active stimulation compared with sham that did not reach statistical significance (variability ratio = 1.03; 95% CI, 0.97, 1.08, P = 0.358). In conclusion, this study found little evidence for treatment effect variability in brain stimulation, suggesting that the need for personalized or stratified medicine is still an open question.

Evaluation of variability in individual response to treatments in the clinical high-risk state for psychosis: A meta-analysis

BACKGROUND

Individuals at Clinical High Risk for Psychosis (CHR-P) may differ considerably in their response to indicated preventive interventions. No studies have tested this.

METHOD

PRISMA-compliant systematic review of the Web of Science (MEDLINE), PsycInfo, CENTRAL and unpublished/gray literature up to 1 September 2019. RCTs in CHR-P individuals, reporting on attenuated positive psychotic symptoms were included. The primary outcome was the variability ratio between the variance of the severity of attenuated positive psychotic symptoms in the indicated intervention condition vs the control condition (needs-based interventions, NBI) at 6 and 12 months. Random effect models, C statistics, meta-regressions/sensitivity analyses and Cochrane Risk of Bias assessment were performed.

RESULTS

Overall, 1707 individuals from 14 RCTs (57% male, mean age = 20) reporting on the impact of preventive interventions on attenuated positive psychotic symptoms were included. At 6 months, the variability ratio was 1 (95% CI 0.89-1.12). At 12 months, the variability ratio was higher in the indicated intervention compared to the NBI condition but not statistically different: 1.09 (95% CI 0.94-1.25). Between-study heterogeneity was serious (I² = 51% and 68%, respectively), but sensitivity analysis suggested it may be related to two outlying studies or larger variability in the response to treatment in small studies.

CONCLUSIONS

There is no evidence for individual differences in CHR-P response to preventive treatments. Although the study cannot exclude that subsets of CHR-P individuals may respond differently to preventive treatments, it indicates that the average effect of preventive interventions is a reasonable estimate for the CHR-P individual.

Methods and Challenges for Assessing Heterogeneity

The widely acknowledged homogeneity assumption limits progress in refining clinical diagnosis, understanding mechanisms, and developing new treatments for mental health disorders. This homogeneity assumption drives both a comorbidity and a heterogeneity problem, where two different approaches tackle the problems. One, a unifying approach, tackles the comorbidity problem by assuming that a single general psychopathology factor underlies multiple disorders. Another, a multifactorial approach, tackles the heterogeneity problem by assuming that disorders comprise multiple subtypes driven by multiple discrete factors. We show how each of these approaches can make useful contributions to mental health-related research and clinical practice. For example, the unifying approach can develop a rapid assessment tool that may be clinically valuable for triaging cases. The multifactorial approach can reveal subtypes that are differentially responsive to treatments and highlight distinct mechanisms leading to similar phenotypes. Because both approaches tackle different problems, both have different limitations. We describe the statistical frameworks that incorporate and adjudicate between both approaches (e.g., the bifactor model, normative modeling, and the functional random forest). Such frameworks can identify whether sets of disorders are more affected by heterogeneity or comorbidity. Therefore, future studies that incorporate such frameworks can provide further insight into the nature of psychopathology.