Is preclinical research in cancer biology reproducible enough?

Artificial intelligence for response prediction and personalisation in radiation oncology

Artificial intelligence (AI) systems may personalise radiotherapy by assessing complex and multifaceted patient data and predicting tumour and normal tissue responses to radiotherapy. Here we describe three distinct generations of AI systems, namely personalised radiotherapy based on pretreatment data, response-driven radiotherapy and dynamically optimised radiotherapy. Finally, we discuss the main challenges in clinical translation of AI systems for radiotherapy personalisation.

A systematic assessment of preclinical multilaboratory studies and a comparison to single laboratory studies

Background

Multicentric approaches are widely used in clinical trials to assess the generalizability of findings, however, they are novel in laboratory-based experimentation. It is unclear how multilaboratory studies may differ in conduct and results from single lab studies. Here, we synthesized the characteristics of these studies and quantitatively compared their outcomes to those generated by single laboratory studies.

Methods

MEDLINE and Embase were systematically searched. Screening and data extractions were completed in duplicate by independent reviewers. Multilaboratory studies investigating interventions using in vivo animal models were included. Study characteristics were extracted. Systematic searches were then performed to identify single lab studies matched by intervention and disease. Difference in standardized mean differences (DSMD) was then calculated across studies to assess differences in effect estimates based on study design (>0 indicates larger effects in single lab studies).

Results

Sixteen multilaboratory studies met inclusion criteria and were matched to 100 single lab studies. The multicenter study design was applied across a diverse range of diseases, including stroke, traumatic brain injury, myocardial infarction, and diabetes. The median number of centers was four (range 2-6) and the median sample size was 111 (range 23-384) with rodents most frequently used. Multilaboratory studies adhered to practices that reduce the risk of bias significantly more often than single lab studies. Multilaboratory studies also demonstrated significantly smaller effect sizes than single lab studies (DSMD 0.72 [95% confidence interval 0.43-1]).

Conclusions

Multilaboratory studies demonstrate trends that have been well recognized in clinical research (i.e. smaller treatment effects with multicentric evaluation and greater rigor in study design). This approach may provide a method to robustly assess interventions and the generalizability of findings between laboratories.

Funding

uOttawa Junior Clinical Research Chair; The Ottawa Hospital Anesthesia Alternate Funds Association; Canadian Anesthesia Research Foundation; Government of Ontario Queen Elizabeth II Graduate Scholarship in Science and Technology.

Large inherent variability in data derived from highly standardised cell culture experiments

Cancer drug development is hindered by high clinical attrition rates, which are blamed on weak predictive power by preclinical models and limited replicability of preclinical findings. However, the technically feasible level of replicability remains unknown. To fill this gap, we conducted an analysis of data from the NCI60 cancer cell line screen (2.8 million compound/cell line experiments), which is to our knowledge the largest depository of experiments that have been repeatedly performed over decades. The findings revealed profound intra-laboratory data variability, although all experiments were executed following highly standardised protocols that avoid all known confounders of data quality. All compound/ cell line combinations with > 100 independent biological replicates displayed maximum GI50 (50% growth inhibition) fold changes (highest/ lowest GI50) > 5% and 70.5% displayed maximum fold changes > 1000. The highest maximum fold change was 3.16 × 10¹⁰ (lowest GI50: 7.93 ×10^-10 µM, highest GI50: 25.0 µM). FDA-approved drugs and experimental agents displayed similar variation. Variability remained high after outlier removal, when only considering experiments that tested drugs at the same concentration range, and when only considering NCI60-provided quality-controlled data. In conclusion, high variability is an intrinsic feature of anti-cancer drug testing, even among standardised experiments in a world-leading research environment. Awareness of this inherent variability will support realistic data interpretation and inspire research to improve data robustness. Further research will have to show whether the inclusion of a wider variety of model systems, such as animal and/ or patient-derived models, may improve data robustness.

Multiple deadlocks in the development of nonprofit drugs

The current Coronavirus 2019 (COVID-19) pandemic has shown us that the pharmaceutical research community can organize and administer large nonprofit clinical trials (RECOVERY and SOLIDARITY) and achieve the swift development of common, unpatentable drugs for a new indication: in this case an old, inexpensive drug, dexamethasone, for COVID-19. Why is it that such nonprofit efforts are so rare and are not organized as a systemic, routine part of drug development in the public interest? Based on my own experience with repurposing the alcohol-abuse drug disulfiram (Antabuse) for cancer, I identify at least four serious deadlocks to development of nonprofit drugs. All of these obstacles should be addressed to leverage the potential of the COVID-19 pandemic for better future healthcare systems in all countries around the world.

Ethical Considerations for Phase I Trials in Oncology

Phase I trials often represent the first occasion where new cancer strategies are tested in patients. Various developments in cancer biology, methodology, regulation, and medical ethics have altered the ethical landscape of such trials. We provide a narrative review of contemporary ethical challenges in design, conduct, and reporting of phase I cancer trials and outline recommendations for addressing each. We organized our review around four topics, supplementing the first three with scoping reviews: (1) benefit/risk, (2) research biopsies, (3) therapeutic misconception and misestimation, and (4) reporting. The main ethical challenges of conducting phase I trials stem from three issues. First, phase I trials often involve higher research burden and scientific uncertainty compared with other cancer trials. Second, many patients arrive at phase I trials at a transitional point in their illness trajectory where they have exhausted standard survival-extending options. Third, phase I trial results play a major role in informing downstream drug development and regulatory decisions. Together, these issues create distinct pressures for study design, ethical review, informed consent, and reporting. Developments in methodology, regulation, cancer biology, and ethical awareness have helped mitigate some of these challenges, while introducing others. We conclude our review with a series of recommendations regarding trial design, ethical review, consent, and reporting. We also outline several unresolved questions that, if addressed, would strengthen the ethical foundation of phase I cancer trials.

What have we learned?

As the final outputs of the Reproducibility Project: Cancer Biology are published, it is clear that preclinical research in cancer biology is not as reproducible as it should be.