1
|
Meinhardt MW, Gerlach B, Spanagel R. Good Practice Guideline for Preclinical Alcohol Research: The STRINGENCY Framework. Curr Top Behav Neurosci 2024. [PMID: 39117860 DOI: 10.1007/7854_2024_484] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/10/2024]
Abstract
Research in the field of preclinical alcohol research, but also science in general, has a problem: Many published scientific results cannot be repeated. As a result, findings from preclinical research often do not translate well to humans, causing increasing disappointment and calls for restructuring of preclinical research, that is, better reproducibility of preclinical research. However, the replication crisis is an inherent problem in biomedical research. Replication failures are not only due to small experimental variations but are often the result of poor methodology. In response to the replication crisis, numerous guidelines and recommendations have been proposed to promote transparency, rigor, and reproducibility in scientific research. What is missing today is a framework that integrates all the confusing information that results from all these guidelines and recommendations. Here we present STRINGENCY, an integrative approach to good practice guidelines for preclinical alcohol research, which can also apply to behavioral research in general and which aims to improve preclinical research to better prepare it for translation and minimize the "valley of death" in translational research. STRINGENCY includes systematic review and, when possible, meta-analysis prior to study design, sample size calculation, preregistration, multisite experiments, scientific data management (FAIR), reporting of data using ARRIVE, generalization of research data, and transparent publications that allow reporting of null results. We invite the scientific community to adopt STRINGENCY to improve the reliability and impact of preclinical alcohol research.
Collapse
Affiliation(s)
- Marcus W Meinhardt
- Institute of Psychopharmacology, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany.
- Department of Molecular Neuroimaging, Medical Faculty Mannheim, Central Institute of Mental Health, University of Heidelberg, Heidelberg, Germany.
| | - Björn Gerlach
- Institute of Psychopharmacology, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany
- Guarantors of EQIPD e.V., Heidelberg, Germany
- PAASP GmbH, Heidelberg, Germany
| | - Rainer Spanagel
- Institute of Psychopharmacology, Central Institute of Mental Health, Medical Faculty Mannheim, University of Heidelberg, Heidelberg, Germany.
- German Center for Mental Health (DZPG), Mannheim, Heidelberg, Ulm, Germany.
| |
Collapse
|
2
|
Sciascia A, Smith M, Holstedt J, Mattingly L, Kibler WB. Utilizing the Pronated Forearm Technique for Measuring Glenohumeral External Rotation in Baseball Players. Sports Health 2024:19417381241235225. [PMID: 38500012 DOI: 10.1177/19417381241235225] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/20/2024] Open
Abstract
BACKGROUND Alterations in glenohumeral internal rotation (GIR), glenohumeral external rotation (GER), and total range of motion (TROM) have been linked with increased injury risk. GER capacity has been measured routinely with the forearm in neutral rotation (GERN), but a recent study reported GERN was greater than GER with the forearm in pronation (GERP) in Minor League pitchers. This work has not yet been replicated or extended to other groups. HYPOTHESIS GERP would be significantly less than GERN in Independent League baseball pitchers, and there would be no difference in GERP or GERN measurements between this new group and the previous group of Minor League pitchers. STUDY DESIGN Cross-sectional study. LEVEL OF EVIDENCE Level 3. METHODS Goniometric measurements were recorded for bilateral GIR, GERN, and GERP, and resulting TROM for 37 Independent League baseball pitchers. These data were compared with the previous study. All motions were compared individually between groups, between throwing and nonthrowing arm, and both within and between techniques (forearm neutral or pronated). RESULTS GERP was significantly less than GERN for both arms within each group tested (P < 0.01). Independent League pitchers had greater between arm differences for GIR (-16.9° vs -6.9°), GERN (+15.1° vs -0.6°), and GERP (+13.1° vs -5.9°) compared with Minor League pitchers. TROM for the Independent League pitchers was not statistically different for either measurement technique, while TROM for the throwing arm of the Minor League pitchers was statistically reduced with varying effect sizes (d = 0.35-0.99) compared with the nonthrowing arm (P < 0.01). CONCLUSION This study confirmed earlier findings that the pronated forearm resulted in decreased GER capacity, illustrating the adaptive response to throwing and the need to evaluate for this variable. CLINICAL RELEVANCE GERP should be evaluated in all groups of pitchers, but there may be variations within tested groups.
Collapse
Affiliation(s)
- Aaron Sciascia
- Institute for Clinical Outcomes and Research, Lexington Clinic, Lexington, Kentucky
| | - Michael Smith
- Department of Orthopedics-Sports Medicine, Lexington Clinic, Lexington, Kentucky
| | - Joseph Holstedt
- Department of Orthopedics-Sports Medicine, Lexington Clinic, Lexington, Kentucky
| | - Logan Mattingly
- Department of Orthopedics-Sports Medicine, Lexington Clinic, Lexington, Kentucky
| | - W Ben Kibler
- Shoulder Center of Kentucky, Lexington Clinic, Lexington, Kentucky
| |
Collapse
|
3
|
Iacobazzi D, Ghorbel MT, Rapetto F, Narayan SA, Deutsch J, Salih T, Harris AG, Skeffington KL, Parry R, Parolari G, Chanoit G, Caputo M. Growth capacity of a Wharton's Jelly derived mesenchymal stromal cells tissue engineered vascular graft used for main pulmonary artery reconstruction in piglets. Front Bioeng Biotechnol 2024; 12:1360221. [PMID: 38464540 PMCID: PMC10920298 DOI: 10.3389/fbioe.2024.1360221] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2023] [Accepted: 02/16/2024] [Indexed: 03/12/2024] Open
Abstract
Background: Surgical treatment of congenital heart defects affecting the right ventricular outflow tract (RVOT) often requires complex reconstruction and multiple reoperations due to structural degeneration and lack of growth of currently available materials. Hence, alternative approaches for RVOT reconstruction, which meet the requirements of biocompatibility and long-term durability of an ideal scaffold, are needed. Through this full scale pre-clinical study, we demonstrated the growth capacity of a Wharton's Jelly derived mesenchymal stromal cells (WJ-MSC) tissue engineered vascular graft used in reconstructing the main pulmonary artery in piglets, providing proof of biocompatibility and efficacy. Methods: Sixteen four-week-old Landrace pigs were randomized to undergo supravalvar Main Pulmonary Artery (MPA) replacement with either unseeded or WJ-MSCs-seeded Small Intestinal Submucosa-derived grafts. Animals were followed up for 6 months by clinical examinations and cardiac imaging. At termination, sections of MPAs were assessed by macroscopic inspection, histology and fluorescent immunohistochemistry. Results: Data collected at 6 months follow up showed no sign of graft thrombosis or calcification. The explanted main pulmonary arteries demonstrated a significantly higher degree of cellular organization and elastin content in the WJ-MSCs seeded grafts compared to the acellular counterparts. Transthoracic echocardiography and cardiovascular magnetic resonance confirmed the superior growth and remodelling of the WJ-MSCs seeded conduit compared to the unseeded. Conclusion: Our findings indicate that the addition of WJ-MSCs to the acellular scaffold can upgrade the material, converting it into a biologically active tissue, with the potential to grow, repair and remodel the RVOT.
Collapse
Affiliation(s)
- Dominga Iacobazzi
- Translational Health Sciences, University of Bristol, Bristol, United Kingdom
| | - Mohamed T. Ghorbel
- Translational Health Sciences, University of Bristol, Bristol, United Kingdom
| | - Filippo Rapetto
- Translational Health Sciences, University of Bristol, Bristol, United Kingdom
- Department of Cardiac Surgery, Bristol Royal Hospital for Children, Bristol, United Kingdom
| | - Srinivas A. Narayan
- Department of Paediatric Cardiology, Bristol Royal Hospital for Children, Bristol, United Kingdom
| | - Julia Deutsch
- Langford Clinical Veterinary Services, University of Bristol, Bristol, United Kingdom
| | - Tasneem Salih
- Translational Health Sciences, University of Bristol, Bristol, United Kingdom
| | - Amy G. Harris
- Translational Health Sciences, University of Bristol, Bristol, United Kingdom
| | | | - Richard Parry
- Translational Health Sciences, University of Bristol, Bristol, United Kingdom
| | - Giulia Parolari
- Translational Health Sciences, University of Bristol, Bristol, United Kingdom
| | | | - Massimo Caputo
- Translational Health Sciences, University of Bristol, Bristol, United Kingdom
- Department of Cardiac Surgery, Bristol Royal Hospital for Children, Bristol, United Kingdom
| |
Collapse
|
4
|
Errington TM. Building reproducible bridges to cross the "valley of death". J Clin Invest 2024; 134:e177383. [PMID: 38165039 PMCID: PMC10760970 DOI: 10.1172/jci177383] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2024] Open
|
5
|
Kelter R. Reducing the false discovery rate of preclinical animal research with Bayesian statistical decision criteria. Stat Methods Med Res 2023; 32:1880-1901. [PMID: 37519294 PMCID: PMC10563376 DOI: 10.1177/09622802231184636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/01/2023]
Abstract
The success of preclinical research hinges on exploratory and confirmatory animal studies. Traditional null hypothesis significance testing is a common approach to eliminate the chaff from a collection of drugs, so that only the most promising treatments are funneled through to clinical research phases. Balancing the number of false discoveries and false omissions is an important aspect to consider during this process. In this paper, we compare several preclinical research pipelines, either based on null hypothesis significance testing or based on Bayesian statistical decision criteria. We build on a recently published large-scale meta-analysis of reported effect sizes in preclinical animal research and elicit a non-informative prior distribution under which both approaches are compared. After correcting for publication bias and shrinkage of effect sizes in replication studies, simulations show that (i) a shift towards statistical approaches which explicitly incorporate the minimum clinically important difference reduces the false discovery rate of frequentist approaches and (ii) a shift towards Bayesian statistical decision criteria can improve the reliability of preclinical animal research by reducing the number of false-positive findings. It is shown that these benefits hold while keeping the number of experimental units low which are required for a confirmatory follow-up study. Results show that Bayesian statistical decision criteria can help in improving the reliability of preclinical animal research and should be considered more frequently in practice.
Collapse
Affiliation(s)
- Riko Kelter
- Department of Mathematics, University of Siegen, Germany
| |
Collapse
|
6
|
Carneiro CFD, Drude N, Hülsemann M, Collazo A, Toelch U. Mapping strategies towards improved external validity in preclinical translational research. Expert Opin Drug Discov 2023; 18:1273-1285. [PMID: 37691294 DOI: 10.1080/17460441.2023.2251886] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2023] [Accepted: 08/22/2023] [Indexed: 09/12/2023]
Abstract
INTRODUCTION Translation is about successfully bringing findings from preclinical contexts into the clinic. This transfer is challenging as clinical trials frequently fail despite positive preclinical results. Limited robustness of preclinical research has been marked as one of the drivers of such failures. One suggested solution is to improve the external validity of in vitro and in vivo experiments via a suite of complementary strategies. AREAS COVERED In this review, the authors summarize the literature available on different strategies to improve external validity in in vivo, in vitro, or ex vivo experiments; systematic heterogenization; generalizability tests; and multi-batch and multicenter experiments. Articles that tested or discussed sources of variability in systematically heterogenized experiments were identified, and the most prevalent sources of variability are reviewed further. Special considerations in sample size planning, analysis options, and practical feasibility associated with each strategy are also reviewed. EXPERT OPINION The strategies reviewed differentially influence variation in experiments. Different research projects, with their unique goals, can leverage the strengths and limitations of each strategy. Applying a combination of these approaches in confirmatory stages of preclinical research putatively increases the chances of success in clinical studies.
Collapse
Affiliation(s)
- Clarissa F D Carneiro
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité, Berlin, Germany
| | - Natascha Drude
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité, Berlin, Germany
| | - Maren Hülsemann
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité, Berlin, Germany
| | - Anja Collazo
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité, Berlin, Germany
| | - Ulf Toelch
- QUEST Center for Responsible Research, Berlin Institute of Health at Charité, Berlin, Germany
| |
Collapse
|
7
|
Alturki NA. Review of the Immune Checkpoint Inhibitors in the Context of Cancer Treatment. J Clin Med 2023; 12:4301. [PMID: 37445336 DOI: 10.3390/jcm12134301] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Revised: 06/14/2023] [Accepted: 06/21/2023] [Indexed: 07/15/2023] Open
Abstract
Checkpoint proteins are an integral part of the immune system and are used by the tumor cells to evade immune response, which helps them grow uncontrollably. By blocking these proteins, immune checkpoint inhibitors can restore the capability of the immune system to attack cancer cells and stop their growth. These findings are backed by adequate clinical trial data and presently, several FDA-approved immune checkpoint inhibitors exist in the market for treating various types of cancers, including melanoma, hepatocellular, endometrial, lung, kidney and others. Their mode of action is inhibition by targeting the checkpoint proteins CTLA-4, PD-1, PD-L1, etc. They can be used alone as well as in amalgamation with other cancer treatments, like surgery, radiation or chemotherapy. Since these drugs target only specific immune system proteins, their side effects are reduced in comparison with the traditional chemotherapy drugs, but may still cause a few affects like fatigue, skin rashes, and fever. In rare cases, these inhibitors are known to have caused more serious side effects, such as cardiotoxicity, and inflammation in the intestines or lungs. Herein, we provide an overview of these inhibitors and their role as biomarkers, immune-related adverse outcomes and clinical studies in the treatment of various cancers, as well as present some future perspectives.
Collapse
Affiliation(s)
- Norah A Alturki
- Clinical Laboratory Science Department, College of Applied Medical Sciences, King Saud University, Riyadh 11433, Saudi Arabia
| |
Collapse
|
8
|
Wen Q, Hong X, He K, Liu B, Li M. Can acupuncture reverse oxidative stress and neuroinflammatory damage in animal models of vascular dementia?: A preclinical systematic review and meta-analysis. Medicine (Baltimore) 2023; 102:e33989. [PMID: 37335660 DOI: 10.1097/md.0000000000033989] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 06/21/2023] Open
Abstract
BACKGROUND Vascular dementia is a cognitive dysfunction syndrome caused by cerebral vascular factors such as ischemic stroke and hemorrhagic stroke. The effect of acupuncture on vascular dementia models is ambiguous, and there is controversy about whether acupuncture has a placebo effect. Oxidative stress and inflammation are the most essential mechanisms in preclinical studies of vascular dementia. However, there is no meta-analysis on the mechanism of vascular dementia in animal models. It is necessary to explore the efficacy of acupuncture through Meta-analysis of preclinical studies. METHODS Three major databases, PubMed, Embase and Web of Science (including medline), were searched in English until December 2022.The quality of the including literature was assessed using SYRCLE's risk of bias tool. Review Manager 5.3 was used to statistically summarize the included studies and the statistical effect values were expressed by SMD. The outcomes included: behavioral tests (escape latency, number of crossings), pathological sections (Nissl and TUNEL staining), oxidative stress markers (ROS, MDA, SOD, GSH-PX) and neuroinflammatory factors (TNF-α, IL-1β, IL-6). RESULTS A total of 31 articles were included in this meta-analysis. The results showed that the escape latency, the contents of ROS, MDA, IL-1β, and IL-6 were decreased, and the contents of SOD and Nissl-positive neurons were increased in the acupuncture group as compared with the non-group (P < .05). Compared with the impaired group, the acupuncture group also had the above advantages (P < .05). In addition, the acupuncture group also increased the number of crossings and GSH-PX content, and decreased the expression of TUNEL-positive neurons and TNF-α (P < .05). CONCLUSIONS From behavioral tests to slices and pathological markers in animal models of vascular dementia, it can be proved that acupuncture is effective in targeting oxidative stress and neuroinflammatory damage, and acupuncture is not a placebo effect. Nevertheless, attention needs to be paid to the gap between animal experiments and clinical applications.
Collapse
Affiliation(s)
- Qin Wen
- Clinical Medical College of Acupuncture Moxibustion and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
- Yuangang Street Community Health Service Center, Guangzhou, China
| | - Xueqin Hong
- Clinical Medical College of Acupuncture Moxibustion and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Kunze He
- Clinical Medical College of Acupuncture Moxibustion and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Buping Liu
- Clinical Medical College of Acupuncture Moxibustion and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Min Li
- Clinical Medical College of Acupuncture Moxibustion and Rehabilitation, Guangzhou University of Chinese Medicine, Guangzhou, China
| |
Collapse
|
9
|
Faisal M, Aid J, Nodirov B, Lee B, Hickey MA. Preclinical trials in Alzheimer's disease: Sample size and effect size for behavioural and neuropathological outcomes in 5xFAD mice. PLoS One 2023; 18:e0281003. [PMID: 37036878 PMCID: PMC10085059 DOI: 10.1371/journal.pone.0281003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Accepted: 01/13/2023] [Indexed: 04/11/2023] Open
Abstract
5xFAD transgenic (TG) mice are used widely in AD preclinical trials; however, data on sample sizes are largely unaddressed. We therefore performed estimates of sample sizes and effect sizes for typical behavioural and neuropathological outcome measures in TG 5xFAD mice, based upon data from single-sex (female) groups. Group-size estimates to detect normalisation of TG body weight to WT littermate levels at 5.5m of age were N = 9-15 depending upon algorithm. However, by 1 year of age, group sizes were small (N = 1 -<6), likely reflecting the large difference between genotypes at this age. To detect normalisation of TG open-field hyperactivity to WT levels at 13-14m, group sizes were also small (N = 6-8). Cued learning in the Morris water maze (MWM) was normal in Young TG mice (5m of age). Mild deficits were noted during MWM spatial learning and memory. MWM reversal learning and memory revealed greater impairment, and groups of up to 22 TG mice were estimated to detect normalisation to WT performance. In contrast, Aged TG mice (tested between 13 and 14m) failed to complete the visual learning (non-spatial) phase of MWM learning, likely due to a failure to recognise the platform as an escape. Estimates of group size to detect normalisation of this severe impairment were small (N = 6-9, depending upon algorithm). Other cognitive tests including spontaneous and forced alternation and novel-object recognition either failed to reveal deficits in TG mice or deficits were negligible. For neuropathological outcomes, plaque load, astrocytosis and microgliosis in frontal cortex and hippocampus were quantified in TG mice aged 2m, 4m and 6m. Sample-size estimates were ≤9 to detect the equivalent of a reduction in plaque load to the level of 2m-old TG mice or the equivalent of normalisation of neuroinflammation outcomes. However, for a smaller effect size of 30%, larger groups of up to 21 mice were estimated. In light of published guidelines on preclinical trial design, these data may be used to provide provisional sample sizes and optimise preclinical trials in 5xFAD TG mice.
Collapse
Affiliation(s)
- Mahvish Faisal
- Department of Pharmacology, Institute of Biomedicine and
Translational Medicine, University of Tartu, Tartu, Estonia
| | - Jana Aid
- Department of Pharmacology, Institute of Biomedicine and
Translational Medicine, University of Tartu, Tartu, Estonia
| | - Bekzod Nodirov
- Department of Pharmacology, Institute of Biomedicine and
Translational Medicine, University of Tartu, Tartu, Estonia
| | - Benjamin Lee
- Department of Pharmacology, Institute of Biomedicine and
Translational Medicine, University of Tartu, Tartu, Estonia
| | - Miriam A. Hickey
- Department of Pharmacology, Institute of Biomedicine and
Translational Medicine, University of Tartu, Tartu, Estonia
| |
Collapse
|
10
|
Piper SK, Zocholl D, Toelch U, Roehle R, Stroux A, Hoessler J, Zinke A, Konietschke F. Statistical review of animal trials-A guideline. Biom J 2023; 65:e2200061. [PMID: 36071025 DOI: 10.1002/bimj.202200061] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2022] [Revised: 07/25/2022] [Accepted: 08/14/2022] [Indexed: 11/11/2022]
Abstract
Any experiment involving living organisms requires justification of the need and moral defensibleness of the study. Statistical planning, design, and sample size calculation of the experiment are no less important review criteria than general medical and ethical points to consider. Errors made in the statistical planning and data evaluation phase can have severe consequences on both results and conclusions. They might proliferate and thus impact future trials-an unintended outcome of fundamental research with profound ethical consequences. Unified statistical standards are currently missing for animal review boards in Germany. In order to accompany, we developed a biometric form to be filled and handed in with the proposal at the concerned local authority on animal welfare. It addresses relevant points to consider for biostatistical planning of animal experiments and can help both the applicants and the reviewers in overseeing the entire experiment(s) planned. Furthermore, the form might also aid in meeting the current standards set by the 3+3R's principle of animal experimentation: Replacement, Reduction, Refinement as well as Robustness, Registration, and Reporting. The form has already been in use by the concerned local authority of animal welfare in Berlin, Germany. In addition, we provide reference to our user guide giving more detailed explanation and examples for each section of the biometric form. Unifying the set of biostatistical aspects will help both the applicants and the reviewers to equal standards and increase quality of preclinical research projects, also for translational, multicenter, or international studies.
Collapse
Affiliation(s)
- Sophie K Piper
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Biometry and Clinical Epidemiology, Berlin, Germany.,Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Medical Informatics, Berlin, Germany.,Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Dario Zocholl
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Biometry and Clinical Epidemiology, Berlin, Germany.,Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Ulf Toelch
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, QUEST Center for Responsible Research, Berlin, Germany
| | - Robert Roehle
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Biometry and Clinical Epidemiology, Berlin, Germany.,Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany.,Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Clinical Trial Office, Berlin, Germany
| | - Andrea Stroux
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Biometry and Clinical Epidemiology, Berlin, Germany.,Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Johanna Hoessler
- Landesamt für Gesundheit und Soziales, Referat für gesundheitlichen Verbraucherschutz, Berlin, Germany
| | - Anne Zinke
- Landesamt für Gesundheit und Soziales, Referat für gesundheitlichen Verbraucherschutz, Berlin, Germany
| | - Frank Konietschke
- Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Institute of Biometry and Clinical Epidemiology, Berlin, Germany.,Berlin Institute of Health at Charité - Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
11
|
Yamada KM, Doyle AD, Lu J. Cell-3D matrix interactions: recent advances and opportunities. Trends Cell Biol 2022; 32:883-895. [PMID: 35410820 PMCID: PMC9464680 DOI: 10.1016/j.tcb.2022.03.002] [Citation(s) in RCA: 57] [Impact Index Per Article: 28.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Revised: 03/12/2022] [Accepted: 03/15/2022] [Indexed: 02/03/2023]
Abstract
Tissues consist of cells and their surrounding extracellular matrix (ECM). Cell-ECM interactions play crucial roles in embryonic development, differentiation, tissue remodeling, and diseases including fibrosis and cancer. Recent research advances in characterizing cell-matrix interactions include detailed descriptions of hundreds of ECM and associated molecules, their complex intermolecular interactions in development and disease, identification of distinctive modes of cell migration in different 3D ECMs, and new insights into mechanisms of organ formation. Exploring the roles of the physical features of different ECM microenvironments and the bidirectional regulation of cell signaling and matrix organization emphasize the dynamic nature of these interactions, which can include feedback loops that exacerbate disease. Understanding mechanisms of cell-matrix interactions can potentially lead to targeted therapeutic interventions.
Collapse
Affiliation(s)
- Kenneth M Yamada
- Cell Biology Section, National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, MD 20892, USA.
| | - Andrew D Doyle
- Cell Biology Section, National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, MD 20892, USA
| | - Jiaoyang Lu
- Cell Biology Section, National Institute of Dental and Craniofacial Research, National Institutes of Health, Bethesda, MD 20892, USA
| |
Collapse
|
12
|
Ellis RJ. Questionable Research Practices, Low Statistical Power, and Other Obstacles to Replicability: Why Preclinical Neuroscience Research Would Benefit from Registered Reports. eNeuro 2022; 9:ENEURO.0017-22.2022. [PMID: 35922130 PMCID: PMC9351632 DOI: 10.1523/eneuro.0017-22.2022] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2022] [Revised: 05/22/2022] [Accepted: 05/31/2022] [Indexed: 02/03/2023] Open
Abstract
Replicability, the degree to which a previous scientific finding can be repeated in a distinct set of data, has been considered an integral component of institutionalized scientific practice since its inception several hundred years ago. In the past decade, large-scale replication studies have demonstrated that replicability is far from favorable, across multiple scientific fields. Here, I evaluate this literature and describe contributing factors including the prevalence of questionable research practices (QRPs), misunderstanding of p-values, and low statistical power. I subsequently discuss how these issues manifest specifically in preclinical neuroscience research. I conclude that these problems are multifaceted and difficult to solve, relying on the actions of early and late career researchers, funding sources, academic publishers, and others. I assert that any viable solution to the problem of substandard replicability must include changing academic incentives, with adoption of registered reports being the most immediately impactful and pragmatic strategy. For animal research in particular, comprehensive reporting guidelines that document potential sources of sensitivity for experimental outcomes is an essential addition.
Collapse
Affiliation(s)
- Randall J Ellis
- Friedman Brain Institute, Department of Neuroscience, Addiction Institute of Mount Sinai, Icahn School of Medicine at Mount Sinai, New York, NY 10029
| |
Collapse
|
13
|
Abstract
The dramatic global consequences of the coronavirus disease 2019 (COVID-19) pandemic soon fueled quests for a suitable model that would facilitate the development and testing of therapies and vaccines. In contrast to other rodents, hamsters are naturally susceptible to infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and the Syrian hamster (Mesocricetus auratus) rapidly developed into a popular model. It recapitulates many characteristic features as seen in patients with a moderate, self-limiting course of the disease such as specific patterns of respiratory tract inflammation, vascular endothelialitis, and age dependence. Among 4 other hamster species examined, the Roborovski dwarf hamster (Phodopus roborovskii) more closely mimics the disease in highly susceptible patients with frequent lethal outcome, including devastating diffuse alveolar damage and coagulopathy. Thus, different hamster species are available to mimic different courses of the wide spectrum of COVID-19 manifestations in humans. On the other hand, fewer diagnostic tools and information on immune functions and molecular pathways are available than in mice, which limits mechanistic studies and inference to humans in several aspects. Still, under pandemic conditions with high pressure on progress in both basic and clinically oriented research, the Syrian hamster has turned into the leading non-transgenic model at an unprecedented pace, currently used in innumerable studies that all aim to combat the impact of the virus with its new variants of concern. As in other models, its strength rests upon a solid understanding of its similarities to and differences from the human disease, which we review here.
Collapse
|
14
|
Boutin ME, Strong CE, Van Hese B, Hu X, Itkin Z, Chen YC, LaCroix A, Gordon R, Guicherit O, Carromeu C, Kundu S, Lee E, Ferrer M. A multiparametric calcium signal screening platform using iPSC-derived cortical neural spheroids. SLAS DISCOVERY : ADVANCING LIFE SCIENCES R & D 2022; 27:209-218. [PMID: 35092840 PMCID: PMC9177534 DOI: 10.1016/j.slasd.2022.01.003] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
Discovery of therapeutics for neurological diseases is hampered by the lack of predictive in vitro and in vivo models. Traditionally, in vitro assays rely on engineered cell lines grown two-dimensionally (2D) outside a physiological tissue context, which makes them very amenable for large scale drug screening but reduces their relevance to in vivo neurophysiology. In recent years, three-dimensional (3D) neural cell culture models derived from human induced pluripotent stem cells (iPSCs) have been developed as an in vitro assay platform to investigate brain development, neurological diseases, and for drug screening. iPSC-derived neural spheroids or organoids can be developed to include complex neuronal and glial cell populations and display spontaneous, synchronous activity, which is a hallmark of in vivo neural communication. In this report we present a proof-of-concept study evaluating 3D iPSC-derived cortical neural spheroids as a physiologically- and pharmacologically-relevant high-throughput screening (HTS) platform and investigate their potential for use for therapeutic development. To this end, a library of 687 neuroactive compounds were tested in a phenotypic screening paradigm which measured calcium activity as a functional biomarker for neural modulation through fluctuations in calcium fluorescence. Pharmacological responses of cortical neural spheroids were analyzed using a multi-parametric approach, whereby seven peak characteristics from the calcium activity in each well were quantified and incorporated into principal component analysis and Sammon mapping to measure compound response. Here, we describe the implementation of the 687-compound library screen and data analysis demonstrating that iPSC-derived cortical spheroids are a robust and information-rich assay platform for HTS.
Collapse
Affiliation(s)
- Molly E Boutin
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, 9800 Medical Center Drive, Rockville, MD, 20850, USA; Ecovative Design, 70 Cohoes Avenue, Green Island, NY, USA
| | - Caroline E Strong
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, 9800 Medical Center Drive, Rockville, MD, 20850, USA
| | | | - Xin Hu
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, 9800 Medical Center Drive, Rockville, MD, 20850, USA
| | - Zina Itkin
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, 9800 Medical Center Drive, Rockville, MD, 20850, USA
| | - Yu-Chi Chen
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, 9800 Medical Center Drive, Rockville, MD, 20850, USA
| | | | | | | | | | - Srikanya Kundu
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, 9800 Medical Center Drive, Rockville, MD, 20850, USA
| | - Emily Lee
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, 9800 Medical Center Drive, Rockville, MD, 20850, USA
| | - Marc Ferrer
- Division of Preclinical Innovation, National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, 9800 Medical Center Drive, Rockville, MD, 20850, USA.
| |
Collapse
|
15
|
Nigri A, Ferraro S, Gandini Wheeler-Kingshott CAM, Tosetti M, Redolfi A, Forloni G, D'Angelo E, Aquino D, Biagi L, Bosco P, Carne I, De Francesco S, Demichelis G, Gianeri R, Lagana MM, Micotti E, Napolitano A, Palesi F, Pirastru A, Savini G, Alberici E, Amato C, Arrigoni F, Baglio F, Bozzali M, Castellano A, Cavaliere C, Contarino VE, Ferrazzi G, Gaudino S, Marino S, Manzo V, Pavone L, Politi LS, Roccatagliata L, Rognone E, Rossi A, Tonon C, Lodi R, Tagliavini F, Bruzzone MG. Quantitative MRI Harmonization to Maximize Clinical Impact: The RIN-Neuroimaging Network. Front Neurol 2022; 13:855125. [PMID: 35493836 PMCID: PMC9047871 DOI: 10.3389/fneur.2022.855125] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2022] [Accepted: 03/17/2022] [Indexed: 11/13/2022] Open
Abstract
Neuroimaging studies often lack reproducibility, one of the cardinal features of the scientific method. Multisite collaboration initiatives increase sample size and limit methodological flexibility, therefore providing the foundation for increased statistical power and generalizable results. However, multisite collaborative initiatives are inherently limited by hardware, software, and pulse and sequence design heterogeneities of both clinical and preclinical MRI scanners and the lack of benchmark for acquisition protocols, data analysis, and data sharing. We present the overarching vision that yielded to the constitution of RIN-Neuroimaging Network, a national consortium dedicated to identifying disease and subject-specific in-vivo neuroimaging biomarkers of diverse neurological and neuropsychiatric conditions. This ambitious goal needs efforts toward increasing the diagnostic and prognostic power of advanced MRI data. To this aim, 23 Italian Scientific Institutes of Hospitalization and Care (IRCCS), with technological and clinical specialization in the neurological and neuroimaging field, have gathered together. Each IRCCS is equipped with high- or ultra-high field MRI scanners (i.e., ≥3T) for clinical or preclinical research or has established expertise in MRI data analysis and infrastructure. The actions of this Network were defined across several work packages (WP). A clinical work package (WP1) defined the guidelines for a minimum standard clinical qualitative MRI assessment for the main neurological diseases. Two neuroimaging technical work packages (WP2 and WP3, for clinical and preclinical scanners) established Standard Operative Procedures for quality controls on phantoms as well as advanced harmonized quantitative MRI protocols for studying the brain of healthy human participants and wild type mice. Under FAIR principles, a web-based e-infrastructure to store and share data across sites was also implemented (WP4). Finally, the RIN translated all these efforts into a large-scale multimodal data collection in patients and animal models with dementia (i.e., case study). The RIN-Neuroimaging Network can maximize the impact of public investments in research and clinical practice acquiring data across institutes and pathologies with high-quality and highly-consistent acquisition protocols, optimizing the analysis pipeline and data sharing procedures.
Collapse
Affiliation(s)
- Anna Nigri
- U.O. Neuroradiologia, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy
| | - Stefania Ferraro
- U.O. Neuroradiologia, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy
- MOE Key Laboratory for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Claudia A. M. Gandini Wheeler-Kingshott
- Unità di Neuroradiologia, IRCCS Mondino Foundation, Pavia, Italy
- NMR Research Unit, Department of Neuroinflammation, Queen Square MS Centre, UCL Queen Square Institute of Neurology, Faculty of Brain Sciences, University College London, London, United Kingdom
- Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
| | - Michela Tosetti
- Medical Physics and MR Lab, Fondazione IRCCS Stella Maris, Pisa, Italy
| | - Alberto Redolfi
- Laboratory of Neuroinformatics, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy
| | - Gianluigi Forloni
- Medical Physics and MR Lab, Fondazione IRCCS Stella Maris, Pisa, Italy
| | - Egidio D'Angelo
- Unità di Neuroradiologia, IRCCS Mondino Foundation, Pavia, Italy
- Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
| | - Domenico Aquino
- U.O. Neuroradiologia, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy
| | - Laura Biagi
- Medical Physics and MR Lab, Fondazione IRCCS Stella Maris, Pisa, Italy
| | - Paolo Bosco
- Medical Physics and MR Lab, Fondazione IRCCS Stella Maris, Pisa, Italy
| | - Irene Carne
- Neuroradiology Unit, IRCCS Istituti Clinici Scientifici Maugeri, Pavia, Italy
| | - Silvia De Francesco
- Laboratory of Neuroinformatics, IRCCS Istituto Centro San Giovanni di Dio Fatebenefratelli, Brescia, Italy
| | - Greta Demichelis
- U.O. Neuroradiologia, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy
| | - Ruben Gianeri
- U.O. Neuroradiologia, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy
| | | | - Edoardo Micotti
- Laboratory of Biology of Neurodegenerative Disorders, Istituto di Ricerche Farmacologiche Mario Negri IRCCS, Milan, Italy
| | - Antonio Napolitano
- Medical Physics, IRCCS Istituto Ospedale Pediatrico Bambino Gesù, Rome, Italy
| | - Fulvia Palesi
- Unità di Neuroradiologia, IRCCS Mondino Foundation, Pavia, Italy
- Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy
| | | | - Giovanni Savini
- Neuroradiology Unit, IRCCS Humanitas Research Hospital, Milan, Italy
| | - Elisa Alberici
- Neuroradiology Unit, IRCCS Istituti Clinici Scientifici Maugeri, Pavia, Italy
| | - Carmelo Amato
- Unit of Neuroradiology, Oasi Research Institute-IRCCS, Troina, Italy
| | - Filippo Arrigoni
- Neuroimaging Unit, Scientific Institute, IRCCS E. Medea, Bosisio Parini, Italy
| | | | - Marco Bozzali
- Neuroimaging Laboratory, Santa Lucia Foundation, IRCCS, Rome, Italy
| | | | | | - Valeria Elisa Contarino
- Unità di Neuroradiologia, Fondazione IRCCS Ca' Granda Ospedale Maggiore Policlinico, Milan, Italy
| | | | - Simona Gaudino
- Istituto di Radiologia, UOC Radiologia e Neuroradiologia, IRCCS Fondazione Policlinico Universitario Agostino Gemelli, Rome, Italy
| | - Silvia Marino
- IRCCS Centro Neurolesi “Bonino-Pulejo”, Messina, Italy
| | - Vittorio Manzo
- Department of Radiology, Istituto Auxologico Italiano, IRCCS, Milan, Italy
| | | | - Letterio S. Politi
- Neuroradiology Unit, IRCCS Humanitas Research Hospital, Milan, Italy
- Department of Biomedical Sciences, Humanitas University, Milan, Italy
| | - Luca Roccatagliata
- Neuroradiologia IRCCS Ospedale Policlinico San Martino, Genoa, Italy
- Dipartimento di Scienze della Salute Università di Genova, Genoa, Italy
| | - Elisa Rognone
- Unità di Neuroradiologia, IRCCS Mondino Foundation, Pavia, Italy
| | - Andrea Rossi
- Dipartimento di Scienze della Salute Università di Genova, Genoa, Italy
- UO Neuroradiologia, IRCCS Istituto Giannina Gaslini, Genoa, Italy
| | - Caterina Tonon
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bologna, Italy
| | - Raffaele Lodi
- Functional and Molecular Neuroimaging Unit, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bologna, Italy
| | - Fabrizio Tagliavini
- Scientific Direction, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy
| | - Maria Grazia Bruzzone
- U.O. Neuroradiologia, Fondazione IRCCS Istituto Neurologico Carlo Besta, Milan, Italy
| | | |
Collapse
|
16
|
Holst MR, Faust A, Strech D. Do German university medical centres promote robust and transparent research? A cross-sectional study of institutional policies. Health Res Policy Syst 2022; 20:39. [PMID: 35413846 PMCID: PMC9004041 DOI: 10.1186/s12961-022-00841-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2021] [Accepted: 03/21/2022] [Indexed: 12/01/2022] Open
Abstract
BACKGROUND In light of replication and translational failures, biomedical research practices have recently come under scrutiny. Experts have pointed out that the current incentive structures at research institutions do not sufficiently incentivise researchers to invest in robustness and transparency and instead incentivise them to optimize their fitness in the struggle for publications and grants. This cross-sectional study aimed to describe whether and how relevant policies of university medical centres in Germany support the robust and transparent conduct of research and how prevalent traditional metrics are. METHODS For 38 German university medical centres, we searched for institutional policies for academic degrees and academic appointments as well as websites for their core facilities and research in general between December 2020 and February 2021. We screened the documents for mentions of indicators of robust and transparent research (study registration; reporting of results; sharing of research data, code and protocols; open access; and measures to increase robustness) and for mentions of more traditional metrics of career progression (number of publications; number and value of awarded grants; impact factors; and authorship order). RESULTS While open access was mentioned in 16% of PhD regulations, other indicators of robust and transparent research were mentioned in less than 10% of institutional policies for academic degrees and academic appointments. These indicators were more frequently mentioned on the core facility and general research websites. Institutional policies for academic degrees and academic appointments had frequent mentions of traditional metrics. CONCLUSIONS References to robust and transparent research practices are, with a few exceptions, generally uncommon in institutional policies at German university medical centres, while traditional criteria for academic promotion and tenure still prevail.
Collapse
Affiliation(s)
- M R Holst
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, QUEST Center for Responsible Research, Charitéplatz 1, 10117, Berlin, Germany.
- Medizinische Hochschule Hannover, Institute of Ethics, History and Philosophy of Medicine, Carl-Neuberg-Str. 1, 30625, Hannover, Germany.
| | - A Faust
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, QUEST Center for Responsible Research, Charitéplatz 1, 10117, Berlin, Germany
- Medizinische Hochschule Hannover, Institute of Ethics, History and Philosophy of Medicine, Carl-Neuberg-Str. 1, 30625, Hannover, Germany
| | - D Strech
- Berlin Institute of Health at Charité - Universitätsmedizin Berlin, QUEST Center for Responsible Research, Charitéplatz 1, 10117, Berlin, Germany
- Medizinische Hochschule Hannover, Institute of Ethics, History and Philosophy of Medicine, Carl-Neuberg-Str. 1, 30625, Hannover, Germany
| |
Collapse
|
17
|
Kane PB, Kimmelman J. Is preclinical research in cancer biology reproducible enough? eLife 2021; 10:67527. [PMID: 34874006 PMCID: PMC8651283 DOI: 10.7554/elife.67527] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2021] [Accepted: 08/27/2021] [Indexed: 12/12/2022] Open
Abstract
The Reproducibility Project: Cancer Biology (RPCB) was established to provide evidence about reproducibility in basic and preclinical cancer research, and to identify the factors that influence reproducibility more generally. In this commentary we address some of the scientific, ethical and policy implications of the project. We liken the basic and preclinical cancer research enterprise to a vast 'diagnostic machine' that is used to determine which clinical hypotheses should be advanced for further development, including clinical trials. The results of the RPCB suggest that this diagnostic machine currently recommends advancing many findings that are not reproducible. While concerning, we believe that more work needs to be done to evaluate the performance of the diagnostic machine. Specifically, we believe three questions remain unanswered: how often does the diagnostic machine correctly recommend against advancing real effects to clinical testing?; what are the relative costs to society of false positive and false negatives?; and how well do scientists and others interpret the outputs of the machine?
Collapse
Affiliation(s)
- Patrick Bodilly Kane
- Studies in Translation, Ethics and Medicine, Biomedical Ethics Unit, McGill University, Montréal, Canada
| | - Jonathan Kimmelman
- Studies in Translation, Ethics and Medicine, Biomedical Ethics Unit, McGill University, Montréal, Canada
| |
Collapse
|
18
|
Abstract
The Reproducibility Project: Cancer Biology (RPCB) was established to provide evidence about reproducibility in basic and preclinical cancer research, and to identify the factors that influence reproducibility more generally. In this commentary we address some of the scientific, ethical and policy implications of the project. We liken the basic and preclinical cancer research enterprise to a vast 'diagnostic machine' that is used to determine which clinical hypotheses should be advanced for further development, including clinical trials. The results of the RPCB suggest that this diagnostic machine currently recommends advancing many findings that are not reproducible. While concerning, we believe that more work needs to be done to evaluate the performance of the diagnostic machine. Specifically, we believe three questions remain unanswered: how often does the diagnostic machine correctly recommend against advancing real effects to clinical testing?; what are the relative costs to society of false positive and false negatives?; and how well do scientists and others interpret the outputs of the machine?
Collapse
Affiliation(s)
- Patrick Bodilly Kane
- Studies in Translation, Ethics and Medicine, Biomedical Ethics Unit, McGill University
| | - Jonathan Kimmelman
- Studies in Translation, Ethics and Medicine, Biomedical Ethics Unit, McGill University
| |
Collapse
|
19
|
Errington TM, Mathur M, Soderberg CK, Denis A, Perfito N, Iorns E, Nosek BA. Investigating the replicability of preclinical cancer biology. eLife 2021; 10:e71601. [PMID: 34874005 PMCID: PMC8651293 DOI: 10.7554/elife.71601] [Citation(s) in RCA: 80] [Impact Index Per Article: 26.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2021] [Accepted: 10/16/2021] [Indexed: 12/18/2022] Open
Abstract
Replicability is an important feature of scientific research, but aspects of contemporary research culture, such as an emphasis on novelty, can make replicability seem less important than it should be. The Reproducibility Project: Cancer Biology was set up to provide evidence about the replicability of preclinical research in cancer biology by repeating selected experiments from high-impact papers. A total of 50 experiments from 23 papers were repeated, generating data about the replicability of a total of 158 effects. Most of the original effects were positive effects (136), with the rest being null effects (22). A majority of the original effect sizes were reported as numerical values (117), with the rest being reported as representative images (41). We employed seven methods to assess replicability, and some of these methods were not suitable for all the effects in our sample. One method compared effect sizes: for positive effects, the median effect size in the replications was 85% smaller than the median effect size in the original experiments, and 92% of replication effect sizes were smaller than the original. The other methods were binary - the replication was either a success or a failure - and five of these methods could be used to assess both positive and null effects when effect sizes were reported as numerical values. For positive effects, 40% of replications (39/97) succeeded according to three or more of these five methods, and for null effects 80% of replications (12/15) were successful on this basis; combining positive and null effects, the success rate was 46% (51/112). A successful replication does not definitively confirm an original finding or its theoretical interpretation. Equally, a failure to replicate does not disconfirm a finding, but it does suggest that additional investigation is needed to establish its reliability.
Collapse
Affiliation(s)
| | - Maya Mathur
- Quantitative Sciences Unit, Stanford UniversityStanfordUnited States
| | | | | | | | | | - Brian A Nosek
- Center for Open ScienceCharlottesvilleUnited States
- University of VirginiaCharlottesvilleUnited States
| |
Collapse
|
20
|
|
21
|
Robust preclinical evidence in somatic cell genome editing: A key driver of responsible and efficient therapeutic innovations. Drug Discov Today 2021; 26:2238-2243. [PMID: 34161846 DOI: 10.1016/j.drudis.2021.06.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/21/2020] [Revised: 04/30/2021] [Accepted: 06/14/2021] [Indexed: 12/27/2022]
Abstract
Somatic cell genome editing (SCGE) is highly promising for therapeutic innovation. This study demonstrates that the majority of 46 preclinical SCGE studies discussed in reviews as particularly promising for clinical translation do not report on key elements for robust and confirmatory research practices: randomization, blinding, sample size calculation, data handling, pre-registration, multi-centric study design, and independent confirmation. We present the here-examined reporting standards and the new National Institutes of Health (NIH) funding criteria for SCGE research as a viable solution to protect this promising field from backlashes. We argue that the implementation of the novel methodological standards provides the opportunity for SCGE research to become a lighthouse example for trustworthy and useful translational research.
Collapse
|
22
|
Hunniford VT, Montroy J, Fergusson DA, Avey MT, Wever KE, McCann SK, Foster M, Fox G, Lafreniere M, Ghaly M, Mannell S, Godwinska K, Gentles A, Selim S, MacNeil J, Sikora L, Sena ES, Page MJ, Macleod M, Moher D, Lalu MM. Epidemiology and reporting characteristics of preclinical systematic reviews. PLoS Biol 2021; 19:e3001177. [PMID: 33951050 PMCID: PMC8128274 DOI: 10.1371/journal.pbio.3001177] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Revised: 05/17/2021] [Accepted: 03/05/2021] [Indexed: 01/10/2023] Open
Abstract
In an effort to better utilize published evidence obtained from animal experiments, systematic reviews of preclinical studies are increasingly more common-along with the methods and tools to appraise them (e.g., SYstematic Review Center for Laboratory animal Experimentation [SYRCLE's] risk of bias tool). We performed a cross-sectional study of a sample of recent preclinical systematic reviews (2015-2018) and examined a range of epidemiological characteristics and used a 46-item checklist to assess reporting details. We identified 442 reviews published across 43 countries in 23 different disease domains that used 26 animal species. Reporting of key details to ensure transparency and reproducibility was inconsistent across reviews and within article sections. Items were most completely reported in the title, introduction, and results sections of the reviews, while least reported in the methods and discussion sections. Less than half of reviews reported that a risk of bias assessment for internal and external validity was undertaken, and none reported methods for evaluating construct validity. Our results demonstrate that a considerable number of preclinical systematic reviews investigating diverse topics have been conducted; however, their quality of reporting is inconsistent. Our study provides the justification and evidence to inform the development of guidelines for conducting and reporting preclinical systematic reviews.
Collapse
Affiliation(s)
- Victoria T. Hunniford
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Joshua Montroy
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Dean A. Fergusson
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | | | - Kimberley E. Wever
- SYstematic Review Center for Laboratory animal Experimentation (SYRCLE), Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, the Netherlands
| | - Sarah K. McCann
- QUEST Center for Transforming Biomedical Research, Berlin Institute of Health (BIH) and Charité—Universitätsmedizin Berlin, Berlin, Germany
| | - Madison Foster
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Grace Fox
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Mackenzie Lafreniere
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Mira Ghaly
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Sydney Mannell
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Karolina Godwinska
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Avonae Gentles
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
| | - Shehab Selim
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Jenna MacNeil
- Department of Medicine, University of Ottawa, Ottawa, Ontario, Canada
| | - Lindsey Sikora
- Health Sciences Library, University of Ottawa, Ottawa, Canada
| | - Emily S. Sena
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - Matthew J. Page
- School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia
| | - Malcolm Macleod
- Centre for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom
| | - David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Manoj M. Lalu
- Clinical Epidemiology Program, Blueprint Translational Research Group, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada
- Department of Anesthesiology and Pain Medicine, The Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada
| |
Collapse
|