Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research

Pipeline to Neonatal Clinical Transformation: The Importance of Preclinical Data

Historically, neonatal neuroscience boasted a robust and successful preclinical pipeline for therapeutic interventions, in particular for the treatment of hypoxic-ischemic encephalopathy (HIE). However, since the successful translation of therapeutic hypothermia (TH), several high-profile failures of promising adjunctive therapies, in addition to the lack of benefit of TH in lower resource settings, have brought to light critical issues in that same pipeline. Using recent data from clinical trials of erythropoietin as an example, the authors highlight several key challenges facing preclinical neonatal neuroscience for HIE therapeutic development and propose key areas where model development and collaboration across the field in general can ensure ongoing success in treatment development for HIE worldwide.

Current evidence on the efficacy of mental health smartphone apps for symptoms of depression and anxiety. A meta-analysis of 176 randomized controlled trials

The mental health care available for depression and anxiety has recently undergone a major technological revolution, with growing interest towards the potential of smartphone apps as a scalable tool to treat these conditions. Since the last comprehensive meta-analysis in 2019 established positive yet variable effects of apps on depressive and anxiety symptoms, more than 100 new randomized controlled trials (RCTs) have been carried out. We conducted an updated meta-analysis with the objectives of providing more precise estimates of effects, quantifying generalizability from this evidence base, and understanding whether major app and trial characteristics moderate effect sizes. We included 176 RCTs that aimed to treat depressive or anxiety symptoms. Apps had overall significant although small effects on symptoms of depression (N=33,567, g=0.28, p<0.001; number needed to treat, NNT=11.5) and generalized anxiety (N=22,394, g=0.26, p<0.001, NNT=12.4) as compared to control groups. These effects were robust at different follow-ups and after removing small sample and higher risk of bias trials. There was less variability in outcome scores at post-test in app compared to control conditions (ratio of variance, RoV=-0.14, 95% CI: -0.24 to -0.05 for depressive symptoms; RoV=-0.21, 95% CI: -0.31 to -0.12 for generalized anxiety symptoms). Effect sizes for depression were significantly larger when apps incorporated cognitive behavioral therapy (CBT) features or included chatbot technology. Effect sizes for anxiety were significantly larger when trials had generalized anxiety as a primary target and administered a CBT app or an app with mood monitoring features. We found evidence of moderate effects of apps on social anxiety (g=0.52) and obsessive-compulsive (g=0.51) symptoms, a small effect on post-traumatic stress symptoms (g=0.12), a large effect on acrophobia symptoms (g=0.90), and a non-significant negative effect on panic symptoms (g=-0.12), although these results should be considered with caution, because most trials had high risk of bias and were based on small sample sizes. We conclude that apps have overall small but significant effects on symptoms of depression and generalized anxiety, and that specific features of apps - such as CBT or mood monitoring features and chatbot technology - are associated with larger effect sizes.

Accelerating the open research agenda to solve global challenges

Harnessing science-based policy is key to addressing global challenges like the biodiversity and climate crises. Open research principles underpin effective science-based policy, but the uptake of these principles is likely constrained by the politicisation, commoditisation and conflicting motives of stakeholders in the research landscape. Here, using the mission and vision statements from 129 stakeholders from across the research landscape, we explore alignment in open research principles between stakeholders. We find poor alignment between stakeholders, largely focussed around journals, societies and funders, all of which have low open research language-use. We argue that this poor alignment stifles knowledge flow within the research landscape, ultimately limiting the mobilisation of impactful science-based policy. We offer recommendations on how the research landscape could embrace open research principles to accelerate societies' ability to solve global challenges.

Systematic heterogenization revisited: Increasing variation in animal experiments to improve reproducibility?

Life sciences are currently facing a reproducibility crisis. Originally, the crisis was born out of single alarming failures to reproduce findings at different times and locations. Nowadays, systematic studies indicate that the prevalence of irreproducible research does in fact exceed 50%. Viewed from a rather cynical perspective, Fett's law of the lab "Never replicate a successful experiment" has thus taken on a completely new meaning. In this respect, animal research has come under particular scrutiny, as the stakes are high in terms of both research ethics and societal impact. To counteract this, it is essential to identify sources of poor reproducibility as well as to iron out these failures. We here review the current debate, briefly discuss potential reasons, and summarize steps that have already been undertaken to improve reproducibility in animal research. By the example of classical behavioural phenotyping studies, we particularly highlight the role strict standardization plays in exacerbating the crisis, and review the concept of systematic heterogenization as an alternative strategy to deal with variation in animal studies. Briefly, we argue that systematic variation rather than strict homogenization of experimental conditions benefits the robustness of research findings, and hence their reproducibility. To this end, we will present concrete examples for systematically heterogenized experiments and provide a practical guide on how to apply systematic heterogenization in experimental practice.

Finding the right power balance: Better study design and collaboration can reduce dependence on statistical power

Power analysis currently dominates sample size determination for experiments, particularly in grant and ethics applications. Yet, this focus could paradoxically result in suboptimal study design because publication biases towards studies with the largest effects can lead to the overestimation of effect sizes. In this Essay, we propose a paradigm shift towards better study designs that focus less on statistical power. We also advocate for (pre)registration and obligatory reporting of all results (regardless of statistical significance), better facilitation of team science and multi-institutional collaboration that incorporates heterogenization, and the use of prospective and living meta-analyses to generate generalizable results. Such changes could make science more effective and, potentially, more equitable, helping to cultivate better collaborations.

Meta-analysis of variation in sport and exercise science: Examples of application within resistance training research

Meta-analysis has become commonplace within sport and exercise science for synthesising and summarising empirical studies. However, most research in the field focuses upon mean effects, particularly the effects of interventions to improve outcomes such as fitness or performance. It is thought that individual responses to interventions vary considerably. Hence, interest has increased in exploring precision or personalised exercise approaches. Not only is the mean often affected by interventions, but variation may also be impacted. Exploration of variation in studies such as randomised controlled trials (RCTs) can yield insight into interindividual heterogeneity in response to interventions and help determine generalisability of effects. Yet, larger samples sizes than those used for typical mean effects are required when probing variation. Thus, in a field with small samples such as sport and exercise science, exploration of variation through a meta-analytic framework is appealing. Despite the value of embracing and exploring variation alongside mean effects in sport and exercise science, it is rarely applied to research synthesis through meta-analysis. We introduce and evaluate different effect size calculations along with models for meta-analysis of variation using relatable examples from resistance training RCTs.

Trace amine-associated receptor 1 (TAAR1) agonists for psychosis: protocol for a living systematic review and meta-analysis of human and non-human studies

BACKGROUND

There is an urgent need to develop more effective and safer antipsychotics beyond dopamine 2 receptor antagonists. An emerging and promising approach is TAAR1 agonism. Therefore, we will conduct a living systematic review and meta-analysis to synthesize and triangulate the evidence from preclinical animal experiments and clinical studies on the efficacy, safety, and underlying mechanism of action of TAAR1 agonism for psychosis.

METHODS

Independent searches will be conducted in multiple electronic databases to identify clinical and animal experimental studies comparing TAAR1 agonists with licensed antipsychotics or other control conditions in individuals with psychosis or animal models for psychosis, respectively. The primary outcomes will be overall psychotic symptoms and their behavioural proxies in animals. Secondary outcomes will include side effects and neurobiological measures. Two independent reviewers will conduct study selection, data extraction using predefined forms, and risk of bias assessment using suitable tools based on the study design. Ontologies will be developed to facilitate study identification and data extraction. Data from clinical and animal studies will be synthesized separately using random-effects meta-analysis if appropriate, or synthesis without meta-analysis. Study characteristics will be investigated as potential sources of heterogeneity. Confidence in the evidence for each outcome and source of evidence will be evaluated, considering the summary of the association, potential concerns regarding internal and external validity, and reporting biases. When multiple sources of evidence are available for an outcome, an overall conclusion will be drawn in a triangulation meeting involving a multidisciplinary team of experts. We plan trimonthly updates of the review, and any modifications in the protocol will be documented. The review will be co-produced by multiple stakeholders aiming to produce impactful and relevant results and bridge the gap between preclinical and clinical research on psychosis.

PROTOCOL REGISTRATION

PROSPERO-ID: CRD42023451628.

Mapping strategies towards improved external validity in preclinical translational research

INTRODUCTION

Translation is about successfully bringing findings from preclinical contexts into the clinic. This transfer is challenging as clinical trials frequently fail despite positive preclinical results. Limited robustness of preclinical research has been marked as one of the drivers of such failures. One suggested solution is to improve the external validity of in vitro and in vivo experiments via a suite of complementary strategies.

AREAS COVERED

In this review, the authors summarize the literature available on different strategies to improve external validity in in vivo, in vitro, or ex vivo experiments; systematic heterogenization; generalizability tests; and multi-batch and multicenter experiments. Articles that tested or discussed sources of variability in systematically heterogenized experiments were identified, and the most prevalent sources of variability are reviewed further. Special considerations in sample size planning, analysis options, and practical feasibility associated with each strategy are also reviewed.

EXPERT OPINION

The strategies reviewed differentially influence variation in experiments. Different research projects, with their unique goals, can leverage the strengths and limitations of each strategy. Applying a combination of these approaches in confirmatory stages of preclinical research putatively increases the chances of success in clinical studies.

Seminatural environments for rodent behavioral testing: a representative design improving animal welfare and enhancing replicability

The low replicability of scientific studies has become an important issue. One possible cause is low representativeness of the experimental design employed. Already in the 1950's, Egon Brunswick pointed out that experimental setups ideally should be based on a random sample of stimuli from the subjects' natural environment or at least include basic features of that environment. Only experimental designs satisfying this criterion, representative designs in Brunswikian terminology, can produce results generalizable beyond the procedure used and to situations outside the laboratory. Such external validity is crucial in preclinical drug studies, for example, and should be important for replicability in general. Popular experimental setups in rodent research on non-human animals, like the tail suspension test or the Geller-Seifter procedure, do not correspond to contexts likely to be encountered in the animals' habitat. Consequently, results obtained in this kind of procedures can be generalized neither to other procedures nor to contexts outside the laboratory. Furthermore, many traditional procedures are incompatible with current notions of animal welfare. An approximation to the natural social and physical context can be provided in the laboratory, in the form of a seminatural environment. In addition to satisfy the basic demands for a representative design, such environments offer a far higher level of animal welfare than the typical small cages. This perspective article will briefly discuss the basic principles of the generalizability of experimental results, the virtues of representative designs and the coincidence of enhanced scientific quality and animal welfare provided by this kind of design.

Quantitative evidence synthesis: a practical guide on meta-analysis, meta-regression, and publication bias tests for environmental sciences

Meta-analysis is a quantitative way of synthesizing results from multiple studies to obtain reliable evidence of an intervention or phenomenon. Indeed, an increasing number of meta-analyses are conducted in environmental sciences, and resulting meta-analytic evidence is often used in environmental policies and decision-making. We conducted a survey of recent meta-analyses in environmental sciences and found poor standards of current meta-analytic practice and reporting. For example, only ~ 40% of the 73 reviewed meta-analyses reported heterogeneity (variation among effect sizes beyond sampling error), and publication bias was assessed in fewer than half. Furthermore, although almost all the meta-analyses had multiple effect sizes originating from the same studies, non-independence among effect sizes was considered in only half of the meta-analyses. To improve the implementation of meta-analysis in environmental sciences, we here outline practical guidance for conducting a meta-analysis in environmental sciences. We describe the key concepts of effect size and meta-analysis and detail procedures for fitting multilevel meta-analysis and meta-regression models and performing associated publication bias tests. We demonstrate a clear need for environmental scientists to embrace multilevel meta-analytic models, which explicitly model dependence among effect sizes, rather than the commonly used random-effects models. Further, we discuss how reporting and visual presentations of meta-analytic results can be much improved by following reporting guidelines such as PRISMA-EcoEvo (Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Ecology and Evolutionary Biology). This paper, along with the accompanying online tutorial, serves as a practical guide on conducting a complete set of meta-analytic procedures (i.e., meta-analysis, heterogeneity quantification, meta-regression, publication bias tests and sensitivity analysis) and also as a gateway to more advanced, yet appropriate, methods.

Estimating Genetic Variance in Life-Span Response to Diet: Insights From Statistical Simulation

Several studies demonstrate genetic variation in response to dietary restriction (DR) by replicating treatments across isogenic lines/strains from genetic reference panels. These studies typically quantify the response to DR as an effect size, estimated for each strain separately (eg, the difference in mean life span between groups). Such "no-pooling" analyses are expected to systematically overestimate variation in response DR, principally by overlooking sampling variance. In contrast, "partial-pooling" analyses using mixed-effects models are less prone to this bias. I demonstrate these issues using simulations, which also show that partial-pooling analyses can improve replicability among studies. Regardless of the analyses used, estimates of among-strain variation will have low precision when sample sizes are small. A worked example using survival data in mice is given. Life-span studies using genetic reference panels always have to trade-off within- and among-strain replication owing to logistical challenges. The simulation presented can also be used to help design such studies through power analysis.

Advanced methods and implementations for the meta-analyses of animal models: Current practices and future recommendations

Meta-analytic techniques have been widely used to synthesize data from animal models of human diseases and conditions, but these analyses often face two statistical challenges due to complex nature of animal data (e.g., multiple effect sizes and multiple species): statistical dependency and confounding heterogeneity. These challenges can lead to unreliable and less informative evidence, which hinders the translation of findings from animal to human studies. We present a literature survey of meta-analysis using animal models (animal meta-analysis), showing that these issues are not adequately addressed in current practice. To address these challenges, we propose a meta-analytic framework based on multilevel (linear mixed-effects) models. Through conceptualization, formulations, and worked examples, we illustrate how this framework can appropriately address these issues while allowing for testing new questions. Additionally, we introduce other advanced techniques such as multivariate models, robust variance estimation, and meta-analysis of emergent effect sizes, which can deliver robust inferences and novel biological insights. We also provide a tutorial with annotated R code to demonstrate the implementation of these techniques.

Embracing Heterogeneity in The Multicenter Stroke Preclinical Assessment Network (SPAN) Trial

The Stroke Preclinical Assessment Network (SPAN) is a multicenter preclinical trial platform using rodent models of transient focal cerebral ischemia to address translational failure in experimental stroke. In addition to centralized randomization and blinding and large samples, SPAN aimed to introduce heterogeneity to simulate the heterogeneity embodied in clinical trials for robust conclusions. Here, we report the heterogeneity introduced by allowing the 6 SPAN laboratories to vary most of the biological and experimental model variables and the impact of this heterogeneity on middle cerebral artery occlusion (MCAo) performance. We included the modified intention-to-treat population of the control mouse cohort of the first SPAN trial (n=421) and examined the biological and procedural independent variables and their covariance. We then determined their impact on the dependent variables cerebral blood flow drop during MCAo, time to achieve MCAo, and total anesthesia duration using multivariable analyses. We found heterogeneity in biological and procedural independent variables introduced mainly by the site. Consequently, all dependent variables also showed heterogeneity among the sites. Multivariable analyses with the site as a random effect variable revealed filament choice as an independent predictor of cerebral blood flow drop after MCAo. Comorbidity, sex, use of laser Doppler flow to monitor cerebral blood flow, days after trial onset, and maintaining anesthesia throughout the MCAo emerged as independent predictors of time to MCAo. Total anesthesia duration was predicted by most independent variables. We present with high granularity the heterogeneity introduced by the biological and model selections by the testing sites in the first trial of cerebroprotection in rodent transient filament MCAo by SPAN. Rather than trying to homogenize all variables across all sites, we embraced the heterogeneity to better approximate clinical trials. Awareness of the heterogeneity, its sources, and how it impacts the study performance may further improve the study design and statistical modeling for future multicenter preclinical trials.

Recommendations for robust and reproducible preclinical research in personalised medicine

BACKGROUND

Personalised medicine is a medical model that aims to provide tailor-made prevention and treatment strategies for defined groups of individuals. The concept brings new challenges to the translational step, both in clinical relevance and validity of models. We have developed a set of recommendations aimed at improving the robustness of preclinical methods in translational research for personalised medicine.

METHODS

These recommendations have been developed following four main steps: (1) a scoping review of the literature with a gap analysis, (2) working sessions with a wide range of experts in the field, (3) a consensus workshop, and (4) preparation of the final set of recommendations.

RESULTS

Despite the progress in developing innovative and complex preclinical model systems, to date there are fundamental deficits in translational methods that prevent the further development of personalised medicine. The literature review highlighted five main gaps, relating to the relevance of experimental models, quality assessment practices, reporting, regulation, and a gap between preclinical and clinical research. We identified five points of focus for the recommendations, based on the consensus reached during the consultation meetings: (1) clinically relevant translational research, (2) robust model development, (3) transparency and education, (4) revised regulation, and (5) interaction with clinical research and patient engagement. Here, we present a set of 15 recommendations aimed at improving the robustness of preclinical methods in translational research for personalised medicine.

CONCLUSIONS

Appropriate preclinical models should be an integral contributor to interventional clinical trial success rates, and predictive translational models are a fundamental requirement to realise the dream of personalised medicine. The implementation of these guidelines is ambitious, and it is only through the active involvement of all relevant stakeholders in this field that we will be able to make an impact and effectuate a change which will facilitate improved translation of personalised medicine in the future.

Replication in Evidence-Based Aphasia Treatments

PURPOSE

A critical review was completed to evaluate replication of aphasia treatments that have been vetted and accepted on the American Speech-Language-Hearing Association Practice Portal.

METHOD

The electronic databases Academic Search Premier, ProQuest Central, CINAHL Complete, and ERIC were searched for relevant articles using treatment names as keywords. Coders compared stimuli, material, design, and statistical analysis to pilot treatment approaches. Each study was coded as direct, conceptual, failed, or no replication.

RESULTS

Eighteen treatment approaches were selected for this review. A total of 188 articles met the inclusion criteria. Results revealed that 14 out of 18 treatment approaches were somehow replicated. Direct replications as the most valuable replication type for affirming previously found results were represented in only 1.5% of all studies (3/188). Failed direct replication were at 2% overall (4/188). Conceptual replications were more common across treatment approaches, but also represented at a low level with 22.8% (43/188). The majority of studies were coded as no replication attempt with 73.4%.

DISCUSSION

A critical factor in developing an evidence-based practice is the existence of replicated results for treatment. Replication evaluates the reproducibility of an author's or authors' research methodology and resulting outcomes and helps to ensure that observed treatment effects are reproducible. For an evidence-based treatment to be implemented or used in any clinical setting, it must be one that can be replicated. Direct and conceptual replications of aphasia treatment approaches were found to be alarmingly low considering the importance of replication in our field. It is recommended that replication should become more valued and mainstreamed in aphasia research. A replication database that compiles and maintains treatment manuals for replication purposes can increase the accessibility and acceptability of replications for researchers.

Sex differences in allometry for phenotypic traits in mice indicate that females are not scaled males

Sex differences in the lifetime risk and expression of disease are well-known. Preclinical research targeted at improving treatment, increasing health span, and reducing the financial burden of health care, has mostly been conducted on male animals and cells. The extent to which sex differences in phenotypic traits are explained by sex differences in body weight remains unclear. We quantify sex differences in the allometric relationship between trait value and body weight for 363 phenotypic traits in male and female mice, recorded in >2 million measurements from the International Mouse Phenotyping Consortium. We find sex differences in allometric parameters (slope, intercept, residual SD) are common (73% traits). Body weight differences do not explain all sex differences in trait values but scaling by weight may be useful for some traits. Our results show sex differences in phenotypic traits are trait-specific, promoting case-specific approaches to drug dosage scaled by body weight in mice.

Early, Intense Rehabilitation Fails to Improve Outcome After Intra-Striatal Hemorrhage in Rats

BACKGROUND

The formation and degradation of an intracerebral hemorrhage causes protracted cell death, and an extended window for intervention. Experimental studies find that rehabilitation mitigates late cell death, with accelerated hematoma clearance as a potential mechanism.

OBJECTIVE

We assessed whether early, intense, enriched rehabilitation (ER, environmental enrichment and massed skills training) enhances functional benefit, reduces brain injury, and augments hematoma clearance.

METHODS

In experiment 1, rats (n = 56) were randomized to intervention in the light (-L) or dark phase (-D) of their housing cycle, then to 10 days of ER or control (CON) treatment after collagenase-induced striatal intracerebral hemorrhage (ICH). ER rats were treated from 5 to 14 days after ICH. Behavior and residual hematoma volume was assessed on day 14. In experiment 2, rats (n = 72) were randomized to ER-D10, ER-D20, or CON-D. ER rats completed 10 or 20 days of training in the dark. Rats were euthanized on day 60 for histology. In both experiments, behavioral assessment was completed pre-ICH, pre-ER (day 4 post-ICH), and post-ER (experiment 1: days 13-14; experiment 2: days 16-17 and 30-31).

RESULTS

Reaching intensity was high but similar between ER-D10 and ER-L10. Unlike previous work, rehabilitation did not alter skilled reaching or hematoma resolution. Varying ER duration also did not affect reaching success or lesion volume.

CONCLUSIONS

In contrast to others, and under these conditions, our findings show that striatal ICH was generally unresponsive to rehabilitation. This highlights the difficulty of replicating and extending published work, perhaps owing to small inter-study differences.

Systematic assessment of the replicability and generalizability of preclinical findings: Impact of protocol harmonization across laboratory sites

The influence of protocol standardization between laboratories on their replicability of preclinical results has not been addressed in a systematic way. While standardization is considered good research practice as a means to control for undesired external noise (i.e., highly variable results), some reports suggest that standardized protocols may lead to idiosyncratic results, thus undermining replicability. Through the EQIPD consortium, a multi-lab collaboration between academic and industry partners, we aimed to elucidate parameters that impact the replicability of preclinical animal studies. To this end, 3 experimental protocols were implemented across 7 laboratories. The replicability of results was determined using the distance travelled in an open field after administration of pharmacological compounds known to modulate locomotor activity (MK-801, diazepam, and clozapine) in C57BL/6 mice as a worked example. The goal was to determine whether harmonization of study protocols across laboratories improves the replicability of the results and whether replicability can be further improved by systematic variation (heterogenization) of 2 environmental factors (time of testing and light intensity during testing) within laboratories. Protocols were tested in 3 consecutive stages and differed in the extent of harmonization across laboratories and standardization within laboratories: stage 1, minimally aligned across sites (local protocol); stage 2, fully aligned across sites (harmonized protocol) with and without systematic variation (standardized and heterogenized cohort); and stage 3, fully aligned across sites (standardized protocol) with a different compound. All protocols resulted in consistent treatment effects across laboratories, which were also replicated within laboratories across the different stages. Harmonization of protocols across laboratories reduced between-lab variability substantially compared to each lab using their local protocol. In contrast, the environmental factors chosen to introduce systematic variation within laboratories did not affect the behavioral outcome. Therefore, heterogenization did not reduce between-lab variability further compared to the harmonization of the standardized protocol. Altogether, these findings demonstrate that subtle variations between lab-specific study protocols may introduce variation across independent replicate studies even after protocol harmonization and that systematic heterogenization of environmental factors may not be sufficient to account for such between-lab variation. Differences in replicability of results within and between laboratories highlight the ubiquity of study-specific variation due to between-lab variability, the importance of transparent and fine-grained reporting of methodologies and research protocols, and the importance of independent study replication.

The rearing environment persistently modulates mouse phenotypes from the molecular to the behavioural level

The phenotype of an organism results from its genotype and the influence of the environment throughout development. Even when using animals of the same genotype, independent studies may test animals of different phenotypes, resulting in poor replicability due to genotype-by-environment interactions. Thus, genetically defined strains of mice may respond differently to experimental treatments depending on their rearing environment. However, the extent of such phenotypic plasticity and its implications for the replicability of research findings have remained unknown. Here, we examined the extent to which common environmental differences between animal facilities modulate the phenotype of genetically homogeneous (inbred) mice. We conducted a comprehensive multicentre study, whereby inbred C57BL/6J mice from a single breeding cohort were allocated to and reared in 5 different animal facilities throughout early life and adolescence, before being transported to a single test laboratory. We found persistent effects of the rearing facility on the composition and heterogeneity of the gut microbial community. These effects were paralleled by persistent differences in body weight and in the behavioural phenotype of the mice. Furthermore, we show that environmental variation among animal facilities is strong enough to influence epigenetic patterns in neurons at the level of chromatin organisation. We detected changes in chromatin organisation in the regulatory regions of genes involved in nucleosome assembly, neuronal differentiation, synaptic plasticity, and regulation of behaviour. Our findings demonstrate that common environmental differences between animal facilities may produce facility-specific phenotypes, from the molecular to the behavioural level. Furthermore, they highlight an important limitation of inferences from single-laboratory studies and thus argue that study designs should take environmental background into account to increase the robustness and replicability of findings.

The phenotype of an organism results not only from its genotype but also the influence of its environment throughout development. This study shows that common environmental differences between animal facilities can induce substantial variation in the phenotype of mice, thereby highlighting an important limitation of inferences from single-laboratory studies in animal research.

Animal pollination increases stability of crop yield across spatial scales

The benefits of animal pollination to crop yield are well known. In contrast, the effects of animal pollination on the spatial or temporal stability (the opposite of variability) of crop yield remain poorly understood. We use meta-analysis to combine variability information from 215 experimental comparisons between animal-pollinated and wind- or self-pollinated control plants in apple, oilseed rape and faba bean. Animal pollination increased yield stability (by an average of 32% per unit of yield) at between-flower, -plant, -plot and -field scales. Evidence suggests this occurs because yield benefits of animal pollination become progressively constrained closer to the maximum potential yield in a given context, causing clustering. The increase in yield stability with animal pollination is greatest when yield benefits of animal pollination are greatest, indicating that managing crop pollination to increase yield also increases yield stability. These additional pollination benefits have not yet been included in economic assessments but provide further justification for policies to protect pollinators.

A composite clinical motor score as a comprehensive and sensitive outcome measure for Parkinson's disease

BACKGROUND

An unmet need remains for sensitive outcome measures in neuroprotective trials. The study aims to determine whether a composite clinical motor score, combining the Movement Disorders Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS) III motor examination score, Purdue Pegboard Test, and Timed Up and Go, provides greater sensitivity in detecting motor change in early disease than the MDS-UPDRS III alone.

METHODS

The Oxford Discovery longitudinal cohort study involves individuals with isolated rapid eye movement sleep behaviour disorder (iRBD) (n=272, confirmed polysomnographically, median follow-up: 1.6 years), idiopathic Parkinson's disease (PD) (n=909, median follow-up: 3.5 years, baseline: <3.5 years disease duration) and controls (n=316, age-matched and sex-matched, without a first-degree family history of PD). Motor and non-motor assessments were performed at each in-person visit.

RESULTS

Compared with the MDS-UPDRS III, the composite clinical motor score demonstrated a wider score distribution in iRBD and controls, lower coefficient of variation (37% vs 67%), and higher correlation coefficients with self-reported measures of motor severity (0.65 vs 0.61) and overall health status (-0.40 vs -0.33). Greater score range in mild to moderate PD, higher magnitude of longitudinal change in iRBD and longitudinal score linearity suggest better sensitivity in detecting subtle motor change. The composite clinical motor score was more accurate than the MDS-UPDRS III in predicting clinical outcomes, requiring 64% fewer participants with PD and 51% fewer participants with iRBD in sample size estimations for a hypothetical 18-month placebo-controlled clinical trial.

CONCLUSION

The composite clinical motor score may offer greater consistency and sensitivity in detecting change than the MDS-UPDRS III.

The Stroke Preclinical Assessment Network: Rationale, Design, Feasibility, and Stage 1 Results

Cerebral ischemia and reperfusion initiate cellular events in brain that lead to neurological disability. Investigating these cellular events provides ample targets for developing new treatments. Despite considerable work, no such therapy has translated into successful stroke treatment. Among other issues-such as incomplete mechanistic knowledge and faulty clinical trial design-a key contributor to prior translational failures may be insufficient scientific rigor during preclinical assessment: nonblinded outcome assessment; missing randomization; inappropriate sample sizes; and preclinical assessments in young male animals that ignore relevant biological variables, such as age, sex, and relevant comorbid diseases. Promising results are rarely replicated in multiple laboratories. We sought to address some of these issues with rigorous assessment of candidate treatments across 6 independent research laboratories. The Stroke Preclinical Assessment Network (SPAN) implements state-of-the-art experimental design to test the hypothesis that rigorous preclinical assessment can successfully reduce or eliminate common sources of bias in choosing treatments for evaluation in clinical studies. SPAN is a randomized, placebo-controlled, blinded, multilaboratory trial using a multi-arm multi-stage protocol to select one or more putative stroke treatments with an implied high likelihood of success in human clinical stroke trials. The first stage of SPAN implemented procedural standardization and experimental rigor. All participating research laboratories performed middle cerebral artery occlusion surgery adhering to a common protocol and rapidly enrolled 913 mice in the first of 4 planned stages with excellent protocol adherence, remarkable data completion and low rates of subject loss. SPAN stage 1 successfully implemented treatment masking, randomization, prerandomization inclusion/exclusion criteria, and blinded assessment to exclude bias. Our data suggest that a large, multilaboratory, preclinical assessment effort to reduce known sources of bias is feasible and practical. Subsequent SPAN stages will evaluate candidate treatments for potential success in future stroke clinical trials using aged animals and animals with comorbid conditions.

Paths towards greater consensus building in experimental biology

In a recent editorial, the Editors-in-Chief of Journal of Experimental Biology argued that consensus building, data sharing, and better integration across disciplines are needed to address the urgent scientific challenges posed by climate change. We agree and expand on the importance of cross-disciplinary integration and transparency to improve consensus building and advance climate change research in experimental biology. We investigated reproducible research practices in experimental biology through a review of open data and analysis code associated with empirical studies on three debated paradigms and for unrelated studies published in leading journals in comparative physiology and behavioural ecology over the last 10 years. Nineteen per cent of studies on the three paradigms had open data, and 3.2% had open code. Similarly, 12.1% of studies in the journals we examined had open data, and 3.1% had open code. Previous research indicates that only 50% of shared datasets are complete and re-usable, suggesting that fewer than 10% of studies in experimental biology have usable open data. Encouragingly, our results indicate that reproducible research practices are increasing over time, with data sharing rates in some journals reaching 75% in recent years. Rigorous empirical research in experimental biology is key to understanding the mechanisms by which climate change affects organisms, and ultimately promotes evidence-based conservation policy and practice. We argue that a greater adoption of open science practices, with a particular focus on FAIR (Findable, Accessible, Interoperable, Re-usable) data and code, represents a much-needed paradigm shift towards improved transparency, cross-disciplinary integration, and consensus building to maximize the contributions of experimental biologists in addressing the impacts of environmental change on living organisms.

Roadbumps at the Crossroads of Integrating Behavioral and In Vitro Approaches for Neurotoxicity Assessment

With the appreciation that behavior represents the integration and complexity of the nervous system, neurobehavioral phenotyping and assessment has seen a renaissance over the last couple of decades, resulting in a robust database on rodent performance within various testing paradigms, possible associations with human disorders, and therapeutic interventions. The interchange of data across behavior and other test modalities and multiple model systems has advanced our understanding of fundamental biology and mechanisms associated with normal functions and alterations in the nervous system. While there is a demonstrated value and power of neurobehavioral assessments for examining alterations due to genetic manipulations, maternal factors, early development environment, the applied use of behavior to assess environmental neurotoxicity continues to come under question as to whether behavior represents a sensitive endpoint for assessment. Why is rodent behavior a sensitive tool to the neuroscientist and yet, not when used in pre-clinical or chemical neurotoxicity studies? Applying new paradigms and evidence on the biological basis of behavior to neurobehavioral testing requires expertise and refinement of how such experiments are conducted to minimize variability and maximize information. This review presents relevant issues of methods used to conduct such test, sources of variability, experimental design, data analysis, interpretation, and reporting. It presents beneficial and critical limitations as they translate to the in vivo environment and considers the need to integrate across disciplines for the best value. It proposes that a refinement of behavioral assessments and understanding of subtle pronounced differences will facilitate the integration of data obtained across multiple approaches and to address issues of translation.

Environmental Enrichment for Rats and Mice Housed in Laboratories: A Metareview

Simple Summary

Environmental enrichment has been widely studied with laboratory rodents, but there is no consensus regarding what counts as enrichment or what it should achieve. Inconsistent use of the term “enrichment” creates challenges in drawing conclusions about the quality of an environment. We conducted a metareview to better understand the definitions and goals of enrichment, perceived risks or requirements of enrichment, and what forms of enrichment have previously been endorsed for use with rodents housed in laboratories. This may help researchers and animal care staff to better define their chosen approach and intended outcomes when providing environmental enrichment.

Abstract

Environmental enrichment has been widely studied in rodents, but there is no consensus on what enrichment should look like or what it should achieve. Inconsistent use of the term “enrichment” creates challenges in drawing conclusions about the quality of an environment, which may slow housing improvements for laboratory animals. Many review articles have addressed environmental enrichment for laboratory rats and mice (Rattus norvegicus and Mus musculus). We conducted a metareview of 29 review articles to assess how enrichment has been defined and what are commonly described as its goals or requirements. Recommendations from each article were summarised to illustrate the conditions generally considered suitable for laboratory rodents. While there is no consensus on alternative terminology, many articles acknowledged that the blanket use of the terms “enriched” and “enrichment” should be avoided. Environmental enrichment was most often conceptualised as a method to increase natural behaviour and improve animal welfare. Authors also commonly outlined perceived risks and requirements of environmental enrichment. We discuss these perceptions, make suggestions for future research, and advocate for the adoption of more specific and value-neutral terminology.

URB597 induces subtle changes to aggression in adult Lister Hooded rats

The endocannabinoid system has been implicated in both social and cognitive processing. The endocannabinoid metabolism inhibitor, URB597, dose-dependently improves non-social memory in adult Wistar and Sprague Dawley rats, whereas its effect on social interaction (SI) is affected by both rat strain and drug dose. Lister Hooded rats consistently respond differently to drug treatment in general compared with albino strains. This study sought to investigate the effects of different doses of URB597 on social and non-social memory in Lister Hooded rats, as well as analyzing the behavioral composition of the SI. Males were tested for novel object recognition (NOR), social preference (between an object and an unfamiliar rat), social novelty recognition (for a familiar vs. unfamiliar rat) and SI with an unfamiliar rat. URB597 (0.1 or 0.3 mg/kg) or vehicle was given 30 min before testing. During SI testing, total interaction time was assessed along with time spent on aggressive and explorative behaviors. Lister Hooded rats displayed expected non-social and social memory and social preference, which was not affected by URB597. During SI, URB597 did not affect total interaction time. However, the high dose increased aggression, compared to vehicle, and decreased anogenital sniffing, compared to the low dose of URB597. In summary, URB597 did not affect NOR, social preference or social recognition memory but did have subtle behavioral effects during SI in Lister hooded rats. Based on our findings we argue for the importance of considering strain as well as the detailed composition of behavior when investigating drug effects on social behavior.

Deregulated Protein Kinases: Friend and Foe in Ischemic Stroke

Ischemic stroke is the third leading cause of mortality worldwide, but its medical management is still limited to the use of thrombolytics as a lifesaving option. Multiple molecular deregulations of the protein kinase family occur during the period of ischemia/reperfusion. However, experimental studies have shown that alterations in the expression of essential protein kinases and their pharmacological modulation can modify the neuropathological milieu and hasten neurophysiological recovery. This review highlights the role of key protein kinase members and their implications in the evolution of stroke pathophysiology. Activation of ROCK-, MAPK-, and GSK-3β-mediated pathways following neuronal ischemia/reperfusion injury in experimental conditions aggravate the neuropathology and delays recovery. Targeting ROCK, MAPK, and GSK-3β will potentially enhance myelin regeneration, improve blood-brain barrier (BBB) function, and suppress inflammation, which ameliorates neuronal survival. Conversely, protein kinases such as PKA, Akt, PKCα, PKCε, Trk, and PERK salvage neurons post-ischemia by mechanisms including enhanced toxin metabolism, restoring BBB integrity, neurotrophic effects, and apoptosis suppression. Certain protein kinases such as ERK1/2, JNK, and AMPK have favourable and unfavourable effects in salvaging ischemia-injured neurons. Targeting multiple protein kinase-mediated pathways simultaneously may improve neuronal recovery post-ischemia.