New Initiatives to Improve the Rigor and Reproducibility of Articles Published in Circulation Research

Patient engagement in designing, conducting, and disseminating clinical pain research: IMMPACT recommended considerations

ABSTRACT

In the traditional clinical research model, patients are typically involved only as participants. However, there has been a shift in recent years highlighting the value and contributions that patients bring as members of the research team, across the clinical research lifecycle. It is becoming increasingly evident that to develop research that is both meaningful to people who have the targeted condition and is feasible, there are important benefits of involving patients in the planning, conduct, and dissemination of research from its earliest stages. In fact, research funders and regulatory agencies are now explicitly encouraging, and sometimes requiring, that patients are engaged as partners in research. Although this approach has become commonplace in some fields of clinical research, it remains the exception in clinical pain research. As such, the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials convened a meeting with patient partners and international representatives from academia, patient advocacy groups, government regulatory agencies, research funding organizations, academic journals, and the biopharmaceutical industry to develop consensus recommendations for advancing patient engagement in all stages of clinical pain research in an effective and purposeful manner. This article summarizes the results of this meeting and offers considerations for meaningful and authentic engagement of patient partners in clinical pain research, including recommendations for representation, timing, continuous engagement, measurement, reporting, and research dissemination.

Nonsense Variant PRDM16-Q187X Causes Impaired Myocardial Development and TGF-β Signaling Resulting in Noncompaction Cardiomyopathy in Humans and Mice

BACKGROUND

PRDM16 plays a role in myocardial development through TGF-β (transforming growth factor-beta) signaling. Recent evidence suggests that loss of PRDM16 expression is associated with cardiomyopathy development in mice, although its role in human cardiomyopathy development is unclear. This study aims to determine the impact of PRDM16 loss-of-function variants on cardiomyopathy in humans.

METHODS

Individuals with PRDM16 variants were identified and consented. Induced pluripotent stem cell-derived cardiomyocytes were generated from a proband hosting a Q187X nonsense variant as an in vitro model and underwent proliferative and transcriptional analyses. CRISPR (clustered regularly interspaced short palindromic repeats)-mediated knock-in mouse model hosting the Prdm16Q187X allele was generated and subjected to ECG, histological, and transcriptional analysis.

RESULTS

We report 2 probands with loss-of-function PRDM16 variants and pediatric left ventricular noncompaction cardiomyopathy. One proband hosts a PRDM16-Q187X variant with left ventricular noncompaction cardiomyopathy and demonstrated infant-onset heart failure, which was selected for further study. Induced pluripotent stem cell-derived cardiomyocytes prepared from the PRDM16-Q187X proband demonstrated a statistically significant impairment in myocyte proliferation and increased apoptosis associated with transcriptional dysregulation of genes implicated in cardiac maturation, including TGF-β-associated transcripts. Homozygous Prdm16Q187X/Q187X mice demonstrated an underdeveloped compact myocardium and were embryonically lethal. Heterozygous Prdm16Q187X/WT mice demonstrated significantly smaller ventricular dimensions, heightened fibrosis, and age-dependent loss of TGF-β expression. Mechanistic studies were undertaken in H9c2 cardiomyoblasts to show that PRDM16 binds TGFB3 promoter and represses its transcription.

CONCLUSIONS

Novel loss-of-function PRDM16 variant impairs myocardial development resulting in noncompaction cardiomyopathy in humans and mice associated with altered TGF-β signaling.

PU.1 inhibition does not attenuate cardiac function deterioration or fibrosis in a murine model of myocardial infarction

Activated cardiac fibroblasts are involved in both reparative wound healing and maladaptive cardiac fibrosis after myocardial infarction (MI). Recent evidence suggests that PU.1 inhibition can enable reprogramming of profibrotic fibroblasts to quiescent fibroblasts, leading to attenuation of pathologic fibrosis in several fibrosis models. The role of PU.1 in acute MI has not been tested. We designed a randomized, blinded study to evaluate whether DB1976, a PU.1 inhibitor, attenuates cardiac function deterioration and fibrosis in a murine model of MI. A total of 44 Ai9 periostin-Cre transgenic mice were subjected to 60 min of coronary occlusion followed by reperfusion. At 7 days after MI, 37 mice were randomly assigned to control (vehicle) or DB1976 treatment and followed for 2 weeks. Left ventricular ejection fraction (EF), assessed by echocardiography, did not differ between the two groups before or after treatment (final EF, 33.3 ± 1.0% in control group and 31.2 ± 1.3% in DB1976 group). Subgroup analysis of female and male mice showed the same results. There were no differences in cardiac scar (trichrome stain) and fibrosis (interstitial/perivascular collagen; picrosirius stain) between groups. Results from the per-protocol dataset (including mice with pre-treatment EF < 35% only) were consistent with the full dataset. In conclusion, this randomized, blinded study demonstrates that DB1976, a PU.1 inhibitor, does not attenuate cardiac functional deterioration or cardiac fibrosis in a mouse model of MI caused by coronary occlusion/reperfusion.

Reduction in Junctophilin 2 Expression in Cardiac Nodal Tissue Results in Intracellular Calcium-Driven Increase in Nodal Cell Automaticity

BACKGROUND

Spontaneously depolarizing nodal cells comprise the pacemaker of the heart. Intracellular calcium (Ca2+) plays a critical role in mediating nodal cell automaticity and understanding this so-called Ca2+ clock is critical to understanding nodal arrhythmias. We previously demonstrated a role for Jph2 (junctophilin 2) in regulating Ca2+-signaling through inhibition of RyR2 (ryanodine receptor 2) Ca2+ leak in cardiac myocytes; however, its role in pacemaker function and nodal arrhythmias remains unknown. We sought to determine whether nodal Jph2 expression silencing causes increased sinoatrial and atrioventricular nodal cell automaticity due to aberrant RyR2 Ca2+ leak.

METHODS

A tamoxifen-inducible, nodal tissue-specific, knockdown mouse of Jph2 was achieved using a Cre-recombinase-triggered short RNA hairpin directed against Jph2 (Hcn4:shJph2). In vivo cardiac rhythm was monitored by surface ECG, implantable cardiac telemetry, and intracardiac electrophysiology studies. Intracellular Ca2+ imaging was performed using confocal-based line scans of isolated nodal cells loaded with fluorescent Ca2+ reporter Cal-520. Whole cell patch clamp was conducted on isolated nodal cells to determine action potential kinetics and sodium-calcium exchanger function.

RESULTS

Hcn4:shJph2 mice demonstrated a 40% reduction in nodal Jph2 expression, resting sinus tachycardia, and impaired heart rate response to pharmacologic stress. In vivo intracardiac electrophysiology studies and ex vivo optical mapping demonstrated accelerated junctional rhythm originating from the atrioventricular node. Hcn4:shJph2 nodal cells demonstrated increased and irregular Ca2+ transient generation with increased Ca2+ spark frequency and Ca2+ leak from the sarcoplasmic reticulum. This was associated with increased nodal cell AP firing rate, faster diastolic repolarization rate, and reduced sodium-calcium exchanger activity during repolarized states compared to control. Phenome-wide association studies of the JPH2 locus identified an association with sinoatrial nodal disease and atrioventricular nodal block.

CONCLUSIONS

Nodal-specific Jph2 knockdown causes increased nodal automaticity through increased Ca2+ leak from intracellular stores. Dysregulated intracellular Ca2+ underlies nodal arrhythmogenesis in this mouse model.

Rigor Me This: What Are the Basic Criteria for a Rigorous, Transparent, and Reproducible Scientific Study?

Scientific advancement is predicated upon the ability of a novel discovery to be independently reproduced and substantiated by others. Despite this inherent necessity, the research community is awash in published studies that cannot be replicated resulting in widespread confusion within the field and waning trust from the general public. In many cases, irreproducibility is the unavoidable consequence of a study that is conducted without the appropriate degree of rigor, typified by fundamental flaws in approach, design, execution, analysis, interpretation, and reporting. Combatting the irreproducibility pandemic in preclinical research is of urgent concern and is the primary responsibility of individual investigators, however there are important roles to be played by institutions, journals, government entities, and funding agencies as well. Herein, we provide an updated review of established rigor criteria pertaining to both in vitro and in vivo studies compiled from multiple sources across the research enterprise and present a practical checklist as a straightforward reference guide. It is our hope that this review may serve as an approachable resource for early career and experienced investigators alike, as they strive to improve all aspects of their scientific endeavors.

Weaknesses in Experimental Design and Reporting Decrease the Likelihood of Reproducibility and Generalization of Recent Cardiovascular Research

Recent evidence indicates that many clinical and preclinical studies are not reproducible. Prominent causes include design and implementation issues, low statistical power, unintentional bias, and incomplete reporting in the published literature. The primary goal of this study was to assess the quality of published research in three prominent cardiovascular research journals by examining statistical power and assessing the adherence to augmented ARRIVE guidelines (Animal Research: Reporting of In Vivo Experiments). For unpaired t-tests, the average median power for a 20% and 50% change was 0.27 ± 0.06 and 0.88 ± 0.08, respectively. For analysis of guidelines, 40 categories were assessed with a 0-2 scale. Although many strengths were observed, several key elements that were needed for reproducibility were inadequate, including differentiation of primary and secondary outcomes, power calculations for group size, allocation methods, use of randomization and blinding, checks for normality, reports of attrition, and adverse events of subjects, and assessment of bias. A secondary goal was to examine whether a required checklist improved the quality of reporting; those results indicated that a checklist improved compliance and quality of reporting, but adequacy levels in key categories were still too low. Overall, the findings of this study indicated that the probability for reproducibility of many clinical and preclinical cardiovascular research studies was low because of incomplete reporting, low statistical power, and lack of research practices that decrease experimental bias. Expansion of group sizes to increase power, use of detailed checklists, and closer monitoring for checklist adherence by editors and journals should remediate many of these deficits and increase the likelihood of reproducibility.

Improving rigor and reproducibility in nonhuman primate research

Nonhuman primates (NHPs) are a critical component of translational/preclinical biomedical research due to the strong similarities between NHP and human physiology and disease pathology. In some cases, NHPs represent the most appropriate, or even the only, animal model for complex metabolic, neurological, and infectious diseases. The increased demand for and limited availability of these valuable research subjects requires that rigor and reproducibility be a prime consideration to ensure the maximal utility of this scarce resource. Here, we discuss a number of approaches that collectively can contribute to enhanced rigor and reproducibility in NHP research.

Single dose of synthetic microRNA-199a or microRNA-149 mimic does not improve cardiac function in a murine model of myocardial infarction

Intramyocardial injection of synthetic microRNAs (miRs) has recently been reported to be beneficial after myocardial infarction (MI). We conducted a randomized blinded study to evaluate the efficacy and reproducibility of this strategy in a mouse model of reperfused MI using rigorous methodology. Mice undergoing a 60-min coronary occlusion followed by reperfusion were randomly assigned to control miR, hsa-miR-199a-3p, hsa-miR-149-3p, or hsa-miR-149-5p mimic treatment. Intramyocardial injections of miRs were performed in the border zone right after reperfusion. At 8 weeks after MI, there were no significant differences in ejection fraction (EF) among groups (EF = 27.1 ± 0.4% in control group [n = 6] and 25.9 ± 0.5%, 26.0 ± 0.8%, and 26.6 ± 0.6% in hsa-miR-199a-3p, hsa-miR-149-3p, or hsa-miR-149-5p groups, respectively [n = 9 each]). Net change (delta) in EF at 8 weeks compared with day 3 after MI was - 4.1% in control and - 3.2%, - 2.4%, and - 0.4% in the miR-treated groups (P = NS). Assessment of cardiac function by hemodynamic studies (a method independent of echocardiography) confirmed that there was no difference in left ventricular systolic or diastolic function among groups. Consistent with the functional data, histological analysis showed no difference in scar size, cardiomyocyte area, capillary density, collagen content, or apoptosis among groups. In conclusion, this randomized, blinded study demonstrates that intramyocardial injection of a single dose of synthetic hsa-miR-199a-3p, hsa-miR-149-3p, or hsa-miR-149-5p mimic does not improve cardiac function or remodeling in a murine model of reperfused MI. The strategy of using synthetic miR mimics for cardiac repair after MI needs to be evaluated with rigorous preclinical studies before its potential clinical translation.

Myocardial Redox Hormesis Protects the Heart of Female Mice in Sepsis

Mice challenged with lipopolysaccharide develop cardiomyopathy in a sex and redox-dependent fashion. Here we extended these studies to the cecal ligation and puncture (CLP) model.We compared male and female FVB mice (wild type, WT) and transgenic littermates overexpressing myocardial catalase (CAT). CLP induced 100% mortality within 4 days, with similar mortality rates in male and female WT and CAT mice. 24 h after CLP, isolated (Langendorff) perfused hearts showed depressed contractility in WT male mice, but not in male CAT or female WT and CAT mice. In WT male mice, CLP induced a depression of cardiomyocyte sarcomere shortening (ΔSS) and calcium transients (ΔCai), and the inhibition of the sarcoplasmic reticulum Ca ATPase (SERCA). These deficits were associated with overexpression of NADPH-dependent oxidase (NOX)-1, NOX-2, and cyclooxygenase 2 (COX-2), and were partially prevented in male CAT mice. Female WT mice showed unchanged ΔSS, ΔCai, and SERCA function after CLP. At baseline, female WT mice showed partially depressed ΔSS, ΔCai, and SERCA function, as compared with male WT mice, which were associated with NOX-1 overexpression and were prevented in CAT female mice.In conclusion, in male WT mice, septic shock induces myocardial NOX-1, NOX-2, and COX-2, and redox-dependent dysregulation of myocardial Ca transporters. Female WT mice are resistant to CLP-induced cardiomyopathy, despite increased NOX-1 and COX-2 expression, suggesting increased antioxidant capacity. Female resistance occurred in association with NOX-1 overexpression and signs of increased oxidative signaling at baseline, indicating the presence of a protective myocardial redox hormesis mechanism.

Journal Initiatives to Enhance Preclinical Research: Analyses of Stroke, Nature Medicine, Science Translational Medicine

Background and Purpose- Preclinical research using animals often informs clinical trials. However, its value is dependent on its scientific validity and reproducibility, which are, in turn, dependent on rigorous study design and reporting. In 2011, Stroke introduced a Basic Science Checklist to enhance the reporting and methodology of its preclinical studies. Except for Nature and Science journals, few others have implemented similar initiatives. We sought to estimate the impact of these journal interventions on the quality of their published reports. Methods- All articles published in Stroke, Nature Medicine, and Science Translational Medicine over 9 to 18 years and in 2 control journals without analogous interventions over a corresponding 11.5 years were reviewed to identify reports of experiments in nonhuman mammals with proposed clinical relevance. The effect of journal interventions on the reporting and use of key study design elements was estimated via interrupted time-series analyses. Results- Of 33 009 articles screened, 4162 studies met inclusion criteria. In the 3.5 to 12 years preceding each journal's intervention, the proportions of studies reporting and using key study design elements were stable except for blinding in Stroke and randomization in Science Translational Medicine, which were both increasing. Post-intervention, abrupt and often marked increases were seen in the reporting of randomization status (level change: +17% to +44%, P≤0.005), blinding (level change: +20% to +40%, P≤0.008), and sample size estimation (level change: 0% to +40%, P≤0.002 in 2 journals). Significant but more modest improvements in the use of these study design elements were also observed. These improvements were not seen in control journals. Conclusions- Journal interventions such as Stroke's author submission checklist can meaningfully improve the quality of published preclinical research and should be considered to enhance study transparency and design. However, such interventions are alone insufficient to fully address widespread shortcomings in preclinical research practices.

Perspectives on Directions and Priorities for Future Preclinical Studies in Regenerative Medicine

The myocardium consists of numerous cell types embedded in organized layers of ECM (extracellular matrix) and requires an intricate network of blood and lymphatic vessels and nerves to provide nutrients and electrical coupling to the cells. Although much of the focus has been on cardiomyocytes, these cells make up <40% of cells within a healthy adult heart. Therefore, repairing or regenerating cardiac tissue by merely reconstituting cardiomyocytes is a simplistic and ineffective approach. In fact, when an injury occurs, cardiac tissue organization is disrupted at the level of the cells, the tissue architecture, and the coordinated interaction among the cells. Thus, reconstitution of a functional tissue must reestablish electrical and mechanical communication between cardiomyocytes and restore their surrounding environment. It is also essential to restore distinctive myocardial features, such as vascular patency and pump function. In this article, we review the current status, challenges, and future priorities in cardiac regenerative or reparative medicine. In the first part, we provide an overview of our current understanding of heart repair and comment on the main contributors and mechanisms involved in innate regeneration. A brief section is dedicated to the novel concept of rejuvenation or regeneration, which we think may impact future development in the field. The last section describes regenerative therapies, where the most advanced and disruptive strategies used for myocardial repair are discussed. Our recommendations for priority areas in studies of cardiac regeneration or repair are summarized in Tables 1 and 2 .

Standards and Methodological Rigor in Pulmonary Arterial Hypertension Preclinical and Translational Research

Despite advances in our understanding of the pathophysiology and the management of pulmonary arterial hypertension (PAH), significant therapeutic gaps remain for this devastating disease. Yet, few innovative therapies beyond the traditional pathways of endothelial dysfunction have reached clinical trial phases in PAH. Although there are inherent limitations of the currently available models of PAH, the leaky pipeline of innovative therapies relates, in part, to flawed preclinical research methodology, including lack of rigour in trial design, incomplete invasive hemodynamic assessment, and lack of careful translational studies that replicate randomized controlled trials in humans with attention to adverse effects and benefits. Rigorous methodology should include the use of prespecified eligibility criteria, sample sizes that permit valid statistical analysis, randomization, blinded assessment of standardized outcomes, and transparent reporting of results. Better design and implementation of preclinical studies can minimize inherent flaws in the models of PAH, reduce the risk of bias, and enhance external validity and our ability to distinguish truly promising therapies form many false-positive or overstated leads. Ideally, preclinical studies should use advanced imaging, study several preclinical pulmonary hypertension models, or correlate rodent and human findings and consider the fate of the right ventricle, which is the major determinant of prognosis in human PAH. Although these principles are widely endorsed, empirical evidence suggests that such rigor is often lacking in pulmonary hypertension preclinical research. The present article discusses the pitfalls in the design of preclinical pulmonary hypertension trials and discusses opportunities to create preclinical trials with improved predictive value in guiding early-phase drug development in patients with PAH, which will need support not only from researchers, peer reviewers, and editors but also from academic institutions, funding agencies, and animal ethics authorities.

Rigor, reproducibility, and novel methodological approaches to eating disorders research

The explosion of new methods in science has put additional pressure on authors, readers, and peer reviewers to evaluate the rigor and reproducibility of these new approaches. This issue brings together a collection of articles aimed to stimulate our thinking about how to evaluate the quality of these new methodologies, in the form of several expert reviews on domains ranging from large scale genetics to clinical interventions. These expert reviews are followed by a collection of papers intended to illustrate new or underrepresented methods in our field and include novel approaches to clinical trials to the quantification of text data collected from public repositories online. The issue concludes with a number of original data papers applying network analysis to study comorbidity and symptom status in eating disorder and related populations and a couple of novel neuroimaging papers demonstrating the use of new and promising methods for studying heterogeneity and the use of psychophysiology to answer questions about emotion in eating disorder populations.

STAT3 as a common signal of ischemic conditioning: a lesson on "rigor and reproducibility" in preclinical studies on cardioprotection

Ischemic conditioning before (ischemic preconditioning, IPC) or after (ischemic postconditioning, POCO) sustained myocardial ischemia/reperfusion (I/R), induced locally or remotely from the heart (remote IPC, RIPC), reduces infarct size. However, none of the identified signaling steps of ischemic conditioning was robust across models and species to be successfully translated to humans. In prior separate studies in pigs, activation of signal transducer and activator of transcription 3 (STAT3) was causal for infarct size reduction by IPC, POCO, and RIPC but it remains unclear whether or not STAT3 is truly a common denominator of cardioprotective signaling. We therefore, now analyzed the phosphorylation of STAT3 and other signaling proteins in left ventricular biopsies from our prior studies on IPC, POCO and RIPC in one approach. We developed a strategy for the quantification of protein phosphorylation in multiple samples from many experiments on different gels/membranes by Western blot. Along with reduced infarct size, the ratio of STAT3_tyr705 phosphorylation/total STAT3 protein at early reperfusion was significantly increased by IPC (IPC 2.0 ± 0.3 vs. I/R 1.2 ± 0.2 arbitrary units), but only trendwise by POCO and RIPC (1.3 ± 0.2; 1.4 ± 0.2 arbitrary units); storage time for IPC samples was shorter than for POCO and RIPC samples. No other signaling protein phosphorylation was associated with reduced infarct size. We confirmed STAT3 phosphorylation with IPC. For POCO and RIPC we could not reproduce the findings from our earlier more focused studies. At this point, we can not distinguish between lack of robustness of the biological signal and methodological issues of our retrospective approach.