On the reproducibility of science: unique identification of research resources in the biomedical literature

Misspellings or "miscellings"-Non-verifiable and unknown cell lines in cancer research publications

Reproducible laboratory research relies on correctly identified reagents. We have previously described gene research papers with wrongly identified nucleotide sequence(s), including papers studying miR-145. Manually verifying reagent identities in 36 recent miR-145 papers found that 56% and 17% of papers described misidentified nucleotide sequences and cell lines, respectively. We also found 5 cell line identifiers in miR-145 papers with misidentified nucleotide sequences and cell lines, and 18 cell line identifiers published elsewhere, that did not represent indexed human cell lines. These 23 identifiers were described as non-verifiable (NV), as their identities were unclear. Studying 420 papers that mentioned 8 NV identifier(s) found 235 papers (56%) that referred to 7 identifiers (BGC-803, BSG-803, BSG-823, GSE-1, HGC-7901, HGC-803, and MGC-823) as independent cell lines. We could not find any publications describing how these cell lines were established. Six cell lines were sourced from cell line repositories with externally accessible online catalogs, but these cell lines were not indexed as claimed. Some papers also stated that short tandem repeat (STR) profiles had been generated for three cell lines, yet no STR profiles could be identified. In summary, as NV cell lines represent new challenges to research integrity and reproducibility, further investigations are required to clarify their status and identities.

Do organisms need an impact factor? Citations of key biological resources including model organisms reveal usage patterns and impact

Research resources like transgenic animals and antibodies are the workhorses of biomedicine, enabling investigators to relatively easily study specific disease conditions. As key biological resources, transgenic animals and antibodies are often validated, maintained, and distributed from university based stock centers. As these centers heavily rely largely on grant funding, it is critical that they are cited by investigators so that usage can be tracked. However, unlike systems for tracking the impact of papers, the conventions and systems for tracking key resource usage and impact lag behind. Previous studies have shown that about 50% of the resources are not findable, making the studies they are supporting irreproducible, but also makes tracking resources difficult. The RRID project is filling this gap by working with journals and resource providers to improve citation practices and to track the usage of these key resources. Here, we reviewed 10 years of citation practices for five university based stock centers, characterizing each reference into two broad categories: findable (authors could use the RRID, stock number, or full name) and not findable (authors could use a nickname or a common name that is not unique to the resource). The data revealed that when stock centers asked their communities to cite resources by RRID, in addition to helping stock centers more easily track resource usage by increasing the number of RRID papers, authors shifted from citing resources predominantly by nickname (~50% of the time) to citing them by one of the findable categories (~85%) in a matter of several years. In the case of one stock center, the MMRRC, the improvement in findability is also associated with improvements in the adherence to NIH rigor criteria, as determined by a significant increase in the Rigor and Transparency Index for studies using MMRRC mice. From this data, it was not possible to determine whether outreach to authors or changes to stock center websites drove better citation practices, but findability of research resources and rigor adherence was improved.

Improving the integrity and reproducibility of research that uses antibodies: a technical, data sharing, behavioral and policy challenge

Antibodies are one of the most important reagents used in biomedical and fundamental research, used to identify, and quantify proteins, contribute to knowledge of disease mechanisms, and validate drug targets. Yet many antibodies used in research do not recognize their intended target, or recognize additional molecules, compromising the integrity of research findings and leading to waste of resources, lack of reproducibility, failure of research projects, and delays in drug development. Researchers frequently use antibodies without confirming that they perform as intended in their application of interest. Here we argue that the determinants of end-user antibody choice and use are critical, and under-addressed, behavioral drivers of this problem. This interacts with the batch-to-batch variability of these biological reagents, and the paucity of available characterization data for most antibodies, making it more difficult for researchers to choose high quality reagents and perform necessary validation experiments. The open-science company YCharOS works with major antibody manufacturers and knockout cell line producers to characterize antibodies, identifying high-performing renewable antibodies for many targets in neuroscience. This shows the progress that can be made by stakeholders working together. However, their work so far applies to only a tiny fraction of available antibodies. Where characterization data exists, end-users need help to find and use it appropriately. While progress has been made in the context of technical solutions and antibody characterization, we argue that initiatives to make best practice behaviors by researchers more feasible, easy, and rewarding are needed. Global cooperation and coordination between multiple partners and stakeholders will be crucial to address the technical, policy, behavioral, and open data sharing challenges. We offer potential solutions by describing our Only Good Antibodies initiative, a community of researchers and partner organizations working toward the necessary change. We conclude with an open invitation for stakeholders, including researchers, to join our cause.

The Status of Data Management Practices Across German Medical Data Integration Centers: Mixed Methods Study

BACKGROUND

In the context of the Medical Informatics Initiative, medical data integration centers (DICs) have implemented complex data flows to transfer routine health care data into research data repositories for secondary use. Data management practices are of importance throughout these processes, and special attention should be given to provenance aspects. Insufficient knowledge can lead to validity risks and reduce the confidence and quality of the processed data. The need to implement maintainable data management practices is undisputed, but there is a great lack of clarity on the status.

OBJECTIVE

Our study examines the current data management practices throughout the data life cycle within the Medical Informatics in Research and Care in University Medicine (MIRACUM) consortium. We present a framework for the maturity status of data management practices and present recommendations to enable a trustful dissemination and reuse of routine health care data.

METHODS

In this mixed methods study, we conducted semistructured interviews with stakeholders from 10 DICs between July and September 2021. We used a self-designed questionnaire that we tailored to the MIRACUM DICs, to collect qualitative and quantitative data. Our study method is compliant with the Good Reporting of a Mixed Methods Study (GRAMMS) checklist.

RESULTS

Our study provides insights into the data management practices at the MIRACUM DICs. We identify several traceability issues that can be partially explained with a lack of contextual information within nonharmonized workflow steps, unclear responsibilities, missing or incomplete data elements, and incomplete information about the computational environment information. Based on the identified shortcomings, we suggest a data management maturity framework to reach more clarity and to help define enhanced data management strategies.

CONCLUSIONS

The data management maturity framework supports the production and dissemination of accurate and provenance-enriched data for secondary use. Our work serves as a catalyst for the derivation of an overarching data management strategy, abiding data integrity and provenance characteristics as key factors. We envision that this work will lead to the generation of fairer and maintained health research data of high quality.

Motivation for using data-driven algorithms in research: A review of machine learning solutions for image analysis of micrographs in neuroscience

Machine learning is a powerful tool that is increasingly being used in many research areas, including neuroscience. The recent development of new algorithms and network architectures, especially in the field of deep learning, has made machine learning models more reliable and accurate and useful for the biomedical research sector. By minimizing the effort necessary to extract valuable features from datasets, they can be used to find trends in data automatically and make predictions about future data, thereby improving the reproducibility and efficiency of research. One application is the automatic evaluation of micrograph images, which is of great value in neuroscience research. While the development of novel models has enabled numerous new research applications, the barrier to use these new algorithms has also decreased by the integration of deep learning models into known applications such as microscopy image viewers. For researchers unfamiliar with machine learning algorithms, the steep learning curve can hinder the successful implementation of these methods into their workflows. This review explores the use of machine learning in neuroscience, including its potential applications and limitations, and provides some guidance on how to select a fitting framework to use in real-life research projects.

The Antibody Registry: ten years of registering antibodies

Antibodies are ubiquitous key biological research resources yet are tricky to use as they are prone to performance issues and represent a major source of variability across studies. Understanding what antibody was used in a published study is therefore necessary to repeat and/or interpret a given study. However, antibody reagents are still frequently not cited with sufficient detail to determine which antibody was used in experiments. The Antibody Registry is a public, open database that enables citation of antibodies by providing a persistent record for any antibody-based reagent used in a publication. The registry is the authority for antibody Research Resource Identifiers, or RRIDs, which are requested or required by hundreds of journals seeking to improve the citation of these key resources. The registry is the most comprehensive listing of persistently identified antibody reagents used in the scientific literature. Data contributors span individual authors who use antibodies to antibody companies, which provide their entire catalogs including discontinued items. Unlike many commercial antibody listing sites which tend to remove reagents no longer sold, registry records persist, providing an interface between a fast-moving commercial marketplace and the static scientific literature. The Antibody Registry (RRID:SCR_006397) https://antibodyregistry.org.

Insufficient reporting of experimental variables as a cause for nonreproducibility in animal physiology? A case study

Nonreproducibility in scientific investigations has been explained by inadequately reporting methodology, honest error, and even misconduct. We hypothesized that, within the field of animal physiology, the most parsimonious explanation for nonreproducibility is inadequate reporting of key methodological details. We further hypothesized that implementation of relatively recently released reporting guidelines has positively impacted journal article quality, as measured by completeness of the methodology descriptions. We analyzed 84 research articles published in five primarily organismal animal physiology journals in 2008-2010 (generally before current guidelines) and 2018-2020. Compliance for reporting 34 variables referring to biology, experiments, and data collection was assessed. Reporting compliance was just ∼61% in 2008-2010, rising only slightly to 67.5% for 2018-2020. Only 21% of the reported variables showed significant differences across the period from 2008-2020. We conclude that, despite attempts by societies and journals to promote greater reporting compliance, such efforts have so far been relatively unsuccessful in the field of animal physiology.

Blind spots on western blots: Assessment of common problems in western blot figures and methods reporting with recommendations to improve them

Western blotting is a standard laboratory method used to detect proteins and assess their expression levels. Unfortunately, poor western blot image display practices and a lack of detailed methods reporting can limit a reader's ability to evaluate or reproduce western blot results. While several groups have studied the prevalence of image manipulation or provided recommendations for improving western blotting, data on the prevalence of common publication practices are scarce. We systematically examined 551 articles published in the top 25% of journals in neurosciences (n = 151) and cell biology (n = 400) that contained western blot images, focusing on practices that may omit important information. Our data show that most published western blots are cropped and blot source data are not made available to readers in the supplement. Publishing blots with visible molecular weight markers is rare, and many blots additionally lack molecular weight labels. Western blot methods sections often lack information on the amount of protein loaded on the gel, blocking steps, and antibody labeling protocol. Important antibody identifiers like company or supplier, catalog number, or RRID were omitted frequently for primary antibodies and regularly for secondary antibodies. We present detailed descriptions and visual examples to help scientists, peer reviewers, and editors to publish more informative western blot figures and methods. Additional resources include a toolbox to help scientists produce more reproducible western blot data, teaching slides in English and Spanish, and an antibody reporting template.

Combining hypothesis- and data-driven neuroscience modeling in FAIR workflows

Modeling in neuroscience occurs at the intersection of different points of view and approaches. Typically, hypothesis-driven modeling brings a question into focus so that a model is constructed to investigate a specific hypothesis about how the system works or why certain phenomena are observed. Data-driven modeling, on the other hand, follows a more unbiased approach, with model construction informed by the computationally intensive use of data. At the same time, researchers employ models at different biological scales and at different levels of abstraction. Combining these models while validating them against experimental data increases understanding of the multiscale brain. However, a lack of interoperability, transparency, and reusability of both models and the workflows used to construct them creates barriers for the integration of models representing different biological scales and built using different modeling philosophies. We argue that the same imperatives that drive resources and policy for data - such as the FAIR (Findable, Accessible, Interoperable, Reusable) principles - also support the integration of different modeling approaches. The FAIR principles require that data be shared in formats that are Findable, Accessible, Interoperable, and Reusable. Applying these principles to models and modeling workflows, as well as the data used to constrain and validate them, would allow researchers to find, reuse, question, validate, and extend published models, regardless of whether they are implemented phenomenologically or mechanistically, as a few equations or as a multiscale, hierarchical system. To illustrate these ideas, we use a classical synaptic plasticity model, the Bienenstock-Cooper-Munro rule, as an example due to its long history, different levels of abstraction, and implementation at many scales.

Establishing Institutional Scores With the Rigor and Transparency Index: Large-scale Analysis of Scientific Reporting Quality

BACKGROUND

Improving rigor and transparency measures should lead to improvements in reproducibility across the scientific literature; however, the assessment of measures of transparency tends to be very difficult if performed manually.

OBJECTIVE

This study addresses the enhancement of the Rigor and Transparency Index (RTI, version 2.0), which attempts to automatically assess the rigor and transparency of journals, institutions, and countries using manuscripts scored on criteria found in reproducibility guidelines (eg, Materials Design, Analysis, and Reporting checklist criteria).

METHODS

The RTI tracks 27 entity types using natural language processing techniques such as Bidirectional Long Short-term Memory Conditional Random Field-based models and regular expressions; this allowed us to assess over 2 million papers accessed through PubMed Central.

RESULTS

Between 1997 and 2020 (where data were readily available in our data set), rigor and transparency measures showed general improvement (RTI 2.29 to 4.13), suggesting that authors are taking the need for improved reporting seriously. The top-scoring journals in 2020 were the Journal of Neurochemistry (6.23), British Journal of Pharmacology (6.07), and Nature Neuroscience (5.93). We extracted the institution and country of origin from the author affiliations to expand our analysis beyond journals. Among institutions publishing >1000 papers in 2020 (in the PubMed Central open access set), Capital Medical University (4.75), Yonsei University (4.58), and University of Copenhagen (4.53) were the top performers in terms of RTI. In country-level performance, we found that Ethiopia and Norway consistently topped the RTI charts of countries with 100 or more papers per year. In addition, we tested our assumption that the RTI may serve as a reliable proxy for scientific replicability (ie, a high RTI represents papers containing sufficient information for replication efforts). Using work by the Reproducibility Project: Cancer Biology, we determined that replication papers (RTI 7.61, SD 0.78) scored significantly higher (P<.001) than the original papers (RTI 3.39, SD 1.12), which according to the project required additional information from authors to begin replication efforts.

CONCLUSIONS

These results align with our view that RTI may serve as a reliable proxy for scientific replicability. Unfortunately, RTI measures for journals, institutions, and countries fall short of the replicated paper average. If we consider the RTI of these replication studies as a target for future manuscripts, more work will be needed to ensure that the average manuscript contains sufficient information for replication attempts.

Genetic quality: a complex issue for experimental study reproducibility

Laboratory animal research involving mice, requires consideration of many factors to be controlled. Genetic quality is one factor that is often overlooked but is essential for the generation of reproducible experimental results. Whether experimental research involves inbred mice, spontaneous mutant, or genetically modified strains, exercising genetic quality through careful breeding, good recordkeeping, and prudent quality control steps such as validation of the presence of mutations and verification of the genetic background, will help ensure that experimental results are accurate and that reference controls are representative for the particular experiment. In this review paper, we will discuss various techniques used for the generation of genetically altered mice, and the different aspects to be considered regarding genetic quality, including inbred strains and substrains used, quality check controls during and after genetic manipulation and breeding. We also provide examples for when to use the different techniques and considerations on genetic quality checks. Further, we emphasize on the importance of establishing an in-house genetic quality program.

A decade of GigaScience: What can be learned from half a million RRIDs in the scientific literature?

Research resource identifiers (RRIDs) are persistent unique identifiers for scientific resources used to conduct studies such as reagents and tools. Inclusion of these identifiers into the scientific literature has been demonstrated to improve the reproducibility of papers because resources, like antibodies, are easier to find, making methods easier to reproduce. RRIDs also dramatically reduce the use of problematic resources, such as contaminated cell lines. The addition of RRIDs into a manuscript means that authors have to look up information that they may have previously omitted or confront information about problems that may have been reported about their resources. The use of RRIDs is primarily driven by champion journals, such as GigaScience and others. Although still nascent, this practice lays important groundwork for citation types that can cover non-traditional scholarly output, such as software tools and key reagents; giving authors of various types of tools scholarly credit for their contributions.

Establishment, authenticity, and characterization of cervical cancer cell lines

Cell lines have been considered excellent research models in many areas of biomedicine and, specifically, in the study of carcinogenesis. However, they cease to be effective models if their behavior changes. Although studies on the cross-contamination of cell lines originating from different tissues have been performed, little is known about cell lines derived from cervical neoplasia. We know that high-risk HPV (HR-HPV) is associated with the development of this type of cancer. This link between HPV infection and cancer was first established over 35 years ago when HPV16 DNA was found to be present in a large proportion of cervical cancer biopsies. The present review paper aims to report the status of the establishment, authenticity, and characterization of cervical cancer (CC) cell lines. This is a systematic review of articles on the establishment, authenticity, and characterization of CC cell lines, published from 1960 to date in the databases and in cell repository databases. 52 cell lines were identified in the literature. Only 25 cell lines were derived from cervical neoplasia, of which only 45.8% have a reported identity test (genomic fingerprint). Despite the increase in the establishment of cell lines of cervical neoplasia and the standards for the regulation of these study models, the criteria for their characterization continue to be diverse.

MALDI-TOF-MS-Based Identification of Monoclonal Murine Anti-SARS-CoV-2 Antibodies within One Hour

During the SARS-CoV-2 pandemic, many virus-binding monoclonal antibodies have been developed for clinical and diagnostic purposes. This underlines the importance of antibodies as universal bioanalytical reagents. However, little attention is given to the reproducibility crisis that scientific studies are still facing to date. In a recent study, not even half of all research antibodies mentioned in publications could be identified at all. This should spark more efforts in the search for practical solutions for the traceability of antibodies. For this purpose, we used 35 monoclonal antibodies against SARS-CoV-2 to demonstrate how sequence-independent antibody identification can be achieved by simple means applied to the protein. First, we examined the intact and light chain masses of the antibodies relative to the reference material NIST-mAb 8671. Already half of the antibodies could be identified based solely on these two parameters. In addition, we developed two complementary peptide mass fingerprinting methods with MALDI-TOF-MS that can be performed in 60 min and had a combined sequence coverage of over 80%. One method is based on the partial acidic hydrolysis of the protein by 5 mM of sulfuric acid at 99 °C. Furthermore, we established a fast way for a tryptic digest without an alkylation step. We were able to show that the distinction of clones is possible simply by a brief visual comparison of the mass spectra. In this work, two clones originating from the same immunization gave the same fingerprints. Later, a hybridoma sequencing confirmed the sequence identity of these sister clones. In order to automate the spectral comparison for larger libraries of antibodies, we developed the online software ABID 2.0. This open-source software determines the number of matching peptides in the fingerprint spectra. We propose that publications and other documents critically relying on monoclonal antibodies with unknown amino acid sequences should include at least one antibody fingerprint. By fingerprinting an antibody in question, its identity can be confirmed by comparison with a library spectrum at any time and context.

The Learning Curve of Murine Subretinal Injection Among Clinically Trained Ophthalmic Surgeons

Purpose

Subretinal injection (SRI) in mice is widely used in retinal research, yet the learning curve (LC) of this surgically challenging technique is unknown.

Methods

To evaluate the LC for SRI in a murine model, we analyzed training data from three clinically trained ophthalmic surgeons from 2018 to 2020. Successful SRI was defined as either the absence of retinal pigment epithelium (RPE) degeneration after phosphate buffered saline injection or the presence of RPE degeneration after Alu RNA injection. Multivariable survival-time regression models were used to evaluate the association between surgeon experience and success rate, with adjustment for injection agents, and to calculate an approximate case number to achieve a 95% success rate. Cumulative sum (CUSUM) analyses were performed and plotted individually to monitor each surgeon's simultaneous performance.

Results

Despite prior microsurgery experience, the combined average success rate of the first 50 cases in mice was only 27%. The predicted SRI success rate did not reach a plateau above 95% until approximately 364 prior cases. Using the 364 training cases as a cutoff point, the predicted probability of success for cases 1 to 364 was 65.38%, and for cases 365 to 455 it was 99.32% (P < 0.0001). CUSUM analysis showed an initial upward slope and then remained within the decision intervals with an acceptable success rate set at 95% in the late stage.

Conclusions

This study demonstrates the complexity and substantial LC for successful SRI in mice with high confidence. A systematic training system could improve the reliability and reproducibility of SRI-related experiments and improve the interpretation of experimental results using this technique.

Translational Relevance

Our prediction model and monitor system allow objective quantification of technical proficiency in the field of subretinal drug delivery and gene therapy for the first time, to the best of our knowledge.

Zebrafish Information Network, the knowledgebase for Danio rerio research

The Zebrafish Information Network (zfin.org) is the central repository for Danio rerio genetic and genomic data. The Zebrafish Information Network has served the zebrafish research community since 1994, expertly curating, integrating, and displaying zebrafish data. Key data types available at the Zebrafish Information Network include, but are not limited to, genes, alleles, human disease models, gene expression, phenotype, and gene function. The Zebrafish Information Network makes zebrafish research data Findable, Accessible, Interoperable, and Reusable through nomenclature, curatorial and annotation activities, web interfaces, and data downloads. Recently, the Zebrafish Information Network and 6 other model organism knowledgebases have collaborated to form the Alliance of Genome Resources, aiming to develop sustainable genome information resources that enable the use of model organisms to understand the genetic and genomic basis of human biology and disease. Here, we provide an overview of the data available at the Zebrafish Information Network including recent updates to the gene page to provide access to single-cell RNA sequencing data, links to Alliance web pages, ribbon diagrams to summarize the biological systems and Gene Ontology terms that have annotations, and data integration with the Alliance of Genome Resources.

Minimal Information about MHC Multimers (MIAMM)

With the goal of improving the reproducibility and annotatability of MHC multimer reagent data, we present the establishment of a new data standard: Minimal Information about MHC Multimers (https://miamm.lji.org/). Multimers are engineered reagents composed of a ligand and a MHC, which can be represented in a standardized format using ontology terminology. We provide an online Web site to host the details of the standard, as well as a validation tool to assist with the adoption of the standard. We hope that this publication will bring increased awareness of Minimal Information about MHC Multimers and drive acceptance, ultimately improving the quality and documentation of multimer data in the scientific literature.

Toward best practice in cancer mutation detection with whole-genome and whole-exome sequencing

Clinical applications of precision oncology require accurate tests that can distinguish true cancer-specific mutations from errors introduced at each step of next-generation sequencing (NGS). To date, no bulk sequencing study has addressed the effects of cross-site reproducibility, nor the biological, technical and computational factors that influence variant identification. Here we report a systematic interrogation of somatic mutations in paired tumor-normal cell lines to identify factors affecting detection reproducibility and accuracy at six different centers. Using whole-genome sequencing (WGS) and whole-exome sequencing (WES), we evaluated the reproducibility of different sample types with varying input amount and tumor purity, and multiple library construction protocols, followed by processing with nine bioinformatics pipelines. We found that read coverage and callers affected both WGS and WES reproducibility, but WES performance was influenced by insert fragment size, genomic copy content and the global imbalance score (GIV; G > T/C > A). Finally, taking into account library preparation protocol, tumor content, read coverage and bioinformatics processes concomitantly, we recommend actionable practices to improve the reproducibility and accuracy of NGS experiments for cancer mutation detection.

Biomonitoring and Digital Data Technology as an Opportunity for Enhancing Animal Study Translation

The failure of animal studies to translate to effective clinical therapeutics has driven efforts to identify underlying cause and develop solutions that improve the reproducibility and translatability of preclinical research. Common issues revolve around study design, analysis, and reporting as well as standardization between preclinical and clinical endpoints. To address these needs, recent advancements in digital technology, including biomonitoring of digital biomarkers, development of software systems and database technologies, as well as application of artificial intelligence to preclinical datasets can be used to increase the translational relevance of preclinical animal research. In this review, we will describe how a number of innovative digital technologies are being applied to overcome recurring challenges in study design, execution, and data sharing as well as improving scientific outcome measures. Examples of how these technologies are applied to specific therapeutic areas are provided. Digital technologies can enhance the quality of preclinical research and encourage scientific collaboration, thus accelerating the development of novel therapeutics.

Antibody Watch: Text mining antibody specificity from the literature

Antibodies are widely used reagents to test for expression of proteins and other antigens. However, they might not always reliably produce results when they do not specifically bind to the target proteins that their providers designed them for, leading to unreliable research results. While many proposals have been developed to deal with the problem of antibody specificity, it is still challenging to cover the millions of antibodies that are available to researchers. In this study, we investigate the feasibility of automatically generating alerts to users of problematic antibodies by extracting statements about antibody specificity reported in the literature. The extracted alerts can be used to construct an “Antibody Watch” knowledge base containing supporting statements of problematic antibodies. We developed a deep neural network system and tested its performance with a corpus of more than two thousand articles that reported uses of antibodies. We divided the problem into two tasks. Given an input article, the first task is to identify snippets about antibody specificity and classify if the snippets report that any antibody exhibits non-specificity, and thus is problematic. The second task is to link each of these snippets to one or more antibodies mentioned in the snippet. The experimental evaluation shows that our system can accurately perform the classification task with 0.925 weighted F1-score, linking with 0.962 accuracy, and 0.914 weighted F1 when combined to complete the joint task. We leveraged Research Resource Identifiers (RRID) to precisely identify antibodies linked to the extracted specificity snippets. The result shows that it is feasible to construct a reliable knowledge base about problematic antibodies by text mining.

Antibodies are widely used reagents to test for the expression of proteins. However, antibodies are also a known source of reproducibility problems in biomedicine, as specificity and other issues can complicate their use. Information about how antibodies perform for specific applications are scattered across the biomedical literature and multiple websites. To alert scientists with reported antibody issues, we develop text mining algorithms that can identify specificity issues reported in the literature. We developed a deep neural network algorithm and performed a feasibility study on 2,223 papers. We leveraged Research Resource Identifiers (RRIDs), unique identifiers for antibodies and other biomedical resources, to match extracted specificity issues with particular antibodies. The results show that our system, called “Antibody Watch,” can accurately perform specificity issue identification and RRID association with a weighted F-score over 0.914. From our test corpus, we identified 37 antibodies with 68 nonspecific issue statements. With Antibody Watch, for example, if one were looking for an antibody targeting beta-Amyloid 1–16, from 74 antibodies at dkNET Resource Reports (on 10/2/20), one would be alerted that “some non-specific bands were detected at 55 kDa in both WT and APP/PS1 mice with the 6E10 antibody…”

Temporal Stability and Genetic Diversity of 48-Year-Old T-Series Phages

T-series phages have been model organisms for molecular biology since the 1940s. Given that these phages have been stocked, distributed, and propagated for decades across the globe, there exists the potential for genetic drift to accumulate between stocks over time. Here, we compared the temporal stability and genetic relatedness of laboratory-maintained phage stocks with a T-series collection from 1972. Only the T-even phages produced viable virions. We obtained complete genomes of these T-even phages, along with two contemporary T4 stocks. Performing comparative genomics, we found 12 and 16 nucleotide variations, respectively, in the genomes of T2 and T6, whereas there were ∼172 nucleotide variations between T4 sublines compared with the NCBI RefSeq genome. To account for the possibility of artifacts in NCBI RefSeq, we used the 1972 T4 stock as a reference and compared genetic and phenotypic variations between T4 sublines. Genomic analysis predicted nucleotide variations in genes associated with DNA metabolism and structural proteins. We did not, however, observe any differences in growth characteristics or host range between the T4 sublines. Our study highlights the potential for genetic drift between individually maintained T-series phage stocks, yet after 48 years, this has not resulted in phenotypic alterations in these important model organisms.

IMPORTANCE T-series bacteriophages have been used throughout the world for various molecular biology researches, which were critical for establishing the fundamentals of molecular biology, from the structure of DNA to advanced gene-editing tools. These model bacteriophages help keep research data consistent and comparable between laboratories. However, we observed genetic variability when we compared contemporary sublines of T4 phages to a 48-year-old stock of T4. This may have effects on the comparability of results obtained using T4 phage. Here, we highlight the genomic differences between T4 sublines and examined phenotypic differences in phage replication parameters. We observed limited genomic changes but no phenotypic variations between T4 sublines. Our research highlights the possibility of genetic drift in model bacteriophages.

The Rigor and Transparency Index Quality Metric for Assessing Biological and Medical Science Methods

The reproducibility crisis is a multifaceted problem involving ingrained practices within the scientific community. Fortunately, some causes are addressed by the author's adherence to rigor and reproducibility criteria, implemented via checklists at various journals. We developed an automated tool (SciScore) that evaluates research articles based on their adherence to key rigor criteria, including NIH criteria and RRIDs, at an unprecedented scale. We show that despite steady improvements, less than half of the scoring criteria, such as blinding or power analysis, are routinely addressed by authors; digging deeper, we examined the influence of specific checklists on average scores. The average score for a journal in a given year was named the Rigor and Transparency Index (RTI), a new journal quality metric. We compared the RTI with the Journal Impact Factor and found there was no correlation. The RTI can potentially serve as a proxy for methodological quality.

Highly Versatile Cloud-Based Automation Solution for the Remote Design and Execution of Experiment Protocols during the COVID-19 Pandemic

There is an urgent need to accelerate the development and validation of both diagnostics and vaccines for COVID-19. These priorities are challenging both public and private sector research groups around the world and have shone a spotlight on both existing bottlenecks in the research workflows involved as well as on the implications of having to do much of this work remotely because of enforced social distancing and lockdown measures. The ability to respond quickly to rapidly evolving events, coupled with an emerging understanding of the disease and its pathology, as well as different mutations of the virus, necessitates a highly flexible liquid-handling automation solution that is amenable to rapid switching between different assay workflows and processes to be exploited tactically as needed. In addition, the use of cloud-based software imparts a unique benefit in enabling multiple research groups and remote technical staff around the world to have ready access to the same protocols in real-time without delays, down to the required level of detail, sharing methods and data (for example, in faster clinical trials). Informed by a recent use case, this article explores these issues alongside the recent development and deployment of an automation solution, whose unique approach in terms of both its cloud-native software and its highly modular hardware aligns especially well with achieving the challenge set by this new frontier in the bioanalytical laboratory.

OpenWorkstation: A modular open-source technology for automated in vitro workflows

Automation liberates scientific staff from repetitive tasks, decreases the probability of human error and consequently enhances the reproducibility of lab experiments. However, the use of laboratory automation in academic laboratories is limited due to high acquisition costs and the inability to customize off-the-shelf hardware. To address these challenges, we present an Open Source Hardware concept, referred to as OpenWorkstation, to build an assembly line-inspired platform consisting of ready-to-use and customizable modules. In contrast to current standalone solutions, the OpenWorkstation concept enables the combination of single hardware modules - each with a specific set of functionalities - to a modular workstation to provide a fully automated setup. The base setup consists of a pipetting and transport module and is designed to execute basic protocol steps for in vitro research applications, including pipetting operations for liquids and viscous substances and transportation of cell culture vessels between the modules. We demonstrate the successful application of this concept within a case study by the development of a storage module to facilitate high-throughput studies and a photo-crosslinker module to initiate photo-induced polymerization of hydrogel solutions. We present a Systems Engineering framework for customized module development, guidance for the design and assembly of the presented modules, and operational instructions on the usage of the workstation. By combining capabilities from various open source instrumentations into a modular technology platform, the OpenWorkstation concept will facilitate efficient and reliable experimentation for in vitro research. Ultimately, this concept will allow academic groups to improve replicability and reproducibility in cell culture process operations towards more economical and innovative research in the future.

Towards FAIR protocols and workflows: the OpenPREDICT use case

It is essential for the advancement of science that researchers share, reuse and reproduce each other's workflows and protocols. The FAIR principles are a set of guidelines that aim to maximize the value and usefulness of research data, and emphasize the importance of making digital objects findable and reusable by others. The question of how to apply these principles not just to data but also to the workflows and protocols that consume and produce them is still under debate and poses a number of challenges. In this paper we describe a two-fold approach of simultaneously applying the FAIR principles to scientific workflows as well as the involved data. We apply and evaluate our approach on the case of the PREDICT workflow, a highly cited drug repurposing workflow. This includes FAIRification of the involved datasets, as well as applying semantic technologies to represent and store data about the detailed versions of the general protocol, of the concrete workflow instructions, and of their execution traces. We propose a semantic model to address these specific requirements and was evaluated by answering competency questions. This semantic model consists of classes and relations from a number of existing ontologies, including Workflow4ever, PROV, EDAM, and BPMN. This allowed us then to formulate and answer new kinds of competency questions. Our evaluation shows the high degree to which our FAIRified OpenPREDICT workflow now adheres to the FAIR principles and the practicality and usefulness of being able to answer our new competency questions.

The effect of repeated passaging on the susceptibility of human proximal tubular HK-2 cells to toxic compounds

The human proximal tubular HK-2 cell line is an immortalized cell line commonly used for studying proximal tubular toxicity. Even as their use is presently increasing, there unfortunately are no studies focused on functional changes in HK-2 cells associated with passaging. The aim of the present study, therefore, was to evaluate the functional stability of HK-2 cells during 13 weeks of continuous passaging after 6 and 24 h of treatment with model nephrotoxic compounds (i.e., acetaminophen, cisplatin, CdCl(2)). Short tandem repeat profile, the doubling time, cell diameter, glutathione concentration, and intracellular dehydrogenase activity were measured in HK-2 cells at each tested passage. The results showed that HK-2 cells exhibit stable morphology, cell size, and cell renewal during passaging. Mean doubling time was determined to be 54 h. On the other hand, we observed a significant effect of passaging on the susceptibility of HK-2 cells to toxic compounds. The largest difference in results was found in both cadmium and cisplatin treated cells across passages. We conclude that the outcomes of scientific studies on HK-2 cells can be affected by the number of passages even after medium-term cultivation and passaging for 13 weeks.

The effect of journal guidelines on the reporting of antibody validation

BACKGROUND

Despite the widespread use of antibodies as a research tool, problems with specificity, lot-to-lot consistency and sensitivity commonly occur and may be important contributing factors to the 'replication crisis' in biomedical research. This makes the validation of antibodies and accurate reporting of this validation in the scientific literature extremely important. Therefore, some journals now require authors to comply with antibody reporting guidelines.

METHODS

We used a quasi-experimental approach to assess the effectiveness of such journal guidelines in improving antibody reporting in the scientific literature. In a sample of 120 publications, we compared the reporting of antibody validation and identification information in two journals with guidelines (Nature and the Journal of Comparative Neurology) with two journals without guidelines (Science and Neuroscience), before and after the introduction of these guidelines.

RESULTS

Our results suggest that the implementation of antibody reporting guidelines might have some influence on the reporting of antibody validation information. The percentage of validated antibodies per article slightly increased from 39% to 57% in journals with guidelines, whereas this percentage decreased from 23% to 14% in journals without guidelines. Furthermore, the reporting of validation information of all primary antibodies increased by 23 percentage points in the journals with guidelines (OR = 2.80, 95% CI = 0.96-INF; adjusted p = 1, one-tailed), compared to a decrease of 13 percentage points in journals without guidelines. Fortunately, the guidelines seem to be more effective in improving the reporting of antibody identification information. The reporting of identification information of all primary antibodies used in a study increased by 58 percentage points (OR = 17.8, 95% CI = 4.8-INF; adjusted p = 0.0003, one-tailed) in journals with guidelines. This percentage also slightly increased in journals without guidelines (by 18 percentage points), suggesting an overall increased awareness of the importance of antibody identifiability. Moreover, this suggests that reporting guidelines mostly have an influence on the reporting of information that is relatively easy to provide. A small increase in the reporting of validation by referencing the scientific literature or the manufacturer's data also indicates this.

CONCLUSION

Combined with the results of previous studies on journal guidelines, our study suggests that the effect of journal antibody guidelines on validation practices by themselves may be limited, since they mostly seem to improve antibody identification instead of actual experimental validation. These guidelines, therefore, may require additional measures to ensure effective implementation. However, due to the explorative nature of our study and our small sample size, we must remain cautious towards other factors that might have played a role in the observed change in antibody reporting behaviour.

Mouse Phenome Database: a data repository and analysis suite for curated primary mouse phenotype data

The Mouse Phenome Database (MPD; https://phenome.jax.org) is a widely accessed and highly functional data repository housing primary phenotype data for the laboratory mouse accessible via APIs and providing tools to analyze and visualize those data. Data come from investigators around the world and represent a broad scope of phenotyping endpoints and disease-related traits in naïve mice and those exposed to drugs, environmental agents or other treatments. MPD houses rigorously curated per-animal data with detailed protocols. Public ontologies and controlled vocabularies are used for annotation. In addition to phenotype tools, genetic analysis tools enable users to integrate and interpret genome–phenome relations across the database. Strain types and populations include inbred, recombinant inbred, F1 hybrid, transgenic, targeted mutants, chromosome substitution, Collaborative Cross, Diversity Outbred and other mapping populations. Our new analysis tools allow users to apply selected data in an integrated fashion to address problems in trait associations, reproducibility, polygenic syndrome model selection and multi-trait modeling. As we refine these tools and approaches, we will continue to provide users a means to identify consistent, quality studies that have high translational relevance.

A snapshot of 3649 Web-based services published between 1994 and 2017 shows a decrease in availability after 2 years

Background

The long-term availability of online Web services is of utmost importance to ensure reproducibility of analytical results. However, because of lack of maintenance following acceptance, many servers become unavailable after a short period of time. Our aim was to monitor the accessibility and the decay rate of published Web services as well as to determine the factors underlying trends changes.

Methods

We searched PubMed to identify publications containing Web server-related terms published between 1994 and 2017. Automatic and manual screening was used to check the status of each Web service. Kruskall–Wallis, Mann–Whitney and Chi-square tests were used to evaluate various parameters, including availability, accessibility, platform, origin of authors, citation, journal impact factor and publication year.

Results

We identified 3649 publications in 375 journals of which 2522 (69%) were currently active. Over 95% of sites were running in the first 2 years, but this rate dropped to 84% in the third year and gradually sank afterwards (P < 1e-16). The mean half-life of Web services is 10.39 years. Working Web services were published in journals with higher impact factors (P = 4.8e-04). Services published before the year 2000 received minimal attention. The citation of offline services was less than for those online (P = 0.022). The majority of Web services provide analytical tools, and the proportion of databases is slowly decreasing. Conclusions. Almost one-third of Web services published to date went out of service. We recommend continued support of Web-based services to increase the reproducibility of published results.

Fast Confirmation of Antibody Identity by MALDI-TOF MS Fingerprints

Thousands of antibodies for diagnostic and other analytical purposes are on the market. However, it is often difficult to identify duplicates, reagent changes, and to assign the correct original publications to an antibody. This slows down scientific progress and might even be a cause of irreproducible research and a waste of resources. Recently, activities were started to suggest the sole use of recombinant antibodies in combination with the open communication of their sequence. In this case, such uncertainties should be eliminated. Unfortunately, this approach seems to be rather a long-term vision since the development and manufacturing of recombinant antibodies remain quite expensive in the foreseeable future. Nearly all commercial antibody suppliers also may be reluctant to publish the sequence of their antibodies, since they fear counterfeiting. De novo sequencing of antibodies is also not feasible today for a reagent user without access to the hybridoma clone. Nevertheless, it seems to be crucial for any scientist to have the opportunity to identify an antibody undoubtedly to guarantee the traceability of any research activity using antibodies from a third party as a tool. For this purpose, we developed a method for the identification of antibodies based on a MALDI-TOF MS fingerprint. To circumvent lengthy denaturation, reduction, alkylation, and enzymatic digestion steps, the fragmentation was performed with a simple formic acid hydrolysis step. Eighty-nine unknown monoclonal antibodies were used for this study to examine the feasibility of this approach. Although the molecular assignment of peaks was rarely possible, antibodies could be easily recognized in a blinded test, simply from their mass-spectral fingerprint. A general protocol is given, which could be used without any optimization to generate fingerprints for a database. We want to propose that, in most scientific projects relying critically on antibody reagents, such a fingerprint should be established to prove and document the identity of the used antibodies, as well as to assign a specific reagent to a datasheet of a commercial supplier, public database record, or antibody ID.

Use of the Template for Intervention Description and Replication (Tidier) Checklist in Social Work Research

Social work has a longstanding commitment to sound research and the development and dissemination of evidence-based practice. To that end, multiple professional groups have developed or refined guidelines for reporting research procedures and findings, with the objectives of enhancing transparency, integrity, and rigor in science. Such guidelines can also facilitate replication and systematic review. The Template for Intervention Description and Replication (TIDieR) checklist represents the culmination of a multi-stage process to expand upon existing reporting guidelines. As such, the checklist provides a framework for more transparent communication about empirically-grounded interventions addressing a broad range of social and behavioral health issues. Use of this checklist can be beneficial for researchers, practitioners, and recipients of social work interventions. After discussing selected background regarding the need for and benefit of reporting standards and describing the TIDieR measure, we outline practical considerations in the checklist's use by those engaged in social work research.

The Antibody Society's antibody validation webinar series

In the wake of the reproducibility crisis and numerous discussions on how commercially available antibodies as research tool contribute to it, The Antibody Society developed a series of 10 webinars to address the issues involved. The webinars were delivered by speakers with both academic and commercial backgrounds. This report highlights the problems, and offers solutions to help the scientific community appropriately identify the right antibodies and to validate them for their research and development projects. Despite the various solutions proposed here, they must be applied on a case-by-case basis. Each antibody must be verified based on the content of the product sheet, and subsequently through experimentation to confirm integrity, specificity and selectivity. Verification needs to focus on the precise application and tissue/cell type for which the antibody will be used, and all verification data must be reported openly. The various approaches discussed here all have caveats, so a combination of solutions must be considered.

snapMRF: GPU-accelerated magnetic resonance fingerprinting dictionary generation and matching using extended phase graphs

PURPOSE

Magnetic resonance fingerprinting (MRF) is a state-of-the-art quantitative MRI technique with a computationally demanding reconstruction process, the accuracy of which depends on the accuracy of the signal model employed. Having a fast, validated, open-source MRF reconstruction would improve the dependability and accuracy of clinical applications of MRF.

METHODS

We parallelized both dictionary generation and signal matching on the GPU by splitting the simulation and matching of dictionary atoms across threads. Signal generation was modeled using both Bloch equation simulation and the extended phase graph (EPG) formalism. Unit tests were implemented to ensure correctness. The new package, snapMRF, was tested with a calibration phantom and an in vivo brain.

RESULTS

Compared with other online open-source packages, dictionary generation was accelerated by 10-1000× and signal matching by 10-100×. On a calibration phantom, T₁ and T₂ values were measured with relative errors that were nearly identical to those from existing packages when using the same sequence and dictionary configuration, but errors were much lower when using variable sequences that snapMRF supports but that competitors do not.

CONCLUSION

Our open-source package snapMRF was significantly faster and retrieved accurate parameters, possibly enabling real-time parameter map generation for small dictionaries. Further refinements to the acquisition scheme and dictionary setup could improve quantitative accuracy.

Semi-automated generation of pictures for the Mouse Grimace Scale: A multi-laboratory analysis (Part 2)

The Mouse Grimace Scale (MGS) is an established method for estimating pain in mice during animal studies. Recently, an improved and standardized MGS set-up and an algorithm for automated and blinded output of images for MGS evaluation were introduced. The present study evaluated the application of this standardized set-up and the robustness of the associated algorithm at four facilities in different locations and as part of varied experimental projects. Experiments using the MGS performed at four facilities (F1-F4) were included in the study; 200 pictures per facility (100 pictures each rated as positive and negative by the algorithm) were evaluated by three raters for image quality and reliability of the algorithm. In three of the four facilities, sufficient image quality and consistency were demonstrated. Intraclass correlation coefficient, calculated to demonstrate the correlation among raters at the three facilities (F1-F3), showed excellent correlation. The specificity and sensitivity of the results obtained by different raters and the algorithm were analysed using Fisher's exact test (p < 0.05). The analysis indicated a sensitivity of 77% and a specificity of 64%. The results of our study showed that the algorithm demonstrated robust performance at facilities in different locations in accordance with the strict application of our MGS setup.

Mouse Phenome Database: an integrative database and analysis suite for curated empirical phenotype data from laboratory mice

The Mouse Phenome Database (MPD; https://phenome.jax.org) is a widely used resource that provides access to primary experimental trait data, genotypic variation, protocols and analysis tools for mouse genetic studies. Data are contributed by investigators worldwide and represent a broad scope of phenotyping endpoints and disease-related traits in naïve mice and those exposed to drugs, environmental agents or other treatments. MPD houses individual animal data with detailed, searchable protocols, and makes these data available to other resources via API. MPD provides rigorous curation of experimental data and supporting documentation using relevant ontologies and controlled vocabularies. Most data in MPD are from inbreds and other reproducible strains such that the data are cumulative over time and across laboratories. The resource has been expanded to include the QTL Archive and other primary phenotype data from mapping crosses as well as advanced high-diversity mouse populations including the Collaborative Cross and Diversity Outbred mice. Furthermore, MPD provides a means of assessing replicability and reproducibility across experimental conditions and protocols, benchmarking assays in users’ own laboratories, identifying sensitized backgrounds for making new mouse models with genome editing technologies, analyzing trait co-inheritance, finding the common genetic basis for multiple traits and assessing sex differences and sex-by-genotype interactions.

An evaluation of different Cripto-1 antibodies and their variable results

Cripto-1 is a protein expressed during embryonal development and has been linked to several malignant processes in cancer. Since the discovery of cripto-1 in the late 1980s, it has become a subject of biomarker investigation in several types of cancer which in many cases relies on immunolocalization of cripto-1 using antibodies. Investigating cripto-1 expression and localization in primary glioblastoma cells, we discovered nonspecific binding of cripto-1 antibody to the extracellular matrix Geltrex. A panel of four cripto-1 antibodies was investigated with respect to their binding to the Geltrex matrix and to the cripto-1 positive control cells NTERA2. The cripto-1 expression was varied for the different antibodies with respect to cellular localization and fixation methods. To further elaborate on these findings, we present a systematic review of cripto-1 antibodies found in the literature and highlight some possible cross reactants with data on sequence alignments and structural comparison of EGF domains.

Improving transparency and scientific rigor in academic publishing

Progress in basic and clinical research is slowed when researchers fail to provide a complete and accurate report of how a study was designed, executed, and the results analyzed. Publishing rigorous scientific research involves a full description of the methods, materials, procedures, and outcomes. Investigators may fail to provide a complete description of how their study was designed and executed because they may not know how to accurately report the information or the mechanisms are not in place to facilitate transparent reporting. Here, we provide an overview of how authors can write manuscripts in a transparent and thorough manner. We introduce a set of reporting criteria that can be used for publishing, including recommendations on reporting the experimental design and statistical approaches. We also discuss how to accurately visualize the results and provide recommendations for peer reviewers to enhance rigor and transparency. Incorporating transparency practices into research manuscripts will significantly improve the reproducibility of the results by independent laboratories. SIGNIFICANCE: Failure to replicate research findings often arises from errors in the experimental design and statistical approaches. By providing a full account of the experimental design, procedures, and statistical approaches, researchers can address the reproducibility crisis and improve the sustainability of research outcomes. In this piece, we discuss the key issues leading to irreproducibility and provide general approaches to improving transparency and rigor in reporting, which could assist in making research more reproducible.

Incidences of problematic cell lines are lower in papers that use RRIDs to identify cell lines

The use of misidentified and contaminated cell lines continues to be a problem in biomedical research. Research Resource Identifiers (RRIDs) should reduce the prevalence of misidentified and contaminated cell lines in the literature by alerting researchers to cell lines that are on the list of problematic cell lines, which is maintained by the International Cell Line Authentication Committee (ICLAC) and the Cellosaurus database. To test this assertion, we text-mined the methods sections of about two million papers in PubMed Central, identifying 305,161 unique cell-line names in 150,459 articles. We estimate that 8.6% of these cell lines were on the list of problematic cell lines, whereas only 3.3% of the cell lines in the 634 papers that included RRIDs were on the problematic list. This suggests that the use of RRIDs is associated with a lower reported use of problematic cell lines.

Validation of anti-glucocerebrosidase antibodies for western blot analysis on protein lysates of murine and human cells

Gaucher disease (GD) is a rare lysosomal storage disorder caused by mutations in the GBA1 gene, encoding the lysosome-resident glucocerebrosidase enzyme involved in the hydrolysis of glucosylceramide. The discovery of an association between mutations in GBA1 and the development of synucleinopathies, including Parkinson disease, has directed attention to glucocerebrosidase as a potential therapeutic target for different synucleinopathies. These findings initiated an exponential growth in research and publications regarding the glucocerebrosidase enzyme. The use of various commercial and custom-made glucocerebrosidase antibodies has been reported, but standardized in-depth validation is still not available for many of these antibodies. This work details the evaluation of several previously reported glucocerebrosidase antibodies for western blot analysis, tested on protein lysates of murine gba+/+ and gba−/− immortalized neurons and primary human wild-type and type 2 GD fibroblasts.

Improving transparency and scientific rigor in academic publishing

Progress in basic and clinical research is slowed when researchers fail to provide a complete and accurate report of how a study was designed, executed, and the results analyzed. Publishing rigorous scientific research involves a full description of the methods, materials, procedures, and outcomes. Investigators may fail to provide a complete description of how their study was designed and executed because they may not know how to accurately report the information or the mechanisms are not in place to facilitate transparent reporting. Here, we provide an overview of how authors can write manuscripts in a transparent and thorough manner. We introduce a set of reporting criteria that can be used for publishing, including recommendations on reporting the experimental design and statistical approaches. We also discuss how to accurately visualize the results and provide recommendations for peer reviewers to enhance rigor and transparency. Incorporating transparency practices into research manuscripts will significantly improve the reproducibility of the results by independent laboratories.

Quantity and Reporting Quality of Kidney Research

BACKGROUND

In 2004, researchers reported that the number of nephrology clinical trials was low and that the reporting quality of such trials was suboptimal. Furthermore, the number or quality of preclinical kidney-related studies has not been systematically evaluated.

METHODS

We performed a systematic review of randomized clinical trials published in 1966-2017 (listed in the Cochrane Library) and preclinical studies published in 1945-2017 (listed in PubMed). For reporting quality analysis, we evaluated the final main paper of 118 clinical trial reports and 135 preclinical studies published in leading journals in 1996, 2006, and 2016 on the basis of criteria from the widely used CONSORT and ARRIVE guidelines.

RESULTS

The annual number of reports of clinical kidney-related trials more than doubled between 2004 and 2014 along with reports in other medical disciplines. Hypertension remains the dominant focus of study, but ongoing trials also center on CKD, ESRD, and AKI. The reporting quality analysis revealed improvements, but deficits in reporting of clinical trial design, mode of randomization, and intention-to-treat analysis remain. Annual numbers of kidney-related preclinical studies remained low between 1945 and 2017 compared with other disciplines. Reporting quality analysis of preclinical studies revealed substantial reporting deficits across all leading journals, with little improvement over the last 20 years, especially for group size calculations, defining primary versus secondary outcomes, and blinded analysis.

CONCLUSIONS

Nephrology studies keep increasing in number but still lag behind other medical disciplines, and the quality of data reporting in kidney research can be further improved.