The Resource Identification Initiative: A Cultural Shift in Publishing

A Standardized Nomenclature Design for Systematic Referencing and Identification of Animal Cellular Material

The documentation, preservation and rescue of biological diversity increasingly uses living biological samples. Persistent associations between species, biosamples, such as tissues and cell lines, and the accompanying data are indispensable for using, exchanging and benefiting from these valuable materials. Explicit authentication of such biosamples by assigning unique and robust identifiers is therefore required to allow for unambiguous referencing, avoid identification conflicts and maintain reproducibility in research. A predefined nomenclature based on uniform rules would facilitate this process. However, such a nomenclature is currently lacking for animal biological material. We here present a first, standardized, human-readable nomenclature design, which is sufficient to generate unique and stable identifying names for animal cellular material with a focus on wildlife species. A species-specific human- and machine-readable syntax is included in the proposed standard naming scheme, allowing for the traceability of donated material and cultured cells, as well as data FAIRification. Only when it is consistently applied in the public domain, as publications and inter-institutional samples and data are exchanged, distributed and stored centrally, can the risks of misidentification and loss of traceability be mitigated. This innovative globally applicable identification system provides a standard for a sustainable structure for the long-term storage of animal bio-samples in cryobanks and hence facilitates current as well as future species conservation and biomedical research.

The Antibody Registry: ten years of registering antibodies

Antibodies are ubiquitous key biological research resources yet are tricky to use as they are prone to performance issues and represent a major source of variability across studies. Understanding what antibody was used in a published study is therefore necessary to repeat and/or interpret a given study. However, antibody reagents are still frequently not cited with sufficient detail to determine which antibody was used in experiments. The Antibody Registry is a public, open database that enables citation of antibodies by providing a persistent record for any antibody-based reagent used in a publication. The registry is the authority for antibody Research Resource Identifiers, or RRIDs, which are requested or required by hundreds of journals seeking to improve the citation of these key resources. The registry is the most comprehensive listing of persistently identified antibody reagents used in the scientific literature. Data contributors span individual authors who use antibodies to antibody companies, which provide their entire catalogs including discontinued items. Unlike many commercial antibody listing sites which tend to remove reagents no longer sold, registry records persist, providing an interface between a fast-moving commercial marketplace and the static scientific literature. The Antibody Registry (RRID:SCR_006397) https://antibodyregistry.org.

New Views of Old Proteins: Clarifying the Enigmatic Proteome

All human diseases involve proteins, yet our current tools to characterize and quantify them are limited. To better elucidate proteins across space, time, and molecular composition, we provide a >10 years of projection for technologies to meet the challenges that protein biology presents. With a broad perspective, we discuss grand opportunities to transition the science of proteomics into a more propulsive enterprise. Extrapolating recent trends, we describe a next generation of approaches to define, quantify, and visualize the multiple dimensions of the proteome, thereby transforming our understanding and interactions with human disease in the coming decade.

The Anti-Glucocorticoid Receptor Antibody Clone 5E4: Raising Awareness of Unspecific Antibody Binding

Unspecific antibody binding takes a significant toll on researchers in the form of both the economic burden and the disappointed hopes of promising new therapeutic targets. Despite recent initiatives promoting antibody validation, a uniform approach addressing this issue has not yet been developed. Here, we demonstrate that the anti-glucocorticoid receptor (GR) antibody clone 5E4 predominantly targets two different proteins of approximately the same size, namely AMP deaminase 2 (AMPD2) and transcription intermediary factor 1-beta (TRIM28). This paper is intended to generate awareness of unspecific binding of well-established reagents and advocate the use of more rigorous verification methods to improve antibody quality in the future.

Toward the explainability, transparency, and universality of machine learning for behavioral classification in neuroscience

The use of rigorous ethological observation via machine learning techniques to understand brain function (computational neuroethology) is a rapidly growing approach that is poised to significantly change how behavioral neuroscience is commonly performed. With the development of open-source platforms for automated tracking and behavioral recognition, these approaches are now accessible to a wide array of neuroscientists despite variations in budget and computational experience. Importantly, this adoption has moved the field toward a common understanding of behavior and brain function through the removal of manual bias and the identification of previously unknown behavioral repertoires. Although less apparent, another consequence of this movement is the introduction of analytical tools that increase the explainabilty, transparency, and universality of the machine-based behavioral classifications both within and between research groups. Here, we focus on three main applications of such machine model explainabilty tools and metrics in the drive toward behavioral (i) standardization, (ii) specialization, and (iii) explainability. We provide a perspective on the use of explainability tools in computational neuroethology, and detail why this is a necessary next step in the expansion of the field. Specifically, as a possible solution in behavioral neuroscience, we propose the use of Shapley values via Shapley Additive Explanations (SHAP) as a diagnostic resource toward explainability of human annotation, as well as supervised and unsupervised behavioral machine learning analysis.

Conference report: Biocuration 2021 Virtual Conference

The International Society for Biocuration (ISB) aims to promote the field of biocuration and provide a community forum for information exchange and networking. Over the past 14 years, the ISB has hosted annual international conferences, entirely dedicated to the field of biocuration, that rotate between regions across the world. These meetings bring together biocurators from various roles, including database curators, bioinformaticians, ontology developers and students. Due to the ongoing global pandemic, the 14th Annual ISB Biocuration Conference (ISB2021) was held virtually in the form of four sessions and one workshop over the course of the year. Each of the four virtual sessions included panel discussions covering (i) The Future of Biocuration, (ii) Career paths and projections in Biocuration, (iii) Addressing Implicit or Unconscious Bias: Equity, Diversity and Inclusion and (iv) Strategic planning. Here we report on highlights from the virtual conference and share some of the ideas and future goals of the ISB.

Database URL:https://www.biocuration.org/14th-annual-biocuration-conference-virtual/

Minimal Information about MHC Multimers (MIAMM)

With the goal of improving the reproducibility and annotatability of MHC multimer reagent data, we present the establishment of a new data standard: Minimal Information about MHC Multimers (https://miamm.lji.org/). Multimers are engineered reagents composed of a ligand and a MHC, which can be represented in a standardized format using ontology terminology. We provide an online Web site to host the details of the standard, as well as a validation tool to assist with the adoption of the standard. We hope that this publication will bring increased awareness of Minimal Information about MHC Multimers and drive acceptance, ultimately improving the quality and documentation of multimer data in the scientific literature.

Misidentified biomedical resources: Journal guidelines are not a quick fix

Biomedical researchers routinely use a variety of biological models and resources, such as cultured cell lines, antibodies and laboratory animals. Unfortunately, these resources are not flawless: cell lines can be misidentified; for antibodies, problems with specificity, lot‐to‐lot consistency and sensitivity are common; and the reliability of animal models is questioned due to poor translation of animal studies to human clinical trials. In some cases, these problems can render the results of a study meaningless. As a response, some journals have implemented guidelines regarding the use and reporting of cell lines, antibodies and laboratory animals. In our study we use a portfolio of existing and newly created datasets to investigate identification and authentication information of cell lines, antibodies and organisms before and after guideline introduction, compared to journals without guidelines. We observed a general improvement of reporting quality over time, which the implementation of guidelines accelerated only in some cases. We therefore conclude that the effectiveness of journal guidelines is likely to be context dependent, affected by factors such as implementation conditions, research community support and monitoring and resource availability. Hence, journal reporting guidelines in themselves are not a quick fix to repair shortcomings in biomedical resource documentation, even though they can be part of the solution.

The science of science: Clinical Science launches a new translational meta-research collection

Clinical Science is proud to launch a new translational meta-research collection. Meta-research, or the science of science, applies the scientific method to study science itself. Meta-research is a powerful tool for identifying common problems in scientific papers, assessing their impact, and testing solutions to improve the transparency, rigor, trustworthiness, and usefulness of biomedical research. The collection welcomes science of science studies that link basic science to disease mechanisms, as well as meta-research articles highlighting opportunities to improve transparency, rigor, and reproducibility among the types of papers published in Clinical Science. Submissions might include science of science studies that explore factors linked to successful translation, or meta-research on experimental methods or study designs that are often used in translational research. We hope that this collection will encourage scientists to think critically about current practices and take advantage of opportunities to make their own research more transparent, rigorous, and reproducible.

Open science saves lives: lessons from the COVID-19 pandemic

In the last decade Open Science principles have been successfully advocated for and are being slowly adopted in different research communities. In response to the COVID-19 pandemic many publishers and researchers have sped up their adoption of Open Science practices, sometimes embracing them fully and sometimes partially or in a sub-optimal manner. In this article, we express concerns about the violation of some of the Open Science principles and its potential impact on the quality of research output. We provide evidence of the misuses of these principles at different stages of the scientific process. We call for a wider adoption of Open Science practices in the hope that this work will encourage a broader endorsement of Open Science principles and serve as a reminder that science should always be a rigorous process, reliable and transparent, especially in the context of a pandemic where research findings are being translated into practice even more rapidly. We provide all data and scripts at https://osf.io/renxy/ .

The Rigor and Transparency Index Quality Metric for Assessing Biological and Medical Science Methods

The reproducibility crisis is a multifaceted problem involving ingrained practices within the scientific community. Fortunately, some causes are addressed by the author's adherence to rigor and reproducibility criteria, implemented via checklists at various journals. We developed an automated tool (SciScore) that evaluates research articles based on their adherence to key rigor criteria, including NIH criteria and RRIDs, at an unprecedented scale. We show that despite steady improvements, less than half of the scoring criteria, such as blinding or power analysis, are routinely addressed by authors; digging deeper, we examined the influence of specific checklists on average scores. The average score for a journal in a given year was named the Rigor and Transparency Index (RTI), a new journal quality metric. We compared the RTI with the Journal Impact Factor and found there was no correlation. The RTI can potentially serve as a proxy for methodological quality.

Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0

Improving the reproducibility of biomedical research is a major challenge. Transparent and accurate reporting is vital to this process; it allows readers to assess the reliability of the findings and repeat or build upon the work of other researchers. The ARRIVE guidelines (Animal Research: Reporting In Vivo Experiments) were developed in 2010 to help authors and journals identify the minimum information necessary to report in publications describing in vivo experiments. Despite widespread endorsement by the scientific community, the impact of ARRIVE on the transparency of reporting in animal research publications has been limited. We have revised the ARRIVE guidelines to update them and facilitate their use in practice. The revised guidelines are published alongside this paper. This explanation and elaboration document was developed as part of the revision. It provides further information about each of the 21 items in ARRIVE 2.0, including the rationale and supporting evidence for their inclusion in the guidelines, elaboration of details to report, and examples of good reporting from the published literature. This document also covers advice and best practice in the design and conduct of animal studies to support researchers in improving standards from the start of the experimental design process through to publication.

Engineered Ovarian Cancer Cell Lines for Validation of CAR T Cell Function

A set of genetically engineered isogenic cell lines is developed to express either folate receptor alpha or mesothelin, and a control cell line negative for both antigens. These cell lines also express fluorescent and bioluminescent reporter transgenes. The cell lines are used to authenticate specificity and function of a T-cell biofactory, a living vector that is developed to express proportionate amounts of engineered proteins upon engaging with disease cells through their specific antigenic biomarkers. The engineered cell lines are also used to assess the cytolytic function and specificity of primary T cells engineered with chimeric antigen receptors; and the specificity of monoclonal antibodies. The strategy described can be used to generate other cell lines to present different disease-specific biomarkers for use as quality control tools.

Revisiting the scientific method to improve rigor and reproducibility of immunohistochemistry in reproductive science

Immunohistochemistry (IHC) is a robust scientific tool whereby cellular components are visualized within a tissue, and this method has been and continues to be a mainstay for many reproductive biologists. IHC is highly informative if performed and interpreted correctly, but studies have shown that the general use and reporting of appropriate controls in IHC experiments is low. This omission of the scientific method can result in data that lack rigor and reproducibility. In this editorial, we highlight key concepts in IHC controls and describe an opportunity for our field to partner with the Histochemical Society to adopt their IHC guidelines broadly as researchers, authors, ad hoc reviewers, editorial board members, and editors-in-chief. Such cross-professional society interactions will ensure that we produce the highest quality data as new technologies emerge that still rely upon the foundations of classic histological and immunohistochemical principles.

Semi-automated fact-checking of nucleotide sequence reagents in biomedical research publications: The Seek & Blastn tool

Nucleotide sequence reagents are verifiable experimental reagents in biomedical publications, because their sequence identities can be independently verified and compared with associated text descriptors. We have previously reported that incorrectly identified nucleotide sequence reagents are characteristic of highly similar human gene knockdown studies, some of which have been retracted from the literature on account of possible research fraud. Because of the throughput limitations of manual verification of nucleotide sequences, we developed a semi-automated fact checking tool, Seek & Blastn, to verify the targeting or non-targeting status of published nucleotide sequence reagents. From previously described and unknown corpora of 48 and 155 publications, respectively, Seek & Blastn correctly extracted 304/342 (88.9%) and 1066/1522 (70.0%) nucleotide sequences and a predicted targeting/ non-targeting status. Seek & Blastn correctly predicted the targeting/ non-targeting status of 293/304 (96.4%) and 988/1066 (92.7%) of the correctly extracted nucleotide sequences. A total of 38/39 (97.4%) or 31/79 (39.2%) Seek & Blastn predictions of incorrect nucleotide sequence reagent use were correct in the two literature corpora. Combined Seek & Blastn and manual analyses identified a list of 91 misidentified nucleotide sequence reagents, which could be built upon through future studies. In summary, incorrect nucleotide sequence reagents represent an under-recognized source of error within the biomedical literature, and fact checking tools such as Seek & Blastn may help to identify papers and manuscripts affected by these errors.

Biomedical text mining for research rigor and integrity: tasks, challenges, directions

An estimated quarter of a trillion US dollars is invested in the biomedical research enterprise annually. There is growing alarm that a significant portion of this investment is wasted because of problems in reproducibility of research findings and in the rigor and integrity of research conduct and reporting. Recent years have seen a flurry of activities focusing on standardization and guideline development to enhance the reproducibility and rigor of biomedical research. Research activity is primarily communicated via textual artifacts, ranging from grant applications to journal publications. These artifacts can be both the source and the manifestation of practices leading to research waste. For example, an article may describe a poorly designed experiment, or the authors may reach conclusions not supported by the evidence presented. In this article, we pose the question of whether biomedical text mining techniques can assist the stakeholders in the biomedical research enterprise in doing their part toward enhancing research integrity and rigor. In particular, we identify four key areas in which text mining techniques can make a significant contribution: plagiarism/fraud detection, ensuring adherence to reporting guidelines, managing information overload and accurate citation/enhanced bibliometrics. We review the existing methods and tools for specific tasks, if they exist, or discuss relevant research that can provide guidance for future work. With the exponential increase in biomedical research output and the ability of text mining approaches to perform automatic tasks at large scale, we propose that such approaches can support tools that promote responsible research practices, providing significant benefits for the biomedical research enterprise.

Enhanced validation of antibodies for research applications

There is a need for standardized validation methods for antibody specificity and selectivity. Recently, five alternative validation pillars were proposed to explore the specificity of research antibodies using methods with no need for prior knowledge about the protein target. Here, we show that these principles can be used in a streamlined manner for enhanced validation of research antibodies in Western blot applications. More than 6,000 antibodies were validated with at least one of these strategies involving orthogonal methods, genetic knockdown, recombinant expression, independent antibodies, and capture mass spectrometry analysis. The results show a path forward for efforts to validate antibodies in an application-specific manner suitable for both providers and users.

Reproducibility in Computational Neuroscience Models and Simulations

OBJECTIVE

Like all scientific research, computational neuroscience research must be reproducible. Big data science, including simulation research, cannot depend exclusively on journal articles as the method to provide the sharing and transparency required for reproducibility.

METHODS

Ensuring model reproducibility requires the use of multiple standard software practices and tools, including version control, strong commenting and documentation, and code modularity.

RESULTS

Building on these standard practices, model-sharing sites and tools have been developed that fit into several categories: 1) standardized neural simulators; 2) shared computational resources; 3) declarative model descriptors, ontologies, and standardized annotations; and 4) model-sharing repositories and sharing standards.

CONCLUSION

A number of complementary innovations have been proposed to enhance sharing, transparency, and reproducibility. The individual user can be encouraged to make use of version control, commenting, documentation, and modularity in development of models. The community can help by requiring model sharing as a condition of publication and funding.

SIGNIFICANCE

Model management will become increasingly important as multiscale models become larger, more detailed, and correspondingly more difficult to manage by any single investigator or single laboratory. Additional big data management complexity will come as the models become more useful in interpreting experiments, thus increasing the need to ensure clear alignment between modeling data, both parameters and results, and experiment.

Data Publications Correlate with Citation Impact

Neuroscience and molecular biology have been generating large datasets over the past years that are reshaping how research is being conducted. In their wake, open data sharing has been singled out as a major challenge for the future of research. We conducted a comparative study of citations of data publications in both fields, showing that the average publication tagged with a data-related term by the NCBI MeSH (Medical Subject Headings) curators achieves a significantly larger citation impact than the average in either field. We introduce a new metric, the data article citation index (DAC-index), to identify the most prolific authors among those data-related publications. The study is fully reproducible from an executable Rmd (R Markdown) script together with all the citation datasets. We hope these results can encourage authors to more openly publish their data.

RRIDs: A Simple Step toward Improving Reproducibility through Rigor and Transparency of Experimental Methods

With the call for more rigorous scientific reporting, authentication, and transparency from the scientific community and funding agencies, one critical step is to make finding and identifying key resources in the published literature tractable. We discuss here the use of Research Resource Identifiers (RRIDs) as one tool to help resolve this tricky problem in reproducibility.

Sustainable funding for biocuration: The Arabidopsis Information Resource (TAIR) as a case study of a subscription-based funding model

Databases and data repositories provide essential functions for the research community by integrating, curating, archiving and otherwise packaging data to facilitate discovery and reuse. Despite their importance, funding for maintenance of these resources is increasingly hard to obtain. Fueled by a desire to find long term, sustainable solutions to database funding, staff from the Arabidopsis Information Resource (TAIR), founded the nonprofit organization, Phoenix Bioinformatics, using TAIR as a test case for user-based funding. Subscription-based funding has been proposed as an alternative to grant funding but its application has been very limited within the nonprofit sector. Our testing of this model indicates that it is a viable option, at least for some databases, and that it is possible to strike a balance that maximizes access while still incentivizing subscriptions. One year after transitioning to subscription support, TAIR is self-sustaining and Phoenix is poised to expand and support additional resources that wish to incorporate user-based funding strategies. Database URL: www.arabidopsis.org.