The effect of journal guidelines on the reporting of antibody validation

What's wrong with my experiment?: The impact of hidden variables on neuropsychopharmacology research

The field of neuropsychopharmacology relies on behavioral assays to quantify behavioral processes related to mental illness and substance use disorders. Although these assays have been highly informative, sometimes laboratories have unpublished datasets from experiments that "didn't work". Often this is because expected outcomes were not observed in positive or negative control groups. While this can be due to experimenter error, an important alternative is that under-appreciated environmental factors can have a major impact on results. "Hidden variables" such as circadian cycles, husbandry, and social environments are often omitted in methods sections, even though there is a strong body of literature documenting their impact on physiological and behavioral outcomes. Applying this knowledge in a more critical manner could provide behavioral neuroscientists with tools to develop better testing methods, improve the external validity of behavioral techniques, and make better comparisons of experimental data across institutions. Here we review the potential impact of "hidden variables" that are commonly overlooked such as light-dark cycles, transport stress, cage ventilation, and social housing structure. While some of these conditions may not be under direct control of investigators, it does not diminish the potential impact of these variables on experimental results. We provide recommendations to investigators on which variables to report in publications and how to address "hidden variables" that impact their experimental results.

The Human Protein Atlas and Antibody-Based Tissue Profiling in Clinical Proteomics

Immunohistochemistry (IHC) is a standard method for spatial proteomics and allows for exploration of protein expression at single-cell resolution within the intact tissue environment. Stringent procedures and proper antibody validation strategies are however needed to ensure reliability of results. Application-specific strategies have been proposed by the scientific community to ensure high quality despite variations in sample preparation between different antibody-based methods. Here, the entire workflow utilized within the Human Protein Atlas, from sample preparation to annotation of the IHC staining patterns is described in detail, with important notes on various factors that can affect the outcome of IHC. Methods include tissue microarray (TMA) production, tissue sectioning, IHC, annotation, and validation. Also, building on previously suggested validation strategies, IHC-specific orthogonal and independent validation methods are outlined.

Misidentified biomedical resources: Journal guidelines are not a quick fix

Biomedical researchers routinely use a variety of biological models and resources, such as cultured cell lines, antibodies and laboratory animals. Unfortunately, these resources are not flawless: cell lines can be misidentified; for antibodies, problems with specificity, lot‐to‐lot consistency and sensitivity are common; and the reliability of animal models is questioned due to poor translation of animal studies to human clinical trials. In some cases, these problems can render the results of a study meaningless. As a response, some journals have implemented guidelines regarding the use and reporting of cell lines, antibodies and laboratory animals. In our study we use a portfolio of existing and newly created datasets to investigate identification and authentication information of cell lines, antibodies and organisms before and after guideline introduction, compared to journals without guidelines. We observed a general improvement of reporting quality over time, which the implementation of guidelines accelerated only in some cases. We therefore conclude that the effectiveness of journal guidelines is likely to be context dependent, affected by factors such as implementation conditions, research community support and monitoring and resource availability. Hence, journal reporting guidelines in themselves are not a quick fix to repair shortcomings in biomedical resource documentation, even though they can be part of the solution.

Antibody Watch: Text mining antibody specificity from the literature

Antibodies are widely used reagents to test for expression of proteins and other antigens. However, they might not always reliably produce results when they do not specifically bind to the target proteins that their providers designed them for, leading to unreliable research results. While many proposals have been developed to deal with the problem of antibody specificity, it is still challenging to cover the millions of antibodies that are available to researchers. In this study, we investigate the feasibility of automatically generating alerts to users of problematic antibodies by extracting statements about antibody specificity reported in the literature. The extracted alerts can be used to construct an “Antibody Watch” knowledge base containing supporting statements of problematic antibodies. We developed a deep neural network system and tested its performance with a corpus of more than two thousand articles that reported uses of antibodies. We divided the problem into two tasks. Given an input article, the first task is to identify snippets about antibody specificity and classify if the snippets report that any antibody exhibits non-specificity, and thus is problematic. The second task is to link each of these snippets to one or more antibodies mentioned in the snippet. The experimental evaluation shows that our system can accurately perform the classification task with 0.925 weighted F1-score, linking with 0.962 accuracy, and 0.914 weighted F1 when combined to complete the joint task. We leveraged Research Resource Identifiers (RRID) to precisely identify antibodies linked to the extracted specificity snippets. The result shows that it is feasible to construct a reliable knowledge base about problematic antibodies by text mining.

Antibodies are widely used reagents to test for the expression of proteins. However, antibodies are also a known source of reproducibility problems in biomedicine, as specificity and other issues can complicate their use. Information about how antibodies perform for specific applications are scattered across the biomedical literature and multiple websites. To alert scientists with reported antibody issues, we develop text mining algorithms that can identify specificity issues reported in the literature. We developed a deep neural network algorithm and performed a feasibility study on 2,223 papers. We leveraged Research Resource Identifiers (RRIDs), unique identifiers for antibodies and other biomedical resources, to match extracted specificity issues with particular antibodies. The results show that our system, called “Antibody Watch,” can accurately perform specificity issue identification and RRID association with a weighted F-score over 0.914. From our test corpus, we identified 37 antibodies with 68 nonspecific issue statements. With Antibody Watch, for example, if one were looking for an antibody targeting beta-Amyloid 1–16, from 74 antibodies at dkNET Resource Reports (on 10/2/20), one would be alerted that “some non-specific bands were detected at 55 kDa in both WT and APP/PS1 mice with the 6E10 antibody…”

YAP vs. TAZ: differences in expression revealed through rigorous validation of target-specific monoclonal antibodies

The ability to reproduce scientific findings is foundational in research; yet, it is compromised in part by poorly characterized reagents, including antibodies. In this report, we describe the application of complementary validation strategies tailored for use in immunohistochemical assays in the characterization of rabbit monoclonal antibodies against YAP and TAZ, homologous and sequentially similar transcriptional effectors of the Hippo signaling pathway. A lack of antibody reagents rigorously validated for immunohistochemistry has limited the Hippo signaling research community's ability to interrogate YAP and TAZ independently in tissue. In a series of normal and diseased human tissues, we were able to demonstrate differential expression patterns of YAP and TAZ, suggesting the potential for functional differences of these proteins. These differences can now be studied in greater detail with these highly validated tools.

Enhanced Validation of Antibodies Enables the Discovery of Missing Proteins

The localization of proteins at a tissue- or cell-type-specific level is tightly linked to the protein function. To better understand each protein’s role in cellular systems, spatial information constitutes an important complement to quantitative data. The standard methods for determining the spatial distribution of proteins in single cells of complex tissue samples make use of antibodies. For a stringent analysis of the human proteome, we used orthogonal methods and independent antibodies to validate 5981 antibodies that show the expression of 3775 human proteins across all major human tissues. This enhanced validation uncovered 56 proteins corresponding to the group of “missing proteins” and 171 proteins of unknown function. The presented strategy will facilitate further discussions around criteria for evidence of protein existence based on immunohistochemistry and serves as a useful guide to identify candidate proteins for integrative studies with quantitative proteomics methods.