[Letter to the Editor] The need for improved education and training in research antibody usage and validation practices

Antibody characterization is critical to enhance reproducibility in biomedical research

Antibodies are used in many areas of biomedical and clinical research, but many of these antibodies have not been adequately characterized, which casts doubt on the results reported in many scientific papers. This problem is compounded by a lack of suitable control experiments in many studies. In this article we review the history of the 'antibody characterization crisis', and we document efforts and initiatives to address the problem, notably for antibodies that target human proteins. We also present recommendations for a range of stakeholders - researchers, universities, journals, antibody vendors and repositories, scientific societies and funders - to increase the reproducibility of studies that rely on antibodies.

Improving the integrity and reproducibility of research that uses antibodies: a technical, data sharing, behavioral and policy challenge

Antibodies are one of the most important reagents used in biomedical and fundamental research, used to identify, and quantify proteins, contribute to knowledge of disease mechanisms, and validate drug targets. Yet many antibodies used in research do not recognize their intended target, or recognize additional molecules, compromising the integrity of research findings and leading to waste of resources, lack of reproducibility, failure of research projects, and delays in drug development. Researchers frequently use antibodies without confirming that they perform as intended in their application of interest. Here we argue that the determinants of end-user antibody choice and use are critical, and under-addressed, behavioral drivers of this problem. This interacts with the batch-to-batch variability of these biological reagents, and the paucity of available characterization data for most antibodies, making it more difficult for researchers to choose high quality reagents and perform necessary validation experiments. The open-science company YCharOS works with major antibody manufacturers and knockout cell line producers to characterize antibodies, identifying high-performing renewable antibodies for many targets in neuroscience. This shows the progress that can be made by stakeholders working together. However, their work so far applies to only a tiny fraction of available antibodies. Where characterization data exists, end-users need help to find and use it appropriately. While progress has been made in the context of technical solutions and antibody characterization, we argue that initiatives to make best practice behaviors by researchers more feasible, easy, and rewarding are needed. Global cooperation and coordination between multiple partners and stakeholders will be crucial to address the technical, policy, behavioral, and open data sharing challenges. We offer potential solutions by describing our Only Good Antibodies initiative, a community of researchers and partner organizations working toward the necessary change. We conclude with an open invitation for stakeholders, including researchers, to join our cause.

Scaling of an antibody validation procedure enables quantification of antibody performance in major research applications

Antibodies are critical reagents to detect and characterize proteins. It is commonly understood that many commercial antibodies do not recognize their intended targets, but information on the scope of the problem remains largely anecdotal, and as such, feasibility of the goal of at least one potent and specific antibody targeting each protein in a proteome cannot be assessed. Focusing on antibodies for human proteins, we have scaled a standardized characterization approach using parental and knockout cell lines (Laflamme et al., 2019) to assess the performance of 614 commercial antibodies for 65 neuroscience-related proteins. Side-by-side comparisons of all antibodies against each target, obtained from multiple commercial partners, have demonstrated that: (i) more than 50% of all antibodies failed in one or more applications, (ii) yet, ~50-75% of the protein set was covered by at least one high-performing antibody, depending on application, suggesting that coverage of human proteins by commercial antibodies is significant; and (iii) recombinant antibodies performed better than monoclonal or polyclonal antibodies. The hundreds of underperforming antibodies identified in this study were found to have been used in a large number of published articles, which should raise alarm. Encouragingly, more than half of the underperforming commercial antibodies were reassessed by the manufacturers, and many had alterations to their recommended usage or were removed from the market. This first study helps demonstrate the scale of the antibody specificity problem but also suggests an efficient strategy toward achieving coverage of the human proteome; mine the existing commercial antibody repertoire, and use the data to focus new renewable antibody generation efforts.

Scaling of an antibody validation procedure enables quantification of antibody performance in major research applications

Antibodies are critical reagents to detect and characterize proteins. It is commonly understood that many commercial antibodies do not recognize their intended targets, but information on the scope of the problem remains largely anecdotal, and as such, feasibility of the goal of at least one potent and specific antibody targeting each protein in a proteome cannot be assessed. Focusing on antibodies for human proteins, we have scaled a standardized characterization approach using parental and knockout cell lines (Laflamme et al., 2019) to assess the performance of 614 commercial antibodies for 65 neuroscience-related proteins. Side-by-side comparisons of all antibodies against each target, obtained from multiple commercial partners, demonstrates that: i) more than 50% of all antibodies failed in one or more tests, ii) yet, ~50-75% of the protein set was covered by at least one high-performing antibody, depending on application, suggesting that coverage of human proteins by commercial antibodies is significant; and iii) recombinant antibodies performed better than monoclonal or polyclonal antibodies. The hundreds of underperforming antibodies identified in this study were found to have been used in a large number of published articles, which should raise alarm. Encouragingly, more than half of the underperforming commercial antibodies were reassessed by the manufacturers, and many had alterations to their recommended usage or were removed from the market. This first such study helps demonstrate the scale of the antibody specificity problem but also suggests an efficient strategy toward achieving coverage of the human proteome; mine the existing commercial antibody repertoire, and use the data to focus new renewable antibody generation efforts.

Affiblot: a dot blot-based screening device for selection of reliable antibodies

The key factor in the development of antibody-based assays is to find an antibody that has an appropriate affinity, high specificity, and low cross-reactivity. However, this task is not easy to carry out since the research antibodies on the market may suffer from low specificity and reproducibility. Here, we report on a palm-sized dot blot-based device, called the affiblot, that has a specially designed lid that allows simultaneous semi-quantitative comparison of up to five antibodies from different suppliers regarding their affinity/avidity, cross-reactivity, and batch-to-batch reliability. The only required peripheral equipment is a vacuum pump, a camera, and densitometry software. The affiblot device was tested for its functionality and its measurements were compared against those obtained by standard dot blot and ELISA. The benefit over these methods, when various antibodies are evaluated, is in its simplicity. It allows easy antigen deposition, fast application and the discarding of the solutions, a compact undivided membrane, and therefore significant decrease of labor. The device was tested with specific anti-ApoE, anti-EpCAM, anti-Salmonella, anti-E. coli, and anti-Listeria antibodies from different suppliers. Their properties were compared for their ability to interact specifically with antigen and/or non-target structures and the best-suited antibody for the intended application was identified.

YAP vs. TAZ: differences in expression revealed through rigorous validation of target-specific monoclonal antibodies

The ability to reproduce scientific findings is foundational in research; yet, it is compromised in part by poorly characterized reagents, including antibodies. In this report, we describe the application of complementary validation strategies tailored for use in immunohistochemical assays in the characterization of rabbit monoclonal antibodies against YAP and TAZ, homologous and sequentially similar transcriptional effectors of the Hippo signaling pathway. A lack of antibody reagents rigorously validated for immunohistochemistry has limited the Hippo signaling research community's ability to interrogate YAP and TAZ independently in tissue. In a series of normal and diseased human tissues, we were able to demonstrate differential expression patterns of YAP and TAZ, suggesting the potential for functional differences of these proteins. These differences can now be studied in greater detail with these highly validated tools.

Antibody validation for protein expression on tissue slides: a protocol for immunohistochemistry

Antibodies play a crucial role in basic research and clinical decision-making. However, there are no standardized algorithms or guidelines to ensure their accuracy and validity. There have been efforts to generate consensus, but, with the exception of clinical labs, antibody validation remains variable in the literature and sometimes in clinical practice. Here we focus on immunohistochemistry, an example of a scientific and clinical tool where validation of antibodies is critical. We describe a protocol that we use to validate antibodies specifically for immunohistochemistry, including some of the pillars of antibody validation from Uhlen et al. 2016, as an example of a rigorous approach to build antibody-based tests for both basic and translational science labs and for the clinic.

Imaging methods are vastly underreported in biomedical research

A variety of microscopy techniques are used by researchers in the life and biomedical sciences. As these techniques become more powerful and more complex, it is vital that scientific articles containing images obtained with advanced microscopes include full details about how each image was obtained. To explore the reporting of such details we examined 240 original research articles published in eight journals. We found that the quality of reporting was poor, with some articles containing no information about how images were obtained, and many articles lacking important basic details. Efforts by researchers, funding agencies, journals, equipment manufacturers and staff at shared imaging facilities are required to improve the reporting of experiments that rely on microscopy techniques.

A novel selection strategy for antibody producing hybridoma cells based on a new transgenic fusion cell line

The use of monoclonal antibodies is ubiquitous in science and biomedicine but the generation and validation process of antibodies is nevertheless complicated and time-consuming. To address these issues we developed a novel selective technology based on an artificial cell surface construct by which secreted antibodies were connected to the corresponding hybridoma cell when they possess the desired antigen-specificity. Further the system enables the selection of desired isotypes and the screening for potential cross-reactivities in the same context. For the design of the construct we combined the transmembrane domain of the EGF-receptor with a hemagglutinin epitope and a biotin acceptor peptide and performed a transposon-mediated transfection of myeloma cell lines. The stably transfected myeloma cell line was used for the generation of hybridoma cells and an antigen- and isotype-specific screening method was established. The system has been validated for globular protein antigens as well as for haptens and enables a fast and early stage selection and validation of monoclonal antibodies in one step.

Anything You Can Do, I Can Do Better: Can Aptamers Replace Antibodies in Clinical Diagnostic Applications?

The mainstay of clinical diagnostics is the use of specialised ligands that can recognise specific biomarkers relating to pathological changes. While protein antibodies have been utilised in these assays for the last 40 years, they have proven to be unreliable due to a number of reasons. The search for the 'perfect' targeting ligand or molecular probe has been slow, though the description of chemical antibodies, also known as aptamers, nearly 30 years ago suggested a replacement reagent. However, uptake has been slow to progress into the clinical environment. In this review, we discuss the issues associated with antibodies and describe some of the applications of aptamers that have relevancy to the clinical diagnostic environment.

Designing a rigorous microscopy experiment: Validating methods and avoiding bias

Images generated by a microscope are never a perfect representation of the biological specimen. Microscopes and specimen preparation methods are prone to error and can impart images with unintended attributes that might be misconstrued as belonging to the biological specimen. In addition, our brains are wired to quickly interpret what we see, and with an unconscious bias toward that which makes the most sense to us based on our current understanding. Unaddressed errors in microscopy images combined with the bias we bring to visual interpretation of images can lead to false conclusions and irreproducible imaging data. Here we review important aspects of designing a rigorous light microscopy experiment: validation of methods used to prepare samples and of imaging system performance, identification and correction of errors, and strategies for avoiding bias in the acquisition and analysis of images.

Validation of antibody-based assays for regulatory peptides: Do it once, get it right, and exploit the under-appreciated benefit of long-term antibody stability

Many submitted manuscripts utilizing antibody-based assays for biologically active peptides frequently neither include nor cite adequate validation data with the risk that the report is adding to the reproducibility crisis in biological research. On the basis of recent experience in re-characterizing in a radioimmunoassay format a polycolonal antibody to gastrin that was first raised nearly five decades ago, it is argued that some antibodies can be stable for very many decades. Researchers concerned about the reproducibility of data using antibodies in assays for regulatory peptides should therefore note that by rigorous validation at an early stage they may not only contribute to the resolution of the reproducibility crisis but also establish a resource that could be useful for very many years.

The 3rd Antibody Validation meeting: Bath UK 20-21 st September 2018

The International Antibody Validation meetings offer a welcome British forum for discussing this important topic, which is existentially crucial for the biological sciences community. Now in its 6th year, the biennial meeting is organized by Andrew Chalmers (University of Bath; CiteAb), this year with Carly Dix (Astra Zeneca).  The organizers gathered some 100 members of industry and academia, producers and users, for a day and a half to describe their efforts to ensure that their antibodies have the desired specificity and selectively for well-defined molecular targets. The meeting is largely available as WebCasts ( http://www.antibodyvalidation.co.uk/past-events/2018).

Enhanced validation of antibodies for research applications

There is a need for standardized validation methods for antibody specificity and selectivity. Recently, five alternative validation pillars were proposed to explore the specificity of research antibodies using methods with no need for prior knowledge about the protein target. Here, we show that these principles can be used in a streamlined manner for enhanced validation of research antibodies in Western blot applications. More than 6,000 antibodies were validated with at least one of these strategies involving orthogonal methods, genetic knockdown, recombinant expression, independent antibodies, and capture mass spectrometry analysis. The results show a path forward for efforts to validate antibodies in an application-specific manner suitable for both providers and users.

Antibody validation: a view from the mountains

Validation of antibodies and other protein binders is a subject of pressing concern for the research community and one which is uppermost in the minds of all who use antibodies as research and diagnostic reagents. Assessing an antibody's fitness for purpose includes accurate ascertainment of its target specificity and suitability for the envisaged task. Moreover, standardised procedures are essential to guarantee sample quality in testing procedures. The problem of defining precise standards for antibody validation has engendered much debate in recent publications and meetings, but gradually a consensus is emerging. At the 8th Alpbach Affinity Proteomics workshop (March 2017), a panel of leaders in the antibody field discussed suggestions which could bring this complex but essential issue a step nearer to a resolution. 'Alpbach recommendations' for best practice include tailoring binder validation processes according to the intended applications and promoting greater transparency in publications and in the information available from commercial antibody developers/providers. A single approach will not fit all applications and end users must ensure that the reported validation holds for their specific requirements, highlighting the need for adequate training in the fundamentals of antibody characterisation and validation across the user community.

The path to VICTORy - a beginner's guide to success using commercial research antibodies

Commercial research antibodies are crucial tools in modern cell biology and biochemistry. In the USA some $2 billion a year are spent on them, but many are apparently not fit-for-purpose, and this may contribute to the 'reproducibility crisis' in biological sciences. Inadequate antibody validation and characterization, lack of user awareness, and occasional incompetence amongst suppliers have had immense scientific and personal costs. In this Opinion, I suggest some paths to make the use of these vital tools more successful. I have attempted to summarize and extend expert views from the literature to suggest that sustained routine efforts should made in: (1) the validation of antibodies, (2) their identification, (3) communication and controls, (4) the training of potential users, (5) the transparency of original equipment manufacturer (OEM) marketing agreements, and (5) in a more widespread use of recombinant antibodies (together denoted the 'VICTOR' approach).

Antibodies and methods for immunohistochemistry of extracellular matrix proteins

The diversity of extracellular matrix (ECM) proteins encoded in mammalian genomes and detected by proteomic analyses generates a need for well validated antibodies against these proteins. We present characterization of a large number of antibodies against ECM proteins, from both commercial and academic sources, together with discussion of methods and strategies for their effective use in immunohistochemistry and illustrations of their efficacy. These data should be of value to investigators seeking well validated antibodies to ECM proteins of interest and save significant time and money tracking down effective reagents.

Ten Basic Rules of Antibody Validation

The quality of research antibodies is an issue for decades. Although several papers have been published to improve the situation, their impact seems to be limited. This publication makes the effort to simplify the description of validation criteria in a way that the occasional antibody user is able to assess the validation level of an immunochemical reagent. A simple, 1-page checklist is supplied for the practical application of these criteria.

The antibody horror show: an introductory guide for the perplexed

The biological literature reverberates with the inadequacies of commercial research-tool antibodies. The scientific community spends some $2 billion per year on such reagents. Excellent accessible scientific platforms exist for reliably making, validating and using antibodies, yet the laboratory end-user reality is somehow depressing - because they often "don't work". This experience is due to a bizarre and variegated spectrum of causes including: inadequately identified antibodies; inappropriate user and supplier validation; poor user training; and overloaded publishers. Colourful as this may appear, the outcomes for the community are uniformly grim, including badly damaged scientific careers, wasted public funding, and contaminated literature. As antibodies are amongst the most important of everyday reagents in cell biology and biochemistry, I have tried here to gently suggest a few possible solutions, including: a move towards using recombinant antibodies; obligatory unique identification of antibodies, their immunogens, and their producers; centralized international banking of standard antibodies and their ligands; routine, accessible open-source documentation of user experience with antibodies; and antibody-user certification.

Evaluation and validation of commercial antibodies for the detection of Shb

Antibodies are among the most important tools for protein detection but, prior to their usage, proper validation of their appropriateness for given applications is required. The utility of an antibody depends on its sensitivity and specificity. We studied these two aspects in a panel of commercial antibodies against Shb, a platform protein involved in receptor tyrosine kinase signalling, but the function of which is still incompletely understood. Several of the antibodies showed shortcomings or were not acceptable for detection of the endogenous protein. The few that could detect Shb were doing so in either western blotting or immunoprecipitation experiments but a given antibody could not work in both applications. This article provides a resource for the available molecular tools that can be used in future research on Shb.

Simultaneous assessment of iodine, iron, vitamin A, malarial antigenemia, and inflammation status biomarkers via a multiplex immunoassay method on a population of pregnant women from Niger

Deficiencies of vitamin A, iron, and iodine are major public health concerns in many low- and middle-income countries, but information on their status in populations is often lacking due to high costs and logistical challenges associated with assessing micronutrient status. Accurate, user-friendly, and low-cost analytical tools are needed to allow large-scale population surveys on micronutrient status. We present the expansion of a 7-plex protein microarray tool for the simultaneous measurement of up to seven biomarkers with relevance to the assessment of the key micronutrients iron, iodine, and vitamin A, and inflammation and malaria biomarkers: α-1-acid glycoprotein, C-reactive protein, ferritin, retinol binding protein 4, soluble transferrin receptor, thyroglobulin, and histidine-rich protein II. Assay performance was assessed using international reference standards and then verified by comparing the multiplexed and conventional immunoassay results on a training panel of plasma samples collected from US adults. These data were used to assign nominal concentrations to the calibrators of the assay to further improve performance which was then assessed by interrogating plasma samples from a cohort of pregnant women from Niger. The correlation between assays for each biomarker measured from this cohort was typically good, with the exception of thyroglobulin, and the sensitivity ranged from 74% to 93%, and specificity from 81% to 98%. The 7-Plex micronutrient assay has the potential for use as an affordable tool for population surveillance of vitamin A, iron, and iodine deficiencies as well as falciparum malarial parasitemia infectivity and inflammation. The assay is easy-to-use, requires minimal sample volume, and is scalable, rapid, and accurate—needing only a low-cost reader and basic equipment present in most reference laboratory settings and so may be employed by low and middle income countries for micronutrient surveillance to inform on status in key populations. Micronutrient deficiencies including iron, iodine, and vitamin A affect a significant portion of the world’s population. Efforts to assess the prevalence of these deficiencies in vulnerable populations are challenging, partly due to measurement tools that are inadequate for assessing multiple micronutrients in large-scale population surveys. We have developed a 7-plex immunoassay for the simultaneous measurement of seven biomarkers relevant to assessing iodine, iron, and vitamin A status, inflammation and Plasmodium falciparum parasitemia by measuring levels of thyroglobulin, ferritin, soluble transferrin receptor, retinol binding protein 4, α-1-acid glycoprotein, C-reactive protein, and histidine-rich protein II. This 7-plex immunoassay technique has potential as a rapid and effective tool for use in large-scale surveys and assessments of nutrition intervention programs in low- and middle-income countries.

Reproducibility2020: Progress and priorities

The preclinical research process is a cycle of idea generation, experimentation, and reporting of results. The biomedical research community relies on the reproducibility of published discoveries to create new lines of research and to translate research findings into therapeutic applications. Since 2012, when scientists from Amgen reported that they were able to reproduce only 6 of 53 "landmark" preclinical studies, the biomedical research community began discussing the scale of the reproducibility problem and developing initiatives to address critical challenges. Global Biological Standards Institute (GBSI) released the "Case for Standards" in 2013, one of the first comprehensive reports to address the rising concern of irreproducible biomedical research. Further attention was drawn to issues that limit scientific self-correction, including reporting and publication bias, underpowered studies, lack of open access to methods and data, and lack of clearly defined standards and guidelines in areas such as reagent validation. To evaluate the progress made towards reproducibility since 2013, GBSI identified and examined initiatives designed to advance quality and reproducibility. Through this process, we identified key roles for funders, journals, researchers and other stakeholders and recommended actions for future progress. This paper describes our findings and conclusions.

The challenges with the validation of research antibodies

This article further discusses the reproducibility crisis in biomedical science and how poor conduct of commercial antibodies contribute to this. In addition, the way quality data are presented on product sheets by antibody vendors is scrutinized. The article proposes that there is a distinction between testing data and validation data, and special attention is asked for consistency between batches and aliquots. Moreover, the article separates the specifics, such as formulation, antigen and price, from the specifics on performance. Finally, a two-tier approach is discussed, enabling scientists to anticipate how an antibody is likely to perform when repeated purchases are required.

Never Waste a Good Crisis: Confronting Reproducibility in Translational Research

The lack of reproducibility of preclinical experimentation has implications for sustaining trust in and ensuring the viability and funding of the academic research enterprise. Here I identify problematic behaviors and practices and suggest solutions to enhance reproducibility in translational research.