A verified genomic reference sample for assessing performance of cancer panels detecting small variants of low allele frequency

NCI workshop on ctDNA in cancer treatment and clinical care

Detection of cell-free circulating tumor DNA (ctDNA) from solid tumors is a fast-evolving field with significant potential for improving patient treatment outcomes. The spectrum of applications for ctDNA assays is broad and includes very diverse intended uses that will require different strategies to demonstrate utility. On September 14-15, 2023, the National Cancer Institute held an in-person workshop in Rockville, MD titled "ctDNA in Cancer Treatment and Clinical Care." The goal of the workshop was to examine what is currently known and what needs to be determined for various ctDNA liquid biopsy use cases related to treatment and management of patients with solid tumors and to explore how the community can best assess the value of ctDNA assays and technology. Additionally, new approaches were presented that may show promise in the future. The information exchanged in this workshop will provide the community with a better understanding of this field and its potential to affect and benefit decision-making in the treatment of patients with solid tumors.

Development and extensive sequencing of a broadly-consented Genome in a Bottle matched tumor-normal pair

The Genome in a Bottle Consortium (GIAB), hosted by the National Institute of Standards and Technology (NIST), is developing new matched tumor-normal samples, the first to be explicitly consented for public dissemination of genomic data and cell lines. Here, we describe a comprehensive genomic dataset from the first individual, HG008, including DNA from an adherent, epithelial-like pancreatic ductal adenocarcinoma (PDAC) tumor cell line and matched normal cells from duodenal and pancreatic tissues. Data for the tumor-normal matched samples comes from thirteen distinct state-of-the-art whole genome measurement technologies, including high depth short and long-read bulk whole genome sequencing (WGS), single cell WGS, and Hi-C, and karyotyping. These data will be used by the GIAB Consortium to develop matched tumor-normal benchmarks for somatic variant detection. We expect these data to facilitate innovation for whole genome measurement technologies, de novo assembly of tumor and normal genomes, and bioinformatic tools to identify small and structural somatic mutations. This first-of-its-kind broadly consented open-access resource will facilitate further understanding of sequencing methods used for cancer biology.

Reference Materials for Improving Reliability of Multiomics Profiling

High-throughput technologies for multiomics or molecular phenomics profiling have been extensively adopted in biomedical research and clinical applications, offering a more comprehensive understanding of biological processes and diseases. Omics reference materials play a pivotal role in ensuring the accuracy, reliability, and comparability of laboratory measurements and analyses. However, the current application of omics reference materials has revealed several issues, including inappropriate selection and underutilization, leading to inconsistencies across laboratories. This review aims to address these concerns by emphasizing the importance of well-characterized reference materials at each level of omics, encompassing (epi-)genomics, transcriptomics, proteomics, and metabolomics. By summarizing their characteristics, advantages, and limitations along with appropriate performance metrics pertinent to study purposes, we provide an overview of how omics reference materials can enhance data quality and data integration, thus fostering robust scientific investigations with omics technologies.

Targeted DNA-seq and RNA-seq of Reference Samples with Short-read and Long-read Sequencing

Next-generation sequencing (NGS) has revolutionized genomic research by enabling high-throughput, cost-effective genome and transcriptome sequencing accelerating personalized medicine for complex diseases, including cancer. Whole genome/transcriptome sequencing (WGS/WTS) provides comprehensive insights, while targeted sequencing is more cost-effective and sensitive. In comparison to short-read sequencing, which still dominates the field due to high speed and cost-effectiveness, long-read sequencing can overcome alignment limitations and better discriminate similar sequences from alternative transcripts or repetitive regions. Hybrid sequencing combines the best strengths of different technologies for a more comprehensive view of genomic/transcriptomic variations. Understanding each technology's strengths and limitations is critical for translating cutting-edge technologies into clinical applications. In this study, we sequenced DNA and RNA libraries of reference samples using various targeted DNA and RNA panels and the whole transcriptome on both short-read and long-read platforms. This study design enables a comprehensive analysis of sequencing technologies, targeting protocols, and library preparation methods. Our expanded profiling landscape establishes a reference point for assessing current sequencing technologies, facilitating informed decision-making in genomic research and precision medicine.

Implementation of an ISO 15189 accredited next generation sequencing service for cell-free total nucleic acid (cfTNA) analysis to facilitate driver mutation reporting in blood: the experience of a clinical diagnostic laboratory

AIMS

Next generation sequencing (NGS) on tumour tissue is integral to the delivery of personalised medicine and targeted therapy. NGS on liquid biopsy, a much less invasive technology, is an emerging clinical tool that has rapidly expanded clinical utility. Gene mutations in cell-free total nucleic acids (cfTNA) circulating in the blood are representative of whole tumour biology and can reveal different mutations from different tumour sites, thus addressing tumour heterogeneity challenges.

METHODS

The novel Ion Torrent Genexus NGS system with automated sample preparation, onboard library preparation, templating, sequencing, data analysis and Oncomine Reporter software was used. cfTNA extracted from plasma was verified with the targeted pan-cancer (~50 genes) Oncomine Precision Assay (OPA). Assessment criteria included analytical sensitivity, specificity, limits of detection (LOD), accuracy, repeatability, reproducibility and the establishment of performance metrics.

RESULTS

An ISO 15189 accredited, minimally invasive cfTNA NGS diagnostic service has been implemented. High sensitivity (>83%) and specificity between plasma and tissue were observed. A sequencing LOD of 1.2% was achieved when the depth of coverage was >22 000×. A reduction (>68%) in turnaround time (TAT) of liquid biopsy results was achieved: 5 days TAT for in-house analysis from sample receipt to a final report issued to oncologists as compared with >15 days from reference laboratories.

CONCLUSION

Tumour-derived somatic variants can now be reliably assessed from plasma to provide minimally invasive tumour profiling. Successful implementation of this accredited service resulted in:Appropriate molecular profiling of patients where tumour tissue is unavailable or inaccessible.Rapid TAT of plasma NGS results.

Multi-omics data integration using ratio-based quantitative profiling with Quartet reference materials

Characterization and integration of the genome, epigenome, transcriptome, proteome and metabolome of different datasets is difficult owing to a lack of ground truth. Here we develop and characterize suites of publicly available multi-omics reference materials of matched DNA, RNA, protein and metabolites derived from immortalized cell lines from a family quartet of parents and monozygotic twin daughters. These references provide built-in truth defined by relationships among the family members and the information flow from DNA to RNA to protein. We demonstrate how using a ratio-based profiling approach that scales the absolute feature values of a study sample relative to those of a concurrently measured common reference sample produces reproducible and comparable data suitable for integration across batches, labs, platforms and omics types. Our study identifies reference-free 'absolute' feature quantification as the root cause of irreproducibility in multi-omics measurement and data integration and establishes the advantages of ratio-based multi-omics profiling with common reference materials.

2023 White Paper on Recent Issues in Bioanalysis: Deuterated Drugs; LNP; Tumor/FFPE Biopsy; Targeted Proteomics; Small Molecule Covalent Inhibitors; Chiral Bioanalysis; Remote Regulatory Assessments; Sample Reconciliation/Chain of Custody (PART 1A - Recommendations on Mass Spectrometry, Chromatography, Sample Preparation Latest Developments, Challenges, and Solutions and BMV/Regulated Bioanalysis PART 1B - Regulatory Agencies' Inputs on Regulated Bioanalysis/BMV, Biomarkers/IVD/CDx/BAV, Immunogenicity, Gene & Cell Therapy and Vaccine)

The 17th Workshop on Recent Issues in Bioanalysis (17th WRIB) took place in Orlando, FL, USA on June 19-23, 2023. Over 1000 professionals representing pharma/biotech companies, CROs, and multiple regulatory agencies convened to actively discuss the most current topics of interest in bioanalysis. The 17th WRIB included 3 Main Workshops and 7 Specialized Workshops that together spanned 1 week to allow an exhaustive and thorough coverage of all major issues in bioanalysis of biomarkers, immunogenicity, gene therapy, cell therapy and vaccines.Moreover, in-depth workshops on "EU IVDR 2017/746 Implementation and impact for the Global Biomarker Community: How to Comply with this NEW Regulation" and on "US FDA/OSIS Remote Regulatory Assessments (RRAs)" were the special features of the 17th edition.As in previous years, WRIB continued to gather a wide diversity of international, industry opinion leaders and regulatory authority experts working on both small and large molecules as well as gene, cell therapies and vaccines to facilitate sharing and discussions focused on improving quality, increasing regulatory compliance, and achieving scientific excellence on bioanalytical issues.This 2023 White Paper encompasses recommendations emerging from the extensive discussions held during the workshop and is aimed to provide the bioanalytical community with key information and practical solutions on topics and issues addressed, in an effort to enable advances in scientific excellence, improved quality and better regulatory compliance. Due to its length, the 2023 edition of this comprehensive White Paper has been divided into three parts for editorial reasons.This publication covers the recommendations on Mass Spectrometry Assays, Regulated Bioanalysis/BMV (Part 1A) and Regulatory Inputs (Part 1B). Part 2 (Biomarkers, IVD/CDx, LBA and Cell-Based Assays) and Part 3 (Gene Therapy, Cell therapy, Vaccines and Biotherapeutics Immunogenicity) are published in volume 16 of Bioanalysis, issues 7 and 8 (2024), respectively.

Towards accurate indel calling for oncopanel sequencing through an international pipeline competition at precisionFDA

Accurately calling indels with next-generation sequencing (NGS) data is critical for clinical application. The precisionFDA team collaborated with the U.S. Food and Drug Administration's (FDA's) National Center for Toxicological Research (NCTR) and successfully completed the NCTR Indel Calling from Oncopanel Sequencing Data Challenge, to evaluate the performance of indel calling pipelines. Top performers were selected based on precision, recall, and F1-score. The performance of many other pipelines was close to the top performers, which produced a top cluster of performers. The performance was significantly higher in high confidence regions and coding regions, and significantly lower in low complexity regions. Oncopanel capture and other issues may have occurred that affected the recall rate. Indels with higher variant allele frequency (VAF) may generally be called with higher confidence. Many of the indel calling pipelines had good performance. Some of them performed generally well across all three oncopanels, while others were better for a specific oncopanel. The performance of indel calling can further be improved by restricting the calls within high confidence intervals (HCIs) and coding regions, and by excluding low complexity regions (LCR) regions. Certain VAF cut-offs could be applied according to the applications.

Extend the benchmarking indel set by manual review using the individual cell line sequencing data from the Sequencing Quality Control 2 (SEQC2) project

Accurate indel calling plays an important role in precision medicine. A benchmarking indel set is essential for thoroughly evaluating the indel calling performance of bioinformatics pipelines. A reference sample with a set of known-positive variants was developed in the FDA-led Sequencing Quality Control Phase 2 (SEQC2) project, but the known indels in the known-positive set were limited. This project sought to provide an enriched set of known indels that would be more translationally relevant by focusing on additional cancer related regions. A thorough manual review process completed by 42 reviewers, two advisors, and a judging panel of three researchers significantly enriched the known indel set by an additional 516 indels. The extended benchmarking indel set has a large range of variant allele frequencies (VAFs), with 87% of them having a VAF below 20% in reference Sample A. The reference Sample A and the indel set can be used for comprehensive benchmarking of indel calling across a wider range of VAF values in the lower range. Indel length was also variable, but the majority were under 10 base pairs (bps). Most of the indels were within coding regions, with the remainder in the gene regulatory regions. Although high confidence can be derived from the robust study design and meticulous human review, this extensive indel set has not undergone orthogonal validation. The extended benchmarking indel set, along with the indels in the previously published known-positive set, was the truth set used to benchmark indel calling pipelines in a community challenge hosted on the precisionFDA platform. This benchmarking indel set and reference samples can be utilized for a comprehensive evaluation of indel calling pipelines. Additionally, the insights and solutions obtained during the manual review process can aid in improving the performance of these pipelines.

Implementation of an ISO15189 accredited next-generation sequencing service with the fully automated Ion Torrent Genexus: the experience of a clinical diagnostic laboratory

AIMS

Next-generation sequencing (NGS) is integral to the delivery of personalised medicine for targeted cancer therapy. Average turnaround times (TAT) from reference laboratories with advanced expertise in sequencing are typically 2-3 weeks. Prolonged TAT for biomarker analysis can adversely affect patient outcomes. The project aim was to establish an accredited NGS service integrated within a routine clinical diagnostic laboratory, in a designated tertiary cancer centre with no previous experience in NGS or bioinformatics.

METHODS

Platform selected was the novel Ion Torrent Genexus Sequencer with automated onboard library preparation, templating, sequencing and data analysis, with subsequent reporting using Oncomine Reporter software.Entire workflow validation was performed with a targeted panel, the Oncomine Precision Assay, on formalin-fixed paraffin embedded clinical tumour samples. Oncomine Reporter software was used to report on variants including mutations, copy number variations and fusions across 50 key genes.Samples included surgical resections, biopsies, cytology and commercial reference material. Assessment of criteria included analytical sensitivity, specificity, limit of detection, accuracy, repeatability and reproducibility, with the establishment of performance metrics and quality parameters.

RESULTS

High sensitivity, specificity and reproducibility were achieved. DNA/RNA input requirements optimised to >10 ng, and sequencing performance established with a limit of detection of 5% when depth of coverage of 2500X was reached. This NGS service attained ISO15189 accreditation with no non-conformances and >56% reduction in TAT.

CONCLUSION

Successful implementation, clinical validation and accreditation of a novel NGS technology was achieved in this institution, with a significantly improved TAT of results to oncologists.

Assessments of tumor mutational burden estimation by targeted panel sequencing: A comprehensive simulation analysis

Tumor mutational burden (TMB), when at a high level, is an emerging indicative factor of sensitivity to immune checkpoint inhibitors. Previous studies have shown that the more affordable and accurate targeted panels can be used to measure TMB as a substitute for whole exome sequencing (WES). However, additional processes, such as hotspot mutations exclusion and TMB adjustment, are usually required to deal with the effect of the limited panel sizes. A comprehensive investigation of the effective factors is needed for accurate TMB estimation by targeted panels. In this study, we quantitatively evaluated the variances of TMB values calculated by WES and targeted panels using 10,000 simulated targeted panels with panel sizes ranging from 0.2 to 3.1 million bases. With The Cancer Genome Atlas (TCGA) cancer samples and mutation profiles, we fixed regressions on WES-TMBs and panel-TMBs to assess the performance of a given targeted panel. Panel size was found as one of the major effective factors of TMB estimation. Meanwhile, by investigating the well-performing small panels that reported TMB values similar to those of WES, we demonstrated the evidence of the cancer type-specific impacts of genes on TMB estimation and identified high-impact gene sets for different cancer types based on the TCGA data. This study revealed the quantitative correlations between TMB variance and panel size, and the potential impacts of individual genes on TMB estimation. Our results suggested that for cancer patients diagnosed using targeted panels, it would be highly beneficial to have the capability to directly measure TMB from the targeted sequencing data. This would greatly assist in making decisions regarding the use of immunotherapies.

2022 White Paper on Recent Issues in Bioanalysis: ICH M10 BMV Guideline & Global Harmonization; Hybrid Assays; Oligonucleotides & ADC; Non-Liquid & Rare Matrices; Regulatory Inputs (Part 1A - Recommendations on Mass Spectrometry, Chromatography and Sample Preparation, Novel Technologies, Novel Modalities, and Novel Challenges, ICH M10 BMV Guideline & Global Harmonization Part 1B - Regulatory Agencies' Inputs on Regulated Bioanalysis/BMV, Biomarkers/CDx/BAV, Immunogenicity, Gene & Cell Therapy and Vaccine)

The 16th Workshop on Recent Issues in Bioanalysis (16th WRIB) took place in Atlanta, GA, USA on September 26-30, 2022. Over 1000 professionals representing pharma/biotech companies, CROs, and multiple regulatory agencies convened to actively discuss the most current topics of interest in bioanalysis. The 16th WRIB included 3 Main Workshops and 7 Specialized Workshops that together spanned 1 week in order to allow exhaustive and thorough coverage of all major issues in bioanalysis, biomarkers, immunogenicity, gene therapy, cell therapy and vaccines. Moreover, in-depth workshops on the ICH M10 BMV final guideline (focused on this guideline training, interpretation, adoption and transition); mass spectrometry innovation (focused on novel technologies, novel modalities, and novel challenges); and flow cytometry bioanalysis (rising of the 3rd most common/important technology in bioanalytical labs) were the special features of the 16th edition. As in previous years, WRIB continued to gather a wide diversity of international, industry opinion leaders and regulatory authority experts working on both small and large molecules as well as gene, cell therapies and vaccines to facilitate sharing and discussions focused on improving quality, increasing regulatory compliance, and achieving scientific excellence on bioanalytical issues. This 2022 White Paper encompasses recommendations emerging from the extensive discussions held during the workshop and is aimed to provide the bioanalytical community with key information and practical solutions on topics and issues addressed, in an effort to enable advances in scientific excellence, improved quality and better regulatory compliance. Due to its length, the 2022 edition of this comprehensive White Paper has been divided into three parts for editorial reasons. This publication (Part 1A) covers the recommendations on Mass Spectrometry and ICH M10. Part 1B covers the Regulatory Agencies' Inputs on Bioanalysis, Biomarkers, Immunogenicity, Gene & Cell Therapy and Vaccine. Part 2 (LBA, Biomarkers/CDx and Cytometry) and Part 3 (Gene Therapy, Cell therapy, Vaccines and Biotherapeutics Immunogenicity) are published in volume 15 of Bioanalysis, issues 15 and 14 (2023), respectively.

Variant calling and benchmarking in an era of complete human genome sequences

Genetic variant calling from DNA sequencing has enabled understanding of germline variation in hundreds of thousands of humans. Sequencing technologies and variant-calling methods have advanced rapidly, routinely providing reliable variant calls in most of the human genome. We describe how advances in long reads, deep learning, de novo assembly and pangenomes have expanded access to variant calls in increasingly challenging, repetitive genomic regions, including medically relevant regions, and how new benchmark sets and benchmarking methods illuminate their strengths and limitations. Finally, we explore the possible future of more complete characterization of human genome variation in light of the recent completion of a telomere-to-telomere human genome reference assembly and human pangenomes, and we consider the innovations needed to benchmark their newly accessible repetitive regions and complex variants.

FVC as an adaptive and accurate method for filtering variants from popular NGS analysis pipelines

The quality control of variants from whole-genome sequencing data is vital in clinical diagnosis and human genetics research. However, current filtering methods (Frequency, Hard-Filter, VQSR, GARFIELD, and VEF) were developed to be utilized on particular variant callers and have certain limitations. Especially, the number of eliminated true variants far exceeds the number of removed false variants using these methods. Here, we present an adaptive method for quality control on genetic variants from different analysis pipelines, and validate it on the variants generated from four popular variant callers (GATK HaplotypeCaller, Mutect2, Varscan2, and DeepVariant). FVC consistently exhibited the best performance. It removed far more false variants than the current state-of-the-art filtering methods and recalled ~51-99% true variants filtered out by the other methods. Once trained, FVC can be conveniently integrated into a user-specific variant calling pipeline.

FVC is a method for calling specific gene variants from whole genome data, for potential use in clinical diagnosis and human genetics research.

Recommendations related to the analytical equivalence assessment of gene panel testing

Advances in cancer genome care over the past few years have included the development of gene panel testing for various biomarkers. This article summarizes issues and provides recommendations related to analytical performance evaluations for new oncology gene panels. The scope of these recommendations includes comprehensive genomic profiling assays related to gene panel testing that uses histological or serum specimens to detect gene mutations. As a research project of the Japan Agency for Medical Research and Development Research on Regulatory Science of Pharmaceuticals and Medical Devices, we convened the working group committee that consisted of more than 30 experts from academia, industry, and government. We have discussed the points that should be considered to allow maximal simplification of assessments using clinical specimens in evaluating accuracy and limit of detection in equivalence and analytical performance for three years. We provide recommendations specific to each type of gene mutation as well as to reference standards or specimens used for evaluations. In addition, in order to facilitate the discussion on the analytical performance of gene panel tests by multidisciplinary tumor boards of hospitals, the present recommendations also describe the items that companies are expected to provide information on in their packaging inserts and reports, and the items that are expected to be discussed by multidisciplinary tumor boards. Our working group document will be important for participants in multidisciplinary tumor boards including medical oncologists and genome scientists, and developers of gene panels not only in Japan but also in other countries.

Deep oncopanel sequencing reveals within block position-dependent quality degradation in FFPE processed samples

BACKGROUND

Clinical laboratories routinely use formalin-fixed paraffin-embedded (FFPE) tissue or cell block cytology samples in oncology panel sequencing to identify mutations that can predict patient response to targeted therapy. To understand the technical error due to FFPE processing, a robustly characterized diploid cell line was used to create FFPE samples with four different pre-tissue processing formalin fixation times. A total of 96 FFPE sections were then distributed to different laboratories for targeted sequencing analysis by four oncopanels, and variants resulting from technical error were identified.

RESULTS

Tissue sections that fail more frequently show low cellularity, lower than recommended library preparation DNA input, or target sequencing depth. Importantly, sections from block surfaces are more likely to show FFPE-specific errors, akin to "edge effects" seen in histology, while the inner samples display no quality degradation related to fixation time.

CONCLUSIONS

To assure reliable results, we recommend avoiding the block surface portion and restricting mutation detection to genomic regions of high confidence.

Ultra-deep multi-oncopanel sequencing of benchmarking samples with a wide range of variant allele frequencies

The lack of suitable reference genomic material to enable a transparent cross-lab study of oncopanels inspired the SEQC2 Oncopanel Sequencing Working Group to develop four reference samples, sequenced with eight oncopanels at independent test laboratories with ultra-deep sequencing depth. This rich, publicly available dataset enabled performance assessment of the clinical applicability of oncopanels. In addition, this dataset present sample opportunities for developing specific and robust bioinformatics pipelines and fine-tuning parameters for more accurate variant calling, investigating ideal sequencing depth for variant calling of a given minimum VAF and variant type, and also recommending best use cases for Unique Molecular Identifier (UMI) technology.

Measurement(s)	Short Somatic Mutation
Technology Type(s)	Tumor Panel Sequencing
Factor Type(s)	Tumor Panel • Genomic Region
Sample Characteristic - Organism	Homo sapiens

Ultra-deep sequencing data from a liquid biopsy proficiency study demonstrating analytic validity

Recently we reported the accuracy and reproducibility of circulating tumor DNA (ctDNA) assays using a unique set of reference materials, associated analytical framework, and suggested best practices. With the rapid adoption of ctDNA sequencing in precision oncology, it is critical to understand the analytical validity and technical limitations of this cutting-edge and medical-practice-changing technology. The SEQC2 Oncopanel Sequencing Working Group has developed a multi-site, cross-platform study design for evaluating the analytical performance of five industry-leading ctDNA assays. The study used tailor-made reference samples at various levels of input material to assess ctDNA sequencing across 12 participating clinical and research facilities. The generated dataset encompasses multiple key variables, including a broad range of mutation frequencies, sequencing coverage depth, DNA input quantity, etc. It is the most comprehensive public-facing dataset of its kind and provides valuable insights into ultra-deep ctDNA sequencing technology. Eventually the clinical utility of ctDNA assays is required and our proficiency study and corresponding dataset are needed steps towards this goal.

Measurement(s)	Somatic Mutation • spike-in quality control role
Technology Type(s)	Tumor Panel Sequencing
Factor Type(s)	Tumor Panel • DNA Library Input Quantity • Variant Allele Frequency
Sample Characteristic - Organism	Homo sapiens

Consistency and reproducibility of large panel next-generation sequencing: Multi-laboratory assessment of somatic mutation detection on reference materials with mismatch repair and proofreading deficiency

INTRODUCTION

Clinical precision oncology increasingly relies on accurate genome-wide profiling using large panel next generation sequencing; however, difficulties in accurate and consistent detection of somatic mutation from individual platforms and pipelines remain an open question.

OBJECTIVES

To obtain paired tumor-normal reference materials that can be effectively constructed and interchangeable with clinical samples, and evaluate the performance of 56 panels under routine testing conditions based on the reference samples.

METHODS

Genes involved in mismatch repair and DNA proofreading were knocked down using the CRISPR-Cas9 technology to accumulate somatic mutations in a defined GM12878 cell line. They were used as reference materials to comprehensively evaluate the reproducibility and accuracy of detection results of oncopanels and explore the potential influencing factors.

RESULTS

In total, 14 paired tumor-normal reference DNA samples from engineered cell lines were prepared, and a reference dataset comprising 168 somatic mutations in a high-confidence region of 1.8 Mb were generated. For mutations with an allele frequency (AF) of more than 5% in reference samples, 56 panels collectively reported 1306 errors, including 729 false negatives (FNs), 179 false positives (FPs) and 398 reproducibility errors. The performance metric varied among panels with precision and recall ranging from 0.773 to 1 and 0.683 to 1, respectively. Incorrect and inadequate filtering accounted for a large proportion of false discovery (including FNs and FPs), while low-quality detection, cross-contamination and other sequencing errors during the wet bench process were other sources of FNs and FPs. In addition, low AF (<5%) considerably influenced the reproducibility and comparability among panels.

CONCLUSIONS

This study provided an integrated practice for developing reference standard to assess oncopanels in detecting somatic mutations and quantitatively revealed the source of detection errors. It will promote optimization, validation, and quality control among laboratories with potential applicability in clinical use.

Towards accurate and reliable resolution of structural variants for clinical diagnosis

Structural variants (SVs) are a major source of human genetic diversity and have been associated with different diseases and phenotypes. The detection of SVs is difficult, and a diverse range of detection methods and data analysis protocols has been developed. This difficulty and diversity make the detection of SVs for clinical applications challenging and requires a framework to ensure accuracy and reproducibility. Here, we discuss current developments in the diagnosis of SVs and propose a roadmap for the accurate and reproducible detection of SVs that includes case studies provided from the FDA-led SEquencing Quality Control Phase II (SEQC-II) and other consortium efforts.

Subclonal landscape of cancer drives resistance to immune therapy

Tumor mutation burden (TMB) is often used as a biomarker for immunogenicity and prerequisite for immune checkpoint inhibitor (ICI) therapy. However, it is becoming increasingly evident that not all tumors with high TMB respond to ICIs as expected. It has been shown that the ability of T-cells to infiltrate the tumor microenvironment and elicit a specific immune response is dependent not only on the TMB, but also on intra-tumor heterogeneity and the fraction of low-frequency subclonal mutations that make up the tumor. High intra-tumor heterogeneity leads to inefficient recognition of tumor neoantigens by T-cells due to their diluted frequency and spatial heterogeneity. Clinical studies have shown that tumors with a high degree of intra-tumor heterogeneity respond poorly to ICI therapy, and previous cytotoxic treatment may increase the intra-tumor heterogeneity and render second-line ICI therapy less effective. This paper reviews the role of ICI therapy when following chemotherapy or radiation to determine if they may be better suited as first-line therapy in patients with high TMB, low intra-tumor heterogeneity, and high PD-1, PD-L1, or CTLA-4 expression.

FDA-led consortium studies advance quality control of targeted next generation sequencing assays for precision oncology

Cancer is the second leading cause of mortality worldwide despite tremendous advances in treatment. The promise of precision oncology depends on accurate characterization of tumor mutations and subsequent therapy selection. The lack of tumor reference samples along with the associated next generation sequencing (NGS) technical assessments has hindered the development of NGS assays and the realization of benefits for precision oncology. The summarized results and recommendations of several seminal SEQC2 studies along with a vision of the changing landscape of precision oncology and anticipated next steps by the SEQC2 consortium are reported. Importantly, these studies utilized a new robust reference sample material which was developed and constructed to support multiple DNA and RNA-based NGS assay studies. These studies focused on a wide variety of precision oncology assay scenarios and provided guidelines for standardized analyses and best practice recommendations. The evolving landscape of precision oncology requires insights into critical factors supporting the sensitivity and reproducibility of clinical NGS assays for continued improvement in patient outcomes. Persistent development of robust reference materials, quantitative performance metrics, and actionable data analysis recommendations are needed. This series of SEQC2 studies serve to advance NGS-based assays for precision oncology and support regulatory science endeavors.

Advancing NGS quality control to enable measurement of actionable mutations in circulating tumor DNA

The primary objective of the FDA-led Sequencing and Quality Control Phase 2 (SEQC2) project is to develop standard analysis protocols and quality control metrics for use in DNA testing to enhance scientific research and precision medicine. This study reports a targeted next-generation sequencing (NGS) method that will enable more accurate detection of actionable mutations in circulating tumor DNA (ctDNA) clinical specimens. To accomplish this, a synthetic internal standard spike-in was designed for each actionable mutation target, suitable for use in NGS following hybrid capture enrichment and unique molecular index (UMI) or non-UMI library preparation. When mixed with contrived ctDNA reference samples, internal standards enabled calculation of technical error rate, limit of blank, and limit of detection for each variant at each nucleotide position in each sample. True-positive mutations with variant allele fraction too low for detection by current practice were detected with this method, thereby increasing sensitivity.

High-Throughput Nucleotide Resolution Predictions of Assay Limitations Increase the Reliability and Concordance of Clinical Tests

PURPOSE

The ability of next-generation sequencing (NGS) assays to interrogate thousands of genomic loci has revolutionized genetic testing. However, translation to the clinic is impeded by false-negative results that pose a risk to patients. In response, regulatory bodies are calling for reliability measures to be reported alongside NGS results. Existing methods to estimate reliability do not account for sample- and position-specific variability, which can be significant. Here, we report an approach that computes reliability metrics for every genomic position and sample interrogated by an NGS assay.

METHODS

Our approach predicts the limit of detection (LOD), the lowest reliably detectable variant fraction, by taking technical factors into account. We initially explored how LOD is affected by input material amount, library conversion rate, sequencing coverage, and sequencing error rate. This revealed that LOD depends heavily on genomic context and sample properties. Using these insights, we developed a computational approach to predict LOD on the basis of a biophysical model of the NGS workflow. We focused on targeted assays for cell-free DNA, but, in principle, this approach applies to any NGS assay.

RESULTS

We validated our approach by showing that it accurately predicts LOD and distinguishes reliable from unreliable results when screening 580 lung cancer samples for actionable mutations. Compared with a standard variant calling workflow, our approach avoided most false negatives and improved interassay concordance from 94% to 99%.

CONCLUSION

Our approach, which we name LAVA (LOD-aware variant analysis), reports the LOD for every position and sample interrogated by an NGS assay. This enables reliable results to be identified and improves the transparency and safety of genetic tests.

Evaluating the analytical validity of circulating tumor DNA sequencing assays for precision oncology

Circulating tumor DNA (ctDNA) sequencing is being rapidly adopted in precision oncology, but the accuracy, sensitivity and reproducibility of ctDNA assays is poorly understood. Here we report the findings of a multi-site, cross-platform evaluation of the analytical performance of five industry-leading ctDNA assays. We evaluated each stage of the ctDNA sequencing workflow with simulations, synthetic DNA spike-in experiments and proficiency testing on standardized, cell-line-derived reference samples. Above 0.5% variant allele frequency, ctDNA mutations were detected with high sensitivity, precision and reproducibility by all five assays, whereas, below this limit, detection became unreliable and varied widely between assays, especially when input material was limited. Missed mutations (false negatives) were more common than erroneous candidates (false positives), indicating that the reliable sampling of rare ctDNA fragments is the key challenge for ctDNA assays. This comprehensive evaluation of the analytical performance of ctDNA assays serves to inform best practice guidelines and provides a resource for precision oncology.

Cross-oncopanel study reveals high sensitivity and accuracy with overall analytical performance depending on genomic regions

BACKGROUND

Targeted sequencing using oncopanels requires comprehensive assessments of accuracy and detection sensitivity to ensure analytical validity. By employing reference materials characterized by the U.S. Food and Drug Administration-led SEquence Quality Control project phase2 (SEQC2) effort, we perform a cross-platform multi-lab evaluation of eight Pan-Cancer panels to assess best practices for oncopanel sequencing.

RESULTS

All panels demonstrate high sensitivity across targeted high-confidence coding regions and variant types for the variants previously verified to have variant allele frequency (VAF) in the 5-20% range. Sensitivity is reduced by utilizing VAF thresholds due to inherent variability in VAF measurements. Enforcing a VAF threshold for reporting has a positive impact on reducing false positive calls. Importantly, the false positive rate is found to be significantly higher outside the high-confidence coding regions, resulting in lower reproducibility. Thus, region restriction and VAF thresholds lead to low relative technical variability in estimating promising biomarkers and tumor mutational burden.

CONCLUSION

This comprehensive study provides actionable guidelines for oncopanel sequencing and clear evidence that supports a simplified approach to assess the analytical performance of oncopanels. It will facilitate the rapid implementation, validation, and quality control of oncopanels in clinical use.