Results of a worldwide external quality assessment of cfDNA testing in lung Cancer

BACKGROUND

Circulating cell free DNA (cfDNA) testing of plasma for EGFR somatic variants in lung cancer patients is being widely implemented and with any new service, external quality assessment (EQA) is required to ensure patient safety. An international consortium, International Quality Network for Pathology (IQNPath), has delivered a second round of assessment to measure the accuracy of cfDNA testing for lung cancer and the interpretation of the results.

METHODS

A collaboration of five EQA provider organisations, all members of IQNPath, have delivered the assessment during 2018-19 to a total of 264 laboratories from 45 countries. Bespoke plasma reference material containing a range of EGFR mutations at varying allelic frequencies were supplied to laboratories for testing and reporting according to routine procedures. The genotyping accuracy and clinical reporting was reviewed against standardised criteria and feedback was provided to participants.

RESULTS

The overall genotyping error rate in the EQA was found to be 11.1%. Low allelic frequency samples were the most challenging and were not detected by some testing methods, resulting in critical genotyping errors. This was reflected in higher false negative rates for samples with variant allele frequencies (VAF) rates less than 1.5% compared to higher frequencies. A sample with two different EGFR mutations gave inconsistent detection of both mutations. However, for one sample, where two variants were present at a VAF of less than 1% then both mutations were correctly detected in 145/263 laboratories. Reports often did not address the risk that tumour DNA may have not been tested and limitations of the methodologies provided by participants were insufficient. This was reflected in the average interpretation score for the EQA being 1.49 out of a maximum of 2.

CONCLUSIONS

The variability in the standard of genotyping and reporting highlighted the need for EQA and educational guidance in this field to ensure the delivery of high-quality clinical services where testing of cfDNA is the only option for clinical management.

Evaluation of a worldwide EQA scheme for complex clonality analysis of clinical lymphoproliferative cases demonstrates a learning effect

Clonality analysis of immunoglobulin (IG) or T-cell receptor (TR) gene rearrangements is routine practice to assist diagnosis of lymphoid malignancies. Participation in external quality assessment (EQA) aids laboratories in identifying systematic shortcomings. The aim of this study was to evaluate laboratories' improvement in IG/TR analysis and interpretation during five EQA rounds between 2014 and 2018. Each year, participants received a total of five cases for IG and five cases for TR testing. Paper-based cases were included for analysis of the final molecular conclusion that should be interpreted based on the integration of the individual PCR results. Wet cases were distributed for analysis of their routine protocol as well as evaluation of the final molecular conclusion. In total, 94.9% (506/533) of wet tests and 97.9% (829/847) of paper tests were correctly analyzed for IG, and 96.8% (507/524) wet tests and 93.2% (765/821) paper tests were correctly analyzed for TR. Analysis scores significantly improved when laboratories participated to more EQA rounds (p=0.001). Overall performance was significantly lower (p=0.008) for non-EuroClonality laboratories (95% for IG and 93% for TR) compared to EuroClonality laboratories (99% for IG and 97% for TR). The difference was not related to the EQA scheme year, anatomic origin of the sample, or final clinical diagnosis. This evaluation showed that repeated EQA participation helps to reduce performance differences between laboratories (EuroClonality versus non-EuroClonality) and between sample types (paper versus wet). The difficulties in interpreting oligoclonal cases highlighted the need for continued education by meetings and EQA schemes.

Biomarker testing in oncology - Requirements for organizing external quality assessment programs to improve the performance of laboratory testing: revision of an expert opinion paper on behalf of IQNPath ABSL

In personalized medicine, predictive biomarker testing is the basis for an appropriate choice of therapy for patients with cancer. An important tool for laboratories to ensure accurate results is participation in external quality assurance (EQA) programs. Several providers offer predictive EQA programs for different cancer types, test methods, and sample types. In 2013, a guideline was published on the requirements for organizing high-quality EQA programs in molecular pathology. Now, after six years, steps were taken to further harmonize these EQA programs as an initiative by IQNPath ABSL, an umbrella organization founded by various EQA providers. This revision is based on current knowledge, adds recommendations for programs developed for predictive biomarkers by in situ methodologies (immunohistochemistry and in situ hybridization), and emphasized transparency and an evidence-based approach. In addition, this updated version also has the aim to give an overview of current practices from various EQA providers.

Staining Performance of ALK and ROS1 Immunohistochemistry and Influence on Interpretation in Non-Small-Cell Lung Cancer

Selection of non-small-cell lung cancer patients for treatment relies on the detection of expression of anaplastic lymphoma kinase (ALK) and ROS proto-oncogene 1 (ROS1) protein by immunohistochemistry (IHC). We evaluated staining performance for different IHC protocols and laboratory characteristics, and their influence on ALK and ROS1 interpretation during external quality assessment schemes between 2015 and 2018. Participants received five formalin-fixed, paraffin-embedded cases for staining by their routine protocol, whereafter at least two pathologists scored them simultaneously under a multihead microscope and awarded a graded expert staining score (ESS) from 1 to 5 points based on staining quality. European Conformity in Vitro Diagnostic kits (such as D5F3) revealed a better ALK ESS compared with laboratory-developed tests. ESS was indifferent to the applied antibody dilution or a recent protocol change. Lower ESSs were observed for higher antibody incubation times and temperatures. ESS for various ROS1 protocols were largely similar. Overall, for both markers, ESS improved over time and for repeated external quality assessment participation but was independent of laboratory setting or experience. Except for ROS1, ESS positively correlated with laboratory accreditation. IHC stains with lower ESS correlated with increased error rates in ALK and ROS1 interpretation and analysis failures. Laboratory characteristics differently affected staining quality and interpretation, and laboratories should assess both aspects, and less common protocols need improvement in staining performance.

The new IVD Regulation 2017/746: a case study at a large university hospital laboratory in Belgium demonstrates the need for clarification on the degrees of freedom laboratories have to use lab-developed tests to improve patient care

Objectives

The new European In Vitro Diagnostic (IVD) Regulation 2017/746 (IVDR) restricts the use of lab-developed tests (LDT) after 26th May 2022. There are no data on the impact of the IVDR on laboratories in the European Union.

Methods

Laboratory tests performed in UZ Leuven were divided in four groups: core laboratory, immunology, special chemistry, and molecular microbiology testing. Each test was classified as Conformité Européenne (CE)-IVD, modified/off-label CE-IVD, commercial Research Use Only (RUO) or LDT. Each matrix was considered a separate test.

Results

We found that 97.6% of the more than 11.5 million results/year were generated with a CE-IVD method. Of the 922 different laboratory tests, however, only 41.8% were CE-IVD, 10.8% modified/off-label CE-IVD, 0.3% RUO, and 47.1% LDT. Off-label CE-IVD was mainly used to test alternative matrices not covered by the claim of the manufacturer (e.g., pleural or peritoneal fluid). LDTs were mainly used for special chemistry, flow cytometry, and molecular testing. Excluding flow cytometry, the main reasons for the use of 377 LDTs were lack of a CE-IVD method (71.9%), analytical requirements (14.3%), and the fact the LDT was in use before CE-IVD available (11.9%).

Conclusions

While the large majority of results (97.6%) were generated with a CE-IVD method, only 41.8% of laboratory tests were CE-IVD. There is currently no alternative on the market for 71.5% of the 537 LDTs performed in our laboratory which do not fall within the scope of the current IVD directive (IVDD). Compliance with the IVDR will require a major investment of time and effort.

Laboratory reporting on the clinical spectrum of CFTR p.Arg117His: Still room for improvement

BACKGROUND

The clinical spectrum associated with cystic fibrosis transmembrane conductance regulator (CFTR) variant p.Arg117His is highly variable, ranging from full-blown cystic fibrosis (CF) in a small number of cases to CFTR-related disorders (CFTR-RDs) or no symptoms at all. Therefore, taking into account phenotype variability is essential for interpretation. External quality assessment (EQA) schemes can help laboratories to objectively assess the quality of genotyping and reporting by the laboratory.

METHODS

We performed a retrospective longitudinal data analysis on laboratory performance regarding the interpretation of p.Arg117His during CF EQA scheme participation. Completeness and accuracy of reporting on two mock clinical cases were each compared over time (case 1: 2005, 2007 and 2012; case 2: 2015 and 2018). These cases concerned subjects compound heterozygous for p.Phe508del and p.Arg117His in cis with 7T, but with different clinical backgrounds (family planning (case 1) versus diagnostic testing for a child (case 2)). Furthermore, we analyzed the influence of previous participations, annual test volume, accreditation status and laboratory setting on overall performance.

RESULTS

Overall performance improved over time, except during the 2007 CF EQA scheme. In addition, previous participations had a beneficial effect on laboratory performance. Accreditation status, annual test volume and laboratory setting did not significantly influence total interpretation scores.

CONCLUSIONS

In general, laboratories performed well on both cases, although reporting on the variable clinical spectrum of p.Arg117His in cis with 7T and on the disease liability of individual CFTR variants can still improve. Moreover, this study underlined the educational role of CF EQA schemes.

Sensitive detection methods are key to identify secondary EGFR c.2369C>T p.(Thr790Met) in non-small cell lung cancer tissue samples

Background

Correct identification of the EGFR c.2369C>T p.(Thr790Met) variant is key to decide on a targeted therapeutic strategy for patients with acquired EGFR TKI resistance in non-small cell lung cancer. The aim of this study was to evaluate the correct detection of this variant in 12 tumor tissue specimens tested by 324 laboratories participating in External Quality Assessment (EQA) schemes.

Methods

Data from EQA schemes were evaluated between 2013 and 2018 from cell lines (6) and resections (6) containing the EGFR c.2369C>T p.(Thr790Met) mutation. Adequate performance was defined as the percentage of tests for which an outcome was available and correct. Additional data on the used test method were collected from the participants. Chi-squared tests on contingency tables and a biserial rank correlation were applied by IBM SPSS Statistics version 25 (IBM, Armonk, NY, USA).

Results

In 26 of the 1190 tests (2.2%) a technical failure occurred. For the remaining 1164 results, 1008 (86.6%) were correct, 151 (12.9%) were false-negative and 5 (0.4%) included incorrect mutations. Correct p.(Thr790Met) detection improved over time and for repeated scheme participations. In-house non-next-generation sequencing (NGS) techniques performed worse (81.1%, n = 293) compared to non-NGS commercial kits (85.2%, n = 656) and NGS (97.0%, n = 239). Over time there was an increase in the users of NGS. Resection specimens performed worse (82.6%, n = 610 tests) compared to cell line material (90.9%, n = 578 tests), except for NGS (96.3%, n = 344 for resections and 98.6%, n = 312 for cell lines). Samples with multiple mutations were more difficult compared to samples with the single p.(Thr790Met) variant. A change of the test method was shown beneficial to reduce errors but introduced additional analysis failures.

Conclusions

A significant number of laboratories that offer p.(Thr790Met) testing did not detect this relevant mutation compared to the other EQA participants. However, correct identification of this variant is improving over time and was higher for NGS users. Revising the methodology might be useful to resolve errors, especially for resection specimens with low frequency or multiple variants. EQA providers should include challenging resections in the scheme.

Variation in nomenclature of somatic variants for selection of oncological therapies: Can we reach a consensus soon?

A standardized nomenclature for reporting oncology biomarker variants is key to avoid misinterpretation of results and unambiguous registration in clinical databases. External quality assessment (EQA) schemes have revealed a need for more consistent nomenclature use in clinical genetics. We evaluated the propensity of EQA for improvement of compliance with Human Genome Variation Society (HGVS) recommendations for reporting of predictive somatic variants in lung and colorectal cancer. Variant entries between 2012 and 2018 were collected from written reports and electronic results sheets. In total, 4,053 variants were assessed, of which 12.1% complied with HGVS recommendations. Compliance improved over time from 2.1% (2012) to 22.3% (2018), especially when laboratories participated in multiple EQA schemes. Compliance was better for next-generation sequencing (20.9%) compared with targeted techniques (9.8%). In the 1792 reports, HGVS recommendations for reference sequences were met for 31.9% of reports, for 36.0% of noncommercial, and 26.5% of commercial test methods. Compliance improved from 16.7% (2012) to 33.1% (2018), and after repeated EQA participation. EQA participation improves compliance with HGVS recommendations. The residual percentage of errors in the most recent schemes suggests that laboratories, companies, and EQA providers need to collaborate for additional improvement of harmonization in clinical test reporting.

IQN path ASBL report from the first European cfDNA consensus meeting: expert opinion on the minimal requirements for clinical ctDNA testing

Liquid biopsy testing is a new laboratory-based method that detects tumour mutations in circulating free DNA (cfDNA) derived from minimally invasive blood sampling techniques. Recognising the significance for clinical testing, in 2017, IQN Path provided external quality assessment for liquid biopsy testing. Representatives of those participating laboratories were invited to attend a workshop to discuss the findings and how to achieve quality implementation of cfDNA testing in the clinical setting, the discussion and outcomes of this consensus meeting are described below. Predictive molecular profiling using tumour tissue in order to select cancer patients eligible for targeted therapy is now routine in diagnostic pathology. If insufficient tumour tissue material is available, in some circumstances, recent European Medicines Agency (EMA) guidance recommends mutation testing with plasma cfDNA. Clinical applications of cfDNA include treatment selection based on clinically relevant mutations derived from pre-treatment samples and the detection of resistant mutations upon progression of the disease. In order to identify tumour-related mutations in amongst other nucleic acid material found in plasma samples, highly sensitive laboratory methods are needed. In the workshop, we discussed the variable approaches taken with regard to cfDNA extraction methods, the tests, and considered the impact of false-negative test results. We explored the lack of standardisation of complex testing procedures ranging from plasma collection, transport, processing and storage, cfDNA extraction, and mutation analysis, to interpretation and reporting of results. We will also address the current status of clinical validation and clinical utility, and its use in current diagnosis. This workshop revealed a need for guidelines on with standardised procedures for clinical cfDNA testing and reporting, and a requirement for cfDNA-based external quality assessment programs.

European follow-up of incorrect biomarker results for colorectal cancer demonstrates the importance of quality improvement projects

Biomarker analysis for colorectal cancer has been shown to be reliable in Europe with 97% of samples tested by EQA participants to be correctly classified. This study focuses on errors during the annual EQA assessment. The aim was to explore the causes and actions related to the observed errors and to provide feedback and assess any improvement between 2016 and 2017. An electronic survey was sent to all laboratories with minimum one genotyping error or technical failure on ten tumor samples. A workshop was organized based on 2016 survey responses. Improvement of performance in 2017 was assessed for returning participants (n = 76), survey respondents (n = 13) and workshop participants (n = 4). Survey respondents and workshop participants improved in terms of (maximum) analysis score, successful participation, and genotyping errors compared to all returning participants. In 2016, mostly pre- and post-analytical errors (both 25%) were observed caused by unsuitability of the tumor tissue for molecular analysis. In 2017, most errors were due to analytical problems (50.0%) caused by methodological problems. The most common actions taken (n = 58) were protocol revisions (34.5%) and staff training (15.5%). In 24.1% of issues identified no action was performed. Corrective actions were linked to an improved performance, especially if performed by the pathologist. Although biomarker testing has improved over time, error occurrence at different phases stresses the need for quality improvement throughout the test process. Participation to quality improvement projects and a close collaboration with the pathologist can have a positive influence on performance.

Accreditation, setting and experience as indicators to assure quality in oncology biomarker testing laboratories

BACKGROUND

Predictive biomarkers allow clinicians to optimise cancer treatment decisions. Therefore, molecular biomarker test results need to be accurate and swiftly available. To ensure quality of oncology biomarker testing, external quality assessments (EQA) for somatic variant analyses were organised. This study hypothesised whether laboratory characteristics influence the performance of laboratories and whether these can be imposed before authorisation of biomarker testing.

METHODS

Longitudinal EQA data from the European Society of Pathology were available over six (metastatic colorectal cancer) and four years (non-small cell lung cancer), including the percentage of analysis errors and technical failures, and information on laboratory characteristics (accreditation status, laboratory setting, number of samples analysed and detection method). Statistical models for repeated measurements were used to analyse the association between the EQA results and the laboratory characteristics.

RESULTS

Laboratory accreditation was associated with fewer analysis errors in early stages of biomarker introduction into the laboratory. Analysing more samples, or university and research laboratories showed better performance. Changing the detection method did not have an effect.

CONCLUSION

The indicators support the clinicians in choosing molecular pathology laboratories by improving quality assurance and guaranteeing patient safety. Accreditation of laboratories, centralisation of biomarker testing or a university and research setting should be stimulated.

Describing the Reportable Range Is Important for Reliable Treatment Decisions: A Multiple Laboratory Study for Molecular Tumor Profiling Using Next-Generation Sequencing

Because interpretation of next-generation sequencing (NGS) data remains challenging, optimization of the NGS process is needed to obtain correct sequencing results. Therefore, extensive validation and continuous monitoring of the quality is essential. NGS performance was compared with traditional detection methods and technical quality of nine NGS technologies was assessed. First, nine formalin-fixed, paraffin-embedded patient samples were analyzed by 114 laboratories by using different detection methods. No significant differences in performance were observed between analyses with NGS and traditional techniques. Second, two DNA control samples were analyzed for a selected number of variants by 26 participants with the use of nine different NGS technologies. Quality control metrics were analyzed from raw data files and a survey about routine procedures. Results showed large differences in coverages, but observed variant allele frequencies in raw data files were in line with predefined variant allele frequencies. Many false negative results were found because of low-quality regions, which were not reported as such. It is recommended to disclose the reportable range, the fraction of targeted genomic regions for which calls of acceptable quality can be generated, to avoid any errors in therapy decisions. NGS can be a reliable technique, only if essential quality control during analysis is applied and reported.

External Quality Assessment Identifies Training Needs to Determine the Neoplastic Cell Content for Biomarker Testing

Neoplastic cell content determination is crucial for biomarker testing. It is known that interobserver variation exists, but largescale data are missing about variation in tumor delineation and cell content determination. Results were obtained from the external quality assessment program for metastatic colorectal cancer from the European Society of Pathology (N = 5776 observations). The study included three parts: current practices were surveyed, neoplastic cell content estimations and delineations were retrieved from stained slides, and clinical reports were analyzed. Seventeen of 43 pathologists determined the neoplastic cell content in a tumor-rich area for DNA extraction and took immune cells (n = 37), tumor cell distribution (n = 33), desmoplastic stroma (n = 30), necrosis (n = 29), and mucus (n = 23) into account. The selected area was highly variable, and the average difference between the highest and lowest estimation ranged between 51% and 78% (2011 to 2017). The number of overestimations was alarmingly high in samples containing <30% tumor cells. Of concern is that 33 of 105 laboratories reported a wild-type result in a sample without tumor in 2017. Standardization of neoplastic cell content determination is needed for test outcome interpretation. The authors' data show variation in estimation practices, tumor delineations and estimations, and interpretation problems (n = 226 reports). Further training for selecting the most suitable block and creating clear reports is urgently needed.

The ins and outs of molecular pathology reporting

The raid evolution in molecular pathology resulting in an increasing complexity requires careful reporting. The need for standardisation is clearer than ever. While synoptic reporting was first used for reporting hereditary genetic diseases, it is becoming more frequent in pathology, especially molecular pathology reports too. The narrative approach is no longer feasible with the growing amount of essential data present on the report, although narrative components are still necessary for interpretation in molecular pathology. On the way towards standardisation of reports, guidelines can be a helpful tool. There are several guidelines that focus on reporting in the field of hereditary diseases, but it is not always feasible to extrapolate these to the reporting of somatic variants in molecular pathology. The rise of multi-gene testing causes challenges for the laboratories. In order to provide a continuous optimisation of the laboratory testing process, including reporting, external quality assessment is essential and has already proven to improve the quality of reports. In general, a clear and concise report for molecular pathology can be created by including elements deemed important by different guidelines, adapting the report to the process flows of the laboratory and integrating the report with the laboratory information management system and the patient record.

External quality assessment unravels interlaboratory differences in quality of RAS testing for anti-EGFR therapy in colorectal cancer

BACKGROUND

Regulations for the selection of patients with metastatic colorectal cancer for anti-EGFR treatment changed at the end of 2013. The set of mutations to be tested extended from KRAS codons 12 and 13 to KRAS and NRAS exons 2, 3, and 4. A European external quality assessment scheme monitored the performance of laboratories and evaluated the implementation of the new regulations.

MATERIALS AND METHODS

The 131 participating laboratories received 10 samples of formalin-fixed paraffin-embedded material, including RAS (exon 2, 3, 4) and BRAF mutations. Mock clinical data were provided for three cases. Using their routine methods, laboratories determined the genotypes and submitted three written reports. Assessors scored the results according to predefined evaluation criteria.

RESULTS

Half of the participants (49.3%) had completely implemented the new test requirements (codons 12, 13, 59, 61, 117, and 146 of KRAS and NRAS), and 96 laboratories (73.3%) made no genotype mistakes. Correct nomenclature, according to the Human Genome Variation Society, was used by 82 laboratories (62.6%).

CONCLUSION

Although regulations were effective for several months, many laboratories were not ready for full RAS testing in the context of anti-EGFR therapy. Nevertheless, in each participating country, there are laboratories that provide complete and correct testing. External quality assessments can be used to monitor implementation of new test regulations and to stimulate the laboratories to improve their testing procedures. Because the results of this program are available on the website of the European Society of Pathology, patients and clinicians can refer test samples to a reliable laboratory.

The relevance of external quality assessment for molecular testing for ALK positive non-small cell lung cancer: results from two pilot rounds show room for optimization

BACKGROUND AND PURPOSE

Molecular profiling should be performed on all advanced non-small cell lung cancer with non-squamous histology to allow treatment selection. Currently, this should include EGFR mutation testing and testing for ALK rearrangements. ROS1 is another emerging target. ALK rearrangement status is a critical biomarker to predict response to tyrosine kinase inhibitors such as crizotinib. To promote high quality testing in non-small cell lung cancer, the European Society of Pathology has introduced an external quality assessment scheme. This article summarizes the results of the first two pilot rounds organized in 2012-2013.

MATERIALS AND METHODS

Tissue microarray slides consisting of cell-lines and resection specimens were distributed with the request for routine ALK testing using IHC or FISH. Participation in ALK FISH testing included the interpretation of four digital FISH images.

RESULTS

Data from 173 different laboratories was obtained. Results demonstrate decreased error rates in the second round for both ALK FISH and ALK IHC, although the error rates were still high and the need for external quality assessment in laboratories performing ALK testing is evident. Error rates obtained by FISH were lower than by IHC. The lowest error rates were observed for the interpretation of digital FISH images.

CONCLUSION

There was a large variety in FISH enumeration practices. Based on the results from this study, recommendations for the methodology, analysis, interpretation and result reporting were issued. External quality assessment is a crucial element to improve the quality of molecular testing.

Higher quality of molecular testing, an unfulfilled priority: Results from external quality assessment for KRAS mutation testing in colorectal cancer

Precision medicine is now a key element in clinical oncology. RAS mutational status is a crucial predictor of responsiveness to anti-epidermal growth factor receptor agents in metastatic colorectal cancer. In an effort to guarantee high-quality testing services in molecular pathology, the European Society of Pathology has been organizing an annual KRAS external quality assessment program since 2009. In 2012, 10 formalin-fixed, paraffin-embedded samples, of which 8 from invasive metastatic colorectal cancer tissue and 2 artificial samples of cell line material, were sent to more than 100 laboratories from 26 countries with a request for routine KRAS testing. Both genotyping and clinical reports were assessed independently. Twenty-seven percent of the participants genotyped at least 1 of 10 samples incorrectly. In total, less than 5% of the distributed specimens were genotyped incorrectly. Genotyping errors consisted of false negatives, false positives, and incorrectly genotyped mutations. Twenty percent of the laboratories reported a technical error for one or more samples. A review of the written reports showed that several essential elements were missing, most notably a clinical interpretation of the test result, the method sensitivity, and the use of a reference sequence. External quality assessment serves as a valuable educational tool in assessing and improving molecular testing quality and is an important asset for monitoring quality assurance upon incorporation of new biomarkers in diagnostic services.