An artificial intelligence approach to variant calling of ALK resistance mutations

3079

Background: ALK tyrosine kinase inhibitors (TKIs) are effective in treating advanced anaplastic lymphoma kinase (ALK) fusion-positive non-small-cell lung cancers (NSCLC), and specific ALK variants are associated with the development of resistance to specific TKIs. Humans struggle to harness the full potential of the highly complex next-generation sequencing bioinformatics pipeline output. As a consequence, the decision to report a variant remains difficult, and we considered the discrete nature of the data and the binary decision (report vs. not-report) as an ideal setting to apply an artificial intelligence (AI) approach for variant reporting. Methods: We assessed diagnostic performance of an AI model in calling ALK-resistance mutations in n = 50 consecutive ALK fusion positive patients who relapsed on TKI-therapy and underwent repeat biopsy at MGH. The random forest model was derived from independent datasets (training and validation) capturing the reporting decision on > 36,000 variants with ~500 features per variant resulting in a matrix of > 18 million data points. The model output is a contiguous prediction score from 0 (not report) to 1 (report) and a visual drill-down functionality allows exploration of the underlying features that contributed to the decision. Results: Examination of n = 76 tests from n = 50 patients with a total of n = 130 reported variants (and = 115 not reported variants) included a total of n = 31 ALK point mutations: p.1156(n = 2), p.1171(n = 8), p.1174(n = 2), p.1180(n = 2), p.1196(n = 1), p.1198(n = 1), p.1202(n = 8), p.1203(n = 1), p.1204(n = 1), p.1206(n = 1), p.1269(n = 4). Setting a screening threshold of the model at > 10% for reporting showed only one false-negative (p.Ile1171Asn) variant and 96.7% sensitivity. The average model score for ALK variants was 0.664 (range: 0.08–0.98; median 0.8) and did not show significant differences from other reported variants (0.636; 0–1; 0.7; t-test 0.66). The model would have called n = 18 of the non-reported control variants (average 0.07; range < 0.001–0.64; P < 0.0001) and was 84% specific. Review of the drill-down function identified prior call frequency, allelic ratio, and predicted transcript consequences as common model features. Importantly, the model is currently agnostic to the medical literature and does not take clinical parameters (e.g. TKI type) into account, which may further improve performance. Conclusions: Applying artificial intelligence to large discrete datasets is one approach to help identify clinically relevant variants in the setting of ALK resistance in ALK-fusion positive NSCLC.

Clinically Integrated Molecular Diagnostics in Adenoid Cystic Carcinoma

Adenoid cystic carcinoma is a rare but aggressive type of salivary gland malignancy. This article addresses the need for more effective, biomarker‐informed therapies in rare cancers, focusing on clinical utility and financial sustainability of integrated next‐generation sequencing in routine practice.

Background.

Adenoid cystic carcinoma (ACC) is an aggressive salivary gland malignancy without effective systemic therapies. Delineation of molecular profiles in ACC has led to an increased number of biomarker‐stratified clinical trials; however, the clinical utility and U.S.‐centric financial sustainability of integrated next‐generation sequencing (NGS) in routine practice has, to our knowledge, not been assessed.

Materials and Methods.

In our practice, NGS genotyping was implemented at the discretion of the primary clinician. We combined NGS‐based mutation and fusion detection, with MYB break‐apart fluorescent in situ hybridization (FISH) and MYB immunohistochemistry. Utility was defined as the fraction of patients with tumors harboring alterations that are potentially amenable to targeted therapies. Financial sustainability was assessed using the fraction of global reimbursement.

Results.

Among 181 consecutive ACC cases (2011–2018), prospective genotyping was performed in 11% (n = 20/181; n = 8 nonresectable). Testing identified 5/20 (25%) NOTCH1 aberrations, 6/20 (30%) MYB‐NFIB fusions (all confirmed by FISH), and 2/20 (10%) MYBL1‐NFIB fusions. Overall, these three alterations (MYB/MYBL1/NOTCH1) made up 65% of patients, and this subset had a more aggressive course with significantly shorter progression‐free survival. In 75% (n = 6/8) of nonresectable patients, we detected potentially actionable alterations. Financial analysis of the global charges, including NGS codes, indicated 63% reimbursement, which is in line with national (U.S.‐based) and international levels of reimbursement.

Conclusion.

Prospective routine clinical genotyping in ACC can identify clinically relevant subsets of patients and is approaching financial sustainability. Demonstrating clinical utility and financial sustainability in an orphan disease (ACC) requires a multiyear and multidimensional program.

Implications for Practice.

Delineation of molecular profiles in adenoid cystic carcinoma (ACC) has been accomplished in the research setting; however, the ability to identify relevant patient subsets in clinical practice has not been assessed. This work presents an approach to perform integrated molecular genotyping of patients with ACC with nonresectable, recurrent, or systemic disease. It was determined that 75% of nonresectable patients harbor potentially actionable alterations and that 63% of charges are reimbursed. This report outlines that orphan diseases such as ACC require a multiyear, multidimensional program to demonstrate utility in clinical practice.

Artificial Intelligence Approach for Variant Reporting

Purpose

Next-generation sequencing technologies are actively applied in clinical oncology. Bioinformatics pipeline analysis is an integral part of this process; however, humans cannot yet realize the full potential of the highly complex pipeline output. As a result, the decision to include a variant in the final report during routine clinical sign-out remains challenging.

Methods

We used an artificial intelligence approach to capture the collective clinical sign-out experience of six board-certified molecular pathologists to build and validate a decision support tool for variant reporting. We extracted all reviewed and reported variants from our clinical database and tested several machine learning models. We used 10-fold cross-validation for our variant call prediction model, which derives a contiguous prediction score from 0 to 1 (no to yes) for clinical reporting.

Results

For each of the 19,594 initial training variants, our pipeline generates approximately 500 features, which results in a matrix of > 9 million data points. From a comparison of naive Bayes, decision trees, random forests, and logistic regression models, we selected models that allow human interpretability of the prediction score. The logistic regression model demonstrated 1% false negativity and 2% false positivity. The final models' Youden indices were 0.87 and 0.77 for screening and confirmatory cutoffs, respectively. Retraining on a new assay and performance assessment in 16,123 independent variants validated our approach (Youden index, 0.93). We also derived individual pathologist-centric models (virtual consensus conference function), and a visual drill-down functionality allows assessment of how underlying features contributed to a particular score or decision branch for clinical implementation.

Conclusion

Our decision support tool for variant reporting is a practically relevant artificial intelligence approach to harness the next-generation sequencing bioinformatics pipeline output when the complexity of data interpretation exceeds human capabilities.