A validated artificial intelligence-based pipeline for population-wide primary immunodeficiency screening

Artificial Intelligence and Machine Learning for Inborn Errors of Immunity: Current State and Future Promise

Artificial intelligence (AI) and machine learning (ML) research within medicine has exponentially increased over the last decade, with studies showcasing the potential of AI/ML algorithms to improve clinical practice and outcomes. Ongoing research and efforts to develop AI-based models have expanded to aid in the identification of inborn errors of immunity (IEI). The use of larger electronic health record data sets, coupled with advances in phenotyping precision and enhancements in ML techniques, has the potential to significantly improve the early recognition of IEI, thereby increasing access to equitable care. In this review, we provide a comprehensive examination of AI/ML for IEI, covering the spectrum from data preprocessing for AI/ML analysis to current applications within immunology, and address the challenges associated with implementing clinical decision support systems to refine the diagnosis and management of IEI.

Charting a course for global progress in PIDs by 2030 - proceedings from the IPOPI global multi-stakeholders' summit (September 2023)

The International Patient Organisation for Primary Immunodeficiencies (IPOPI) held its second Global Multi-Stakeholders' Summit, an annual stimulating and forward-thinking meeting uniting experts to anticipate pivotal upcoming challenges and opportunities in the field of primary immunodeficiency (PID). The 2023 summit focused on three key identified discussion points: (i) How can immunoglobulin (Ig) therapy meet future personalized patient needs? (ii) Pandemic preparedness: what's next for public health and potential challenges for the PID community? (iii) Diagnosing PIDs in 2030: what needs to happen to diagnose better and to diagnose more? Clinician-Scientists, patient representatives and other stakeholders explored avenues to improve Ig therapy through mechanistic insights and tailored Ig preparations/products according to patient-specific needs and local exposure to infectious agents, amongst others. Urgency for pandemic preparedness was discussed, as was the threat of shortage of antibiotics and increasing antimicrobial resistance, emphasizing the need for representation of PID patients and other vulnerable populations throughout crisis and care management. Discussion also covered the complexities of PID diagnosis, addressing issues such as global diagnostic disparities, the integration of patient-reported outcome measures, and the potential of artificial intelligence to increase PID diagnosis rates and to enhance diagnostic precision. These proceedings outline the outcomes and recommendations arising from the 2023 IPOPI Global Multi-Stakeholders' Summit, offering valuable insights to inform future strategies in PID management and care. Integral to this initiative is its role in fostering collaborative efforts among stakeholders to prepare for the multiple challenges facing the global PID community.

Validating inborn error of immunity prevalence and risk with nationally representative electronic health record data

BACKGROUND

The 10 Warning Signs of Primary Immunodeficiency were created 30 years ago to advance recognition of inborn errors of immunity (IEI). However, no population-level assessment of their utility applied to electronic health record (EHR) data has been conducted.

OBJECTIVE

We sought to quantify the value of having ≥2 warning signs (WS) toward diagnosing IEI using a highly representative real-world US cohort. A secondary goal was estimating the US prevalence of IEI.

METHODS

In this cohort study, we accessed normalized and de-identified EHR data on 152 million US patients. An IEI cohort (n = 41,080), in which patients were defined by having at least 1 verifiable IEI diagnosis placed ≥2 times in their record, was compared with a matched set of controls (n = 250,262). WS were encoded along with relevant diagnoses, relative weights were calculated, and the proportion of IEI cases versus controls with ≥2 WS was compared.

RESULTS

The proportion of IEI cases with ≥2 WS significantly differed from controls (0.33 vs 0.031; P < .0005, χ2 test). We also estimated a US IEI prevalence of 6 per 10,000 individuals (41,080/73,165,655; 0.056%). WS 9 (≥2 deep-seated infections), 7 (fungal infections), 5 (failure to thrive) and 4 (≥2 pneumonias in 1 year) were the most heavily weighted among the IEI cohort.

CONCLUSIONS

This nationally representative US-based cohort study demonstrates that presence of WS and associated clinical diagnoses can facilitate identification of patients with IEI from EHR data. In addition, we estimate that 6 in 10,000, or approximately 150,000 to 200,000 individuals are affected by IEI across the United States.

Electronic health record signatures identify undiagnosed patients with common variable immunodeficiency disease

Human inborn errors of immunity include rare disorders entailing functional and quantitative antibody deficiencies due to impaired B cells called the common variable immunodeficiency (CVID) phenotype. Patients with CVID face delayed diagnoses and treatments for 5 to 15 years after symptom onset because the disorders are rare (prevalence of ~1/25,000), and there is extensive heterogeneity in CVID phenotypes, ranging from infections to autoimmunity to inflammatory conditions, overlapping with other more common disorders. The prolonged diagnostic odyssey drives excessive system-wide costs before diagnosis. Because there is no single causal mechanism, there are no genetic tests to definitively diagnose CVID. Here, we present PheNet, a machine learning algorithm that identifies patients with CVID from their electronic health records (EHRs). PheNet learns phenotypic patterns from verified CVID cases and uses this knowledge to rank patients by likelihood of having CVID. PheNet could have diagnosed more than half of our patients with CVID 1 or more years earlier than they had been diagnosed. When applied to a large EHR dataset, followed by blinded chart review of the top 100 patients ranked by PheNet, we found that 74% were highly probable to have CVID. We externally validated PheNet using >6 million records from disparate medical systems in California and Tennessee. As artificial intelligence and machine learning make their way into health care, we show that algorithms such as PheNet can offer clinical benefits by expediting the diagnosis of rare diseases.

Natural language processing of clinical notes enables early inborn error of immunity risk ascertainment

Background

There are now approximately 450 discrete inborn errors of immunity (IEI) described; however, diagnostic rates remain suboptimal. Use of structured health record data has proven useful for patient detection but may be augmented by natural language processing (NLP). Here we present a machine learning model that can distinguish patients from controls significantly in advance of ultimate diagnosis date.

Objective

We sought to create an NLP machine learning algorithm that could identify IEI patients early during the disease course and shorten the diagnostic odyssey.

Methods

Our approach involved extracting a large corpus of IEI patient clinical-note text from a major referral center's electronic health record (EHR) system and a matched control corpus for comparison. We built text classifiers with simple machine learning methods and trained them on progressively longer time epochs before date of diagnosis.

Results

The top performing NLP algorithm effectively distinguished cases from controls robustly 36 months before ultimate clinical diagnosis (area under precision recall curve > 0.95). Corpus analysis demonstrated that statistically enriched, IEI-relevant terms were evident 24+ months before diagnosis, validating that clinical notes can provide a signal for early prediction of IEI.

Conclusion

Mining EHR notes with NLP holds promise for improving early IEI patient detection.

Current genetic diagnostics in inborn errors of immunity

New technologies in genetic diagnostics have revolutionized the understanding and management of rare diseases. This review highlights the significant advances and latest developments in genetic diagnostics in inborn errors of immunity (IEI), which encompass a diverse group of disorders characterized by defects in the immune system, leading to increased susceptibility to infections, autoimmunity, autoinflammatory diseases, allergies, and malignancies. Various diagnostic approaches, including targeted gene sequencing panels, whole exome sequencing, whole genome sequencing, RNA sequencing, or proteomics, have enabled the identification of causative genetic variants of rare diseases. These technologies not only facilitated the accurate diagnosis of IEI but also provided valuable insights into the underlying molecular mechanisms. Emerging technologies, currently mainly used in research, such as optical genome mapping, single cell sequencing or the application of artificial intelligence will allow even more insights in the aetiology of hereditary immune defects in the near future. The integration of genetic diagnostics into clinical practice significantly impacts patient care. Genetic testing enables early diagnosis, facilitating timely interventions and personalized treatment strategies. Additionally, establishing a genetic diagnosis is necessary for genetic counselling and prognostic assessments. Identifying specific genetic variants associated with inborn errors of immunity also paved the way for the development of targeted therapies and novel therapeutic approaches. This review emphasizes the challenges related with genetic diagnosis of rare diseases and provides future directions, specifically focusing on IEI. Despite the tremendous progress achieved over the last years, several obstacles remain or have become even more important due to the increasing amount of genetic data produced for each patient. This includes, first and foremost, the interpretation of variants of unknown significance (VUS) in known IEI genes and of variants in genes of unknown significance (GUS). Although genetic diagnostics have significantly contributed to the understanding and management of IEI and other rare diseases, further research, exchange between experts from different clinical disciplines, data integration and the establishment of comprehensive guidelines are crucial to tackle the remaining challenges and maximize the potential of genetic diagnostics in the field of rare diseases, such as IEI.

Proceedings from the inaugural Artificial Intelligence in Primary Immune Deficiencies (AIPID) conference

Here, we summarize the proceedings of the inaugural Artificial Intelligence in Primary Immune Deficiencies conference, during which experts and advocates gathered to advance research into the applications of artificial intelligence (AI), machine learning, and other computational tools in the diagnosis and management of inborn errors of immunity (IEIs). The conference focused on the key themes of expediting IEI diagnoses, challenges in data collection, roles of natural language processing and large language models in interpreting electronic health records, and ethical considerations in implementation. Innovative AI-based tools trained on electronic health records and claims databases have discovered new patterns of warning signs for IEIs, facilitating faster diagnoses and enhancing patient outcomes. Challenges in training AIs persist on account of data limitations, especially in cases of rare diseases, overlapping phenotypes, and biases inherent in current data sets. Furthermore, experts highlighted the significance of ethical considerations, data protection, and the necessity for open science principles. The conference delved into regulatory frameworks, equity in access, and the imperative for collaborative efforts to overcome these obstacles and harness the transformative potential of AI. Concerted efforts to successfully integrate AI into daily clinical immunology practice are still needed.

Clinical Validation of a Primary Antibody Deficiency Screening Algorithm for Primary Care

PURPOSE

The diagnostic delay of primary antibody deficiencies (PADs) is associated with increased morbidity, mortality, and healthcare costs. Therefore, a screening algorithm was previously developed for the early detection of patients at risk of PAD in primary care. We aimed to clinically validate and optimize the PAD screening algorithm by applying it to a primary care database in the Netherlands.

METHODS

The algorithm was applied to a data set of 61,172 electronic health records (EHRs). Four hundred high-scoring EHRs were screened for exclusion criteria, and remaining patients were invited for serum immunoglobulin analysis and referred if clinically necessary.

RESULTS

Of the 104 patients eligible for inclusion, 16 were referred by their general practitioner for suspected PAD, of whom 10 had a PAD diagnosis. In patients selected by the screening algorithm and included for laboratory analysis, prevalence of PAD was ~ 1:10 versus 1:1700-1:25,000 in the general population. To optimize efficiency of the screening process, we refitted the algorithm with the subset of high-risk patients, which improved the area under the curve-receiver operating characteristics curve value to 0.80 (95% confidence interval 0.63-0.97). We propose a two-step screening process, first applying the original algorithm to distinguish high-risk from low-risk patients, then applying the optimized algorithm to select high-risk patients for serum immunoglobulin analysis.

CONCLUSION

Using the screening algorithm, we were able to identify 10 new PAD patients from a primary care population, thus reducing diagnostic delay. Future studies should address further validation in other populations and full cost-effectiveness analyses.

REGISTRATION

Clinicaltrials.gov record number NCT05310604, first submitted 25 March 2022.

Who's your data? Primary immune deficiency differential diagnosis prediction via machine learning and data mining of the USIDNET registry

PURPOSE

There are currently more than 480 primary immune deficiency (PID) diseases and about 7000 rare diseases that together afflict around 1 in every 17 humans. Computational aids based on data mining and machine learning might facilitate the diagnostic task by extracting rules from large datasets and making predictions when faced with new problem cases. In a proof-of-concept data mining study, we aimed to predict PID diagnoses using a supervised machine learning algorithm based on classification tree boosting.

METHODS

Through a data query at the USIDNET registry we obtained a database of 2396 patients with common diagnoses of PID, including their clinical and laboratory features. We kept 286 features and all 12 diagnoses to include in the model. We used the XGBoost package with parallel tree boosting for the supervised classification model, and SHAP for variable importance interpretation, on Python v3.7. The patient database was split into training and testing subsets, and after boosting through gradient descent, the predictive model provides measures of diagnostic prediction accuracy and individual feature importance. After a baseline performance test, we used the Class Weighting Hyperparameter, or scale_pos_weight to correct for imbalanced classification.

RESULTS

The twelve PID diagnoses were CVID (1098 patients), DiGeorge syndrome, Chronic granulomatous disease, Congenital agammaglobulinemia, PID not otherwise classified, Specific antibody deficiency, Complement deficiency, Hyper-IgM, Leukocyte adhesion deficiency, ectodermal dysplasia with immune deficiency, Severe combined immune deficiency, and Wiskott-Aldrich syndrome. For CVID, the model found an accuracy on the train sample of 0.80, with an area under the ROC curve (AUC) of 0.80, and a Gini coefficient of 0.60. In the test subset, accuracy was 0.76, AUC 0.75, and Gini 0.51. The positive feature value to predict CVID was highest for upper respiratory infections, asthma, autoimmunity and hypogammaglobulinemia. Features with the highest negative predictive value were high IgE, growth delay, abscess, lymphopenia, and congenital heart disease. For the rest of the diagnoses, accuracy stayed between 0.75 and 0.99, AUC 0.46-0.87, Gini 0.07-0.75, and LogLoss 0.09-8.55.

DISCUSSION

Clinicians should remember to consider the negative predictive features together with the positives. We are calling this a proof-of-concept study to continue with our explorations. A good performance is encouraging, and feature importance might aid feature selection for future endeavors. In the meantime, we can learn from the rules derived by the model and build a user-friendly decision tree to generate differential diagnoses.