Systematic review of clinical prediction models to support the diagnosis of asthma in primary care

Machine Learning-Based Asthma Attack Prediction Models From Routinely Collected Electronic Health Records: Systematic Scoping Review

BACKGROUND

An early warning tool to predict attacks could enhance asthma management and reduce the likelihood of serious consequences. Electronic health records (EHRs) providing access to historical data about patients with asthma coupled with machine learning (ML) provide an opportunity to develop such a tool. Several studies have developed ML-based tools to predict asthma attacks.

OBJECTIVE

This study aims to critically evaluate ML-based models derived using EHRs for the prediction of asthma attacks.

METHODS

We systematically searched PubMed and Scopus (the search period was between January 1, 2012, and January 31, 2023) for papers meeting the following inclusion criteria: (1) used EHR data as the main data source, (2) used asthma attack as the outcome, and (3) compared ML-based prediction models' performance. We excluded non-English papers and nonresearch papers, such as commentary and systematic review papers. In addition, we also excluded papers that did not provide any details about the respective ML approach and its result, including protocol papers. The selected studies were then summarized across multiple dimensions including data preprocessing methods, ML algorithms, model validation, model explainability, and model implementation.

RESULTS

Overall, 17 papers were included at the end of the selection process. There was considerable heterogeneity in how asthma attacks were defined. Of the 17 studies, 8 (47%) studies used routinely collected data both from primary care and secondary care practices together. Extreme imbalanced data was a notable issue in most studies (13/17, 76%), but only 38% (5/13) of them explicitly dealt with it in their data preprocessing pipeline. The gradient boosting-based method was the best ML method in 59% (10/17) of the studies. Of the 17 studies, 14 (82%) studies used a model explanation method to identify the most important predictors. None of the studies followed the standard reporting guidelines, and none were prospectively validated.

CONCLUSIONS

Our review indicates that this research field is still underdeveloped, given the limited body of evidence, heterogeneity of methods, lack of external validation, and suboptimally reported models. We highlighted several technical challenges (class imbalance, external validation, model explanation, and adherence to reporting guidelines to aid reproducibility) that need to be addressed to make progress toward clinical adoption.

Deriving and validating an asthma diagnosis prediction model for children and young people in primary care

Introduction: Accurately diagnosing asthma can be challenging. We aimed to derive and validate a prediction model to support primary care clinicians assess the probability of an asthma diagnosis in children and young people. Methods: The derivation dataset was created from the Avon Longitudinal Study of Parents and Children (ALSPAC) linked to electronic health records. Participants with at least three inhaled corticosteroid prescriptions in 12-months and a coded asthma diagnosis were designated as having asthma. Demographics, symptoms, past medical/family history, exposures, investigations, and prescriptions were considered as candidate predictors. Potential candidate predictors were included if data were available in ≥60% of participants. Multiple imputation was used to handle remaining missing data. The prediction model was derived using logistic regression. Internal validation was completed using bootstrap re-sampling. External validation was conducted using health records from the Optimum Patient Care Research Database (OPCRD). Results: Predictors included in the final model were wheeze, cough, breathlessness, hay-fever, eczema, food allergy, social class, maternal asthma, childhood exposure to cigarette smoke, prescription of a short acting beta agonist and the past recording of lung function/reversibility testing. In the derivation dataset, which comprised 11,972 participants aged <25 years (49% female, 8% asthma), model performance as indicated by the C-statistic and calibration slope was 0.86, 95% confidence interval (CI) 0.85-0.87 and 1.00, 95% CI 0.95-1.05 respectively. In the external validation dataset, which included 2,670 participants aged <25 years (50% female, 10% asthma), the C-statistic was 0.85, 95% CI 0.83-0.88, and calibration slope 1.22, 95% CI 1.09-1.35. Conclusions: We derived and validated a prediction model for clinicians to calculate the probability of asthma diagnosis for a child or young person up to 25 years of age presenting to primary care. Following further evaluation of clinical effectiveness, the prediction model could be implemented as a decision support software.

Deriving and validating an asthma diagnosis prediction model for children and young people in primary care

Introduction: Accurately diagnosing asthma can be challenging. We aimed to derive and validate a prediction model to support primary care clinicians assess the probability of an asthma diagnosis in children and young people. Methods: The derivation dataset was created from the Avon Longitudinal Study of Parents and Children (ALSPAC) linked to electronic health records. Participants with at least three inhaled corticosteroid prescriptions in 12-months and a coded asthma diagnosis were designated as having asthma. Demographics, symptoms, past medical/family history, exposures, investigations, and prescriptions were considered as candidate predictors. Potential candidate predictors were included if data were available in ≥60% of participants. Multiple imputation was used to handle remaining missing data. The prediction model was derived using logistic regression. Internal validation was completed using bootstrap re-sampling. External validation was conducted using health records from the Optimum Patient Care Research Database (OPCRD). Results: Predictors included in the final model were wheeze, cough, breathlessness, hay-fever, eczema, food allergy, social class, maternal asthma, childhood exposure to cigarette smoke, prescription of a short acting beta agonist and the past recording of lung function/reversibility testing. In the derivation dataset, which comprised 11,972 participants aged <25 years (49% female, 8% asthma), model performance as indicated by the C-statistic and calibration slope was 0.86, 95% confidence interval (CI) 0.85-0.87 and 1.00, 95% CI 0.95-1.05 respectively. In the external validation dataset, which included 2,670 participants aged <25 years (50% female, 10% asthma), the C-statistic was 0.85, 95% CI 0.83-0.88, and calibration slope 1.22, 95% CI 1.09-1.35. Conclusions: We derived and validated a prediction model for clinicians to calculate the probability of asthma diagnosis for a child or young person up to 25 years of age presenting to primary care. Following further evaluation of clinical effectiveness, the prediction model could be implemented as a decision support software.

Clinical predictive models in equine medicine: A systematic review

Clinical predictive models use a patient's baseline demographic and clinical data to make predictions about patient outcomes and have the potential to aid clinical decision making. The extent of equine clinical predictive models is unknown in the literature. Using PubMed and Google Scholar, we systematically reviewed the predictive models currently described for use in equine patients. Models were eligible for inclusion if they were published in a peer-reviewed article as a multivariable model used to predict a clinical/laboratory/imaging outcome in an individual horse or herd. The agreement of at least two authors was required for model inclusion. We summarised the patient populations, model development methods, performance metric reporting, validation efforts, and, using the Predictive model Risk of Bias Assessment Tool (PROBAST), assessed the risk of bias and applicability concerns for these models. In addition, we summarised the index conditions for which models were developed and provided detailed information on included models. A total of 90 predictive models and 9 external validation studies were included in the final systematic review. A plurality of models (41%) was developed to predict outcomes associated with colic, for example, need for surgery or survival to discharge. All included models were at high risk of bias, defined as failing one or more PROBAST signalling questions, primarily for analysis-related reasons. Importantly, a high risk of bias does not necessarily mean that models are unusable, but that they require more careful consideration prior to clinical use. Concerns about applicability were low for the majority of models. Systematic reviews such as this can serve to increase veterinarians' awareness of predictive models, including evaluation of their performance and their use in different patient populations.

Clinical decision support in primary care for better diagnosis and management of retinal disease

Misdiagnosis of retinal disease is a common problem in primary care that can lead to irreversible vision loss and false-positive referrals, resulting in inappropriate use of health services. Clinical decision support systems describe tools that leverage information technology to provide timely recommendations that assist clinicians in the decisions they make about the care of a patient. They, therefore, have the potential to reduce the rate of misdiagnosis by promoting evidence-based medicine and more effective and efficient healthcare. This narrative review aims to support primary care practitioners in better understanding the current and emerging capacity of clinical decision support systems in eye care. Different types of clinical decision support systems are discussed, using current examples and evidence from the available literature to demonstrate how they may improve diagnostic effectiveness and aid the management of retinal disease. Comments are made on the future directions of clinical decision support in primary eye care and the potential applications of artificial intelligence.

Predictive models for personalized asthma attacks based on patient's biosignals and environmental factors: a systematic review

BACKGROUND

Asthma is a chronic disease that exacerbates due to various risk factors, including the patient's biosignals and environmental conditions. It is affecting on average 7% of the world population. Preventing an asthma attack is the main challenge for asthma patients, which requires keeping track of any risk factor that can cause a seizure. Many researchers developed asthma attacks prediction models that used various asthma biosignals and environmental factors. These predictive models can help asthmatic patients predict asthma attacks in advance, and thus preventive measures can be taken. This paper introduces a review of these models to evaluate the used methods, model's performance, and determine the need to improve research in this field.

METHOD

A systematic review was conducted for the research articles introducing asthma attack prediction models for children and adults. We searched the PubMed, ScienceDirect, Springer, and IEEE databases from January 2000 to December 2020. The search includes the prediction models that used biosignal, environmental, and both risk factors. The research article's quality was assessed and scored based on two checklists, the Checklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) and the Critical Appraisal Skills Programme clinical prediction rule checklist (CASP). The highest scored articles were selected to review.

RESULT

From 1068 research articles we reviewed, we found that most of the studies used asthma biosignal factors only for prediction, few of the studies used environmental factors, and limited studies used both of these factors. Fifteen different asthma attack predictive models were selected for this review. we found that most of the studies used traditional prediction methods, like Support Vector Machine and regression. We have identified the pros and cons of the reviewed asthma attack prediction models and propose solutions to advance the studies in this field.

CONCLUSION

Asthma attack predictive models become more significant when using both patient's biosignal and environmental factors. There is a lack of utilizing advanced machine learning methods, like deep learning techniques. Besides, there is a need to build smart healthcare systems that provide patients with decision-making systems to identify risk and visualize high-risk regions.

Artificial Intelligence Approach to the Monitoring of Respiratory Sounds in Asthmatic Patients

Background: Effective and reliable monitoring of asthma at home is a relevant factor that may reduce the need to consult a doctor in person.

Aim: We analyzed the possibility to determine intensities of pathological breath phenomena based on artificial intelligence (AI) analysis of sounds recorded during standard stethoscope auscultation.

Methods: The evaluation set comprising 1,043 auscultation examinations (9,319 recordings) was collected from 899 patients. Examinations were assigned to one of four groups: asthma with and without abnormal sounds (AA and AN, respectively), no-asthma with and without abnormal sounds (NA and NN, respectively). Presence of abnormal sounds was evaluated by a panel of 3 physicians that were blinded to the AI predictions. AI was trained on an independent set of 9,847 recordings to determine intensity scores (indexes) of wheezes, rhonchi, fine and coarse crackles and their combinations: continuous phenomena (wheezes + rhonchi) and all phenomena. The pair-comparison of groups of examinations based on Area Under ROC-Curve (AUC) was used to evaluate the performance of each index in discrimination between groups.

Results: Best performance in separation between AA and AN was observed with Continuous Phenomena Index (AUC 0.94) while for NN and NA. All Phenomena Index (AUC 0.91) showed the best performance. AA showed slightly higher prevalence of wheezes compared to NA.

Conclusions: The results showed a high efficiency of the AI to discriminate between the asthma patients with normal and abnormal sounds, thus this approach has a great potential and can be used to monitor asthma symptoms at home.

Implementing asthma management guidelines in public primary care clinics in Malaysia

Implementing asthma guideline recommendations is challenging in low- and middle-income countries. We aimed to explore healthcare provider (HCP) perspectives on the provision of recommended care. Twenty-six HCPs from six public primary care clinics in a semi-urban district of Malaysia were purposively sampled based on roles and experience. Focus group discussions were guided by a semi-structured interview guide and analysed thematically. HCPs had access to guidelines and training but highlighted multiple infrastructure-related challenges to implementing recommended care. Diagnosis and review of asthma control were hampered by limited access to spirometry and limited asthma control test (ACT) use, respectively. Treatment decisions were limited by poor availability of inhaled combination therapy (ICS/LABA) and free spacer devices. Imposed Ministry of Health programmes involving other non-communicable diseases were prioritised over asthma. Ministerial policies need practical resources and organisational support if quality improvement programmes are to facilitate better management of asthma in public primary care clinics.

Data-Influence Analytics in Predictive Models Applied to Asthma Disease

Asthma is one of the most common chronic diseases around the world and represents a serious problem in human health. Predictive models have become important in medical sciences because they provide valuable information for data-driven decision-making. In this work, a methodology of data-influence analytics based on mixed-effects logistic regression models is proposed for detecting potentially influential observations which can affect the quality of these models. Global and local influence diagnostic techniques are used simultaneously in this detection, which are often used separately. In addition, predictive performance measures are considered for this analytics. A study with children and adolescent asthma real data, collected from a public hospital of São Paulo, Brazil, is conducted to illustrate the proposed methodology. The results show that the influence diagnostic methodology is helpful for obtaining an accurate predictive model that provides scientific evidence when data-driven medical decision-making.

Technology Solutions for Nurse Leaders

Over the next 10 years, the World Health Organization estimates that there will be global shortage of 18 000 000 health care workers. A perfect storm of an aging demographic, long-term drop in birth rate, and a retiring workforce has all the factors contributing to this impending crisis. In nursing, efforts to narrow the shortage gap through strategies that increase the number of admissions to nursing schools, such as increasing faculty members, clinical sites, preceptors, and scholarships, will likely not be enough to offset the shortfall. Solutions, in part, will be to make nurses more productive by reducing waste through the use of technology. This article evaluates how various types of technology such as electronic health records, data analytics, predictive modeling, artificial intelligence, speech recognition, natural language processing, robotics, the Internet of Things, and others can improve nurse productivity by using a Lean framework to eliminate waste and create value.

Establishment and evaluation of a multicenter collaborative prediction model construction framework supporting model generalization and continuous improvement: A pilot study

BACKGROUND AND OBJECTIVE

In recent years, an increasing number of clinical prediction models have been developed to serve clinical care. Establishing a data-driven prediction model based on large-scale electronic health record (EHR) data can provide a more empirical basis for clinical decision making. However, research on model generalization and continuous improvement is insufficiently focused, which also hinders the application and evaluation of prediction models in real clinical environments. Therefore, this study proposes a multicenter collaborative prediction model construction framework to build a prediction model with greater generalizability and continuous improvement capabilities while preserving patient data security and privacy.

MATERIALS AND METHODS

Based on a multicenter collaborative research network, such as the Observational Health Data Sciences and Informatics (OHDSI), a multicenter collaborative prediction model construction framework is proposed. Based on the idea of multi-source transfer learning, in each source hospital, a base classifier was trained according to the model research setting. Then, in the target hospital with missing calibration data, a prediction model was established through weighted integration of base classifiers from source hospitals based on the smoothness assumption. Moreover, a passive-aggressive online learning algorithm was used for continuous improvement of the prediction model, which can help to maintain a high predictive performance to provide reliable clinical decision-making abilities. To evaluate the proposed prediction model construction framework, a prototype system for colorectal cancer prognosis prediction was developed. To evaluate the performance of models, 70,906 patients were screened, including 70,090 from 5 US hospital-specific datasets and 816 from a Chinese hospital-specific dataset. The area under the receiver operating characteristic curve (AUC) and the estimated calibration index (ECI) were used to evaluate the discrimination and calibration of models.

RESULTS

Regarding the colorectal cancer prognosis prediction in our prototype system, compared with the reference models, our model achieved a better performance in model calibration (ECI = 9.294 [9.146, 9.441]) and a similar ability in model discrimination (AUC = 0.783 [0.780, 0.786]). Furthermore, the online learning process provided in this study can continuously improve the performance of the prediction model when patient data with specified labels arrive (the AUC value increased from 0.709 to 0.715 and the ECI value decreased from 13.013 to 9.634 after 650 patient instances with specified labels from the Chinese hospital arrived), enabling the prediction model to maintain a good predictive performance during clinical application.

CONCLUSIONS

This study proposes and evaluates a multicenter collaborative prediction model construction framework that can support the construction of prediction models with better generalizability and continuous improvement capabilities without the need to aggregate multicenter patient-level data.

Defining high probability when making a diagnosis of asthma in primary care: mixed-methods consensus workshop

OBJECTIVE

Making the diagnosis of asthma is challenging. Guidelines recommend that clinicians identify a group at 'high probability' of asthma. High probability, however, is not numerically defined giving rise to uncertainty. The aim of this work was to build consensus on what constitutes a high probability of asthma in primary care. High probability was defined as the probability threshold at which there is enough information to make a firm diagnosis of asthma, and a subsequent negative test would not alter that opinion (assumed to be a false negative).

DESIGN

Mixed-methods study.

SETTING

A consensus workshop using modified nominal group technique was held during an international respiratory conference.

PARTICIPANTS

International conference attendees eligible if they had knowledge/experience of working in primary care, respiratory medicine and spoke English.

METHODS

Participants took part in facilitated discussions and voted over three rounds on what constituted a high probability of asthma diagnosis. The workshop was audio-recorded, transcribed and qualitatively analysed.

RESULTS

Based on final votes, the mean value for a high probability of asthma in primary care was 75% (SD 7.6), representing a perceived trade-off between limiting the number of false positives (more likely if a lower threshold was used) and pragmatism on the basis that first-line preventive therapies (ie, low-dose inhaled corticosteroids) are relatively low risk. The need to review response to treatment was strongly emphasised for detecting non-responders and reviewing the diagnosis.

CONCLUSION

A consensus probability of 75% was the threshold at which the primary care participants in this workshop felt confident to establish the diagnosis of asthma, albeit with the caveat that a review of treatment response was essential. Contextual factors, including availability and timing of tests and the ease with which patients could be reviewed, influenced participants' decision making.

Protocol for the derivation and validation of a clinical prediction model to support the diagnosis of asthma in children and young people in primary care

Background: Accurately diagnosing asthma can be challenging. Uncertainty about the best combination of clinical features and investigations for asthma diagnosis is reflected in conflicting recommendations from international guidelines. One solution could be a clinical prediction model to support health professionals estimate the probability of an asthma diagnosis. However, systematic review evidence identifies that existing models for asthma diagnosis are at high risk of bias and unsuitable for clinical use. Being mindful of previous limitations, this protocol describes plans to derive and validate a prediction model for use by healthcare professionals to aid diagnostic decision making during assessment of a child or young person with symptoms suggestive of asthma in primary care. Methods: A prediction model will be derived using data from the Avon Longitudinal Study of Parents and Children (ALSPAC) and linked primary care electronic health records (EHR). Data will be included from study participants up to 25 years of age where permissions exist to use their linked EHR. Participants will be identified as having asthma if they received at least three prescriptions for an inhaled corticosteroid within a one-year period and have an asthma code in their EHR. To deal with missing data we will consider conducting a complete case analysis. However, if the exclusion of cases with missing data substantially reduces the total sample size, multiple imputation will be used. A multivariable logistic regression model will be fitted with backward stepwise selection of candidate predictors.  Apparent model performance will be assessed before internal validation using bootstrapping techniques. The model will be adjusted for optimism before external validation in a dataset created from the Optimum Patient Care Research Database. Discussion: This protocol describes a robust strategy for the derivation and validation of a prediction model to support the diagnosis of asthma in children and young people in primary care.