Automated Spirometry Quality Assurance: Supervised Learning From Multiple Experts

Deep learning algorithm for visual quality assessment of the spirograms

Objective. The quality of spirometry manoeuvres is crucial for correctly interpreting the values of spirometry parameters. A fundamental guideline for proper quality assessment is the American Thoracic Society and European Respiratory Society (ATS/ERS) Standards for spirometry, updated in 2019, which describe several start-of-test and end-of-test criteria which can be assessed automatically. However, the spirometry standards also require a visual evaluation of the spirometry curve to determine the spirograms' acceptability or usability. In this study, we present an automatic algorithm based on a convolutional neural network (CNN) for quality assessment of the spirometry curves as an alternative to manual verification performed by specialists.Approach. The algorithm for automatic assessment of spirometry measurements was created using a set of randomly selected 1998 spirograms which met all quantitative criteria defined by ATS/ERS Standards. Each spirogram was annotated as 'confirm' (remaining acceptable or usable status) or 'reject' (change the status to unacceptable) by four pulmonologists, separately for FEV1 and FVC parameters. The database was split into a training (80%) and test set (20%) for developing the CNN classification algorithm. The algorithm was optimised using a cross-validation method.Main results. The accuracy, sensitivity and specificity obtained for the algorithm were 92.6%, 93.1% and 90.0% for FEV1 and 94.1%, 95.6% and 88.3% for FVC, respectively.Significance.The algorithm provides an opportunity to significantly improve the quality of spirometry tests, especially during unsupervised spirometry. It can also serve as an additional tool in clinical trials to quickly assess the quality of a large group of tests.

Fractional Dynamics Foster Deep Learning of COPD Stage Prediction

Chronic obstructive pulmonary disease (COPD) is one of the leading causes of death worldwide. Current COPD diagnosis (i.e., spirometry) could be unreliable because the test depends on an adequate effort from the tester and testee. Moreover, the early diagnosis of COPD is challenging. The authors address COPD detection by constructing two novel physiological signals datasets (4432 records from 54 patients in the WestRo COPD dataset and 13824 medical records from 534 patients in the WestRo Porti COPD dataset). The authors demonstrate their complex coupled fractal dynamical characteristics and perform a fractional-order dynamics deep learning analysis to diagnose COPD. The authors found that the fractional-order dynamical modeling can extract distinguishing signatures from the physiological signals across patients with all COPD stages-from stage 0 (healthy) to stage 4 (very severe). They use the fractional signatures to develop and train a deep neural network that predicts COPD stages based on the input features (such as thorax breathing effort, respiratory rate, or oxygen saturation). The authors show that the fractional dynamic deep learning model (FDDLM) achieves a COPD prediction accuracy of 98.66% and can serve as a robust alternative to spirometry. The FDDLM also has high accuracy when validated on a dataset with different physiological signals.

Artificial intelligence based software facilitates spirometry quality control in asthma and COPD clinical trials

Rationale

Acquiring high-quality spirometry data in clinical trials is important, particularly when using forced expiratory volume in 1 s or forced vital capacity as primary end-points. In addition to quantitative criteria, the American Thoracic Society (ATS)/European Respiratory Society (ERS) standards include subjective evaluation which introduces inter-rater variability and potential mistakes. We explored the value of artificial intelligence (AI)-based software (ArtiQ.QC) to assess spirometry quality and compared it to traditional over-reading control.

Methods

A random sample of 2000 sessions (8258 curves) was selected from Chiesi COPD and asthma trials (n=1000 per disease). Acceptability using the 2005 ATS/ERS standards was determined by over-reader review and by ArtiQ.QC. Additionally, three respiratory physicians jointly reviewed a subset of curves (n=150).

Results

The majority of curves (n=7267, 88%) were of good quality. The AI agreed with over-readers in 91% of cases, with 97% sensitivity and 93% positive predictive value. Performance was significantly better in the asthma group. In the revised subset, n=50 curves were repeated to assess intra-rater reliability (κ=0.83, 0.86 and 0.80 for each of the three reviewers). All reviewers agreed on 63% of 100 unique tests (κ=0.5). When reviewers set the consensus (gold standard), individual agreement with it was 88%, 94% and 70%. The agreement between AI and "gold-standard" was 73%; over-reader agreement was 46%.

Conclusion

AI-based software can be used to measure spirometry data quality with comparable accuracy as experts. The assessment is a subjective exercise, with intra- and inter-rater variability even when the criteria are defined very precisely and objectively. By providing consistent results and immediate feedback to the sites, AI may benefit clinical trial conduct and variability reduction.

Deep learning for spirometry quality assurance with spirometric indices and curves

Background

Spirometry quality assurance is a challenging task across levels of healthcare tiers, especially in primary care. Deep learning may serve as a support tool for enhancing spirometry quality. We aimed to develop a high accuracy and sensitive deep learning-based model aiming at assisting high-quality spirometry assurance.

Methods

Spirometry PDF files retrieved from one hospital between October 2017 and October 2020 were labeled according to ATS/ERS 2019 criteria and divided into training and internal test sets. Additional files from three hospitals were used for external testing. A deep learning-based model was constructed and assessed to determine acceptability, usability, and quality rating for FEV₁ and FVC. System warning messages and patient instructions were also generated for general practitioners (GPs).

Results

A total of 16,502 files were labeled. Of these, 4592 curves were assigned to the internal test set, the remaining constituted the training set. In the internal test set, the model generated 95.1%, 92.4%, and 94.3% accuracy for FEV₁ acceptability, usability, and rating. The accuracy for FVC acceptability, usability, and rating were 93.6%, 94.3%, and 92.2%. With the assistance of the model, the performance of GPs in terms of monthly percentages of good quality (A, B, or C grades) tests for FEV₁ and FVC was higher by ~ 21% and ~ 36%, respectively.

Conclusion

The proposed model assisted GPs in spirometry quality assurance, resulting in enhancing the performance of GPs in quality control of spirometry.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12931-022-02014-9.

The effects of different body positions on pulmonary function in healthy adults

Abstract Introduction: Pulmonary function testing, or spirometry, is a validated, globally recognized test that contributes to the diagnosis, staging, and longitudinal follow-up of lung diseases. The exam is most often performed in a sitting position in clinical practice; hence, there are no predicted values for its performance in other positions, such as in different decubitus. Objective: The present study aimed to evaluate the effects of position on pulmonary function test results in healthy adults. Methods: Forty-two healthy adults of both sexes, divided into male (MG) and female groups (FG), were provided respiratory questionnaires. Subsequently, the pulmonary function test was conducted to evaluate the ventilatory parameters of forced vital capacity (FVC), forced expiratory volume in 1 second (FEV1), and FEV1/FVC ratio in the sitting (S), dorsal decubitus (DD), right lateral decubitus (RLD), and left lateral decubitus (LLD) positions. A comparison of the parametric data was performed via one-way analysis of variance followed by Tukey post-hoc tests. Correlations between the S position variables along with the other positions were evaluated using the Pearson test. Results: The mean and standard error for the FVC values of the MG at positions DD (4.3 ± 0.7/L), RLD (4.1 ± 0.6/L) and LLD (4.1 ± 0.6/L) were lower when compared to S (5.05 ± 0.6 L). There was a strong positive correlation between the values of FVC, FEV1, and FEV1/FVC in the S position compared to other positions analyzed in both groups. Conclusion: Body positioning altered the parameters of the pulmonary function test in healthy adults.

Area under the expiratory flow-volume curve: predicted values by artificial neural networks

Area under expiratory flow-volume curve (AEX) has been proposed recently to be a useful spirometric tool for assessing ventilatory patterns and impairment severity. We derive here normative reference values for AEX, based on age, gender, race, height and weight, and by using artificial neural network (ANN) algorithms. We analyzed 3567 normal spirometry tests with available AEX values, performed on subjects from two countries (United States and Spain). Regular linear or optimized regression and ANN models were built using traditional predictors of lung function. The ANN-based models outperformed the de novo regression-based equations for AEX_predicted and AEX z scores using race, gender, age, height and weight as predictor factors. We compared these reference values with previously developed equations for AEX (by gender and race), and found that the ANN models led to the most accurate predictions. When we compared the performance of ANN-based models in derivation/training, internal validation/testing, and external validation random groups, we found that the models based on pooling samples from various geographic areas outperformed the other models (in both central tendency and dispersion of the residuals, ameliorating any cohort effects). In a geographically diverse cohort of subjects with normal spirometry, we computed by both regression and ANN models several predicted equations and z scores for AEX, an alternative measurement of respiratory function. We found that the dynamic nature of the ANN allows for continuous improvement of the predictive models’ performance, thus promising that the AEX could become an essential tool in assessing respiratory impairment.

Standardization of Spirometry 2019 Update. An Official American Thoracic Society and European Respiratory Society Technical Statement

Background: Spirometry is the most common pulmonary function test. It is widely used in the assessment of lung function to provide objective information used in the diagnosis of lung diseases and monitoring lung health. In 2005, the American Thoracic Society and the European Respiratory Society jointly adopted technical standards for conducting spirometry. Improvements in instrumentation and computational capabilities, together with new research studies and enhanced quality assurance approaches, have led to the need to update the 2005 technical standards for spirometry to take full advantage of current technical capabilities.Methods: This spirometry technical standards document was developed by an international joint task force, appointed by the American Thoracic Society and the European Respiratory Society, with expertise in conducting and analyzing pulmonary function tests, laboratory quality assurance, and developing international standards. A comprehensive review of published evidence was performed. A patient survey was developed to capture patients' experiences.Results: Revisions to the 2005 technical standards for spirometry were made, including the addition of factors that were not previously considered. Evidence to support the revisions was cited when applicable. The experience and expertise of task force members were used to develop recommended best practices.Conclusions: Standards and consensus recommendations are presented for manufacturers, clinicians, operators, and researchers with the aims of increasing the accuracy, precision, and quality of spirometric measurements and improving the patient experience. A comprehensive guide to aid in the implementation of these standards was developed as an online supplement.