Automated ongoing data validation and quality control of multi-institutional studies

Automated prediction of extubation success in extremely preterm infants: the APEX multicenter study

BACKGROUND

Extremely preterm infants are frequently subjected to mechanical ventilation. Current prediction tools of extubation success lacks accuracy.

METHODS

Multicenter study including infants with birth weight ≤1250 g undergoing their first extubation attempt. Clinical data and cardiorespiratory signals were acquired before extubation. Primary outcome was prediction of extubation success. Automated analysis of cardiorespiratory signals, development of clinical and cardiorespiratory features, and a 2-stage Clinical Decision-Balanced Random Forest classifier were used. A leave-one-out cross-validation was done. Performance was analyzed by ROC curves and determined by balanced accuracy. An exploratory analysis was performed for extubations before 7 days of age.

RESULTS

A total of 241 infants were included and 44 failed (18%) extubation. The classifier had a balanced accuracy of 73% (sensitivity 70% [95% CI: 63%, 76%], specificity 75% [95% CI: 62%, 88%]). As an additional clinical-decision tool, the classifier would have led to an increase in extubation success from 82% to 93% but misclassified 60 infants who would have been successfully extubated. In infants extubated before 7 days of age, the classifier identified 16/18 failures (specificity 89%) and 73/105 infants with success (sensitivity 70%).

CONCLUSIONS

Machine learning algorithms may improve a balanced prediction of extubation outcomes, but further refinement and validation is required.

IMPACT

A machine learning-derived predictive model combining clinical data with automated analyses of individual cardiorespiratory signals may improve the prediction of successful extubation and identify infants at higher risk of failure with a good balanced accuracy. Such multidisciplinary approach including medicine, biomedical engineering and computer science is a step forward as current tools investigated to predict extubation outcomes lack sufficient balanced accuracy to justify their use in future trials or clinical practice. Thus, this individualized assessment can optimize patient selection for future trials of extubation readiness by decreasing exposure of low-risk infants to interventions and maximize the benefits of those at high risk.

Parental Engagement of a Prototype Electronic Diary in an Ambulatory Setting Following Adenotonsillectomy in Children: A Prospective Cohort Study

Adenotonsillectomy is performed in children on an outpatient basis, and pain is managed by parents. A pain diary would facilitate pain management in the ambulatory setting. Our objective was to evaluate the parental response rate and the compliance of a prototype electronic pain diary (e-diary) with cloud storage in children aged 2–12 years recovering from adenotonsillectomy and to compare the e-diary with a paper diary (p-diary). Parents recorded pain scores twice daily in a pain diary for 2 weeks post-operation. Parents were given the choice of an e-diary or p-diary with picture message. A total of 208 patients were recruited, of which 35 parents (16.8%) chose the e-diary. Most parents (98%) chose to be contacted by text message. Eighty-one families (47%) returned p-diaries to us by mail. However, the response rate increased to 77% and was similar to that of the e-diary (80%) when we included data texted to the research phone from 53 families. The proportion of diaries with Complete (e-diary:0.37 vs. p-diary:0.4) and Incomplete (e-diary:0.43 vs. p-diary:0.38) data entries were similar. E-diaries provide a means to follow patients in real time after discharge. Our findings suggest that a smartphone-based medical health application coupled with a cloud would meet the needs of families and health care providers alike.

Patterns of reintubation in extremely preterm infants: a longitudinal cohort study

BackgroundThe optimal approach for reporting reintubation rates in extremely preterm infants is unknown. This study aims to longitudinally describe patterns of reintubation in this population over a broad range of observation windows following extubation.MethodsTiming and reasons for reintubation following a first planned extubation were collected from infants with birth weight ≤1,250 g. An algorithm was generated to discriminate between reintubations attributable to respiratory and non-respiratory causes. Frequency and cumulative distribution curves were constructed for each category using 24 h intervals. The ability of observation windows to capture respiratory-related reintubations while limiting non-respiratory reasons was assessed using a receiver operating characteristic curve.ResultsOut of 194 infants, 91 (47%) were reintubated during hospitalization; 68% for respiratory and 32% for non-respiratory reasons. Respiratory-related reintubation rates steadily increased from 0 to 14 days post-extubation before reaching a plateau. In contrast, non-respiratory reintubations were negligible in the first post-extubation week, but became predominant after 14 days. An observation window of 7 days captured 77% of respiratory-related reintubations while only including 14% of non-respiratory cases.ConclusionReintubation patterns are highly variable and affected by the reasons for reintubation and observation window used. Ideally, reintubation rates should be reported using a cumulative distribution curve over time.

Prediction of Extubation readiness in extremely preterm infants by the automated analysis of cardiorespiratory behavior: study protocol

Background

Extremely preterm infants (≤ 28 weeks gestation) commonly require endotracheal intubation and mechanical ventilation (MV) to maintain adequate oxygenation and gas exchange. Given that MV is independently associated with important adverse outcomes, efforts should be made to limit its duration. However, current methods for determining extubation readiness are inaccurate and a significant number of infants fail extubation and require reintubation, an intervention that may be associated with increased morbidities. A variety of objective measures have been proposed to better define the optimal time for extubation, but none have proven clinically useful. In a pilot study, investigators from this group have shown promising results from sophisticated, automated analyses of cardiorespiratory signals as a predictor of extubation readiness. The aim of this study is to develop an automated predictor of extubation readiness using a combination of clinical tools along with novel and automated measures of cardiorespiratory behavior, to assist clinicians in determining when extremely preterm infants are ready for extubation.

Methods

In this prospective, multicenter observational study, cardiorespiratory signals will be recorded from 250 eligible extremely preterm infants with birth weights ≤1250 g immediately prior to their first planned extubation. Automated signal analysis algorithms will compute a variety of metrics for each infant, and machine learning methods will then be used to find the optimal combination of these metrics together with clinical variables that provide the best overall prediction of extubation readiness. Using these results, investigators will develop an Automated system for Prediction of EXtubation (APEX) readiness that will integrate the software for data acquisition, signal analysis, and outcome prediction into a single application suitable for use by medical personnel in the neonatal intensive care unit. The performance of APEX will later be prospectively validated in 50 additional infants.

Discussion

The results of this research will provide the quantitative evidence needed to assist clinicians in determining when to extubate a preterm infant with the highest probability of success, and could produce significant improvements in extubation outcomes in this population.

Trial registration

Clinicaltrials.gov identifier: NCT01909947. Registered on July 17 2013.

Trial sponsor: Canadian Institutes of Health Research (CIHR).

Electronic supplementary material

The online version of this article (doi:10.1186/s12887-017-0911-z) contains supplementary material, which is available to authorized users.

APEX_SCOPE: A graphical user interface for visualization of multi-modal data in inter-disciplinary studies

In multi-disciplinary studies, different forms of data are often collected for analysis. For example, APEX, a study on the automated prediction of extubation readiness in extremely preterm infants, collects clinical parameters and cardiorespiratory signals. A variety of cardiorespiratory metrics are computed from these signals and used to assign a cardiorespiratory pattern at each time. In such a situation, exploratory analysis requires a visualization tool capable of displaying these different types of acquired and computed signals in an integrated environment. Thus, we developed APEX_SCOPE, a graphical tool for the visualization of multi-modal data comprising cardiorespiratory signals, automated cardiorespiratory metrics, automated respiratory patterns, manually classified respiratory patterns, and manual annotations by clinicians during data acquisition. This MATLAB-based application provides a means for collaborators to view combinations of signals to promote discussion, generate hypotheses and develop features.