Diagnosing Middle Ear Dysfunction in 10- to 16-Month-Old Infants Using Wideband Absorbance: An Ordinal Prediction Model

AutoScore-Ordinal: an interpretable machine learning framework for generating scoring models for ordinal outcomes

Background

Risk prediction models are useful tools in clinical decision-making which help with risk stratification and resource allocations and may lead to a better health care for patients. AutoScore is a machine learning–based automatic clinical score generator for binary outcomes. This study aims to expand the AutoScore framework to provide a tool for interpretable risk prediction for ordinal outcomes.

Methods

The AutoScore-Ordinal framework is generated using the same 6 modules of the original AutoScore algorithm including variable ranking, variable transformation, score derivation (from proportional odds models), model selection, score fine-tuning, and model evaluation. To illustrate the AutoScore-Ordinal performance, the method was conducted on electronic health records data from the emergency department at Singapore General Hospital over 2008 to 2017. The model was trained on 70% of the data, validated on 10% and tested on the remaining 20%.

Results

This study included 445,989 inpatient cases, where the distribution of the ordinal outcome was 80.7% alive without 30-day readmission, 12.5% alive with 30-day readmission, and 6.8% died inpatient or by day 30 post discharge. Two point-based risk prediction models were developed using two sets of 8 predictor variables identified by the flexible variable selection procedure. The two models indicated reasonably good performance measured by mean area under the receiver operating characteristic curve (0.758 and 0.793) and generalized c-index (0.737 and 0.760), which were comparable to alternative models.

Conclusion

AutoScore-Ordinal provides an automated and easy-to-use framework for development and validation of risk prediction models for ordinal outcomes, which can systematically identify potential predictors from high-dimensional data.

Diagnosing Conductive Dysfunction in Infants Using Wideband Acoustic Immittance: Validation and Development of Predictive Models

Purpose The aims of this study were (a) to validate the wideband acoustic immittance (WAI) model developed by Myers et al. (2018a) in a new sample of neonates and (b) to develop a prediction model for diagnosing middle ear dysfunction in infants aged 6-18 months using wideband absorbance, controlling for the effect of age. Method Tympanometry, distortion product otoacoustic emissions, and WAI were measured in 124 neonates and longitudinally in 357 infants at 6, 12, and 18 months of age. Results of tympanometry and distortion product otoacoustic emissions were used to assess middle ear function of each infant. For the first study, results from the neonates were applied to the diagnostic WAI model developed by Myers et al. (2018a). For the second study, a prediction model was developed using results from the 6- to 18-month-old infants. Results from 1 ear of infants in each age group (6, 12, and 18 months) were used to develop the model. The amount of bias (overfitting) was estimated with bootstrap resampling and by applying the model to the opposite ears (the test sample). Performance was assessed using measures of discrimination (c-index) and calibration (calibration curves). Results For the validation study, the Myers et al. (2018a) model was well calibrated and had a c-index of 0.837 when applied to a new sample of neonates. Although this was lower than the apparent performance c-index of 0.876 reported by Myers et al., it was close to the bias-corrected estimate of 0.845. The model developed for 6- to 18-month-old infants had satisfactory calibration and apparent, bias-corrected, and test sample c-index of 0.884, 0.867, and 0.887, respectively. Conclusions The validated and developed models may be clinically useful, and further research validating, updating, and assessing the clinical impact of the models is warranted.