Development and validation of risk models for mortality and morbidity in 12 major pediatric surgical procedures: A study from the National Clinical Database-Pediatric of Japan

Influence of surgical volume on the mortality and morbidity of gastrointestinal perforation in children

PURPOSE

To clarify the influence of surgical volume on the mortality and morbidity of gastrointestinal perforation in children in Japan.

METHODS

We collected data on pediatric patients with gastrointestinal perforation between 2017 and 2019, from the National Clinical Database. The surgical volumes of various institutions were classified into three groups: low (average number of surgeries for gastrointestinal perforation/year < 1), medium (≥ 1, < 6), and high (≥ 6). The observed-to-expected (o/e) ratios of 30-day mortality and morbidity were calculated for each group using an existing risk model.

RESULTS

Among 1641 patients (median age, 0.0 years), the 30-day mortality and morbidity rates were 5.2% and 37.7%, respectively. The 30-day mortality rates in the low-, medium-, and high-volume institutions were 4.9%, 5.3%, and 5.1% (p = 0.94), and the 30-day morbidity rates in the three groups were 26.8%, 39.7%, and 37.7% (p < 0.01), respectively. The o/e ratios of 30-day mortality were 1.05 (95% confidence interval [CI] 0.83-1.26), 1.08 (95% CI 1.01-1.15), and 1.02 (95% CI 0.91-1.13), and those of 30-day morbidity were 1.72 (95% CI 0.93-2.51), 1.03 (95% CI 0.79-1.28), and 0.95 (95% CI 0.56-1.33), respectively.

CONCLUSION

Surgical volume does not have significant impact on the outcomes of pediatric gastrointestinal perforation in Japan.

Validation of Automated Data Extraction From the Electronic Medical Record to Provide a Pediatric Risk Assessment Score

BACKGROUND

Although the rate of pediatric postoperative mortality is low, the development and validation of perioperative risk assessment models have allowed for the stratification of those at highest risk, including the Pediatric Risk Assessment (PRAm) score. The clinical application of such tools requires manual data entry, which may be inaccurate or incomplete, compromise efficiency, and increase physicians' clerical obligations. We aimed to create an electronically derived, automated PRAm score and to evaluate its agreement with the original American College of Surgery National Surgical Quality Improvement Program (ACS NSQIP)-derived and validated score.

METHODS

We performed a retrospective observational study of children <18 years who underwent noncardiac surgery from 2017 through 2021 at Boston Children's Hospital (BCH). An automated PRAm score was developed via electronic derivation of International Classification of Disease (ICD) -9 and -10 codes. The primary outcome was agreement and correlation among PRAm scores obtained via automation, NSQIP data, and manual physician entry from the same BCH cohort. The secondary outcome was discriminatory ability of the 3 PRAm versions. Fleiss Kappa, Spearman correlation (rho), and intraclass correlation coefficient (ICC) and receiver operating characteristic (ROC) curve analyses with area under the curve (AUC) were applied accordingly.

RESULTS

Of the 6014 patients with NSQIP and automated PRAm scores (manual scores: n = 5267), the rate of 30-day mortality was 0.18% (n = 11). Agreement and correlation were greater between the NSQIP and automated scores (rho = 0.78; 95% confidence interval [CI], 0.76-0.79; P <.001; ICC = 0.80; 95% CI, 0.79-0.81; Fleiss kappa = 0.66; 95% CI, 0.65-0.67) versus the NSQIP and manual scores (rho = 0.73; 95% CI, 0.71-0.74; P < .001; ICC = 0.78; 95% CI, 0.77-0.79; Fleiss kappa = 0.56; 95% CI, 0.54-0.57). ROC analysis with AUC showed the manual score to have the greatest discrimination (AUC = 0.976; 95% CI, 0.959,0.993) compared to the NSQIP (AUC = 0.904; 95% CI, 0.792-0.999) and automated (AUC = 0.880; 95% CI, 0.769-0.999) scores.

CONCLUSIONS

Development of an electronically derived, automated PRAm score that maintains good discrimination for 30-day mortality in neonates, infants, and children after noncardiac surgery is feasible. The automated PRAm score may reduce the preoperative clerical workload and provide an efficient and accurate means by which to risk stratify neonatal and pediatric surgical patients with the goal of improving clinical outcomes and resource utilization.

Perioperative mortality in pediatric patients: a systematic review of risk assessment tools for use in the preoperative setting

BACKGROUND

There are multiple preoperative risk scores for pediatric mortality. The aim of this study was to systematically describe and compare the existing studies of patient-specific multispecialty risk prediction scores for perioperative mortality in pediatric populations, with the goal of guiding clinicians on which may be most appropriate for use in the preoperative setting.

METHODS

We conducted a systematic literature review of published journal articles that presented the development, extension/updating, and/or validation of a risk score which predicted all-cause mortality (up to 30 days postoperatively) in pediatric patients undergoing a procedure in which anesthesia was used. Scores needed to be applicable to surgeries in more than one non-cardiac surgical specialty and had to be able to be calculated by the anesthesiologist at the time of the pre-anesthetic assessment. Two investigators independently screened studies for inclusion and assessed study quality in the domains of clinical applicability, feasibility/ease of use in the clinical setting, and risk of bias.

RESULTS

A total of 1,681 titles were retrieved. Of these, 10 studies met inclusion criteria: nine reported the development and validation of scores, and one was an external validation of an existing score. Seven studies used varying years of multicenter data from the National Surgical Quality Improvement Program - Pediatric Participant Use File for development and/or validation. The unadjusted rate of mortality in the studies ranged from 0.3% to 3.6%. The preoperative predictors of mortality used in score development included patient demographics, preoperative therapies, and chronic conditions, among others. All models showed good discrimination upon validation (AUC > 0.8). Most risk scores had high or unclear risks of bias.

CONCLUSION

There are numerous scores available for the prediction of mortality in pediatric populations, all of which exhibited good performance. However, many have high or unclear risks of bias, and most have not undergone external validation.