Uncertain diagnoses in a children's hospital: patient characteristics and outcomes

Developing a machine learning model to detect diagnostic uncertainty in clinical documentation

BACKGROUND AND OBJECTIVE

Diagnostic uncertainty, when unrecognized or poorly communicated, can result in diagnostic error. However, diagnostic uncertainty is challenging to study due to a lack of validated identification methods. This study aims to identify distinct linguistic patterns associated with diagnostic uncertainty in clinical documentation.

DESIGN, SETTING AND PARTICIPANTS

This case-control study compares the clinical documentation of hospitalized children who received a novel uncertain diagnosis (UD) diagnosis label during their admission to a set of matched controls. Linguistic analyses identified potential linguistic indicators (i.e., words or phrases) of diagnostic uncertainty that were then manually reviewed by a linguist and clinical experts to identify those most relevant to diagnostic uncertainty. A natural language processing program categorized medical terminology into semantic types (i.e., sign or symptom), from which we identified a subset of these semantic types that both categorized reliably and were relevant to diagnostic uncertainty. Finally, a competitive machine learning modeling strategy utilizing the linguistic indicators and semantic types compared different predictive models for identifying diagnostic uncertainty.

RESULTS

Our cohort included 242 UD-labeled patients and 932 matched controls with a combination of 3070 clinical notes. The best-performing model was a random forest, utilizing a combination of linguistic indicators and semantic types, yielding a sensitivity of 89.4% and a positive predictive value of 96.7%.

CONCLUSION

Expert labeling, natural language processing, and machine learning methods combined with human validation resulted in highly predictive models to detect diagnostic uncertainty in clinical documentation and represent a promising approach to detecting, studying, and ultimately mitigating diagnostic uncertainty in clinical practice.

Factors Associated With Inpatient Subspecialty Consultation Patterns Among Pediatric Hospitalists

IMPORTANCE

Subspecialty consultation is a frequent, consequential practice in the pediatric inpatient setting. Little is known about factors affecting consultation practices.

OBJECTIVES

To identify patient, physician, admission, and systems characteristics that are independently associated with subspecialty consultation among pediatric hospitalists at the patient-day level and to describe variation in consultation utilization among pediatric hospitalist physicians.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study of hospitalized children used electronic health record data from October 1, 2015, through December 31, 2020, combined with a cross-sectional physician survey completed between March 3 and April 11, 2021. The study was conducted at a freestanding quaternary children's hospital. Physician survey participants were active pediatric hospitalists. The patient cohort included children hospitalized with 1 of 15 common conditions, excluding patients with complex chronic conditions, intensive care unit stay, or 30-day readmission for the same condition. Data were analyzed from June 2021 to January 2023.

EXPOSURES

Patient (sex, age, race and ethnicity), admission (condition, insurance, year), physician (experience, anxiety due to uncertainty, gender), and systems (hospitalization day, day of week, inpatient team, and prior consultation) characteristics.

MAIN OUTCOMES AND MEASURES

The primary outcome was receipt of inpatient consultation on each patient-day. Risk-adjusted consultation rates, expressed as number of patient-days consulting per 100, were compared between physicians.

RESULTS

We evaluated 15 922 patient-days attributed to 92 surveyed physicians (68 [74%] women; 74 [80%] with ≥3 years' attending experience) caring for 7283 unique patients (3955 [54%] male patients; 3450 [47%] non-Hispanic Black and 2174 [30%] non-Hispanic White patients; median [IQR] age, 2.5 ([0.9-6.5] years). Odds of consultation were higher among patients with private insurance compared with those with Medicaid (adjusted odds ratio [aOR], 1.19 [95% CI, 1.01-1.42]; P = .04) and physicians with 0 to 2 years of experience vs those with 3 to 10 years of experience (aOR, 1.42 [95% CI, 1.08-1.88]; P = .01). Hospitalist anxiety due to uncertainty was not associated with consultation. Among patient-days with at least 1 consultation, non-Hispanic White race and ethnicity was associated with higher odds of multiple consultations vs non-Hispanic Black race and ethnicity (aOR, 2.23 [95% CI, 1.20-4.13]; P = .01). Risk-adjusted physician consultation rates were 2.1 times higher in the top quartile of consultation use (mean [SD], 9.8 [2.0] patient-days consulting per 100) compared with the bottom quartile (mean [SD], 4.7 [0.8] patient-days consulting per 100; P < .001).

CONCLUSIONS AND RELEVANCE

In this cohort study, consultation use varied widely and was associated with patient, physician, and systems factors. These findings offer specific targets for improving value and equity in pediatric inpatient consultation.

Diagnosis Code and Health Care Utilization Patterns Associated With Diagnostic Uncertainty

BACKGROUND AND OBJECTIVES

Diagnostic uncertainty is challenging to identify and study in clinical practice. This study compares differences in diagnosis code and health care utilization between a unique cohort of hospitalized children with uncertain diagnoses (UD) and matched controls.

PATIENTS AND METHODS

This case-control study was conducted at Cincinnati Children's Hospital Medical Center. Cases were defined as patients admitted to the pediatric hospital medicine service and having UDs during their hospitalization. Control patients were matched on age strata, biological sex, and time of year. Outcomes included type of diagnosis codes used (ie, disease- or nondisease-based) and change in code from admission to discharge. Differences in diagnosis codes were evaluated using conditional logistic regression. Health care utilization outcomes included hospital length of stay (LOS), hospital transfer, consulting service utilization, rapid response team activations, escalation to intensive care, and 30-day health care reutilization. Differences in health care utilization were assessed using bivariate statistics.

RESULTS

Our final cohort included 240 UD cases and 911 matched controls. Compared with matched controls, UD cases were 8 times more likely to receive a nondisease-based diagnosis code (odds ratio [OR], 8.0; 95% confidence interval [CI], 5.7-11.2) and 2.5 times more likely to have a change in their primary International Classification of Disease, 10th revision, diagnosis code between admission and discharge (OR, 2.5; 95% CI, 1.9-3.4). UD cases had a longer average LOS and higher transfer rates to our main hospital campus, consulting service use, and 30-day readmission rates.

CONCLUSIONS

Hospitalized children with UDs have meaningfully different patterns of diagnosis code use and increased health care utilization compared with matched controls.

Diagnostic Error in Pediatrics: A Narrative Review

A priority topic for patient safety research is diagnostic errors. However, despite the significant growth in awareness of their unacceptably high incidence and associated harm, a relative paucity of large, high-quality studies of diagnostic error in pediatrics exists. In this narrative review, we present what is known about the incidence and epidemiology of diagnostic error in pediatrics as well as the established research methods for identifying, evaluating, and reducing diagnostic errors, including their strengths and weaknesses. Additionally, we highlight that pediatric diagnostic error remains an area in need of both innovative research and quality improvement efforts to apply learnings from a rapidly growing evidence base. We propose several key research questions aimed at addressing persistent gaps in the pediatric diagnostic error literature that focus on the foundational knowledge needed to inform effective interventions to reduce the incidence of diagnostic errors and their associated harm. Additional research is needed to better establish the epidemiology of diagnostic error in pediatrics, including identifying high-risk clinical scenarios, patient populations, and groups of diagnoses. A critical need exists for validated measures of both diagnostic errors and diagnostic processes that can be adapted for different clinical settings and standardized for use across varying institutions. Pediatric researchers will need to work collaboratively on large-scale, high-quality studies to accomplish the ultimate goal of reducing diagnostic errors and their associated harm in children by addressing these fundamental gaps in knowledge.