Moving Biosurveillance Beyond Coded Data Using AI for Symptom Detection From Physician Notes: Retrospective Cohort Study

BACKGROUND

Real-time surveillance of emerging infectious diseases necessitates a dynamically evolving, computable case definition, which frequently incorporates symptom-related criteria. For symptom detection, both population health monitoring platforms and research initiatives primarily depend on structured data extracted from electronic health records.

OBJECTIVE

This study sought to validate and test an artificial intelligence (AI)-based natural language processing (NLP) pipeline for detecting COVID-19 symptoms from physician notes in pediatric patients. We specifically study patients presenting to the emergency department (ED) who can be sentinel cases in an outbreak.

METHODS

Subjects in this retrospective cohort study are patients who are 21 years of age and younger, who presented to a pediatric ED at a large academic children's hospital between March 1, 2020, and May 31, 2022. The ED notes for all patients were processed with an NLP pipeline tuned to detect the mention of 11 COVID-19 symptoms based on Centers for Disease Control and Prevention (CDC) criteria. For a gold standard, 3 subject matter experts labeled 226 ED notes and had strong agreement (F1-score=0.986; positive predictive value [PPV]=0.972; and sensitivity=1.0). F1-score, PPV, and sensitivity were used to compare the performance of both NLP and the International Classification of Diseases, 10th Revision (ICD-10) coding to the gold standard chart review. As a formative use case, variations in symptom patterns were measured across SARS-CoV-2 variant eras.

RESULTS

There were 85,678 ED encounters during the study period, including 4% (n=3420) with patients with COVID-19. NLP was more accurate at identifying encounters with patients that had any of the COVID-19 symptoms (F1-score=0.796) than ICD-10 codes (F1-score =0.451). NLP accuracy was higher for positive symptoms (sensitivity=0.930) than ICD-10 (sensitivity=0.300). However, ICD-10 accuracy was higher for negative symptoms (specificity=0.994) than NLP (specificity=0.917). Congestion or runny nose showed the highest accuracy difference (NLP: F1-score=0.828 and ICD-10: F1-score=0.042). For encounters with patients with COVID-19, prevalence estimates of each NLP symptom differed across variant eras. Patients with COVID-19 were more likely to have each NLP symptom detected than patients without this disease. Effect sizes (odds ratios) varied across pandemic eras.

CONCLUSIONS

This study establishes the value of AI-based NLP as a highly effective tool for real-time COVID-19 symptom detection in pediatric patients, outperforming traditional ICD-10 methods. It also reveals the evolving nature of symptom prevalence across different virus variants, underscoring the need for dynamic, technology-driven approaches in infectious disease surveillance.

Biomarker Assessment of a High-Risk, Data-Driven Pediatric Sepsis Phenotype Characterized by Persistent Hypoxemia, Encephalopathy, and Shock

OBJECTIVES

Identification of children with sepsis-associated multiple organ dysfunction syndrome (MODS) at risk for poor outcomes remains a challenge. We sought to the determine reproducibility of the data-driven "persistent hypoxemia, encephalopathy, and shock" (PHES) phenotype and determine its association with inflammatory and endothelial biomarkers, as well as biomarker-based pediatric risk strata.

DESIGN

We retrained and validated a random forest classifier using organ dysfunction subscores in the 2012-2018 electronic health record (EHR) dataset used to derive the PHES phenotype. We used this classifier to assign phenotype membership in a test set consisting of prospectively (2003-2023) enrolled pediatric septic shock patients. We compared profiles of the PERSEVERE family of biomarkers among those with and without the PHES phenotype and determined the association with established biomarker-based mortality and MODS risk strata.

SETTING

Twenty-five PICUs across the United States.

PATIENTS

EHR data from 15,246 critically ill patients with sepsis-associated MODS split into derivation and validation sets and 1,270 pediatric septic shock patients in the test set of whom 615 had complete biomarker data.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The area under the receiver operator characteristic curve of the modified classifier to predict PHES phenotype membership was 0.91 (95% CI, 0.90-0.92) in the EHR validation set. In the test set, PHES phenotype membership was associated with both increased adjusted odds of complicated course (adjusted odds ratio [aOR] 4.1; 95% CI, 3.2-5.4) and 28-day mortality (aOR of 4.8; 95% CI, 3.11-7.25) after controlling for age, severity of illness, and immunocompromised status. Patients belonging to the PHES phenotype were characterized by greater degree of systemic inflammation and endothelial activation, and were more likely to be stratified as high risk based on PERSEVERE biomarkers predictive of death and persistent MODS.

CONCLUSIONS

The PHES trajectory-based phenotype is reproducible, independently associated with poor clinical outcomes, and overlapped with higher risk strata based on prospectively validated biomarker approaches.

Comparing the Quality of Domain-Specific Versus General Language Models for Artificial Intelligence-Generated Differential Diagnoses in PICU Patients

OBJECTIVES

Generative language models (LMs) are being evaluated in a variety of tasks in healthcare, but pediatric critical care studies are scant. Our objective was to evaluate the utility of generative LMs in the pediatric critical care setting and to determine whether domain-adapted LMs can outperform much larger general-domain LMs in generating a differential diagnosis from the admission notes of PICU patients.

DESIGN

Single-center retrospective cohort study.

SETTING

Quaternary 40-bed PICU.

PATIENTS

Notes from all patients admitted to the PICU between January 2012 and April 2023 were used for model development. One hundred thirty randomly selected admission notes were used for evaluation.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Five experts in critical care used a 5-point Likert scale to independently evaluate the overall quality of differential diagnoses: 1) written by the clinician in the original notes, 2) generated by two general LMs (BioGPT-Large and LLaMa-65B), and 3) generated by two fine-tuned models (fine-tuned BioGPT-Large and fine-tuned LLaMa-7B). Differences among differential diagnoses were compared using mixed methods regression models. We used 1,916,538 notes from 32,454 unique patients for model development and validation. The mean quality scores of the differential diagnoses generated by the clinicians and fine-tuned LLaMa-7B, the best-performing LM, were 3.43 and 2.88, respectively (absolute difference 0.54 units [95% CI, 0.37-0.72], p < 0.001). Fine-tuned LLaMa-7B performed better than LLaMa-65B (absolute difference 0.23 unit [95% CI, 0.06-0.41], p = 0.009) and BioGPT-Large (absolute difference 0.86 unit [95% CI, 0.69-1.0], p < 0.001). The differential diagnosis generated by clinicians and fine-tuned LLaMa-7B were ranked as the highest quality in 144 (55%) and 74 cases (29%), respectively.

CONCLUSIONS

A smaller LM fine-tuned using notes of PICU patients outperformed much larger models trained on general-domain data. Currently, LMs remain inferior but may serve as an adjunct to human clinicians in real-world tasks using real-world data.

Distinct clinical phenotypes in paediatric cancer patients with sepsis are associated with different outcomes-an international multicentre retrospective study

Background

Identifying phenotypes in sepsis patients may enable precision medicine approaches. However, the generalisability of these phenotypes to specific patient populations is unclear. Given that paediatric cancer patients with sepsis have different host response and pathogen profiles and higher mortality rates when compared to non-cancer patients, we determined whether unique, reproducible, and clinically-relevant sepsis phenotypes exist in this specific patient population.

Methods

We studied patients with underlying malignancies admitted with sepsis to one of 25 paediatric intensive care units (PICUs) participating in two large, multi-centre, observational cohorts from the European SCOTER study (n = 383 patients; study period between January 1, 2018 and January 1, 2020) and the U.S. Novel Data-Driven Sepsis Phenotypes in Children study (n = 1898 patients; study period between January 1, 2012 and January 1, 2018). We independently used latent class analysis (LCA) in both cohorts to identify phenotypes using demographic, clinical, and laboratory data from the first 24 h of PICU admission. We then tested the association of the phenotypes with clinical outcomes in both cohorts.

Findings

LCA identified two distinct phenotypes that were comparable across both cohorts. Phenotype 1 was characterised by lower serum bicarbonate and albumin, markedly increased lactate and hepatic, renal, and coagulation abnormalities when compared to phenotype 2. Patients with phenotype 1 had a higher 90-day mortality (European cohort 29.2% versus 13.4%, U.S. cohort 27.3% versus 11.4%, p < 0.001) and received more vasopressor and renal replacement therapy than patients with phenotype 2. After adjusting for severity of organ dysfunction, haematological cancer, prior stem cell transplantation and age, phenotype 1 was associated with an adjusted OR of death at 90-day of 1.9 (1.04-3.34) in the European cohort and 1.6 (1.2-2.2) in the U.S. cohort.

Interpretation

We identified two clinically-relevant sepsis phenotypes in paediatric cancer patients that are reproducible across two international, multicentre cohorts with prognostic implications. These results may guide further research regarding therapeutic approaches for these specific phenotypes.

Funding

Part of this study is funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development.

Emergency department visits and boarding for pediatric patients with suicidality before and during the COVID-19 pandemic

OBJECTIVE

To quantify the increase in pediatric patients presenting to the emergency department with suicidality before and during the COVID-19 pandemic, and the subsequent impact on emergency department length of stay and boarding.

METHODS

This retrospective cohort study from June 1, 2016, to October 31, 2022, identified patients ages 6 to 21 presenting to the emergency department at a pediatric academic medical center with suicidality using ICD-10 codes. Number of emergency department encounters for suicidality, demographic characteristics of patients with suicidality, and emergency department length of stay were compared before and during the COVID-19 pandemic. Unobserved components models were used to describe monthly counts of emergency department encounters for suicidality.

RESULTS

There were 179,736 patient encounters to the emergency department during the study period, 6,215 (3.5%) for suicidality. There were, on average, more encounters for suicidality each month during the COVID-19 pandemic than before the COVID-19 pandemic. A time series unobserved components model demonstrated a temporary drop of 32.7 encounters for suicidality in April and May of 2020 (p<0.001), followed by a sustained increase of 31.2 encounters starting in July 2020 (p = 0.003). The average length of stay for patients that boarded in the emergency department with a diagnosis of suicidality was 37.4 hours longer during the COVID-19 pandemic compared to before the COVID-19 pandemic (p<0.001).

CONCLUSIONS

The number of encounters for suicidality among pediatric patients and the emergency department length of stay for psychiatry boarders has increased during the COVID-19 pandemic. There is a need for acute care mental health services and solutions to emergency department capacity issues.

Knowledge-Driven Online Multimodal Automated Phenotyping System

Though electronic health record (EHR) systems are a rich repository of clinical information with large potential, the use of EHR-based phenotyping algorithms is often hindered by inaccurate diagnostic records, the presence of many irrelevant features, and the requirement for a human-labeled training set. In this paper, we describe a knowledge-driven online multimodal automated phenotyping (KOMAP) system that i) generates a list of informative features by an online narrative and codified feature search engine (ONCE) and ii) enables the training of a multimodal phenotyping algorithm based on summary data. Powered by composite knowledge from multiple EHR sources, online article corpora, and a large language model, features selected by ONCE show high concordance with the state-of-the-art AI models (GPT4 and ChatGPT) and encourage large-scale phenotyping by providing a smaller but highly relevant feature set. Validation of the KOMAP system across four healthcare centers suggests that it can generate efficient phenotyping algorithms with robust performance. Compared to other methods requiring patient-level inputs and gold-standard labels, the fully online KOMAP provides a significant opportunity to enable multi-center collaboration.

Derivation, Validation, and Clinical Relevance of a Pediatric Sepsis Phenotype With Persistent Hypoxemia, Encephalopathy, and Shock

OBJECTIVES

Untangling the heterogeneity of sepsis in children and identifying clinically relevant phenotypes could lead to the development of targeted therapies. Our aim was to analyze the organ dysfunction trajectories of children with sepsis-associated multiple organ dysfunction syndrome (MODS) to identify reproducible and clinically relevant sepsis phenotypes and determine if they are associated with heterogeneity of treatment effect (HTE) to common therapies.

DESIGN

Multicenter observational cohort study.

SETTING

Thirteen PICUs in the United States.

PATIENTS

Patients admitted with suspected infections to the PICU between 2012 and 2018.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We used subgraph-augmented nonnegative matrix factorization to identify candidate trajectory-based phenotypes based on the type, severity, and progression of organ dysfunction in the first 72 hours. We analyzed the candidate phenotypes to determine reproducibility as well as prognostic, therapeutic, and biological relevance. Overall, 38,732 children had suspected infection, of which 15,246 (39.4%) had sepsis-associated MODS with an in-hospital mortality of 10.1%. We identified an organ dysfunction trajectory-based phenotype (which we termed persistent hypoxemia, encephalopathy, and shock) that was highly reproducible, had features of systemic inflammation and coagulopathy, and was independently associated with higher mortality. In a propensity score-matched analysis, patients with persistent hypoxemia, encephalopathy, and shock phenotype appeared to have HTE and benefit from adjuvant therapy with hydrocortisone and albumin. When compared with other high-risk clinical syndromes, the persistent hypoxemia, encephalopathy, and shock phenotype only overlapped with 50%-60% of patients with septic shock, moderate-to-severe pediatric acute respiratory distress syndrome, or those in the top tier of organ dysfunction burden, suggesting that it represents a nonsynonymous clinical phenotype of sepsis-associated MODS.

CONCLUSIONS

We derived and validated the persistent hypoxemia, encephalopathy, and shock phenotype, which is highly reproducible, clinically relevant, and associated with HTE to common adjuvant therapies in children with sepsis.

A computable case definition for patients with SARS-CoV2 testing that occurred outside the hospital

Objective

To identify a cohort of COVID-19 cases, including when evidence of virus positivity was only mentioned in the clinical text, not in structured laboratory data in the electronic health record (EHR).

Materials and Methods

Statistical classifiers were trained on feature representations derived from unstructured text in patient EHRs. We used a proxy dataset of patients with COVID-19 polymerase chain reaction (PCR) tests for training. We selected a model based on performance on our proxy dataset and applied it to instances without COVID-19 PCR tests. A physician reviewed a sample of these instances to validate the classifier.

Results

On the test split of the proxy dataset, our best classifier obtained 0.56 F1, 0.6 precision, and 0.52 recall scores for SARS-CoV2 positive cases. In an expert validation, the classifier correctly identified 97.6% (81/84) as COVID-19 positive and 97.8% (91/93) as not SARS-CoV2 positive. The classifier labeled an additional 960 cases as not having SARS-CoV2 lab tests in hospital, and only 177 of those cases had the ICD-10 code for COVID-19.

Discussion

Proxy dataset performance may be worse because these instances sometimes include discussion of pending lab tests. The most predictive features are meaningful and interpretable. The type of external test that was performed is rarely mentioned.

Conclusion

COVID-19 cases that had testing done outside of the hospital can be reliably detected from the text in EHRs. Training on a proxy dataset was a suitable method for developing a highly performant classifier without labor-intensive labeling efforts.

Validation and Extension of the Association Between Potentially Excess Oxygen Exposure and Death in Mechanically Ventilated Children

OBJECTIVES

"Cumulative excess oxygen exposure" (CEOE)-previously defined as the mean hourly administered Fio2 above 0.21 when the corresponding hourly Spo2 was 95% or above-was previously shown to be associated with mortality. The objective of this study was to examine the relationship among Fio2, Spo2, and mortality in an independent cohort of mechanically ventilated children.

DESIGN

Retrospective cross-sectional study.

SETTING

Quaternary-care PICU.

PATIENTS

All patients admitted to the PICU between 2012 and 2021 and mechanically ventilated via endotracheal tube for at least 24 hours.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Among 3,354 patients, 260 (8%) died. Higher CEOE quartile was associated with increased mortality (p = 0.001). The highest CEOE quartile had an 87% increased risk of mortality (95% CI, 7-236) compared with the first CEOE quartile. The hazard ratio for extended CEOE exposure, which included mechanical ventilation data from throughout the patients' mechanical ventilation time rather than only from the first 24 hours of mechanical ventilation, was 1.03 (95% CI, 1.02-1.03).

CONCLUSIONS

Potentially excess oxygen exposure in patients whose oxygen saturation was at least 95% was associated with increased mortality.

External validation and biomarker assessment of a high-risk, data-driven pediatric sepsis phenotype characterized by persistent hypoxemia, encephalopathy, and shock

Objective

Identification of children with sepsis-associated multiple organ dysfunction syndrome (MODS) at risk for poor outcomes remains a challenge. Data-driven phenotyping approaches that leverage electronic health record (EHR) data hold promise given the widespread availability of EHRs. We sought to externally validate the data-driven 'persistent hypoxemia, encephalopathy, and shock' (PHES) phenotype and determine its association with inflammatory and endothelial biomarkers, as well as biomarker-based pediatric risk-strata.

Design

We trained and validated a random forest classifier using organ dysfunction subscores in the EHR dataset used to derive the PHES phenotype. We used the classifier to assign phenotype membership in a test set consisting of prospectively enrolled pediatric septic shock patients. We compared biomarker profiles of those with and without the PHES phenotype and determined the association with established biomarker-based mortality and MODS risk-strata.

Setting

25 pediatric intensive care units (PICU) across the U.S.

Patients

EHR data from 15,246 critically ill patients sepsis-associated MODS and 1,270 pediatric septic shock patients in the test cohort of whom 615 had biomarker data.

Interventions

None.

Measurements and Main Results

The area under the receiver operator characteristic curve (AUROC) of the new classifier to predict PHES phenotype membership was 0.91(95%CI, 0.90-0.92) in the EHR validation set. In the test set, patients with the PHES phenotype were independently associated with both increased odds of complicated course (adjusted odds ratio [aOR] of 4.1, 95%CI: 3.2-5.4) and 28-day mortality (aOR of 4.8, 95%CI: 3.11-7.25) after controlling for age, severity of illness, and immuno-compromised status. Patients belonging to the PHES phenotype were characterized by greater degree of systemic inflammation and endothelial activation, and overlapped with high risk-strata based on PERSEVERE biomarkers predictive of death and persistent MODS.

Conclusions

The PHES trajectory-based phenotype is reproducible, independently associated with poor clinical outcomes, and overlap with higher risk-strata based on validated biomarker approaches.

Examining the Evidence for Escalating Antimicrobial Regimens in Febrile Oncology and Hematopoietic Stem Cell Transplant Patients Admitted to the PICU: An Observational Study

OBJECTIVES

To examine whether escalating antimicrobial treatment in pediatric oncology and hematopoietic cell transplantation (HSCT) patients admitted to the PICU is supported by culture data or affects patient outcomes.

DESIGN

Retrospective cross-sectional study.

SETTING

Quaternary care PICU.

PATIENTS

Patients younger than 18 years old who were admitted to the PICU at Boston Children's Hospital from 2012 to 2017 with a diagnosis of cancer or who had received HSCT and who had suspected sepsis at the time of PICU admission.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Of 791 PICU admissions for 544 patients that met inclusion criteria, 71 (9%) had escalation of antimicrobial therapy. Median Pediatric Logistic Organ Dysfunction (PELOD) score was higher in the escalation group (4 vs 3; p = 0.01). There were 14 admissions (20%) with a positive culture in the escalation group and 110 (15%) in the no escalation group ( p = 0.31). In the escalation group, there were only 2 (3%) cultures with organisms resistant to the initial antimicrobial regimen, compared with 28 (4%) cultures with resistant organisms in the no escalation group ( p = 1). Mortality in the escalation group was higher (17%) compared with the nonescalation group (5%; p < 0.001). The escalation group had more acute kidney injury (AKI) (25%) during treatment compared with the no escalation group (15%; p = 0.04), although this difference was not statistically significant when controlling for age, neutropenia, and PELOD-2 score (odds ratio, 1.75; 95% CI, 0.95-3.08; p = 0.06).

CONCLUSIONS

Few patients who had escalation of antimicrobials proved on culture data to have an organism resistant to the initial antimicrobials, and more patients developed AKI during escalated treatment. While the escalation group likely represents a sicker population, whether some of these patients would be safer without escalation of antimicrobial therapy warrants further study.

Heart Rate Change as a Potential Digital Biomarker of Brain Death in Critically Ill Children With Acute Catastrophic Brain Injury

Bedside measurement of heart rate (HR) change (HRC) may provide an objective physiologic marker for when brain death (BD) may have occurred, and BD testing is indicated in children.

OBJECTIVES

To determine whether HRC, calculated using numeric HR measurements sampled every 5 seconds, can identify patients with BD among patients with catastrophic brain injury (CBI).

DESIGN SETTING AND PARTICIPANTS

Single-center, retrospective study (2008-2020) of critically ill children with acute CBI. Patients with CBI had a neurocritical care consultation, were admitted to an ICU, had acute neurologic injury on presentation or during hospitalization based on clinical and/or imaging findings, and died or survived with Glasgow Coma Scale (GCS) less than 13 at hospital discharge. Patients meeting BD criteria (BD group) were compared with those with cardiopulmonary death (CD group) or those who survived to discharge.

MAIN OUTCOMES AND MEASURES

HRC was calculated as the interquartile range of HR divided by median HR using 5-minute windows with 50% overlap for up to 5 days before death or end of recording. HRC was compared among the BD, CD, and survivor groups.

RESULTS

Of 96 patients with CBI (69% male, median age 4 years), 28 died (8 BD, 20 CD) and 20 survived (median GCS 9 at discharge). Within 24 hours before death, HRC was lower in BD compared with CD patients or survivors (0.01 vs 0.03 vs 0.04, p = 0.001). In BD patients, HRC decreased at least 1 day before death. HRC discriminated BD from CD patients and survivors with 90% sensitivity, 70% specificity, 44% positive predictive value, 96% negative predictive value (area under the receiver operating characteristic curve 0.88, 95% CI, 0.80-0.93).

CONCLUSIONS AND RELEVANCE

HRC is a novel digital biomarker that, with further validation, may be useful as a classifier for BD in the overall course of patients with CBI.

A computable phenotype for patients with SARS-CoV2 testing that occurred outside the hospital

Objective

To identify a cohort of COVID-19 cases, including when evidence of virus positivity was only mentioned in the clinical text, not in structured laboratory data in the electronic health record (EHR).

Materials and Methods

Statistical classifiers were trained on feature representations derived from unstructured text in patient electronic health records (EHRs). We used a proxy dataset of patients with COVID-19 polymerase chain reaction (PCR) tests for training. We selected a model based on performance on our proxy dataset and applied it to instances without COVID-19 PCR tests. A physician reviewed a sample of these instances to validate the classifier.

Results

On the test split of the proxy dataset, our best classifier obtained 0.56 F1, 0.6 precision, and 0.52 recall scores for SARS-CoV2 positive cases. In an expert validation, the classifier correctly identified 90.8% (79/87) as COVID-19 positive and 97.8% (91/93) as not SARS-CoV2 positive. The classifier identified an additional 960 positive cases that did not have SARS-CoV2 lab tests in hospital, and only 177 of those cases had the ICD-10 code for COVID-19.

Discussion

Proxy dataset performance may be worse because these instances sometimes include discussion of pending lab tests. The most predictive features are meaningful and interpretable. The type of external test that was performed is rarely mentioned.

Conclusion

COVID-19 cases that had testing done outside of the hospital can be reliably detected from the text in EHRs. Training on a proxy dataset was a suitable method for developing a highly performant classifier without labor intensive labeling efforts.

Weakly Semi-supervised phenotyping using Electronic Health records

OBJECTIVE

Electronic Health Record (EHR) based phenotyping is a crucial yet challenging problem in the biomedical field. Though clinicians typically determine patient-level diagnoses via manual chart review, the sheer volume and heterogeneity of EHR data renders such tasks challenging, time-consuming, and prohibitively expensive, thus leading to a scarcity of clinical annotations in EHRs. Weakly supervised learning algorithms have been successfully applied to various EHR phenotyping problems, due to their ability to leverage information from large quantities of unlabeled samples to better inform predictions based on a far smaller number of patients. However, most weakly supervised methods are subject to the challenge to choose the right cutoff value to generate an optimal classifier. Furthermore, since they only utilize the most informative features (i.e., main ICD and NLP counts) they may fail for episodic phenotypes that cannot be consistently detected via ICD and NLP data. In this paper, we propose a label-efficient, weakly semi-supervised deep learning algorithm for EHR phenotyping (WSS-DL), which overcomes the limitations above.

MATERIALS AND METHODS

WSS-DL classifies patient-level disease status through a series of learning stages: 1) generating silver standard labels, 2) deriving enhanced-silver-standard labels by fitting a weakly supervised deep learning model to data with silver standard labels as outcomes and high dimensional EHR features as input, and 3) obtaining the final prediction score and classifier by fitting a supervised learning model to data with a minimal number of gold standard labels as the outcome, and the enhanced-silver-standard labels and a minimal set of most informative EHR features as input. To assess the generalizability of WSS-DL across different phenotypes and medical institutions, we apply WSS-DL to classify a total of 17 diseases, including both acute and chronic conditions, using EHR data from three healthcare systems. Additionally, we determine the minimum quantity of training labels required by WSS-DL to outperform existing supervised and semi-supervised phenotyping methods.

RESULTS

The proposed method, in combining the strengths of deep learning and weakly semi-supervised learning, successfully leverages the crucial phenotyping information contained in EHR features from unlabeled samples. Indeed, the deep learning model's ability to handle high-dimensional EHR features allows it to generate strong phenotype status predictions from silver standard labels. These predictions, in turn, provide highly effective features in the final logistic regression stage, leading to high phenotyping accuracy in notably small subsets of labeled data (e.g. n = 40 labeled samples).

CONCLUSION

Our method's high performance in EHR datasets with very small numbers of labels indicates its potential value in aiding doctors to diagnose rare diseases as well as conditions susceptible to misdiagnosis.

Scalable relevance ranking algorithm via semantic similarity assessment improves efficiency of medical chart review

OBJECTIVE

Accurately assigning phenotype information to individual patients via computational phenotyping using Electronic Health Records (EHRs) has been seen as the first step towards enabling EHRs for precision medicine research. Chart review labels annotated by clinical experts, also known as "gold standard" labels, are essential for the development and validation of computational phenotyping algorithms. However, given the complexity of EHR systems, the process of chart review is both labor intensive and time consuming. We propose a fully automated algorithm, referred to as pGUESS, to rank EHR notes according to their relevance to a given phenotype. By identifying the most relevant notes, pGUESS can greatly improve the efficiency and accuracy of chart reviews.

METHOD

pGUESS uses prior guided semantic similarity to measure the informativeness of a clinical note to a given phenotype. We first select candidate clinical concepts from a pool of comprehensive medical concepts using public knowledge sources and then derive the semantic embedding vector (SEV) for a reference article (SEV_ref) and each note (SEV_note). The algorithm scores the relevance of a note as the cosine similarity between SEV_note and SEV_ref.

RESULTS

The algorithm was validated against four sets of 200 notes that were manually annotated by clinical experts to assess their informativeness to one of three disease phenotypes. pGUESS algorithm substantially outperforms existing unsupervised approaches for classifying the relevance status with respect to both accuracy and scalability across phenotypes. Averaging over the three phenotypes, the rank correlation between the algorithm ranking and gold standard label was 0.64 for pGUESS, but only 0.47 and 0.35 for the next two best performing algorithms. pGUESS is also much more computationally scalable compared to existing algorithms.

CONCLUSION

pGUESS algorithm can substantially reduce the burden of chart review and holds potential in improving the efficiency and accuracy of human annotation.

Implementation Science in Pediatric Critical Care - Sedation and Analgesia Practices as a Case Study

Sedation and analgesia (SA) management is essential practice in the pediatric intensive care unit (PICU). Over the past decade, there has been significant interest in optimal SA management strategy, due to reports of the adverse effects of SA medications and their relationship to ICU delirium. We reviewed 13 studies examining SA practices in the PICU over the past decade for the purposes of reporting the study design, outcomes of interest, SA protocols used, strategies for implementation, and the patient-centered outcomes. We highlighted the paucity of evidence-base for these practices and also described the existing gaps in the intersection of implementation science (IS) and SA protocols in the PICU. Future studies would benefit from a focus on effective implementation strategies to introduce and sustain evidence-based SA protocols, as well as novel quasi-experimental study designs that will help determine their impact on relevant clinical outcomes, such as the occurrence of ICU delirium. Adoption of the available evidence-based practices into routine care in the PICU remains challenging. Using SA practice as an example, we illustrated the need for a structured approach to the implementation science in pediatric critical care. Key components of the successful adoption of evidence-based best practice include the assessment of the local context, both resources and barriers, followed by a context-specific strategy for implementation and a focus on sustainability and integration of the practice into the permanent workflow.

Clinical Characteristics and Outcomes of Children with Acute Catastrophic Brain Injury: A 13-Year Retrospective Cohort Study

BACKGROUND

The purpose of this study was to describe and analyze clinical characteristics and outcomes in children with acute catastrophic brain injury (CBI).

METHODS

This was a single-center, 13-year (2008-2020) retrospective cohort study of children in the pediatric and cardiac intensive care units with CBI, defined as (1) acute neurologic injury based on clinical and/or imaging findings, (2) the need for life-sustaining intensive care unit therapies, and (3) death or survival with a Glasgow Coma Scale score < 13 at discharge. Patients were excluded if they were discharged directly to home < 14 days from admission or had a chronic neurologic condition with a baseline Glasgow Coma Scale score < 13. The association between the primary outcome of death and clinical variables was analyzed by using Kaplan-Meier estimates and multivariable Cox proportional hazard models. Outcomes assessed after discharge were technology dependence, neurologic deficits, and Functional Status Score. Improved functional status was defined as a change in total Functional Status Score [Formula: see text] 2.

RESULTS

Of 106 patients (58% boys, median age 3.9 years) with CBI, 86 (81%) died. Withdrawal of life-sustaining therapies was the most common cause of death (60 of 86, 70%). In our multivariable analysis, each unit increase in admission pediatric sequential organ failure assessment score was associated with 10% greater hazard of death (hazard ratio 1.10, 95% confidence interval 1.04-1.17, p < .01). After controlling for admission pediatric sequential organ failure assessment scores, compared with those of patients with traumatic brain injury, all other etiologies of CBI were associated with a greater hazard of death (p = .02; hazard ratio 3.76-10). The median survival time for the cohort was 22 days (95% confidence interval 14-37 days). Of 23 survivors to hospital discharge, 20 were still alive after a median of 2 years (interquartile range 1-3 years), 6 of 20 (30%) did not have any technology dependence, 12 of 20 (60%) regained normal levels of alertness and responsiveness, and 15 of 20 (75%) had improved functional status.

CONCLUSIONS

Most children with acute CBI died within 1 month of hospitalization. Having traumatic brain injury as the etiology of CBI was associated with greater survival, whereas increased organ dysfunction score on admission was associated with a higher hazard of mortality. Of the survivors, some recovered consciousness and functional status and did not require permanent technology dependence. Larger prospective studies are needed to improve prediction of CBI among critically ill children, understand factors guiding clinician and family decisions on the continuation or withdrawal of life-sustaining treatments, and characterize the natural history and long-term outcomes among CBI survivors.

Data-driven clustering identifies features distinguishing multisystem inflammatory syndrome from acute COVID-19 in children and adolescents

BACKGROUND

Multisystem inflammatory syndrome in children (MIS-C) consensus criteria were designed for maximal sensitivity and therefore capture patients with acute COVID-19 pneumonia.

METHODS

We performed unsupervised clustering on data from 1,526 patients (684 labeled MIS-C by clinicians) <21 years old hospitalized with COVID-19-related illness admitted between 15 March 2020 and 31 December 2020. We compared prevalence of assigned MIS-C labels and clinical features among clusters, followed by recursive feature elimination to identify characteristics of potentially misclassified MIS-C-labeled patients.

FINDINGS

Of 94 clinical features tested, 46 were retained for clustering. Cluster 1 patients (N = 498; 92% labeled MIS-C) were mostly previously healthy (71%), with mean age 7·2 ± 0·4 years, predominant cardiovascular (77%) and/or mucocutaneous (82%) involvement, high inflammatory biomarkers, and mostly SARS-CoV-2 PCR negative (60%). Cluster 2 patients (N = 445; 27% labeled MIS-C) frequently had pre-existing conditions (79%, with 39% respiratory), were similarly 7·4 ± 2·1 years old, and commonly had chest radiograph infiltrates (79%) and positive PCR testing (90%). Cluster 3 patients (N = 583; 19% labeled MIS-C) were younger (2·8 ± 2·0 y), PCR positive (86%), with less inflammation. Radiographic findings of pulmonary infiltrates and positive SARS-CoV-2 PCR accurately distinguished cluster 2 MIS-C labeled patients from cluster 1 patients.

INTERPRETATION

Using a data driven, unsupervised approach, we identified features that cluster patients into a group with high likelihood of having MIS-C. Other features identified a cluster of patients more likely to have acute severe COVID-19 pulmonary disease, and patients in this cluster labeled by clinicians as MIS-C may be misclassified. These data driven phenotypes may help refine the diagnosis of MIS-C.

Implementation of an Analgesia-Sedation Protocol Is Associated With Reduction in Midazolam Usage in the PICU

OBJECTIVES

Examine the association of a revised analgesia-sedation protocol with midazolam usage in the PICU.

DESIGN

A single-center nonrandomized before-after study.

SETTING

PICU at a quaternary pediatric hospital (Boston Children's Hospital, Boston, MA).

PATIENTS

Children admitted to the PICU who were mechanically ventilated for greater than 24 hours. The preimplementation cohort included 190 eligible patients admitted between July 29, 2017, and February 28, 2018, and the postimplementation cohort included 144 patients admitted between July 29, 2019, and February 28, 2020.

INTERVENTIONS

Implementation of a revised analgesia-sedation protocol.

MEASUREMENTS AND MAIN RESULTS

Our primary outcome, total dose of IV midazolam administered in mechanically ventilated patients up to day 14 of ventilation, decreased by 72% (95% CI [61-80%]; p < 0.001) in the postimplementation cohort. Dexmedetomidine usage increased 230% (95% CI [145-344%]) in the postimplementation cohort. Opioid usage, our balancing metric, was not significantly different between the two cohorts. There were no significant differences in ventilator-free days, PICU length of stay, rate of unplanned extubations, failed extubations, cardiorespiratory arrest events, and 24-hour readmissions to the PICU.

CONCLUSIONS

We successfully implemented an analgesia-sedation protocol that primarily uses dexmedetomidine and intermittent opioids, and it was associated with significant decrease in overall midazolam usage in mechanically ventilated patients in the PICU. The intervention was not associated with changes in opioid usage or prevalence of adverse events.

eSIMPLER: A Dynamic, Electronic Health Record-Integrated Checklist for Clinical Decision Support During PICU Daily Rounds

OBJECTIVES

Design, implement, and evaluate a rounding checklist with deeply embedded, dynamic electronic health record integration.

DESIGN

Before-after quality-improvement study.

SETTING

Quaternary PICU in an academic, free-standing children's hospital.

PATIENTS

All patients in the PICU during daily morning rounds.

INTERVENTIONS

Implementation of an updated dynamic checklist (eSIMPLER) providing clinical decision support prompts with display of relevant data automatically pulled from the electronic health record.

MEASUREMENTS AND MAIN RESULTS

The prior daily rounding checklist, eSIMPLE, was implemented for 49,709 patient-days (7,779 patients) between October 30, 2011, and October 7, 2018. eSIMPLER was implemented for 5,306 patient-days (971 patients) over 6 months. Checklist completion rates were similar (eSIMPLE: 95% [95% CI, 88-98%] vs eSIMPLER: 98% [95% CI, 92-100%] of patient-days; p = 0.40). eSIMPLER required less time per patient (28 ± 1 vs 47 ± 24 s; p < 0.001). Users reported improved satisfaction with eSIMPLER (p = 0.009). Several checklist-driven process measures-discordance between electronic health record orders for stress ulcer prophylaxis and user-recorded indication for stress ulcer prophylaxis, rate of venous thromboembolism prophylaxis prescribing, and recognition of reduced renal function-improved during the eSIMPLER phase.

CONCLUSIONS

eSIMPLER, a dynamic, electronic health record-informed checklist, required less time to complete and improved certain care processes compared with a prior, static checklist with limited electronic health record data. By focusing on the "Five Rights" of clinical decision support, we created a well-accepted clinical decision support tool that was integrated efficiently into daily rounds. Generalizability of eSIMPLER's effectiveness and its impact on patient outcomes need to be examined.

Validation of an internationally derived patient severity phenotype to support COVID-19 analytics from electronic health record data

OBJECTIVE

The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing coronavirus disease 2019 (COVID-19) with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity.

MATERIALS AND METHODS

Twelve 4CE sites participated. First, we developed an EHR-based severity phenotype consisting of 6 code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of intensive care unit (ICU) admission and/or death. We also piloted an alternative machine learning approach and compared selected predictors of severity with the 4CE phenotype at 1 site.

RESULTS

The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability-up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean area under the curve of 0.903 (95% confidence interval, 0.886-0.921), compared with an area under the curve of 0.956 (95% confidence interval, 0.952-0.959) for the machine learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared with chart review.

DISCUSSION

We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly owing to heterogeneous pandemic conditions.

CONCLUSIONS

We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.

Coagulations Studies Do Not Correlate With Each Other or With Hematologic Complications During Pediatric Extracorporeal Membrane Oxygenation

OBJECTIVES

Anticoagulation plays a key role in the management of children supported with extracorporeal membrane oxygenation. However, the ideal strategy for monitoring anticoagulation remains unclear. Our objective was to evaluate the utility of laboratory measures of anticoagulation in pediatric extracorporeal membrane oxygenation.

DESIGN

Retrospective cohort study.

SETTING

Quaternary care academic children's hospital.

PATIENTS

Children in a noncardiac PICU cannulated to extracorporeal membrane oxygenation in 2010-2016.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Demographic data, laboratory values, and heparin doses were extracted from the enterprise data warehouse. Primary diagnoses, indications for cannulation, hemorrhagic and thrombotic complications, and survival outcomes were abstracted from the local registry used for Extracorporeal Life Support Organization reporting. Statistical models accounting for repeated measures using generalized estimating equations were constructed to evaluate correlations between heparin doses and laboratory values; among laboratory values; and between heparin dose or laboratory values and clinical outcomes. One hundred thirty-three unique patients-78 neonates and 55 older patients-were included in the study. There was no significant association between antifactor Xa level, activated partial thromboplastin time, activated clotting time, or heparin dose with hemorrhage or thrombosis (odds ratio ≅ 1 for all associations). There was weak-to-moderate correlation between antifactor Xa, activated partial thromboplastin time, and activated clotting time in both neonates and older pediatric patients (R2 < 0.001 to 0.456). Heparin dose correlated poorly with laboratory measurements in both age groups (R2 = 0.010-0.063).

CONCLUSIONS

In children supported with extracorporeal membrane oxygenation, heparin dose correlates poorly with common laboratory measures of anticoagulation, and these laboratory measures correlate poorly with each other. Neither heparin dose nor laboratory measures correlate with hemorrhage or thrombosis. Further work is needed to identify better measures of anticoagulation in order to minimize morbidity and mortality associated with extracorporeal membrane oxygenation.

International Analysis of Electronic Health Records of Children and Youth Hospitalized With COVID-19 Infection in 6 Countries

IMPORTANCE

Additional sources of pediatric epidemiological and clinical data are needed to efficiently study COVID-19 in children and youth and inform infection prevention and clinical treatment of pediatric patients.

OBJECTIVE

To describe international hospitalization trends and key epidemiological and clinical features of children and youth with COVID-19.

DESIGN, SETTING, AND PARTICIPANTS

This retrospective cohort study included pediatric patients hospitalized between February 2 and October 10, 2020. Patient-level electronic health record (EHR) data were collected across 27 hospitals in France, Germany, Spain, Singapore, the UK, and the US. Patients younger than 21 years who tested positive for COVID-19 and were hospitalized at an institution participating in the Consortium for Clinical Characterization of COVID-19 by EHR were included in the study.

MAIN OUTCOMES AND MEASURES

Patient characteristics, clinical features, and medication use.

RESULTS

There were 347 males (52%; 95% CI, 48.5-55.3) and 324 females (48%; 95% CI, 44.4-51.3) in this study's cohort. There was a bimodal age distribution, with the greatest proportion of patients in the 0- to 2-year (199 patients [30%]) and 12- to 17-year (170 patients [25%]) age range. Trends in hospitalizations for 671 children and youth found discrete surges with variable timing across 6 countries. Data from this cohort mirrored national-level pediatric hospitalization trends for most countries with available data, with peaks in hospitalizations during the initial spring surge occurring within 23 days in the national-level and 4CE data. A total of 27 364 laboratory values for 16 laboratory tests were analyzed, with mean values indicating elevations in markers of inflammation (C-reactive protein, 83 mg/L; 95% CI, 53-112 mg/L; ferritin, 417 ng/mL; 95% CI, 228-607 ng/mL; and procalcitonin, 1.45 ng/mL; 95% CI, 0.13-2.77 ng/mL). Abnormalities in coagulation were also evident (D-dimer, 0.78 ug/mL; 95% CI, 0.35-1.21 ug/mL; and fibrinogen, 477 mg/dL; 95% CI, 385-569 mg/dL). Cardiac troponin, when checked (n = 59), was elevated (0.032 ng/mL; 95% CI, 0.000-0.080 ng/mL). Common complications included cardiac arrhythmias (15.0%; 95% CI, 8.1%-21.7%), viral pneumonia (13.3%; 95% CI, 6.5%-20.1%), and respiratory failure (10.5%; 95% CI, 5.8%-15.3%). Few children were treated with COVID-19-directed medications.

CONCLUSIONS AND RELEVANCE

This study of EHRs of children and youth hospitalized for COVID-19 in 6 countries demonstrated variability in hospitalization trends across countries and identified common complications and laboratory abnormalities in children and youth with COVID-19 infection. Large-scale informatics-based approaches to integrate and analyze data across health care systems complement methods of disease surveillance and advance understanding of epidemiological and clinical features associated with COVID-19 in children and youth.

Adverse drug event rates in pediatric pulmonary hypertension: a comparison of real-world data sources

Objective

Real-world data (RWD) are increasingly used for pharmacoepidemiology and regulatory innovation. Our objective was to compare adverse drug event (ADE) rates determined from two RWD sources, electronic health records and administrative claims data, among children treated with drugs for pulmonary hypertension.

Materials and Methods

Textual mentions of medications and signs/symptoms that may represent ADEs were identified in clinical notes using natural language processing. Diagnostic codes for the same signs/symptoms were identified in our electronic data warehouse for the patients with textual evidence of taking pulmonary hypertension-targeted drugs. We compared rates of ADEs identified in clinical notes to those identified from diagnostic code data. In addition, we compared putative ADE rates from clinical notes to those from a healthcare claims dataset from a large, national insurer.

Results

Analysis of clinical notes identified up to 7-fold higher ADE rates than those ascertained from diagnostic codes. However, certain ADEs (eg, hearing loss) were more often identified in diagnostic code data. Similar results were found when ADE rates ascertained from clinical notes and national claims data were compared.

Discussion

While administrative claims and clinical notes are both increasingly used for RWD-based pharmacovigilance, ADE rates substantially differ depending on data source.

Conclusion

Pharmacovigilance based on RWD may lead to discrepant results depending on the data source analyzed. Further work is needed to confirm the validity of identified ADEs, to distinguish them from disease effects, and to understand tradeoffs in sensitivity and specificity between data sources.

Adding Continuous Vital Sign Information to Static Clinical Data Improves the Prediction of Length of Stay After Intubation: A Data-Driven Machine Learning Approach

BACKGROUND

Bedside monitors in the ICU routinely measure and collect patients' physiologic data in real time to continuously assess the health status of patients who are critically ill. With the advent of increased computational power and the ability to store and rapidly process big data sets in recent years, these physiologic data show promise in identifying specific outcomes and/or events during patients' ICU hospitalization.

METHODS

We introduced a methodology designed to automatically extract information from continuous-in-time vital sign data collected from bedside monitors to predict if a patient will experience a prolonged stay (length of stay) on mechanical ventilation, defined as >4 d, in a pediatric ICU.

RESULTS

Continuous-in-time vital signs information and clinical history data were retrospectively collected for 284 ICU subjects from their first 24 h on mechanical ventilation from a medical-surgical pediatric ICU at Boston Children's Hospital. Multiple machine learning models were trained on multiple subsets of these subjects to predict the likelihood that each of these subjects would experience a long stay. We evaluated the predictive power of our models strictly on unseen hold-out validation sets of subjects. Our methodology achieved model performance of >83% (area under the curve) by using only vital sign information as input, and performances of 90% (area under the curve) by combining vital sign information with subjects' static clinical data readily available in electronic health records. We implemented this approach on 300 independently trained experiments with different choices of training and hold-out validation sets to ensure the consistency and robustness of our results in our study sample. The predictive power of our approach outperformed recent efforts that used deep learning to predict a similar task.

CONCLUSIONS

Our proposed workflow may prove useful in the design of scalable approaches for real-time predictive systems in ICU environments, exploiting real-time vital sign information from bedside monitors. (ClinicalTrials.gov registration NCT02184208.).

A high-throughput phenotyping algorithm is portable from adult to pediatric populations

OBJECTIVE

Multimodal automated phenotyping (MAP) is a scalable, high-throughput phenotyping method, developed using electronic health record (EHR) data from an adult population. We tested transportability of MAP to a pediatric population.

MATERIALS AND METHODS

Without additional feature engineering or supervised training, we applied MAP to a pediatric population enrolled in a biobank and evaluated performance against physician-reviewed medical records. We also compared performance of MAP at the pediatric institution and the original adult institution where MAP was developed, including for 6 phenotypes validated at both institutions against physician-reviewed medical records.

RESULTS

MAP performed equally well in the pediatric setting (average AUC 0.98) as it did at the general adult hospital system (average AUC 0.96). MAP's performance in the pediatric sample was similar across the 6 specific phenotypes also validated against gold-standard labels in the adult biobank.

CONCLUSIONS

MAP is highly transportable across diverse populations and has potential for wide-scale use.

Adverse drug event presentation and tracking (ADEPT): semiautomated, high throughput pharmacovigilance using real-world data

Objective

To advance use of real-world data (RWD) for pharmacovigilance, we sought to integrate a high-sensitivity natural language processing (NLP) pipeline for detecting potential adverse drug events (ADEs) with easily interpretable output for high-efficiency human review and adjudication of true ADEs.

Materials and methods

The adverse drug event presentation and tracking (ADEPT) system employs an open source NLP pipeline to identify in clinical notes mentions of medications and signs and symptoms potentially indicative of ADEs. ADEPT presents the output to human reviewers by highlighting these drug-event pairs within the context of the clinical note. To measure incidence of seizures associated with sildenafil, we applied ADEPT to 149 029 notes for 982 patients with pediatric pulmonary hypertension.

Results

Of 416 patients identified as taking sildenafil, NLP found 72 [17%, 95% confidence interval (CI) 14–21] with seizures as a potential ADE. Upon human review and adjudication, only 4 (0.96%, 95% CI 0.37–2.4) patients with seizures were determined to have true ADEs. Reviewers using ADEPT required a median of 89 s (interquartile range 57–142 s) per patient to review potential ADEs.

Discussion

ADEPT combines high throughput NLP to increase sensitivity of ADE detection and human review, to increase specificity by differentiating true ADEs from signs and symptoms related to comorbidities, effects of other medications, or other confounders.

Conclusion

ADEPT is a promising tool for creating gold standard, patient-level labels for advancing NLP-based pharmacovigilance. ADEPT is a potentially time savings platform for computer-assisted pharmacovigilance based on RWD.

Noninvasive Ventilation Is Interrupted Frequently and Mostly Used at Night in the Pediatric Intensive Care Unit

BACKGROUND

Noninvasive ventilation (NIV) is commonly used to support children with respiratory failure, but detailed patterns of real-world use are lacking. The aim of our study was to describe use patterns of NIV via electronic medical record (EMR) data.

METHODS

We performed a retrospective electronic chart review in a tertiary care pediatric ICU in the United States. Subjects admitted to the pediatric ICU from 2014 to 2017 who were mechanically ventilated were included in the study.

RESULTS

The median number of discrete device episodes, defined as a time on support without interruption, was 20 (interquartile range [IQR] 8-49) per subject. The median duration of bi-level positive airway pressure (BPAP) support prior to interruption was 6.3 h (IQR 2.4-10.4); the median duration of CPAP was 6 h (IQR 2.1-10.4). Interruptions to BPAP had a median duration of 6.3 h (IQR 2-15.5); interruptions to CPAP had a median duration of 8.6 h (IQR 2.2-16.8). Use of NIV followed a diurnal pattern, with 44% of BPAP and 42% of CPAP subjects initiating support between 7:00 pm and midnight, and 49% of BPAP and 46% of CPAP subjects stopping support between 5:00 am and 10:00 am.

CONCLUSIONS

NIV was frequently interrupted, and initiation and discontinuation of NIV follows a diurnal pattern. Use of EMR data collected for routine clinical care allowed the analysis of granular details of typical use patterns. Understanding NIV use patterns may be particularly important to understanding the burden of pediatric ICU bed utilization for nocturnal NIV. To our knowledge, this is the first study to examine in detail the use of pediatric NIV and to define diurnal use and frequent interruptions to support.

Feature extraction for phenotyping from semantic and knowledge resources

OBJECTIVE

Phenotyping algorithms can efficiently and accurately identify patients with a specific disease phenotype and construct electronic health records (EHR)-based cohorts for subsequent clinical or genomic studies. Previous studies have introduced unsupervised EHR-based feature selection methods that yielded algorithms with high accuracy. However, those selection methods still require expert intervention to tweak the parameter settings according to the EHR data distribution for each phenotype. To further accelerate the development of phenotyping algorithms, we propose a fully automated and robust unsupervised feature selection method that leverages only publicly available medical knowledge sources, instead of EHR data.

METHODS

SEmantics-Driven Feature Extraction (SEDFE) collects medical concepts from online knowledge sources as candidate features and gives them vector-form distributional semantic representations derived with neural word embedding and the Unified Medical Language System Metathesaurus. A number of features that are semantically closest and that sufficiently characterize the target phenotype are determined by a linear decomposition criterion and are selected for the final classification algorithm.

RESULTS

SEDFE was compared with the EHR-based SAFE algorithm and domain experts on feature selection for the classification of five phenotypes including coronary artery disease, rheumatoid arthritis, Crohn's disease, ulcerative colitis, and pediatric pulmonary arterial hypertension using both supervised and unsupervised approaches. Algorithms yielded by SEDFE achieved comparable accuracy to those yielded by SAFE and expert-curated features. SEDFE is also robust to the input semantic vectors.

CONCLUSION

SEDFE attains satisfying performance in unsupervised feature selection for EHR phenotyping. Both fully automated and EHR-independent, this method promises efficiency and accuracy in developing algorithms for high-throughput phenotyping.

A Computable Phenotype Improves Cohort Ascertainment in a Pediatric Pulmonary Hypertension Registry

OBJECTIVES

To compare registry and electronic health record (EHR) data mining approaches for cohort ascertainment in patients with pediatric pulmonary hypertension (PH) in an effort to overcome some of the limitations of registry enrollment alone in identifying patients with particular disease phenotypes.

STUDY DESIGN

This study was a single-center retrospective analysis of EHR and registry data at Boston Children's Hospital. The local Informatics for Integrating Biology and the Bedside (i2b2) data warehouse was queried for billing codes, prescriptions, and narrative data related to pediatric PH. Computable phenotype algorithms were developed by fitting penalized logistic regression models to a physician-annotated training set. Algorithms were applied to a candidate patient cohort, and performance was evaluated using a separate set of 136 records and 179 registry patients. We compared clinical and demographic characteristics of patients identified by computable phenotype and the registry.

RESULTS

The computable phenotype had an area under the receiver operating characteristics curve of 90% (95% CI, 85%-95%), a positive predictive value of 85% (95% CI, 77%-93%), and identified 413 patients (an additional 231%) with pediatric PH who were not enrolled in the registry. Patients identified by the computable phenotype were clinically distinct from registry patients, with a greater prevalence of diagnoses related to perinatal distress and left heart disease.

CONCLUSIONS

Mining of EHRs using computable phenotypes identified a large cohort of patients not recruited using a classic registry. Fusion of EHR and registry data can improve cohort ascertainment for the study of rare diseases.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT02249923.

Learning a Comorbidity-Driven Taxonomy of Pediatric Pulmonary Hypertension

RATIONALE

Pediatric pulmonary hypertension (PH) is a heterogeneous condition with varying natural history and therapeutic response. Precise classification of PH subtypes is, therefore, crucial for individualizing care. However, gaps remain in our understanding of the spectrum of PH in children.

OBJECTIVE

We seek to study the manifestations of PH in children and to assess the feasibility of applying a network-based approach to discern disease subtypes from comorbidity data recorded in longitudinal data sets.

METHODS AND RESULTS

A retrospective cohort study comprising 6 943 263 children (<18 years of age) enrolled in a commercial health insurance plan in the United States, between January 2010 and May 2013. A total of 1583 (0.02%) children met the criteria for PH. We identified comorbidities significantly associated with PH compared with the general population of children without PH. A Bayesian comorbidity network was constructed to model the interdependencies of these comorbidities, and network-clustering analysis was applied to derive disease subtypes comprising subgraphs of highly connected comorbid conditions. A total of 186 comorbidities were found to be significantly associated with PH. Network analysis of comorbidity patterns captured most of the major PH subtypes with known pathological basis defined by the World Health Organization and Panama classifications. The analysis further identified many subtypes documented in only a few case studies, including rare subtypes associated with several well-described genetic syndromes.

CONCLUSIONS

Application of network science to model comorbidity patterns recorded in longitudinal data sets can facilitate the discovery of disease subtypes. Our analysis relearned established subtypes, thus validating the approach, and identified rare subtypes that are difficult to discern through clinical observations, providing impetus for deeper investigation of the disease subtypes that will enrich current disease classifications.

Provider Connectedness to Other Providers Reduces Risk of Readmission After Hospitalization for Heart Failure

Provider interactions other than explicit care coordination, which is challenging to measure, may influence practice and outcomes. We performed a network analysis using claims data from a commercial payor. Networks were identified based on provider pairs billing outpatient care for the same patient. We compared network variables among patients who had and did not have a 30-day readmission after hospitalization for heart failure. After adjusting for comorbidities, high median provider connectedness-normalized degree, which for each provider is the number of connections to other providers normalized to the number of providers in the region-was the network variable associated with reduced odds of readmission after heart failure hospitalization (odds ratio = 0.55; 95% confidence interval [0.35, 0.86]). We conclude that heart failure patients with high provider connectedness are less likely to require readmission. The structure and importance of provider relationships using claims data merits further study.

Diagnostic utility of brain activity flow patterns analysis in attention deficit hyperactivity disorder

BACKGROUND

A previous small study suggested that Brain Network Activation (BNA), a novel ERP-based brain network analysis, may have diagnostic utility in attention deficit hyperactivity disorder (ADHD). In this study we examined the diagnostic capability of a new advanced version of the BNA methodology on a larger population of adults with and without ADHD.

METHOD

Subjects were unmedicated right-handed 18- to 55-year-old adults of both sexes with and without a DSM-IV diagnosis of ADHD. We collected EEG while the subjects were performing a response inhibition task (Go/NoGo) and then applied a spatio-temporal Brain Network Activation (BNA) analysis of the EEG data. This analysis produced a display of qualitative measures of brain states (BNA scores) providing information on cortical connectivity. This complex set of scores was then fed into a machine learning algorithm.

RESULTS

The BNA analysis of the EEG data recorded during the Go/NoGo task demonstrated a high discriminative capacity between ADHD patients and controls (AUC = 0.92, specificity = 0.95, sensitivity = 0.86 for the Go condition; AUC = 0.84, specificity = 0.91, sensitivity = 0.76 for the NoGo condition).

CONCLUSIONS

BNA methodology can help differentiate between ADHD and healthy controls based on functional brain connectivity. The data support the utility of the tool to augment clinical examinations by objective evaluation of electrophysiological changes associated with ADHD. Results also support a network-based approach to the study of ADHD.

Preliminary investigation of Brain Network Activation (BNA) and its clinical utility in sport-related concussion

Background: The clinical diagnosis and management of patients with sport-related concussion is largely dependent on subjectively reported symptoms, clinical examinations, cognitive, balance, vestibular and oculomotor testing. Consequently, there is an unmet need for objective assessment tools that can identify the injury from a physiological perspective and add an important layer of information to the clinician’s decision-making process.

Objective: The goal of the study was to evaluate the clinical utility of the EEG-based tool named Brain Network Activation (BNA) as a longitudinal assessment method of brain function in the management of young athletes with concussion.

Methods: Athletes with concussion (n = 86) and age-matched controls (n = 81) were evaluated at four time points with symptom questionnaires and BNA. BNA scores were calculated by comparing functional networks to a previously defined normative reference brain network model to the same cognitive task.

Results: Subjects above 16 years of age exhibited a significant decrease in BNA scores immediately following injury, as well as notable changes in functional network activity, relative to the controls. Three representative case studies of the tested population are discussed in detail, to demonstrate the clinical utility of BNA.

Conclusion: The data support the utility of BNA to augment clinical examinations, symptoms and additional tests by providing an effective method for evaluating objective electrophysiological changes associated with sport-related concussions.

Long-term outcomes and risk factors for aortic regurgitation after discrete subvalvular aortic stenosis resection in children

OBJECTIVES

To characterise long-term outcomes after discrete subaortic stenosis (DSS) resection and to identify risk factors for reoperation and aortic regurgitation (AR) requiring repair or replacement.

METHODS

All patients who underwent DSS resection between 1984 and 2009 at our institution with at least 36 months' follow-up were included. Demographic, surgical and echocardiographic data were reviewed. Outcomes were reoperation for recurrent DSS, surgery for AR, death and morbidities, including heart transplant, endocarditis and complete heart block.

RESULTS

Median length of postoperative follow-up was 10.9 years (3-27.2 years). Reoperation occurred in 32 patients (21%) and plateaued 10 years after initial resection. Survival at 10 years and 20 years was 98.6% and 86.3%, respectively. Aortic valve (AoV) repair or replacement for predominant AR occurred in 31 patients (20%) during or after DSS resection. By multivariable analysis, prior aortic stenosis (AS) intervention (HR 22.4, p<0.001) was strongly associated with AoV repair or replacement. Risk factors for reoperation by multivariable analysis included younger age at resection (HR 1.24, p=0.003), preoperative gradient ≥60 mm Hg (HR 2.23, p=0.04), peeling of membrane off AoV or mitral valve (HR 2.52, p=0.01), distance of membrane to AoV <7.0 mm (HR 4.03, p=0.03) and AS (HR 2.58, p=0.01).

CONCLUSIONS

In this cohort, the incidence of reoperations after initial DSS resection plateaued after 10 years. Despite a significant rate of reoperation, overall survival was good. Concomitant congenital AS and its associated interventions significantly increased the risk of AR requiring surgical intervention.

Spread of methicillin-resistant Staphylococcus aureus in a large tertiary NICU: network analysis

OBJECTIVE

Methicillin-resistant Staphylococcus aureus (MRSA) colonization in NICUs increases the risk of nosocomial infection. Network analysis provides tools to examine the interactions among patients and staff members that put patients at risk of colonization.

METHODS

Data from MRSA surveillance cultures were combined with patient room locations, nursing assignments, and sibship information to create patient- and unit-based networks. Multivariate models were constructed to quantify the risk of incident MRSA colonization as a function of exposure to MRSA-colonized infants in these networks.

RESULTS

A MRSA-negative infant in the NICU simultaneously with a MRSA-positive infant had higher odds of becoming colonized when the colonized infant was a sibling, compared with an unrelated patient (odds ratio: 8.8 [95% confidence interval [CI]: 5.3-14.8]). Although knowing that a patient was MRSA-positive and was placed on contact precautions reduced the overall odds of another patient becoming colonized by 35% (95% CI: 20%-47%), having a nurse in common with that patient still increased the odds of colonization by 43% (95% CI: 14%-80%). Normalized group degree centrality, a unitwide network measure of connectedness between colonized and uncolonized patients, was a significant predictor of incident MRSA cases (odds ratio: 18.1 [95% CI: 3.6-90.0]).

CONCLUSIONS

Despite current infection-control strategies, patients remain at significant risk of MRSA colonization from MRSA-positive siblings and from other patients with whom they share nursing care. Strategies that minimize the frequency of staff members caring for both colonized and uncolonized infants may be beneficial in reducing the spread of MRSA colonization.

A quantitative analysis of optimal treatment capacity for perinatal asphyxia

OBJECTIVE

In centers electing to offer therapeutic hypothermia for treating hypoxic-ischemic encephalopathy (HIE), determining the optimal number of cooling devices is not straightforward. The authors used computer-based modeling to determine the level of service as a function of local HIE caseload and number of cooling devices available.

METHODS

The authors used discrete event simulation to create a model that varied the number of HIE cases and number of cooling devices available. Outcomes of interest were percentage of HIE-affected infants not cooled, number of infants not cooled, and percentage of time that all cooling devices were in use.

RESULTS

With 1 cooling device, even the smallest perinatal center did not achieve a cooling rate of 99% of eligible infants. In contrast, 2 devices ensured 99% service in centers treating as many as 20 infants annually. In centers averaging no more than 1 HIE infant monthly, the addition of a third cooling device did not result in a substantial reduction in the number of infants who would not be cooled.

CONCLUSION

Centers electing to offer therapeutic hypothermia with only a single cooling device are at significant risk of being unable to provide treatment to eligible infants, whereas 2 devices appear to suffice for most institutions treating as many as 20 annual HIE cases. Three devices would rarely be needed given current caseloads seen at individual institutions. The quantitative nature of this analysis allows decision makers to determine the number of devices necessary to ensure adequate availability of therapeutic hypothermia given the HIE caseload of a particular institution.

Dexamethasone and Tonsillectomy Bleeding

Objective. To use meta-analytic techniques to examine the effect of dexamethasone on the risk of postoperative bleeding following tonsillectomy.

Data Sources. PubMed and Embase databases accessed on April 23, 2009, and April 28, 2009.

Review Methods. Using principles of meta-analysis, inclusion and exclusion criteria were developed to identify all randomized controlled trials of patients undergoing tonsillectomy in which perioperative intravenous dexamethasone was administered in at least 1 treatment arm and bleeding complications were reported. Electronic databases were searched to identify candidate articles. Two authors independently abstracted data from each article. Discrepancies were resolved by consensus. A fixed-effects model was used to pool relative risks among studies using the Mantel-Haenszel method. Studies were assessed for publication bias using a funnel plot of studies’ effect size vs standard error of the effect size as well as Begg test and Egger test. A P value <.05 was considered significant.

Results. The primary search identified 85 potential studies. Fourteen met inclusion criteria and were selected for meta-analysis. No significant heterogeneity was found among studies (I2< 0.1%; 95% confidence interval [CI], 0%-55%; P = .68). The pooled relative risk (RR) of postoperative bleeding did not differ significantly between patients receiving dexamethasone and controls (RR, 1.02; 95% CI, 0.65-1.61; P = .92). When studies were stratified by age, primary vs secondary hemorrhage, and follow-up duration, no further significant differences in bleeding rate were identified. No evidence of publication bias was found using Begg ( P = .70) or Egger ( P = .73) tests.

Conclusion. The results of this meta-analysis indicate that perioperative dexamethasone does not confer an increased risk of postoperative bleeding in patients undergoing tonsillectomy.

Network analysis of team structure in the neonatal intensive care unit

OBJECTIVE

The goal was to examine nursing team structure and its relationship with family satisfaction.

METHODS

We used electronic health records to create patient-based, 1-mode networks of nursing handoffs. In these networks, nurses were represented as nodes and handoffs as edges. For each patient, we calculated network statistics including team size and diameter, network centrality index, proportion of newcomers to care teams according to day of hospitalization, and a novel measure of the average number of shifts between repeat caregivers, which was meant to quantify nursing continuity. We assessed parental satisfaction by using a standardized survey.

RESULTS

Team size increased with increasing length of stay. At 2 weeks of age, 50% of shifts were staffed by a newcomer nurse who had not previously cared for the index patient. The patterns of newcomers to teams did not differ according to birth weight. When the population was dichotomized according to median mean repeat caregiver interval value, increased reports of problems with nursing care were seen with less-consistent staffing by familiar nurses. This relationship persisted after controlling for factors including birth weight, length of stay, and team size.

CONCLUSIONS

Family perceptions of nursing care quality are more strongly associated with team structure and the sequence of nursing participation than with team size. Objective measures of health care team structure and function can be examined by applying network analytic techniques to information contained in electronic health records.

CoolSim: using industrial modeling techniques to examine the impact of selective head cooling in a model of perinatal regionalization

OBJECTIVE

A selective head-cooling device for the treatment of moderate to severe hypoxic-ischemic encephalopathy has been approved by the Food and Drug Administration for use in the United States. To reflect the complexity of health care delivery at the systems level, we used the industrial modeling technique of discrete event simulation to analyze the impact of various deployment strategies for selective head cooling and its partner technology, amplitude-integrated electroencephalography.

METHODS

We modeled the course through the perinatal system of all births in Massachusetts over a 1-year period. Cohort and care characteristics were drawn from existing databases. Results of a recently published trial were used to estimate the effects of selective head cooling. One thousand cohort replications were conducted to assess uncertainty. Several policy alternatives were examined, including no use of selective head cooling and scenarios that altered the number and placement of selective head-cooling and amplitude-integrated electroencephalography units throughout the state. Patient-level outcome and cost data were assessed.

RESULTS

For all scenarios tested, the use of amplitude-integrated electroencephalography/selective head cooling resulted in better outcomes at lower cost. However, substantial differences in transfer rates, failure-to-cool rates, and total costs were seen across scenarios. Optimal decision-making regarding the number and placement of devices led to a 16% improvement in cost savings and a 10-fold decrease in failure-to-cool rates, compared with other deployment scenarios. These results were insensitive to significant changes in model inputs.

CONCLUSIONS

On the basis of currently available data, the package of amplitude-integrated electroencephalography and selective head cooling seems to be an economically desirable intervention. Quantifiable techniques to assess system-wide technology performance provide a powerful approach to informing decisions regarding the structure and function of health care systems.

Risk factors for reoperation after repair of discrete subaortic stenosis in children

OBJECTIVES

This study aimed to identify independent predictors of reoperation after successful resection of discrete subaortic stenosis (DSS).

BACKGROUND

Recurrence of DSS has been reported to range from 0% to 55% of patients. Factors associated with recurrence have not been adequately defined.

METHODS

Patients were included if they had a diagnosis of DSS, normal segmental cardiac anatomy, previous resection of DSS, and at least 36 months' follow-up. Demographic, surgical, and echocardiographic data were analyzed. Primary outcome was repeat resection of DSS in patients after successful primary resection.

RESULTS

Of 111 subjects who had successful surgical resection of DSS, 16 patients (14%) required reoperation. Median follow-up time was 8.2 years. Form of DSS and gender did not differ significantly between those with reoperation and those without. In multivariate analysis, independent predictors of reoperation that would be available before first surgery were <6 mm distance between the aortic valve (AoV) and the obstruction (hazard ratio [HR] 5.1; p = 0.013) and peak gradient by Doppler > or =60 mm Hg (HR 4.2; p = 0.016). If intraoperative variables are also considered, peeling of the membrane from the AoV or mitral valve at first surgery, <6 mm distance between the DSS and AoV, and peak gradient by Doppler > or =60 mm Hg were independent predictors of reoperation.

CONCLUSIONS

Proximity of the obstructive lesion to the AoV and severe obstruction determined by preoperative echocardiography, as well as involvement of valve leaflets requiring surgical peeling, predict recurrent DSS requiring reoperation.

Hemoglobin Jamaica plain--a sickling hemoglobin with reduced oxygen affinity

A baby girl presented with symptomatic sickle cell disease exacerbated by mild hypoxemia, despite a newborn-screening diagnosis of sickle cell trait. DNA sequencing of the beta globin gene revealed that her maternal beta globin allele was normal. Her paternal allele had not only the expected sickle-trait mutation, betaGlu6Val, but also a second, charge-neutral mutation, betaLeu68Phe. Analysis of the patient's hemoglobin revealed that the double-mutant protein, which we called "hemoglobin Jamaica Plain," had severely reduced oxygen affinity. Structural modeling suggested destabilization of the oxy conformation as a molecular mechanism for sickling in a heterozygote at an ambient partial pressure of oxygen.

Hemolytic anemia and severe rhabdomyolysis caused by compound heterozygous mutations of the gene for erythrocyte/muscle isozyme of aldolase, ALDOA(Arg303X/Cys338Tyr)

Aldolase (E.C. 4.1.2.13), a homotetrameric protein encoded by the ALDOA gene, converts fructose-1,6-bisphosphate to dihydroxyacetone phosphate and glyceraldehyde-3-phosphate. Three isozymes are encoded by distinct genes. The sole aldolase present in red blood cells and skeletal muscle is the A isozyme. We report here the case of a girl of Sicilian descent with aldolase A deficiency. Clinical manifestations included transfusion-dependent anemia until splenectomy at age 3 and increasing muscle weakness, with death at age 4 associated with rhabdomyolysis and hyperkalemia. Sequence analysis of the ALDOA coding regions revealed 2 novel heterozygous ALDOA mutations in conserved regions of the protein. The paternal allele encoded a nonsense mutation, Arg303X, in the enzyme-active site. The maternal allele encoded a missense mutation, Cys338Tyr, predicted to cause enzyme instability. This is the most severely affected patient reported to date and only the second with both rhabdomyolysis and hemolysis.

Chronic disseminated intravascular coagulation and childhood-onset skin necrosis resulting from homozygosity for a protein C Gla domain mutation, Arg15Trp

A toddler of Haitian descent presented with an 18-month history of chronic consumption coagulopathy, followed by catastrophic skin necrosis. Protein C deficiency (1% to 3% of control) was noted by functional assay; chromogenic assay and antigen levels were 30% of control. Plasma infusion abrogated the disseminated intravascular coagulation-like state. The authors identified a homozygous mutation, C1432T, resulting in a missense, Arg15Trp, in the gamma-carboxyglutamate domain of the protein. Chronic consumption coagulopathy without purpura fulminans or venous thrombosis is a rare presentation of defective protein C pathway. The result of this mutation is a mixed type I (low antigen) and type II (low function) phenotype.

Attitude towards health and hormone replacement therapy among female obstetrician-gynecologists in Israel

OBJECTIVE

To assess Israeli women gynecologists toward their own health, their health related behaviors and to assess attitude towards and the use of hormone replacement therapy (HRT).

METHODS

Ninety five actively employed hospital and community women gynecologist completed a questionnaire on attitude towards self-health, way of life, smoking habits, and regular breast, blood, pap smear examinations and HRT.

RESULTS

Mean BMI was 25.3 Kg/m(2)+/-4.2, 61% considered themselves above average weight, and only 39% estimated their weight as appropriate. Fifty six percent were on active weight-loss diets, and 35% were current smokers. Blood tests, pap smears and breast evaluations were regularly done by 73.4, 91.5, and 64.1%, respectively. Overall, 74% of the gynecologists had a positive opinion about HRT; 70% of the menopausal subgroup had ever used HRT, and 93.3% of the perimenopausal subgroup intended to use it. The main reason for starting HRT was climacteric symptoms, and for stopping or avoiding HRT were equally bleeding, fear of cancer and adverse reactions towards HRT. By far the oral HRT mode was the most popular and 90% of users expressed satisfaction with treatment.

CONCLUSIONS

Israeli women gynecologists are aware and maintain carefully their health, excluding cigarette smoking. The high rate of use and awareness of HRT among this group is encouraging considering that gynecologist serves as role model for the public and maintains the main source of HRT in the community.

Attitudes of mothers toward their daughters' use of contraceptives in Israel

The objective of the study was to examine mothers' knowledge about contraception, their attitudes to their daughters' use of contraception, and their communication with their daughters on the subject. A 20-item questionnaire was distributed in gynecological clinics throughout Israel for completion by women who had daughters over the age of 14 years. The questionnaire covered the mothers' demographic data, use of contraception, knowledge of contraception, attitude to their daughters' contraceptive use and sexual relationships, and communication with their daughters about contraception. Only 36% of the women received contraceptive information from physicians. Almost half felt their daughter should begin sexual relations when she felt she was ready; over two-thirds felt she should begin using contraception before or at the time of beginning sexual relations. Over three-quarters spoke with their daughters about contraception. Higher educational level of the mother was associated with high rate use of contraception by the mother, her support of earlier use of contraception by her daughter, a greater likelihood of her discussing contraception with her daughter, and a lesser likelihood to view contraceptives as dangerous to one's health. It is concluded that mothers of teenage daughters in Israel are involved in their daughters' decisions to begin sexual relations and the use of contraceptives. Their knowledge of contraception is adequate, although some gaps are still apparent. Clear correlation is found between higher educational level of the mothers and a more liberal attitude toward their daughters' sex life.