Social and Behavioral Determinants of Health in the Era of Artificial Intelligence with Electronic Health Records: A Scoping Review

Assessing the capture of sociodemographic information in electronic medical records to inform clinical decision making

There is a growing need to document sociodemographic factors in electronic medical records to produce representative cohorts for medical research and to perform focused research for potentially vulnerable populations. The objective of this work was to assess the content of family physicians' electronic medical records and characterize the quality of the documentation of sociodemographic characteristics. Descriptive statistics were reported for each sociodemographic characteristic. The association between the completeness rates of the sociodemographic data and the various clinics, electronic medical record vendors, and physician characteristics was analyzed. Supervised machine learning models were used to determine the absence or presence of each characteristic for all adult patients over the age of 18 in the database. Documentation of marital status (51.0%) and occupation (47.2%) were significantly higher compared to the rest of the variables. Race (1.4%), sexual orientation (2.5%), and gender identity (0.8%) had the lowest documentation rates with a 97.5% missingness rate or higher. The correlation analysis for vendor type demonstrated that there was significant variation in the availability of marital and occupation information between vendors (χ2 > 6.0, P < 0.05). Variability in documentation between clinics indicated that the majority of characteristics exhibited high variation in completeness rates with the highest variation for occupation (median: 47.2, interquartile range: 60.6%) and marital status (median: 45.6, interquartile: 59.7%). Finally, physician sex, years since a physician graduated, and whether a physician was a foreign vs a Canadian medical graduate were significantly associated with documentation rates of place of birth, citizenship status, occupation, and education in the electronic medical records. Our findings suggest a crucial need to implement better documentation strategies for sociodemographic information in the healthcare setting. To improve completeness rates, healthcare systems should monitor, encourage, enforce, or incentivize sociodemographic data collection standards.

Documentation of social determinants of health across individuals from different racial and ethnic groups in home healthcare

INTRODUCTION

Home healthcare (HHC) enables patients to receive healthcare services within their homes to manage chronic conditions and recover from illnesses. Recent research has identified disparities in HHC based on race or ethnicity. Social determinants of health (SDOH) describe the external factors influencing a patient's health, such as access to care and social support. Individuals from racially or ethnically minoritized communities are known to be disproportionately affected by SDOH. Existing evidence suggests that SDOH are documented in clinical notes. However, no prior study has investigated the documentation of SDOH across individuals from different racial or ethnic backgrounds in the HHC setting. This study aimed to (1) describe frequencies of SDOH documented in clinical notes by race or ethnicity and (2) determine associations between race or ethnicity and SDOH documentation.

DESIGN

Retrospective data analysis.

METHODS

We conducted a cross-sectional secondary data analysis of 86,866 HHC episodes representing 65,693 unique patients from one large HHC agency in New York collected between January 1, 2015, and December 31, 2017. We reported the frequency of six SDOH (physical environment, social environment, housing and economic circumstances, food insecurity, access to care, and education and literacy) documented in clinical notes across individuals reported as Asian/Pacific Islander, Black, Hispanic, multi-racial, Native American, or White. We analyzed differences in SDOH documentation by race or ethnicity using logistic regression models.

RESULTS

Compared to patients reported as White, patients across other racial or ethnic groups had higher frequencies of SDOH documented in their clinical notes. Our results suggest that race or ethnicity is associated with SDOH documentation in HHC.

CONCLUSION

As the study of SDOH in HHC continues to evolve, our results provide a foundation to evaluate social information in the HHC setting and understand how it influences the quality of care provided.

CLINICAL RELEVANCE

The results of this exploratory study can help clinicians understand the differences in SDOH across individuals from different racial and ethnic groups and serve as a foundation for future research aimed at fostering more inclusive HHC documentation practices.

Model-based estimation of individual-level social determinants of health and its applications in All of Us

OBJECTIVES

We introduce a widely applicable model-based approach for estimating individual-level Social Determinants of Health (SDoH) and evaluate its effectiveness using the All of Us Research Program.

MATERIALS AND METHODS

Our approach utilizes aggregated SDoH datasets to estimate individual-level SDoH, demonstrated with examples of no high school diploma (NOHSDP) and no health insurance (UNINSUR) variables. Models are estimated using American Community Survey data and applied to derive individual-level estimates for All of Us participants. We assess concordance between model-based SDoH estimates and self-reported SDoHs in All of Us and examine associations with undiagnosed hypertension and diabetes.

RESULTS

Compared to self-reported SDoHs, the area under the curve for NOHSDP is 0.727 (95% CI, 0.724-0.730) and for UNINSUR is 0.730 (95% CI, 0.727-0.733) among the 329 074 All of Us participants, both significantly higher than aggregated SDoHs. The association between model-based NOHSDP and undiagnosed hypertension is concordant with those estimated using self-reported NOHSDP, with a correlation coefficient of 0.649. Similarly, the association between model-based NOHSDP and undiagnosed diabetes is concordant with those estimated using self-reported NOHSDP, with a correlation coefficient of 0.900.

DISCUSSION AND CONCLUSION

The model-based SDoH estimation method offers a scalable and easily standardized approach for estimating individual-level SDoHs. Using the All of Us dataset, we demonstrate reasonable concordance between model-based SDoH estimates and self-reported SDoHs, along with consistent associations with health outcomes. Our findings also underscore the critical role of geographic contexts in SDoH estimation and in evaluating the association between SDoHs and health outcomes.

Predicting Self-Reported Social Risk in Medically Complex Adults Using Electronic Health Data

BACKGROUND

Social barriers to health care, such as food insecurity, financial distress, and housing instability, may impede effective clinical management for individuals with chronic illness. Systematic strategies are needed to more efficiently identify at-risk individuals who may benefit from proactive outreach by health care systems for screening and referral to available social resources.

OBJECTIVE

To create a predictive model to identify a higher likelihood of food insecurity, financial distress, and/or housing instability among adults with multiple chronic medical conditions.

RESEARCH DESIGN AND SUBJECTS

We developed and validated a predictive model in adults with 2 or more chronic conditions who were receiving care within Kaiser Permanente Northern California (KPNC) between January 2017 and February 2020. The model was developed to predict the likelihood of a "yes" response to any of 3 validated self-reported survey questions related to current concerns about food insecurity, financial distress, and/or housing instability. External model validation was conducted in a separate cohort of adult non-Medicaid KPNC members aged 35-85 who completed a survey administered to a random sample of health plan members between April and June 2021 (n = 2820).

MEASURES

We examined the performance of multiple model iterations by comparing areas under the receiver operating characteristic curves (AUCs). We also assessed algorithmic bias related to race/ethnicity and calculated model performance at defined risk thresholds for screening implementation.

RESULTS

Patients in the primary modeling cohort (n = 11,999) had a mean age of 53.8 (±19.3) years, 64.7% were women, and 63.9% were of non-White race/ethnicity. The final, simplified model with 30 predictors (including utilization, diagnosis, behavior, insurance, neighborhood, and pharmacy-based variables) had an AUC of 0.68. The model remained robust within different race/ethnic strata.

CONCLUSIONS

Our results demonstrated that a predictive model developed using information gleaned from the medical record and from public census tract data can be used to identify patients who may benefit from proactive social needs assessment. Depending on the prevalence of social needs in the target population, different risk output thresholds could be set to optimize positive predictive value for successful outreach. This predictive model-based strategy provides a pathway for prioritizing more intensive social risk outreach and screening efforts to the patients who may be in greatest need.

Mapping the regulatory landscape for artificial intelligence in health within the European Union

Regulatory frameworks for artificial intelligence (AI) are needed to mitigate risks while ensuring the ethical, secure, and effective implementation of AI technology in healthcare and population health. In this article, we present a synthesis of 141 binding policies applicable to AI in healthcare and population health in the EU and 10 European countries. The EU AI Act sets the overall regulatory framework for AI, while other legislations set social, health, and human rights standards, address the safety of technologies and the implementation of innovation, and ensure the protection and safe use of data. Regulation specifically pertaining to AI is still nascent and scarce, though a combination of data, technology, innovation, and health and human rights policy has already formed a baseline regulatory framework for AI in health. Future work should explore specific regulatory challenges, especially with respect to AI medical devices, data protection, and data enablement.

Artificial intelligence in primary care practice: Qualitative study to understand perspectives on using AI to derive patient social data

OBJECTIVE

To understand the perspectives of primary care clinicians and health system leaders on the use of artificial intelligence (AI) to derive information about patients' social determinants of health.

DESIGN

Qualitative study.

SETTING

Ontario, Canada.

METHODS

Semistructured, 30-minute virtual interviews were conducted with eligible participants across Ontario wherein they were asked about their perceptions of using AI to derive social data for patients. A descriptive content analysis was used to elicit themes from the data.

MAIN FINDINGS

A total of 12 interviews were conducted with 7 family physicians, 3 clinical team members of various health professions, and 2 health system leaders. Five main themes described the current state of social determinants of health information, perceived benefits of and concerns with using AI to derive social data, how participants would want to see and use AI-derived social data, and suggestions for ethical principles that should underpin the development of this AI tool.

CONCLUSION

Most participants were enthusiastic about the possibility of using AI to derive social data for patients in primary care but noted concerns that should be addressed first. These findings can guide the development of AI-based tools for use in primary care settings.

Large Language Models for Social Determinants of Health Information Extraction from Clinical Notes - A Generalizable Approach across Institutions

The consistent and persuasive evidence illustrating the influence of social determinants on health has prompted a growing realization throughout the health care sector that enhancing health and health equity will likely depend, at least to some extent, on addressing detrimental social determinants. However, detailed social determinants of health (SDoH) information is often buried within clinical narrative text in electronic health records (EHRs), necessitating natural language processing (NLP) methods to automatically extract these details. Most current NLP efforts for SDoH extraction have been limited, investigating on limited types of SDoH elements, deriving data from a single institution, focusing on specific patient cohorts or note types, with reduced focus on generalizability. This study aims to address these issues by creating cross-institutional corpora spanning different note types and healthcare systems, and developing and evaluating the generalizability of classification models, including novel large language models (LLMs), for detecting SDoH factors from diverse types of notes from four institutions: Harris County Psychiatric Center, University of Texas Physician Practice, Beth Israel Deaconess Medical Center, and Mayo Clinic. Four corpora of deidentified clinical notes were annotated with 21 SDoH factors at two levels: level 1 with SDoH factor types only and level 2 with SDoH factors along with associated values. Three traditional classification algorithms (XGBoost, TextCNN, Sentence BERT) and an instruction tuned LLM-based approach (LLaMA) were developed to identify multiple SDoH factors. Substantial variation was noted in SDoH documentation practices and label distributions based on patient cohorts, note types, and hospitals. The LLM achieved top performance with micro-averaged F1 scores over 0.9 on level 1 annotated corpora and an F1 over 0.84 on level 2 annotated corpora. While models performed well when trained and tested on individual datasets, cross-dataset generalization highlighted remaining obstacles. To foster collaboration, access to partial annotated corpora and models trained by merging all annotated datasets will be made available on the PhysioNet repository.

Cross-institution natural language processing for reliable clinical association studies: a methodological exploration

OBJECTIVES

Natural language processing (NLP) of clinical notes in electronic medical records is increasingly used to extract otherwise sparsely available patient characteristics, to assess their association with relevant health outcomes. Manual data curation is resource intensive and NLP methods make these studies more feasible. However, the methodology of using NLP methods reliably in clinical research is understudied. The objective of this study is to investigate how NLP models could be used to extract study variables (specifically exposures) to reliably conduct exposure-outcome association studies.

STUDY DESIGN AND SETTING

In a convenience sample of patients admitted to the intensive care unit of a US academic health system, multiple association studies are conducted, comparing the association estimates based on NLP-extracted vs. manually extracted exposure variables. The association studies varied in NLP model architecture (Bidirectional Encoder Decoder from Transformers, Long Short-Term Memory), training paradigm (training a new model, fine-tuning an existing external model), extracted exposures (employment status, living status, and substance use), health outcomes (having a do-not-resuscitate/intubate code, length of stay, and in-hospital mortality), missing data handling (multiple imputation vs. complete case analysis), and the application of measurement error correction (via regression calibration).

RESULTS

The study was conducted on 1,174 participants (median [interquartile range] age, 61 [50, 73] years; 60.6% male). Additionally, up to 500 discharge reports of participants from the same health system and 2,528 reports of participants from an external health system were used to train the NLP models. Substantial differences were found between the associations based on NLP-extracted and manually extracted exposures under all settings. The error in association was only weakly correlated with the overall F1 score of the NLP models.

CONCLUSION

Associations estimated using NLP-extracted exposures should be interpreted with caution. Further research is needed to set conditions for reliable use of NLP in medical association studies.

Artificial intelligence, ChatGPT, and other large language models for social determinants of health: Current state and future directions

This perspective highlights the importance of addressing social determinants of health (SDOH) in patient health outcomes and health inequity, a global problem exacerbated by the COVID-19 pandemic. We provide a broad discussion on current developments in digital health and artificial intelligence (AI), including large language models (LLMs), as transformative tools in addressing SDOH factors, offering new capabilities for disease surveillance and patient care. Simultaneously, we bring attention to challenges, such as data standardization, infrastructure limitations, digital literacy, and algorithmic bias, that could hinder equitable access to AI benefits. For LLMs, we highlight potential unique challenges and risks including environmental impact, unfair labor practices, inadvertent disinformation or "hallucinations," proliferation of bias, and infringement of copyrights. We propose the need for a multitiered approach to digital inclusion as an SDOH and the development of ethical and responsible AI practice frameworks globally and provide suggestions on bridging the gap from development to implementation of equitable AI technologies.

Natural language processing to identify social determinants of health in Alzheimer's disease and related dementia from electronic health records

OBJECTIVE

To develop a natural language processing (NLP) algorithm that identifies social determinants of health (SDoH), including housing, transportation, food, and medication insecurities, social isolation, abuse, neglect, or exploitation, and financial difficulties for patients with Alzheimer's disease and related dementias (ADRD) from unstructured electronic health records (EHRs).

DATA SOURCES AND STUDY SETTING

We leveraged 1000 medical notes randomly selected from 7401 emergency department and inpatient social worker notes generated between 2015 and 2019 for 231 unique patients diagnosed with ADRD at Michigan Medicine.

STUDY DESIGN

We developed a rule-based NLP algorithm for the identification of seven domains of SDoH noted above. We also compared the rule-based algorithm with deep learning and regularized logistic regression approaches. These models were compared using accuracy, sensitivity, specificity, F1 score, and the area under the receiver operating characteristic curve (AUC). All notes were split into 700 notes for training NLP algorithms, and 300 notes for validation.

DATA COLLECTION/EXTRACTION METHODS

Social worker notes used in this study were extracted from the Michigan Medicine EHR database.

PRINCIPAL FINDINGS

Of the 700 notes for training, F1 and AUC for the rule-based algorithm were at least 0.94 and 0.95, respectively, for all SDoH categories. Of the 300 notes for validation, F1 and AUC were at least 0.80 and 0.97, respectively, for all SDoH except housing and medication insecurities. The deep learning and regularized logistic regression algorithms had unsatisfactory performance.

CONCLUSIONS

The rule-based algorithm can accurately extract SDoH information in all seven domains of SDoH except housing and medication insecurities. Findings from the algorithm can be used by clinicians and social workers to proactively address social needs of patients with ADRD and other vulnerable patient populations.

Systematic design and data-driven evaluation of social determinants of health ontology (SDoHO)

OBJECTIVE

Social determinants of health (SDoH) play critical roles in health outcomes and well-being. Understanding the interplay of SDoH and health outcomes is critical to reducing healthcare inequalities and transforming a "sick care" system into a "health-promoting" system. To address the SDOH terminology gap and better embed relevant elements in advanced biomedical informatics, we propose an SDoH ontology (SDoHO), which represents fundamental SDoH factors and their relationships in a standardized and measurable way.

MATERIAL AND METHODS

Drawing on the content of existing ontologies relevant to certain aspects of SDoH, we used a top-down approach to formally model classes, relationships, and constraints based on multiple SDoH-related resources. Expert review and coverage evaluation, using a bottom-up approach employing clinical notes data and a national survey, were performed.

RESULTS

We constructed the SDoHO with 708 classes, 106 object properties, and 20 data properties, with 1,561 logical axioms and 976 declaration axioms in the current version. Three experts achieved 0.967 agreement in the semantic evaluation of the ontology. A comparison between the coverage of the ontology and SDOH concepts in 2 sets of clinical notes and a national survey instrument also showed satisfactory results.

DISCUSSION

SDoHO could potentially play an essential role in providing a foundation for a comprehensive understanding of the associations between SDoH and health outcomes and paving the way for health equity across populations.

CONCLUSION

SDoHO has well-designed hierarchies, practical objective properties, and versatile functionalities, and the comprehensive semantic and coverage evaluation achieved promising performance compared to the existing ontologies relevant to SDoH.

Identifying High-Need Primary Care Patients Using Nursing Knowledge and Machine Learning Methods

BACKGROUND

Patient cohorts generated by machine learning can be enhanced with clinical knowledge to increase translational value and provide a practical approach to patient segmentation based on a mix of medical, behavioral, and social factors.

OBJECTIVES

This study aimed to generate a pragmatic example of how machine learning could be used to quickly and meaningfully cohort patients using unsupervised classification methods. Additionally, to demonstrate increased translational value of machine learning models through the integration of nursing knowledge.

METHODS

A primary care practice dataset (N = 3,438) of high-need patients defined by practice criteria was parsed to a subset population of patients with diabetes (n = 1233). Three expert nurses selected variables for k-means cluster analysis using knowledge of critical factors for care coordination. Nursing knowledge was again applied to describe the psychosocial phenotypes in four prominent clusters, aligned with social and medical care plans.

RESULTS

Four distinct clusters interpreted and mapped to psychosocial need profiles, allowing for immediate translation to clinical practice through the creation of actionable social and medical care plans. (1) A large cluster of racially diverse female, non-English speakers with low medical complexity, and history of childhood illness; (2) a large cluster of English speakers with significant comorbidities (obesity and respiratory disease); (3) a small cluster of males with substance use disorder and significant comorbidities (mental health, liver and cardiovascular disease) who frequently visit the hospital; and (4) a moderate cluster of older, racially diverse patients with renal failure.

CONCLUSION

This manuscript provides a practical method for analysis of primary care practice data using machine learning in tandem with expert clinical knowledge.

Capturing Concerns about Patient Deterioration in Narrative Documentation in Home Healthcare

Home healthcare (HHC) agencies provide care to more than 3.4 million adults per year. There is value in studying HHC narrative notes to identify patients at risk for deterioration. This study aimed to build machine learning algorithms to identify "concerning" narrative notes of HHC patients and identify emerging themes. Six algorithms were applied to narrative notes (n = 4,000) from a HHC agency to classify notes as either "concerning" or "not concerning." Topic modeling using Latent Dirichlet Allocation bag of words was conducted to identify emerging themes from the concerning notes. Gradient Boosted Trees demonstrated the best performance with a F-score = 0.74 and AUC = 0.96. Emerging themes were related to patient-clinician communication, HHC services provided, gait challenges, mobility concerns, wounds, and caregivers. Most themes have been cited by previous literature as increasing risk for adverse events. In the future, such algorithms can support early identification of patients at risk for deterioration.

Natural language processing for clinical notes in dentistry: A systematic review

OBJECTIVE

To identify and synthesise research on applications of natural language processing (NLP) for information extraction and retrieval from clinical notes in dentistry.

MATERIALS AND METHODS

A predefined search strategy was applied in EMBASE, CINAHL and Medline. Studies eligible for inclusion were those that that described, evaluated, or applied NLP to clinical notes containing either human or simulated patient information. Quality of the study design and reporting was independently assessed based on a set of questions derived from relevant tools including CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS). A narrative synthesis was conducted to present the results.

RESULTS

Of the 17 included studies, 10 developed and evaluated NLP methods and 7 described applications of NLP-based information retrieval methods in dental records. Studies were published between 2015 and 2021, most were missing key details needed for reproducibility, and there was no consistency in design or reporting. The 10 studies developing or evaluating NLP methods used document classification or entity extraction, and 4 compared NLP methods to non-NLP methods. The quality of reporting on NLP studies in dentistry has modestly improved over time.

CONCLUSIONS

Study design heterogeneity and incomplete reporting of studies currently limits our ability to synthesise NLP applications in dental records. Standardisation of reporting and improved connections between NLP methods and applied NLP in dentistry may improve how we can make use of clinical notes from dentistry in population health or decision support systems.

PROTOCOL REGISTRATION

PROSPERO CRD42021227823.

Machine learning applied to electronic health record data in home healthcare: A scoping review

OBJECTIVE

Despite recent calls for home healthcare (HHC) to integrate informatics, the application of machine learning in HHC is relatively unknown. Thus, this study aimed to synthesize and appraise the literature describing the application of machine learning to predict adverse outcomes (e.g., hospitalization, mortality) using electronic health record (EHR) data in the HHC setting. Our secondary aim was to evaluate the comprehensiveness of predictors used in the machine learning algorithms guided by the Biopsychosocial Model.

METHODS

During March 2022 we conducted a literature search in four databases: PubMed, Embase, CINAHL, and Scopus. Inclusion criteria were 1) describing services provided in the HHC setting, 2) applying machine learning algorithms to predict adverse outcomes, defined as outcomes related to patient deterioration, 3) using EHR data and 4) focusing on the adult population. Predictors were mapped to the Biopsychosocial Model. A risk of bias analysis was conducted using the Prediction Model Risk Of Bias Assessment Tool.

RESULTS

The final sample included 20 studies. Eighteen studies used predictors from standardized assessments integrated in the EHR. The most common outcome of interest was hospitalization (55%), followed by mortality (25%). Psychological predictors were frequently excluded (35%). Tree based algorithms were most frequently applied (75%). Most studies demonstrated high or unclear risk of bias (75%).

CONCLUSION

Future studies in HHC should consider incorporating machine learning algorithms into clinical decision support systems to identify patients at risk. Based on the Biopsychosocial model, psychological and interpersonal characteristics should be used along with biological characteristics to enhance risk prediction. To facilitate the widespread adoption of machine learning, stakeholders should encourage standardization in the HHC setting.

The Impact of Artificial Intelligence on Health Equity in Oncology: Scoping Review

BACKGROUND

The field of oncology is at the forefront of advances in artificial intelligence (AI) in health care, providing an opportunity to examine the early integration of these technologies in clinical research and patient care. Hope that AI will revolutionize health care delivery and improve clinical outcomes has been accompanied by concerns about the impact of these technologies on health equity.

OBJECTIVE

We aimed to conduct a scoping review of the literature to address the question, "What are the current and potential impacts of AI technologies on health equity in oncology?"

METHODS

Following PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines for scoping reviews, we systematically searched MEDLINE and Embase electronic databases from January 2000 to August 2021 for records engaging with key concepts of AI, health equity, and oncology. We included all English-language articles that engaged with the 3 key concepts. Articles were analyzed qualitatively for themes pertaining to the influence of AI on health equity in oncology.

RESULTS

Of the 14,011 records, 133 (0.95%) identified from our review were included. We identified 3 general themes in the literature: the use of AI to reduce health care disparities (58/133, 43.6%), concerns surrounding AI technologies and bias (16/133, 12.1%), and the use of AI to examine biological and social determinants of health (55/133, 41.4%). A total of 3% (4/133) of articles focused on many of these themes.

CONCLUSIONS

Our scoping review revealed 3 main themes on the impact of AI on health equity in oncology, which relate to AI's ability to help address health disparities, its potential to mitigate or exacerbate bias, and its capability to help elucidate determinants of health. Gaps in the literature included a lack of discussion of ethical challenges with the application of AI technologies in low- and middle-income countries, lack of discussion of problems of bias in AI algorithms, and a lack of justification for the use of AI technologies over traditional statistical methods to address specific research questions in oncology. Our review highlights a need to address these gaps to ensure a more equitable integration of AI in cancer research and clinical practice. The limitations of our study include its exploratory nature, its focus on oncology as opposed to all health care sectors, and its analysis of solely English-language articles.

Classifying Social Determinants of Health from Unstructured Electronic Health Records Using Deep Learning-based Natural Language Processing

OBJECTIVE

Social determinants of health (SDOH) are non-medical factors that can profoundly impact patient health outcomes. However, SDOH are rarely available in structured electronic health record (EHR) data such as diagnosis codes, and more commonly found in unstructured narrative clinical notes. Hence, identifying social context from unstructured EHR data has become increasingly important. Yet, previous work on using natural language processing to automate extraction of SDOH from text (a) usually focuses on an ad hoc selection of SDOH, and (b) does not use the latest advances in deep learning. Our objective was to advance automatic extraction of SDOH from clinical text by (a) systematically creating a set of SDOH based on standard biomedical and psychiatric ontologies, and (b) training state-of-the-art deep neural networks to extract mentions of these SDOH from clinical notes.

DESIGN

A retrospective cohort study.

SETTING AND PARTICIPANTS

Data were extracted from the Medical Information Mart for Intensive Care (MIMIC-III) database. The corpus comprised 3,504 social related sentences from 2,670 clinical notes.

METHODS

We developed a framework for automated classification of multiple SDOH categories. Our dataset comprised narrative clinical notes under the "Social Work" category in the MIMIC-III Clinical Database. Using standard terminologies, SNOMED-CT and DSM-IV, we systematically curated a set of 13 SDOH categories and created annotation guidelines for these. After manually annotating the 3,504 sentences, we developed and tested three deep neural network (DNN) architectures - convolutional neural network (CNN), long short-term memory (LSTM) network, and the Bidirectional Encoder Representations from Transformers (BERT) - for automated detection of eight SDOH categories. We also compared these DNNs to three baselines models: (1) cTAKES, as well as (2) L2-regularized logistic regression and (3) random forests on bags-of-words. Model evaluation metrics included micro- and macro- F1, and area under the receiver operating characteristic curve (AUC).

RESULTS

All three DNN models accurately classified all SDOH categories (minimum micro-F1 =.632, minimum macro-AUC=.854). Compared to the CNN and LSTM, BERT performed best in most key metrics (micro-F1 = 0.690, macro-AUC=0.907). The BERT model most effectively identified the "occupational" category (F1=.774, AUC=.965) and least effectively identified the "non-SDOH" category (F=.491, AUC=.788). BERT outperformed cTAKES in distinguishing social vs non-social sentences (BERT F1 = .87 vs. cTAKES F1=.06), and outperformed logistic regression (micro-F1=0.649, macro-AUC=0.696) and random forest (micro-F1=0.502, macro-AUC=0.523) trained on bag-of-words.

CONCLUSIONS

Our study framework with DNN models demonstrated improved performance for efficiently identifying a systematic range of SDOH categories from clinical notes in the EHR. Improved identification of patient SDOH may further improve healthcare outcomes.