1
|
Chen B, Alrifai W, Gao C, Jones B, Novak L, Lorenzi N, France D, Malin B, Chen Y. Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning. J Am Med Inform Assoc 2021; 28:1168-1177. [PMID: 33576432 DOI: 10.1093/jamia/ocaa338] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2020] [Accepted: 12/17/2020] [Indexed: 11/12/2022] Open
Abstract
OBJECTIVE The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics. MATERIALS AND METHODS We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit. RESULTS We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57 234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness. CONCLUSIONS The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.
Collapse
Affiliation(s)
- Bob Chen
- Epithelial Biology Center, Vanderbilt University Medical Center, Nashville, Tennessee, USA.,Program in Chemical and Physical Biology, Vanderbilt University, Nashville, Tennessee, USA
| | - Wael Alrifai
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA.,Department of Pediatrics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Cheng Gao
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Barrett Jones
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Laurie Novak
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Nancy Lorenzi
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Daniel France
- Department of Anesthesiology, Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Bradley Malin
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA.,Department of Biostatistics, School of Medicine, Vanderbilt University Medical Center, Nashville, Tennessee, USA.,Department of Electrical Engineering and Computer Science, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA
| | - You Chen
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA.,Department of Electrical Engineering and Computer Science, School of Engineering, Vanderbilt University, Nashville, Tennessee, USA
| |
Collapse
|
2
|
Rule A, Chiang MF, Hribar MR. Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods. J Am Med Inform Assoc 2021; 27:480-490. [PMID: 31750912 DOI: 10.1093/jamia/ocz196] [Citation(s) in RCA: 61] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2019] [Revised: 10/07/2019] [Accepted: 10/18/2019] [Indexed: 01/13/2023] Open
Abstract
OBJECTIVE To systematically review published literature and identify consistency and variation in the aims, measures, and methods of studies using electronic health record (EHR) audit logs to observe clinical activities. MATERIALS AND METHODS In July 2019, we searched PubMed for articles using EHR audit logs to study clinical activities. We coded and clustered the aims, measures, and methods of each article into recurring categories. We likewise extracted and summarized the methods used to validate measures derived from audit logs and limitations discussed of using audit logs for research. RESULTS Eighty-five articles met inclusion criteria. Study aims included examining EHR use, care team dynamics, and clinical workflows. Studies employed 6 key audit log measures: counts of actions captured by audit logs (eg, problem list viewed), counts of higher-level activities imputed by researchers (eg, chart review), activity durations, activity sequences, activity clusters, and EHR user networks. Methods used to preprocess audit logs varied, including how authors filtered extraneous actions, mapped actions to higher-level activities, and interpreted repeated actions or gaps in activity. Nineteen studies validated results (22%), but only 9 (11%) through direct observation, demonstrating varying levels of measure accuracy. DISCUSSION While originally designed to aid access control, EHR audit logs have been used to observe diverse clinical activities. However, most studies lack sufficient discussion of measure definition, calculation, and validation to support replication, comparison, and cross-study synthesis. CONCLUSION EHR audit logs have potential to scale observational research but the complexity of audit log measures necessitates greater methodological transparency and validated standards.
Collapse
Affiliation(s)
- Adam Rule
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA
| | - Michael F Chiang
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA.,Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
| | - Michelle R Hribar
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon, USA.,Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon, USA
| |
Collapse
|
3
|
Patient Privacy Violation Detection in Healthcare Critical Infrastructures: An Investigation Using Density-Based Benchmarking. FUTURE INTERNET 2020. [DOI: 10.3390/fi12060100] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Hospital critical infrastructures have a distinct threat vector, due to (i) a dependence on legacy software; (ii) the vast levels of interconnected medical devices; (iii) the use of multiple bespoke software and that (iv) electronic devices (e.g., laptops and PCs) are often shared by multiple users. In the UK, hospitals are currently upgrading towards the use of electronic patient record (EPR) systems. EPR systems and their data are replacing traditional paper records, providing access to patients’ test results and details of their overall care more efficiently. Paper records are no-longer stored at patients’ bedsides, but instead are accessible via electronic devices for the direct insertion of data. With over 83% of hospitals in the UK moving towards EPRs, access to this healthcare data needs to be monitored proactively for malicious activity. It is paramount that hospitals maintain patient trust and ensure that the information security principles of integrity, availability and confidentiality are upheld when deploying EPR systems. In this paper, an investigation methodology is presented towards the identification of anomalous behaviours within EPR datasets. Many security solutions focus on a perimeter-based approach; however, this approach alone is not enough to guarantee security, as can be seen from the many examples of breaches. Our proposed system can be complementary to existing security perimeter solutions. The system outlined in this research employs an internal-focused methodology for anomaly detection by using the Local Outlier Factor (LOF) and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithms for benchmarking behaviour, for assisting healthcare data analysts. Out of 90,385 unique IDs, DBSCAN finds 102 anomalies, whereas 358 are detected using LOF.
Collapse
|
4
|
Chen Y, Lehmann CU, Hatch LD, Schremp E, Malin BA, France DJ. Modeling Care Team Structures in the Neonatal Intensive Care Unit through Network Analysis of EHR Audit Logs. Methods Inf Med 2020; 58:109-123. [PMID: 32170716 DOI: 10.1055/s-0040-1702237] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
BACKGROUND In the neonatal intensive care unit (NICU), predefined acuity-based team care models are restricted to core roles and neglect interactions with providers outside of the team, such as interactions that transpire via electronic health record (EHR) systems. These unaccounted interactions may be related to the efficiency of resource allocation, information flow, communication, and thus impact patient outcomes. This study applied network analysis methods to EHR audit logs to model the interactions of providers beyond their core roles to better understand the interaction network patterns of acuity-based teams and relationships of the network structures with postsurgical length of stay (PSLOS). METHODS The study used the EHR log data of surgical neonates from a large academic medical center. The study included 104 surgical neonates, for whom 9,206 unique actions were performed by 457 providers in their EHRs. We applied network analysis methods to model EHR provider interaction networks of acuity-based teams in NICU postoperative care. We partitioned each EHR network into three subnetworks based on interaction types: (1) interactions between known core providers who were documented in scheduling records (core subnetwork); (2) interactions between core and noncore providers (extended subnetwork); and (3) interactions between noncore providers (extended subnetwork). For each core subnetwork, we assessed its capability to replicate predefined core-provider relations as documented in scheduling records. We further compared each EHR network, as well as its subnetworks, using standard network measures to determine its differences in network topologies. We conducted a case study to learn provider interaction networks taking care of 15 neonates who underwent gastrostomy tube placement surgery from EHR log data and measure the effectiveness of the interaction networks on PSLOS by the proportional-odds model. RESULTS The provider networks of four acuity-based teams (two high and two low acuity), along with their subnetworks, were discovered. We found that beyond capturing the predefined core-provider relations, EHR audit logs can also learn a large number of relations between core and noncore providers or among noncore providers. Providers in the core subnetwork exhibited a greater number of connections with each other than with providers in the extended subnetworks. Many more providers in the core subnetwork serve as a hub than those in the other types of subnetworks. We also found that high-acuity teams exhibited more complex network structures than low-acuity teams, with high-acuity team generating 6,416 interactions between 407 providers compared with 931 interactions between 124 providers, respectively. In addition, we discovered that high-acuity and low-acuity teams shared more than 33 and 25% of providers with each other, respectively, but exhibited different collaborative structures demonstrating that NICU providers shift across different acuity teams and exhibit different network characteristics. Results of case study show that providers, whose patients had lower PSLOS, tended to disperse patient-related information to more colleagues within their network than those who treated higher PSLOS patients (p = 0.03). CONCLUSION Network analysis can be applied to EHR log data to model acuity-based NICU teams capturing interactions between providers within the predesigned core team as well as those outside of the core team. In the NICU, dissemination of information may be linked to reduced PSLOS. EHR log data provide an efficient, accessible, and research-friendly way to study provider interaction networks. Findings should guide improvements in the EHR system design to facilitate effective interactions between providers.
Collapse
Affiliation(s)
- You Chen
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States.,Department of Electrical Engineering and Computer Science, School of Engineering, Vanderbilt University, Nashville, Tennessee, United States
| | - Christoph U Lehmann
- Departments of Pediatrics, Bioinformatics, and Population & Data Sciences, University of Texas Southwestern Medical Center, Dallas, Texas, United States
| | - Leon D Hatch
- Department of Pediatrics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
| | - Emma Schremp
- Department of Anesthesiology, Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, Tennessee, United States
| | - Bradley A Malin
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, United States.,Department of Electrical Engineering and Computer Science, School of Engineering, Vanderbilt University, Nashville, Tennessee, United States.,Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, United States
| | - Daniel J France
- Department of Anesthesiology, Center for Research and Innovation in Systems Safety, Vanderbilt University Medical Center, Nashville, Tennessee, United States
| |
Collapse
|
5
|
Kim C, Lehmann CU, Hatch D, Schildcrout JS, France DJ, Chen Y. Provider Networks in the Neonatal Intensive Care Unit Associate with Length of Stay. ... IEEE CONFERENCE ON COLLABORATION AND INTERNET COMPUTING. IEEE CONFERENCE ON COLLABORATION AND INTERNET COMPUTING 2019; 2019:127-134. [PMID: 32637942 PMCID: PMC7339831 DOI: 10.1109/cic48465.2019.00024] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/02/2023]
Abstract
We strive to understand care coordination structures of multidisciplinary teams and to evaluate their effect on post-surgical length of stay (PSLOS) in the Neonatal Intensive Care Unit (NICU). Electronic health record (EHR) data were extracted for 18 neonates, who underwent gastrostomy tube placement surgery at the Vanderbilt University Medical Center NICU. Based on providers' interactions with the EHR (e.g. viewing, documenting, ordering), provider-provider relations were learned and used to build patient-specific provider networks representing the care coordination structure. We quantified the networks using standard network analysis metrics (e.g., in-degree, out-degree, betweenness centrality, and closeness centrality). Coordination structure effectiveness was measured as the association between the network metrics and PSLOS, as modeled by a proportional-odds, logistical regression model. The 18 provider networks exhibited various team compositions and various levels of structural complexity. Providers, whose patients had lower PSLOS, tended to disperse patient-related information to more colleagues within their network than those, who treated higher PSLOS patients (P = 0.0294). In the NICU, improved dissemination of information may be linked to reduced PSLOS. EHR data provides an efficient, accessible, and resource-friendly way to study care coordination using network analysis tools. This novel methodology offers an objective way to identify key performance and safety indicators of care coordination and to study dissemination of patient-related information within care provider networks and its effect on care. Findings should guide improvements in the EHR system design to facilitate effective clinical communications among providers.
Collapse
Affiliation(s)
- Cindy Kim
- Department of Mathematics, Vanderbilt University, Nashville, TN
| | | | - Dupree Hatch
- Department of Pediatrics, Vanderbilt University Medical Center, Nashville, TN
| | | | - Daniel J France
- Department of Anesthesiology, Vanderbilt University Medical Center, Nashville, TN
| | - You Chen
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN
| |
Collapse
|
6
|
Chen Y, Patel MB, McNaughton CD, Malin BA. Interaction patterns of trauma providers are associated with length of stay. J Am Med Inform Assoc 2019; 25:790-799. [PMID: 29481625 DOI: 10.1093/jamia/ocy009] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2017] [Accepted: 01/26/2018] [Indexed: 01/08/2023] Open
Abstract
Background Trauma-related hospitalizations drive a high percentage of health care expenditure and inpatient resource consumption, which is directly related to length of stay (LOS). Robust and reliable interactions among health care employees can reduce LOS. However, there is little known about whether certain patterns of interactions exist and how they relate to LOS and its variability. The objective of this study is to learn interaction patterns and quantify the relationship to LOS within a mature trauma system and long-standing electronic medical record (EMR). Methods We adapted a spectral co-clustering methodology to infer the interaction patterns of health care employees based on the EMR of 5588 hospitalized adult trauma survivors. The relationship between interaction patterns and LOS was assessed via a negative binomial regression model. We further assessed the influence of potential confounders by age, number of health care encounters to date, number of access action types care providers committed to patient EMRs, month of admission, phenome-wide association study codes, procedure codes, and insurance status. Results Three types of interaction patterns were discovered. The first pattern exhibited the most collaboration between employees and was associated with the shortest LOS. Compared to this pattern, LOS for the second and third patterns was 0.61 days (P = 0.014) and 0.43 days (P = 0.037) longer, respectively. Although the 3 interaction patterns dealt with different numbers of patients in each admission month, our results suggest that care was provided for similar patients. Discussion The results of this study indicate there is an association between LOS and the extent to which health care employees interact in the care of an injured patient. The findings further suggest that there is merit in ascertaining the content of these interactions and the factors that induce these differences in interaction patterns within a trauma system.
Collapse
Affiliation(s)
- You Chen
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Mayur B Patel
- Department of Surgery, Division of Trauma, Surgical Critical Care, and Emergency General Surgery, Vanderbilt University Medical Center, Nashville, TN, USA.,Department of Neurosurgery, Vanderbilt University Medical Center, Nashville, TN, USA.,Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Candace D McNaughton
- Department of Emergency Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Bradley A Malin
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA.,Department of Biostatistics, School of Medicine, Vanderbilt University, Nashville, TN, USA.,Department of Electrical Engineering and Computer Science, School of Engineering, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
7
|
Zhu X, Tu SP, Sewell D, Yao NA, Mishra V, Dow A, Banas C. Measuring electronic communication networks in virtual care teams using electronic health records access-log data. Int J Med Inform 2019; 128:46-52. [PMID: 31160011 DOI: 10.1016/j.ijmedinf.2019.05.012] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2018] [Revised: 01/01/2019] [Accepted: 05/11/2019] [Indexed: 12/22/2022]
Abstract
OBJECTIVE To develop methods for measuring electronic communication networks in virtual care teams using electronic health records (EHR) access-log data. METHODS For a convenient sample of 100 surgical colorectal cancer patients, we used time-stamped EHR access-log data extracted from an academic medical center's EHR system to construct communication networks among healthcare professionals (HCPs) in each patient's virtual care team. We measured communication linkages between HCPs using the inverse of the average time between access events in which the source HCPs sent information to and the destination HCPs retrieved information from the EHR system. Social network analysis was used to examine and visualize communication network structures, identify principal care teams, and detect meaningful structural differences across networks. We conducted a non-parametric multivariate analysis of variance (MANOVA) to test the association between care teams' communication network structures and patients' cancer stage and site. RESULTS The 100 communication networks showed substantial variations in size and structures. Principal care teams, the subset of HCPs who formed the core of the communication networks, had higher proportions of nurses, physicians, and pharmacists and a lower proportion of laboratory medical technologists than the overall networks. The distributions of conditional uniform graph quantiles suggested that our network-construction technique captured meaningful underlying structures that were different from random unstructured networks. MANOVA results found that the networks' topologies were associated with patients' cancer stage and site. CONCLUSIONS This study demonstrates that it is feasible to use EHR access-log data to measure and examine communication networks in virtual care teams. The proposed methods captured salient communication patterns in care teams that were associated with patients' clinical differences.
Collapse
Affiliation(s)
- Xi Zhu
- University of Iowa, Department of Health Management and Policy, 145 N Riverside Dr, N222, Iowa City, IA 52242, United States.
| | - Shin-Ping Tu
- University of California Davis, Department of Internal Medicine, Davis, CA, United States
| | - Daniel Sewell
- University of Iowa, Department of Biostatistics, Iowa City, IA, United States
| | - Nengliang Aaron Yao
- University of Virginia, Department of Public Health Sciences, Charlottesville, VA, United States
| | - Vimal Mishra
- Virginia Commonwealth University, Department of Internal Medicine, Richmond, VA, United States
| | - Alan Dow
- Virginia Commonwealth University, Department of Internal Medicine, Richmond, VA, United States
| | - Colin Banas
- Virginia Commonwealth University, Department of Internal Medicine, Richmond, VA, United States
| |
Collapse
|
8
|
Gao C, Kho AN, Osmundson S, Malin BA, Chen Y. Obstetric Patients with Repetitious Hospital Location Transfers Have Prolonged Stays. IEEE INTERNATIONAL CONFERENCE ON HEALTHCARE INFORMATICS. IEEE INTERNATIONAL CONFERENCE ON HEALTHCARE INFORMATICS 2019; 2019:10.1109/ICHI.2019.8904557. [PMID: 32524087 PMCID: PMC7286595 DOI: 10.1109/ichi.2019.8904557] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
There is a general belief that the workflow of surrounding location transfers between locations documented in electronic health record (EHR) during hospitalization is associated with a patient's length of stay (LOS). However, this belief has had little formal investigation in a data-driven manner. Location transfers in patients' hospitalization are hypothesized to be related to LOS. The objective of this study is to assess this relationship, using data derived from the EHR system of a large hospital system, with a focus on the obstetric setting - a clinical environment that exhibits wide swing in resource utilization. We designed a data-driven framework to infer patterns of location transfers and developed a zero-truncated negative binomial model, adjusting for demographics and billed diagnoses, to learn the association between patterns of location transfers and LOS. Indicative factors found to be of indicative of location transfer patterns were further investigated via their odds ratios, Pearson Correlation Coefficients, and Chi-squared test. We evaluated our approach with two years of data on from 5,774 obstetric inpatient encounters from the EHR system of Northwestern Memorial Hospital. The results indicated that the average LOS for patients with patterns of repetitious location transfers (RLTs) was 4.25 days (95% confidence interval [4.02, 4.47]) longer than patients with no RLT. This difference reduced to 3.62 days (95% confidence interval [3.61, 3.64]) after adjusting for age, race and billed diagnoses. We further discovered 21 indicative factors of RLT (statistically significant with a significance level of 0.05), in the form of billed diagnosis codes, each of which exhibited an odds ratio larger than 4. This study suggests that RLT patterns are associated with a prolonged LOS in the obstetric setting. As such, healthcare organizations may need to pay more attention to patients with RLTs to refine location transfers workflow and to boost efficiency in obstetric care.
Collapse
Affiliation(s)
- Cheng Gao
- Department of Biomedical Informatics Vanderbilt University
| | - Abel N Kho
- Institute for Public Health and Medicine Northwestern University
| | - Sarah Osmundson
- Department of Obstetrics and Gynecology Vanderbilt University Medical Center
| | | | - You Chen
- Department of Biomedical Informatics Vanderbilt University
| |
Collapse
|
9
|
Brunson JC, Laubenbacher RC. Applications of network analysis to routinely collected health care data: a systematic review. J Am Med Inform Assoc 2018; 25:210-221. [PMID: 29025116 PMCID: PMC6664849 DOI: 10.1093/jamia/ocx052] [Citation(s) in RCA: 41] [Impact Index Per Article: 5.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2017] [Revised: 04/18/2017] [Accepted: 04/23/2017] [Indexed: 01/21/2023] Open
Abstract
Objective To survey network analyses of datasets collected in the course of routine operations in health care settings and identify driving questions, methods, needs, and potential for future research. Materials and Methods A search strategy was designed to find studies that applied network analysis to routinely collected health care datasets and was adapted to 3 bibliographic databases. The results were grouped according to a thematic analysis of their settings, objectives, data, and methods. Each group received a methodological synthesis. Results The search found 189 distinct studies reported before August 2016. We manually partitioned the sample into 4 groups, which investigated institutional exchange, physician collaboration, clinical co-occurrence, and workplace interaction networks. Several robust and ongoing research programs were discerned within (and sometimes across) the groups. Little interaction was observed between these programs, despite conceptual and methodological similarities. Discussion We use the literature sample to inform a discussion of good practice at this methodological interface, including the concordance of motivations, study design, data, and tools and the validation and standardization of techniques. We then highlight instances of positive feedback between methodological development and knowledge domains and assess the overall cohesion of the sample.
Collapse
|
10
|
Chen Y, Kho AN, Liebovitz D, Ivory C, Osmundson S, Bian J, Malin BA. Learning bundled care opportunities from electronic medical records. J Biomed Inform 2018; 77:1-10. [PMID: 29174994 PMCID: PMC5771885 DOI: 10.1016/j.jbi.2017.11.014] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2017] [Revised: 10/30/2017] [Accepted: 11/21/2017] [Indexed: 01/29/2023]
Abstract
OBJECTIVE The traditional fee-for-service approach to healthcare can lead to the management of a patient's conditions in a siloed manner, inducing various negative consequences. It has been recognized that a bundled approach to healthcare - one that manages a collection of health conditions together - may enable greater efficacy and cost savings. However, it is not always evident which sets of conditions should be managed in a bundled manner. In this study, we investigate if a data-driven approach can automatically learn potential bundles. METHODS We designed a framework to infer health condition collections (HCCs) based on the similarity of their clinical workflows, according to electronic medical record (EMR) utilization. We evaluated the framework with data from over 16,500 inpatient stays from Northwestern Memorial Hospital in Chicago, Illinois. The plausibility of the inferred HCCs for bundled care was assessed through an online survey of a panel of five experts, whose responses were analyzed via an analysis of variance (ANOVA) at a 95% confidence level. We further assessed the face validity of the HCCs using evidence in the published literature. RESULTS The framework inferred four HCCs, indicative of (1) fetal abnormalities, (2) late pregnancies, (3) prostate problems, and (4) chronic diseases, with congestive heart failure featuring prominently. Each HCC was substantiated with evidence in the literature and was deemed plausible for bundled care by the experts at a statistically significant level. CONCLUSIONS The findings suggest that an automated EMR data-driven framework conducted can provide a basis for discovering bundled care opportunities. Still, translating such findings into actual care management will require further refinement, implementation, and evaluation.
Collapse
Affiliation(s)
- You Chen
- Dept. of Biomedical Informatics, School of Medicine, Vanderbilt University, Nashville, TN, USA.
| | - Abel N Kho
- Institute for Public Health and Medicine, Northwestern University, Chicago, IL, USA
| | | | - Catherine Ivory
- School of Nursing, Vanderbilt University, Nashville, TN, USA
| | - Sarah Osmundson
- Dept. of Obstetrics and Gynecology, School of Medicine, Vanderbilt University, Nashville, TN, USA
| | - Jiang Bian
- Dept. of Health Outcomes and Policy, University of Florida, Gainesville, FL, USA
| | - Bradley A Malin
- Dept. of Biomedical Informatics, School of Medicine, Vanderbilt University, Nashville, TN, USA; Dept. of Biostatistics, School of Medicine, Vanderbilt University, Nashville, TN, USA; Dept. of Electrical Engineering & Computer Science, School of Engineering, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
11
|
Vainiomäki S, Aalto AM, Lääveri T, Sinervo T, Elovainio M, Mäntyselkä P, Hyppönen H. Better Usability and Technical Stability Could Lead to Better Work-Related Well-Being among Physicians. Appl Clin Inform 2017; 8:1057-1067. [PMID: 29241245 DOI: 10.4338/aci-2017-06-ra-0094] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Abstract
Background and Objective Finnish physicians have been increasingly dissatisfied with poor usability of the electronic patient record (EPR) systems, which they have identified as an overload factor in their work. Our aim is to specify which factors in EPRs are associated with work-related well-being of physicians.
Methods A web-based questionnaire was sent to Finnish physicians younger than 65 years; the responses (n = 3,781) represent one-fourth of these. This was a repetition of a survey in 2010, where this questionnaire was used for the first time. In addition to statements assessing usability, there were questions measuring time pressure and job control. The relation between usability and work well-being was investigated with hierarchical multivariate regression analyses: With time pressure and job control as dependent variables, EPR usability assessments and physicians' background information were used as independent variables.
Results In the multivariate analyses, technical problems that are often experienced in the EPR were related to higher time pressure and lower job control. Active participation in the development of the EPR system was related to stronger time pressure and stronger job control. In addition, use of several systems daily and the experience of time-consuming documentation of patient information for statistical purposes (billing, national registries, and reporting) were related to higher time pressure, while those with longer experience with the EPR system and those experiencing easy-to-read nursing records reported higher job control.
Conclusion To relieve time pressure and increase sense of job control experienced by physicians, usability, integrations, and stability of the EPR systems should be improved: fewer login procedures, easier readability of nursing records, and decreased need for separate documentation for statistical purposes. Physician participation in the EPR development would increase the feeling of job control, but would add the time pressure. Hence, time for developmental work should be arranged.
Collapse
Affiliation(s)
- Suvi Vainiomäki
- Welfare Division, City of Turku, Turku, Finland.,Unit of General Practice, University of Turku, Turku, Finland
| | - Anna-Mari Aalto
- Unit of Social and Health Systems Research, Department of Health and Social Care Systems, National Institute for Health and Welfare, Helsinki, Finland
| | - Tinja Lääveri
- Division of Infectious Diseases, Inflammation Center, Helsinki University Hospital, Helsinki, Finland.,Oy Apotti Ab, Helsinki, Finland
| | - Timo Sinervo
- Unit of Social and Health Systems Research, Department of Health and Social Care Systems, National Institute for Health and Welfare, Helsinki, Finland
| | - Marko Elovainio
- Unit of Social and Health Systems Research, Department of Health and Social Care Systems, National Institute for Health and Welfare, Helsinki, Finland.,Institute of Behavioural Sciences, University of Helsinki, Helsinki, Finland
| | - Pekka Mäntyselkä
- School of Medicine, General Practice, University of Eastern Finland, Finland.,Primary Health Care Unit, Kuopio University Hospital, Kuopio, Finland
| | - Hannele Hyppönen
- Unit of Social and Health Systems Research, Department of Health and Social Care Systems, National Institute for Health and Welfare, Helsinki, Finland
| |
Collapse
|
12
|
Chen Y, Lorenzi NM, Sandberg WS, Wolgast K, Malin BA. Identifying collaborative care teams through electronic medical record utilization patterns. J Am Med Inform Assoc 2017; 24:e111-e120. [PMID: 27570217 PMCID: PMC6080725 DOI: 10.1093/jamia/ocw124] [Citation(s) in RCA: 26] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2016] [Revised: 07/15/2016] [Accepted: 07/20/2016] [Indexed: 02/05/2023] Open
Abstract
OBJECTIVE The goal of this investigation was to determine whether automated approaches can learn patient-oriented care teams via utilization of an electronic medical record (EMR) system. MATERIALS AND METHODS To perform this investigation, we designed a data-mining framework that relies on a combination of latent topic modeling and network analysis to infer patterns of collaborative teams. We applied the framework to the EMR utilization records of over 10 000 employees and 17 000 inpatients at a large academic medical center during a 4-month window in 2010. Next, we conducted an extrinsic evaluation of the patterns to determine the plausibility of the inferred care teams via surveys with knowledgeable experts. Finally, we conducted an intrinsic evaluation to contextualize each team in terms of collaboration strength (via a cluster coefficient) and clinical credibility (via associations between teams and patient comorbidities). RESULTS The framework discovered 34 collaborative care teams, 27 (79.4%) of which were confirmed as administratively plausible. Of those, 26 teams depicted strong collaborations, with a cluster coefficient > 0.5. There were 119 diagnostic conditions associated with 34 care teams. Additionally, to provide clarity on how the survey respondents arrived at their determinations, we worked with several oncologists to develop an illustrative example of how a certain team functions in cancer care. DISCUSSION Inferred collaborative teams are plausible; translating such patterns into optimized collaborative care will require administrative review and integration with management practices. CONCLUSIONS EMR utilization records can be mined for collaborative care patterns in large complex medical centers.
Collapse
Affiliation(s)
- You Chen
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA
| | - Nancy M Lorenzi
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA
- School of Nursing, Vanderbilt University
| | - Warren S Sandberg
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA
- Department of Anesthesiology, Vanderbilt University
| | - Kelly Wolgast
- School of Nursing, Vanderbilt University
- Healthcare Leadership Program, School of Nursing, Vanderbilt University
| | - Bradley A Malin
- Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee, USA
- Department of Electrical Engineering and Computer Science, Vanderbilt University
| |
Collapse
|
13
|
Kricke GS, Carson MB, Lee YJ, Benacka C, Mutharasan RK, Ahmad FS, Kansal P, Yancy CW, Anderson AS, Soulakis ND. Leveraging electronic health record documentation for Failure Mode and Effects Analysis team identification. J Am Med Inform Assoc 2017; 24:288-294. [PMID: 27589944 PMCID: PMC5391722 DOI: 10.1093/jamia/ocw083] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2016] [Revised: 04/26/2016] [Accepted: 04/30/2016] [Indexed: 11/15/2022] Open
Abstract
OBJECTIVE Using Failure Mode and Effects Analysis (FMEA) as an example quality improvement approach, our objective was to evaluate whether secondary use of orders, forms, and notes recorded by the electronic health record (EHR) during daily practice can enhance the accuracy of process maps used to guide improvement. We examined discrepancies between expected and observed activities and individuals involved in a high-risk process and devised diagnostic measures for understanding discrepancies that may be used to inform quality improvement planning. METHODS Inpatient cardiology unit staff developed a process map of discharge from the unit. We matched activities and providers identified on the process map to EHR data. Using four diagnostic measures, we analyzed discrepancies between expectation and observation. RESULTS EHR data showed that 35% of activities were completed by unexpected providers, including providers from 12 categories not identified as part of the discharge workflow. The EHR also revealed sub-components of process activities not identified on the process map. Additional information from the EHR was used to revise the process map and show differences between expectation and observation. CONCLUSION Findings suggest EHR data may reveal gaps in process maps used for quality improvement and identify characteristics about workflow activities that can identify perspectives for inclusion in an FMEA. Organizations with access to EHR data may be able to leverage clinical documentation to enhance process maps used for quality improvement. While focused on FMEA protocols, findings from this study may be applicable to other quality activities that require process maps.
Collapse
Affiliation(s)
- Gayle Shier Kricke
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, 60611, USA
| | - Matthew B Carson
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, 60611, USA
| | - Young Ji Lee
- School of Nursing, University of Pittsburgh, Pittsburgh, PA, 15261, USA
| | - Corrine Benacka
- Bluhm Cardiovascular Institute, Northwestern Memorial Hospital, Chicago, IL, 60611, USA
| | - R. Kannan Mutharasan
- Bluhm Cardiovascular Institute, Northwestern Memorial Hospital, Chicago, IL, 60611, USA
| | - Faraz S Ahmad
- Bluhm Cardiovascular Institute, Northwestern Memorial Hospital, Chicago, IL, 60611, USA
| | - Preeti Kansal
- Bluhm Cardiovascular Institute, Northwestern Memorial Hospital, Chicago, IL, 60611, USA
| | - Clyde W Yancy
- Bluhm Cardiovascular Institute, Northwestern Memorial Hospital, Chicago, IL, 60611, USA
| | - Allen S Anderson
- Bluhm Cardiovascular Institute, Northwestern Memorial Hospital, Chicago, IL, 60611, USA
| | - Nicholas D Soulakis
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Feinberg School of Medicine, Northwestern University, Chicago, IL, 60611, USA
| |
Collapse
|
14
|
Yan C, Chen Y, Li B, Liebovitz D, Malin B. Learning Clinical Workflows to Identify Subgroups of Heart Failure Patients. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2017; 2016:1248-1257. [PMID: 28269922 PMCID: PMC5333346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/06/2023]
Abstract
Heart Failure (HF) is one of the most common indications for readmission to the hospital among elderly patients. This is due to the progressive nature of the disease, as well as its association with complex comorbidities (e.g., anemia, chronic kidney disease, chronic obstructive pulmonary disease, hyper- and hypothyroidism), which contribute to increased morbidity and mortality, as well as a reduced quality of life. Healthcare organizations (HCOs) have established diverse treatment plans for HF patients, but such routines are not always formalized and may, in fact, arise organically as a patient's management evolves over time. This investigation was motivated by the hypothesis that patients associated with a certain subgroup of HF should follow a similar workflow that, once made explicit, could be leveraged by an HCO to more effectively allocate resources and manage HF patients. Thus, in this paper, we introduce a method to identify subgroups of HF through a similarity analysis of event sequences documented in the clinical setting. Specifically, we 1) structure event sequences for HF patients based on the patterns of electronic medical record (EMR) system utilization, 2) identify subgroups of HF patients by applying a k-means clustering algorithm on utilization patterns, 3) learn clinical workflows for each subgroup, and 4) label each subgroup with diagnosis and procedure codes that are distinguishing in the set of all subgroups. To demonstrate its potential, we applied our method to EMR event logs for 785 HF inpatient stays over a 4 month period at a large academic medical center. Our method identified 8 subgroups of HF, each of which was found to associate with a canonical workflow inferred through an inductive mining algorithm. Each subgroup was further confirmed to be affiliated with specific comorbidities, such as hyperthyroidism and hypothyroidism.
Collapse
Affiliation(s)
- Chao Yan
- Vanderbilt University, Nashville, TN
| | - You Chen
- Vanderbilt University, Nashville, TN
| | - Bo Li
- Vanderbilt University, Nashville, TN
| | | | | |
Collapse
|
15
|
Sacchi L, Holmes JH. Progress in Biomedical Knowledge Discovery: A 25-year Retrospective. Yearb Med Inform 2016; Suppl 1:S117-29. [PMID: 27488403 PMCID: PMC5171499 DOI: 10.15265/iys-2016-s033] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022] Open
Abstract
OBJECTIVES We sought to explore, via a systematic review of the literature, the state of the art of knowledge discovery in biomedical databases as it existed in 1992, and then now, 25 years later, mainly focused on supervised learning. METHODS We performed a rigorous systematic search of PubMed and latent Dirichlet allocation to identify themes in the literature and trends in the science of knowledge discovery in and between time periods and compare these trends. We restricted the result set using a bracket of five years previous, such that the 1992 result set was restricted to articles published between 1987 and 1992, and the 2015 set between 2011 and 2015. This was to reflect the current literature available at the time to researchers and others at the target dates of 1992 and 2015. The search term was framed as: Knowledge Discovery OR Data Mining OR Pattern Discovery OR Pattern Recognition, Automated. RESULTS A total 538 and 18,172 documents were retrieved for 1992 and 2015, respectively. The number and type of data sources increased dramatically over the observation period, primarily due to the advent of electronic clinical systems. The period 1992- 2015 saw the emergence of new areas of research in knowledge discovery, and the refinement and application of machine learning approaches that were nascent or unknown in 1992. CONCLUSIONS Over the 25 years of the observation period, we identified numerous developments that impacted the science of knowledge discovery, including the availability of new forms of data, new machine learning algorithms, and new application domains. Through a bibliometric analysis we examine the striking changes in the availability of highly heterogeneous data resources, the evolution of new algorithmic approaches to knowledge discovery, and we consider from legal, social, and political perspectives possible explanations of the growth of the field. Finally, we reflect on the achievements of the past 25 years to consider what the next 25 years will bring with regard to the availability of even more complex data and to the methods that could be, and are being now developed for the discovery of new knowledge in biomedical data.
Collapse
Affiliation(s)
| | - J H Holmes
- John H Holmes, Institute for Biomedical Informatics, University of Pennsylvania School of Medicine, 717 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104, USA, Tel: 215-898-4833, Fax: 215-573-5325, E-Mail:
| |
Collapse
|
16
|
Chen Y, Ghosh J, Bejan CA, Gunter CA, Gupta S, Kho A, Liebovitz D, Sun J, Denny J, Malin B. Building bridges across electronic health record systems through inferred phenotypic topics. J Biomed Inform 2015; 55:82-93. [PMID: 25841328 DOI: 10.1016/j.jbi.2015.03.011] [Citation(s) in RCA: 30] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/02/2014] [Revised: 03/24/2015] [Accepted: 03/25/2015] [Indexed: 10/23/2022]
Abstract
OBJECTIVE Data in electronic health records (EHRs) is being increasingly leveraged for secondary uses, ranging from biomedical association studies to comparative effectiveness. To perform studies at scale and transfer knowledge from one institution to another in a meaningful way, we need to harmonize the phenotypes in such systems. Traditionally, this has been accomplished through expert specification of phenotypes via standardized terminologies, such as billing codes. However, this approach may be biased by the experience and expectations of the experts, as well as the vocabulary used to describe such patients. The goal of this work is to develop a data-driven strategy to (1) infer phenotypic topics within patient populations and (2) assess the degree to which such topics facilitate a mapping across populations in disparate healthcare systems. METHODS We adapt a generative topic modeling strategy, based on latent Dirichlet allocation, to infer phenotypic topics. We utilize a variance analysis to assess the projection of a patient population from one healthcare system onto the topics learned from another system. The consistency of learned phenotypic topics was evaluated using (1) the similarity of topics, (2) the stability of a patient population across topics, and (3) the transferability of a topic across sites. We evaluated our approaches using four months of inpatient data from two geographically distinct healthcare systems: (1) Northwestern Memorial Hospital (NMH) and (2) Vanderbilt University Medical Center (VUMC). RESULTS The method learned 25 phenotypic topics from each healthcare system. The average cosine similarity between matched topics across the two sites was 0.39, a remarkably high value given the very high dimensionality of the feature space. The average stability of VUMC and NMH patients across the topics of two sites was 0.988 and 0.812, respectively, as measured by the Pearson correlation coefficient. Also the VUMC and NMH topics have smaller variance of characterizing patient population of two sites than standard clinical terminologies (e.g., ICD9), suggesting they may be more reliably transferred across hospital systems. CONCLUSIONS Phenotypic topics learned from EHR data can be more stable and transferable than billing codes for characterizing the general status of a patient population. This suggests that EHR-based research may be able to leverage such phenotypic topics as variables when pooling patient populations in predictive models.
Collapse
Affiliation(s)
- You Chen
- Dept. of Biomedical Informatics, School of Medicine, Vanderbilt University, Nashville, TN, USA.
| | - Joydeep Ghosh
- Dept. of Electrical & Computer Engineering, University of Texas, Austin, TX, USA
| | - Cosmin Adrian Bejan
- Dept. of Biomedical Informatics, School of Medicine, Vanderbilt University, Nashville, TN, USA
| | - Carl A Gunter
- Dept. of Computer Science, University of Illinois at Urbana-Champagne, Champaign, IL, USA
| | - Siddharth Gupta
- Dept. of Computer Science, University of Illinois at Urbana-Champagne, Champaign, IL, USA
| | - Abel Kho
- School of Medicine, Northwestern University, Chicago, IL, USA
| | - David Liebovitz
- School of Medicine, Northwestern University, Chicago, IL, USA
| | - Jimeng Sun
- School of Computational Science & Engineering, Georgia Institute of Technology, Atlanta, GA, USA
| | - Joshua Denny
- Dept. of Biomedical Informatics, School of Medicine, Vanderbilt University, Nashville, TN, USA; Department of Medicine, Vanderbilt University, Nashville, TN, USA
| | - Bradley Malin
- Dept. of Biomedical Informatics, School of Medicine, Vanderbilt University, Nashville, TN, USA; Dept. of Electrical Engineering & Computer Science, School of Engineering, Vanderbilt University, Nashville, TN, USA
| |
Collapse
|
17
|
Soulakis ND, Carson MB, Lee YJ, Schneider DH, Skeehan CT, Scholtens DM. Visualizing collaborative electronic health record usage for hospitalized patients with heart failure. J Am Med Inform Assoc 2015; 22:299-311. [PMID: 25710558 PMCID: PMC4394967 DOI: 10.1093/jamia/ocu017] [Citation(s) in RCA: 41] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022] Open
Abstract
Objective To visualize and describe collaborative electronic health record (EHR) usage for hospitalized patients with heart failure. Materials and methods We identified records of patients with heart failure and all associated healthcare provider record usage through queries of the Northwestern Medicine Enterprise Data Warehouse. We constructed a network by equating access and updates of a patient’s EHR to a provider-patient interaction. We then considered shared patient record access as the basis for a second network that we termed the provider collaboration network. We calculated network statistics, the modularity of provider interactions, and provider cliques. Results We identified 548 patient records accessed by 5113 healthcare providers in 2012. The provider collaboration network had 1504 nodes and 83 998 edges. We identified 7 major provider collaboration modules. Average clique size was 87.9 providers. We used a graph database to demonstrate an ad hoc query of our provider-patient network. Discussion Our analysis suggests a large number of healthcare providers across a wide variety of professions access records of patients with heart failure during their hospital stay. This shared record access tends to take place not only in a pairwise manner but also among large groups of providers. Conclusion EHRs encode valuable interactions, implicitly or explicitly, between patients and providers. Network analysis provided strong evidence of multidisciplinary record access of patients with heart failure across teams of 100+ providers. Further investigation may lead to clearer understanding of how record access information can be used to strategically guide care coordination for patients hospitalized for heart failure.
Collapse
Affiliation(s)
- Nicholas D Soulakis
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Northwestern University, Chicago, IL, USA
| | - Matthew B Carson
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Northwestern University, Chicago, IL, USA Center For Healthcare Studies, Institute for Public Health and Medicine, Northwestern University, Chicago, IL, USA
| | - Young Ji Lee
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Northwestern University, Chicago, IL, USA
| | - Daniel H Schneider
- Northwestern University Clinical and Translational Sciences Institute, Chicago, IL, USA
| | - Connor T Skeehan
- Division of Health and Biomedical Informatics, Department of Preventive Medicine, Northwestern University, Chicago, IL, USA
| | - Denise M Scholtens
- Division of Biostatistics, Department of Preventive Medicine, Northwestern University, Chicago, IL, USA
| |
Collapse
|