1
|
Reyes CN, Zheng K, Hanauer DA. Design, Implementation, and Usability of the Electronic Medical Record Search Engine (EMERSE) Tool. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2023; 2022:932-941. [PMID: 37128440 PMCID: PMC10148345] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 05/03/2023]
Abstract
Free text forms of clinical documentation stored in electronic health records contain a trove of data for researchers and clinicians alike. However, often these data are challenging to use and not easily accessible. EMERSE, a clinical documentation search and data abstraction tool developed by the University of Michigan, helps users in the task of searching through free text notes in clinical documentation. This study evaluates the usability and user experience of the EMERSE system, and draws inferences for the design of such systems. The study was conducted in 3 phases. In Phase 1, interviews with site administrators investigated factors that facilitate or hinder the implementation and adoption of EMERSE. Phase 2 employed semi-structured interviews to understand the uses, benefits, and limitations of the system from the perspective of experienced users. In Phase 3, system-naive users performed a set of basic workflow tasks, then completed post-activity questions and surveys to evaluate the intuitiveness and usability of the system. Participants rated the system exceptionally high on usability, user interface satisfaction, and perceived usefulness. Feedback also indicated that improvements could be made in visual contrast, affordances, and scope of notes indexed. These results indicate that tools such as EMERSE should be highly intuitive, attractive, and moderately customizable. This paper discusses some aspects of what may contribute to a system having such characteristics.
Collapse
Affiliation(s)
| | - Kai Zheng
- University of California, Irvine, Irvine, CA, USA
| | | |
Collapse
|
2
|
Hill JR, Visweswaran S, Ning X, Schleyer TK. Use, Impact, Weaknesses, and Advanced Features of Search Functions for Clinical Use in Electronic Health Records: A Scoping Review. Appl Clin Inform 2021; 12:417-428. [PMID: 34261171 PMCID: PMC8279817 DOI: 10.1055/s-0041-1730033] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/16/2022] Open
Abstract
Objective
Although vast amounts of patient information are captured in electronic health records (EHRs), effective clinical use of this information is challenging due to inadequate and inefficient access to it at the point of care. The purpose of this study was to conduct a scoping review of the literature on the use of EHR search functions within a single patient's record in clinical settings to characterize the current state of research on the topic and identify areas for future study.
Methods
We conducted a literature search of four databases to identify articles on within-EHR search functions or the use of EHR search function in the context of clinical tasks. After reviewing titles and abstracts and performing a full-text review of selected articles, we included 17 articles in the analysis. We qualitatively identified themes in those articles and synthesized the literature for each theme.
Results
Based on the 17 articles analyzed, we delineated four themes: (1) how clinicians use search functions, (2) impact of search functions on clinical workflow, (3) weaknesses of current search functions, and (4) advanced search features. Our review found that search functions generally facilitate patient information retrieval by clinicians and are positively received by users. However, existing search functions have weaknesses, such as yielding false negatives and false positives, which can decrease trust in the results, and requiring a high cognitive load to perform an inclusive search of a patient's record.
Conclusion
Despite the widespread adoption of EHRs, only a limited number of articles describe the use of EHR search functions in a clinical setting, despite evidence that they benefit clinician workflow and productivity. Some of the weaknesses of current search functions may be addressed by enhancing EHR search functions with collaborative filtering.
Collapse
Affiliation(s)
- Jordan R Hill
- Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana, United States
| | - Shyam Visweswaran
- Department of Biomedical Informatics, University of Pittsburgh, Pittsburgh, Pennsylvania, United States
| | - Xia Ning
- Department of Biomedical Informatics, The Ohio State University, Columbus, Ohio, United States.,Department of Computer Science and Engineering, The Ohio State University, Columbus, Ohio, United States.,Translational Data Analytics Institute, The Ohio State University, Ohio, United States
| | - Titus K Schleyer
- Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana, United States.,Center for Biomedical Informatics, Regenstrief Institute, Indianapolis, Indiana, United States
| |
Collapse
|
3
|
Yao N, Zhu X, Dow A, Mishra VK, Phillips A, Tu SP. An exploratory study of networks constructed using access data from an electronic health record. J Interprof Care 2018; 32:666-673. [PMID: 30015537 PMCID: PMC6344307 DOI: 10.1080/13561820.2018.1496902] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2017] [Accepted: 06/26/2018] [Indexed: 10/28/2022]
Abstract
Network analysis may be a powerful tool for studying interprofessional practice. Using electronic health record data and social network analysis, the network of healthcare professionals involved in colorectal cancer care at a large, urban academic medical center were mapped and studied. A total of 100 surgical colorectal cancer patients receiving treatment in 2013 and 2014 were selected at random. We used detailed access logs for the EHR to map the network of all healthcare professionals for each patient, including inpatient and outpatient settings. Approximately 2.45 million records of access logs from more than 6,800 unique users, representing over 150 roles or occupations were analyzed. Across all networks, professionals were connected to an average of 5.8 other professionals, but some were rarely connected with others while over 20 were very highly connected (> 100 other professionals). Housestaff, attending physicians, and nurses played central roles in the global network with a high number of inter- and intra-professional connections. Clusters of professionals with frequent interaction were demonstrated but, based on the size and complexity of the network, serendipitous interactions were unlikely. Settings for care seemed to influence these clusters. Patient-centric care networks were similar to the global network with some potentially important differences. Access-log information from electronic health records can be an important source of information about relationships between healthcare professionals. Findings from analyses such as this one may help define the state of current networks and potential targets for interventions to improve the quality of care.
Collapse
Affiliation(s)
- Nengliang Yao
- Department of Public Health Sciences, University of Virginia, Charlottesville, Virginia
| | - Xi Zhu
- Department of Health Management and Policy, University of Iowa, Iowa City, Iowa
| | - Alan Dow
- Department of Internal Medicine, Virginia Commonwealth University, Richmond, VA
| | - Vimal K Mishra
- Department of Internal Medicine, Virginia Commonwealth University, Richmond, VA
| | - Allison Phillips
- Department of Internal Medicine, Virginia Commonwealth University, Richmond, VA
| | - Shin-Ping Tu
- Department of Internal Medicine, Virginia Commonwealth University, Richmond, VA
| |
Collapse
|
4
|
Lin FPY, Pokorny A, Teng C, Epstein RJ. TEPAPA: a novel in silico feature learning pipeline for mining prognostic and associative factors from text-based electronic medical records. Sci Rep 2017; 7:6918. [PMID: 28761061 PMCID: PMC5537364 DOI: 10.1038/s41598-017-07111-0] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2016] [Accepted: 06/21/2017] [Indexed: 12/13/2022] Open
Abstract
Vast amounts of clinically relevant text-based variables lie undiscovered and unexploited in electronic medical records (EMR). To exploit this untapped resource, and thus facilitate the discovery of informative covariates from unstructured clinical narratives, we have built a novel computational pipeline termed Text-based Exploratory Pattern Analyser for Prognosticator and Associator discovery (TEPAPA). This pipeline combines semantic-free natural language processing (NLP), regular expression induction, and statistical association testing to identify conserved text patterns associated with outcome variables of clinical interest. When we applied TEPAPA to a cohort of head and neck squamous cell carcinoma patients, plausible concepts known to be correlated with human papilloma virus (HPV) status were identified from the EMR text, including site of primary disease, tumour stage, pathologic characteristics, and treatment modalities. Similarly, correlates of other variables (including gender, nodal status, recurrent disease, smoking and alcohol status) were also reliably recovered. Using highly-associated patterns as covariates, a patient's HPV status was classifiable using a bootstrap analysis with a mean area under the ROC curve of 0.861, suggesting its predictive utility in supporting EMR-based phenotyping tasks. These data support using this integrative approach to efficiently identify disease-associated factors from unstructured EMR narratives, and thus to efficiently generate testable hypotheses.
Collapse
Affiliation(s)
- Frank Po-Yen Lin
- Department of Oncology, St Vincent's Hospital & The Kinghorn Cancer Centre, Darlinghurst, NSW, Australia.
- Garvan Institute of Medical Research, Darlinghurst, NSW, Australia.
| | - Adrian Pokorny
- Department of Oncology, St Vincent's Hospital & The Kinghorn Cancer Centre, Darlinghurst, NSW, Australia
| | - Christina Teng
- Department of Medical Oncology, Liverpool Hospital, Liverpool, Sydney, NSW, Australia
| | - Richard J Epstein
- Department of Oncology, St Vincent's Hospital & The Kinghorn Cancer Centre, Darlinghurst, NSW, Australia
- Garvan Institute of Medical Research, Darlinghurst, NSW, Australia
| |
Collapse
|
5
|
Hanauer DA, Wu DTY, Yang L, Mei Q, Murkowski-Steffy KB, Vydiswaran VGV, Zheng K. Development and empirical user-centered evaluation of semantically-based query recommendation for an electronic health record search engine. J Biomed Inform 2017; 67:1-10. [PMID: 28131722 DOI: 10.1016/j.jbi.2017.01.013] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2016] [Revised: 12/21/2016] [Accepted: 01/23/2017] [Indexed: 02/01/2023]
Abstract
OBJECTIVE The utility of biomedical information retrieval environments can be severely limited when users lack expertise in constructing effective search queries. To address this issue, we developed a computer-based query recommendation algorithm that suggests semantically interchangeable terms based on an initial user-entered query. In this study, we assessed the value of this approach, which has broad applicability in biomedical information retrieval, by demonstrating its application as part of a search engine that facilitates retrieval of information from electronic health records (EHRs). MATERIALS AND METHODS The query recommendation algorithm utilizes MetaMap to identify medical concepts from search queries and indexed EHR documents. Synonym variants from UMLS are used to expand the concepts along with a synonym set curated from historical EHR search logs. The empirical study involved 33 clinicians and staff who evaluated the system through a set of simulated EHR search tasks. User acceptance was assessed using the widely used technology acceptance model. RESULTS The search engine's performance was rated consistently higher with the query recommendation feature turned on vs. off. The relevance of computer-recommended search terms was also rated high, and in most cases the participants had not thought of these terms on their own. The questions on perceived usefulness and perceived ease of use received overwhelmingly positive responses. A vast majority of the participants wanted the query recommendation feature to be available to assist in their day-to-day EHR search tasks. DISCUSSION AND CONCLUSION Challenges persist for users to construct effective search queries when retrieving information from biomedical documents including those from EHRs. This study demonstrates that semantically-based query recommendation is a viable solution to addressing this challenge.
Collapse
Affiliation(s)
- David A Hanauer
- Department of Pediatrics, University of Michigan Medical School, 5312 CC, SPC 5940, 1500 East Medical Center Drive, Ann Arbor, MI 48109, USA; School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA.
| | - Danny T Y Wu
- School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA; Department of Pediatrics, University of Michigan Medical School, 5312 CC, SPC 5940, 1500 East Medical Center Drive, Ann Arbor, MI 48109, USA.
| | - Lei Yang
- School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA.
| | - Qiaozhu Mei
- School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA; Department of Electrical Engineering and Computer Science, University of Michigan, 2260 Hayward Street, Ann Arbor, MI 48109, USA.
| | - Katherine B Murkowski-Steffy
- Department of Health Management and Policy, School of Public Health, 1415 Washington Heights, Ann Arbor, MI 48109, USA.
| | - V G Vinod Vydiswaran
- Department of Learning Health Sciences, University of Michigan Medical School, 1111 East Catherine Street, Ann Arbor, MI 48109, USA; School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA.
| | - Kai Zheng
- Department of Health Management and Policy, School of Public Health, 1415 Washington Heights, Ann Arbor, MI 48109, USA; School of Information, University of Michigan, 105 South State Street, Ann Arbor, MI 48109, USA.
| |
Collapse
|
6
|
Moen H, Ginter F, Marsi E, Peltonen LM, Salakoski T, Salanterä S. Care episode retrieval: distributional semantic models for information retrieval in the clinical domain. BMC Med Inform Decis Mak 2015; 15 Suppl 2:S2. [PMID: 26099735 PMCID: PMC4474584 DOI: 10.1186/1472-6947-15-s2-s2] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022] Open
Abstract
Patients' health related information is stored in electronic health records (EHRs) by health service providers. These records include sequential documentation of care episodes in the form of clinical notes. EHRs are used throughout the health care sector by professionals, administrators and patients, primarily for clinical purposes, but also for secondary purposes such as decision support and research. The vast amounts of information in EHR systems complicate information management and increase the risk of information overload. Therefore, clinicians and researchers need new tools to manage the information stored in the EHRs. A common use case is, given a--possibly unfinished--care episode, to retrieve the most similar care episodes among the records. This paper presents several methods for information retrieval, focusing on care episode retrieval, based on textual similarity, where similarity is measured through domain-specific modelling of the distributional semantics of words. Models include variants of random indexing and the semantic neural network model word2vec. Two novel methods are introduced that utilize the ICD-10 codes attached to care episodes to better induce domain-specificity in the semantic model. We report on experimental evaluation of care episode retrieval that circumvents the lack of human judgements regarding episode relevance. Results suggest that several of the methods proposed outperform a state-of-the art search engine (Lucene) on the retrieval task.
Collapse
|
7
|
Supporting information retrieval from electronic health records: A report of University of Michigan's nine-year experience in developing and using the Electronic Medical Record Search Engine (EMERSE). J Biomed Inform 2015; 55:290-300. [PMID: 25979153 DOI: 10.1016/j.jbi.2015.05.003] [Citation(s) in RCA: 291] [Impact Index Per Article: 32.3] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2015] [Revised: 03/31/2015] [Accepted: 05/05/2015] [Indexed: 12/18/2022]
Abstract
OBJECTIVE This paper describes the University of Michigan's nine-year experience in developing and using a full-text search engine designed to facilitate information retrieval (IR) from narrative documents stored in electronic health records (EHRs). The system, called the Electronic Medical Record Search Engine (EMERSE), functions similar to Google but is equipped with special functionalities for handling challenges unique to retrieving information from medical text. MATERIALS AND METHODS Key features that distinguish EMERSE from general-purpose search engines are discussed, with an emphasis on functions crucial to (1) improving medical IR performance and (2) assuring search quality and results consistency regardless of users' medical background, stage of training, or level of technical expertise. RESULTS Since its initial deployment, EMERSE has been enthusiastically embraced by clinicians, administrators, and clinical and translational researchers. To date, the system has been used in supporting more than 750 research projects yielding 80 peer-reviewed publications. In several evaluation studies, EMERSE demonstrated very high levels of sensitivity and specificity in addition to greatly improved chart review efficiency. DISCUSSION Increased availability of electronic data in healthcare does not automatically warrant increased availability of information. The success of EMERSE at our institution illustrates that free-text EHR search engines can be a valuable tool to help practitioners and researchers retrieve information from EHRs more effectively and efficiently, enabling critical tasks such as patient case synthesis and research data abstraction. CONCLUSION EMERSE, available free of charge for academic use, represents a state-of-the-art medical IR tool with proven effectiveness and user acceptance.
Collapse
|
8
|
Unbiased identification of patients with disorders of sex development. PLoS One 2014; 9:e108702. [PMID: 25268640 PMCID: PMC4182545 DOI: 10.1371/journal.pone.0108702] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/28/2014] [Accepted: 08/29/2014] [Indexed: 01/01/2023] Open
Abstract
Disorders of sex development (DSD) represent a collection of rare diseases that generate substantial controversy regarding best practices for diagnosis and treatment. A significant barrier preventing a better understanding of how patients with these conditions should be evaluated and treated, especially from a psychological standpoint, is the lack of systematic and standardized approaches to identify cases for study inclusion. Common approaches include "hand-picked" subjects already known to the practice, which could introduce bias. We implemented an informatics-based approach to identify patients with DSD from electronic health records (EHRs) at three large, academic children's hospitals. The informatics approach involved comprehensively searching EHRs at each hospital using a combination of structured billing codes as an initial filtering strategy followed by keywords applied to the free text clinical documentation. The informatics approach was implemented to replicate the functionality of an EHR search engine (EMERSE) available at one of the hospitals. At the two hospitals that did not have EMERSE, we compared case ascertainment using the informatics method to traditional approaches employed for identifying subjects. Potential cases identified using all approaches were manually reviewed by experts in DSD to verify eligibility criteria. At the two institutions where both the informatics and traditional approaches were applied, the informatics approach identified substantially higher numbers of potential study subjects. The traditional approaches yielded 14 and 28 patients with DSD, respectively; the informatics approach yielded 226 and 77 patients, respectively. The informatics approach missed only a few cases that the traditional approaches identified, largely because those cases were known to the study team, but patient data were not in the particular children's hospital EHR. The use of informatics approaches to search electronic documentation can result in substantially larger numbers of subjects identified for studies of rare diseases such as DSD, and these approaches can be applied across hospitals.
Collapse
|
9
|
Biron P, Metzger MH, Pezet C, Sebban C, Barthuet E, Durand T. An information retrieval system for computerized patient records in the context of a daily hospital practice: the example of the Léon Bérard Cancer Center (France). Appl Clin Inform 2014; 5:191-205. [PMID: 24734133 DOI: 10.4338/aci-2013-08-cr-0065] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/28/2013] [Accepted: 12/31/2013] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND A full-text search tool was introduced into the daily practice of Léon Bérard Center (France), a health care facility devoted to treatment of cancer. This tool was integrated into the hospital information system by the IT department having been granted full autonomy to improve the system. OBJECTIVES To describe the development and various uses of a tool for full-text search of computerized patient records. METHODS The technology is based on Solr, an open-source search engine. It is a web-based application that processes HTTP requests and returns HTTP responses. A data processing pipeline that retrieves data from different repositories, normalizes, cleans and publishes it to Solr, was integrated in the information system of the Leon Bérard center. The IT department developed also user interfaces to allow users to access the search engine within the computerized medical record of the patient. RESULTS From January to May 2013, 500 queries were launched per month by an average of 140 different users. Several usages of the tool were described, as follows: medical management of patients, medical research, and improving the traceability of medical care in medical records. The sensitivity of the tool for detecting the medical records of patients diagnosed with both breast cancer and diabetes was 83.0%, and its positive predictive value was 48.7% (gold standard: manual screening by a clinical research assistant). CONCLUSION The project demonstrates that the introduction of full-text-search tools allowed practitioners to use unstructured medical information for various purposes.
Collapse
Affiliation(s)
- P Biron
- Léon Bérard Cancer Center , Lyon, France
| | - M H Metzger
- Université Lyon I - CNRS-UMR 5558 , Lyon, France
| | - C Pezet
- Léon Bérard Cancer Center , Lyon, France
| | - C Sebban
- Léon Bérard Cancer Center , Lyon, France
| | - E Barthuet
- SWORD, Saint Didier au Mont d'Or , France
| | - T Durand
- Léon Bérard Cancer Center , Lyon, France
| |
Collapse
|
10
|
Pathak J, Kho AN, Denny JC. Electronic health records-driven phenotyping: challenges, recent advances, and perspectives. J Am Med Inform Assoc 2014; 20:e206-11. [PMID: 24302669 DOI: 10.1136/amiajnl-2013-002428] [Citation(s) in RCA: 165] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Affiliation(s)
- Jyotishman Pathak
- Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA
| | | | | |
Collapse
|
11
|
Hanauer DA, Ramakrishnan N, Seyfried LS. Describing the relationship between cat bites and human depression using data from an electronic health record. PLoS One 2013; 8:e70585. [PMID: 23936453 PMCID: PMC3731284 DOI: 10.1371/journal.pone.0070585] [Citation(s) in RCA: 23] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2013] [Accepted: 06/20/2013] [Indexed: 01/09/2023] Open
Abstract
Data mining approaches have been increasingly applied to the electronic health record and have led to the discovery of numerous clinical associations. Recent data mining studies have suggested a potential association between cat bites and human depression. To explore this possible association in more detail we first used administrative diagnosis codes to identify patients with either depression or bites, drawn from a population of 1.3 million patients. We then conducted a manual chart review in the electronic health record of all patients with a code for a bite to accurately determine which were from cats or dogs. Overall there were 750 patients with cat bites, 1,108 with dog bites, and approximately 117,000 patients with depression. Depression was found in 41.3% of patients with cat bites and 28.7% of those with dog bites. Furthermore, 85.5% of those with both cat bites and depression were women, compared to 64.5% of those with dog bites and depression. The probability of a woman being diagnosed with depression at some point in her life if she presented to our health system with a cat bite was 47.0%, compared to 24.2% of men presenting with a similar bite. The high proportion of depression in patients who had cat bites, especially among women, suggests that screening for depression could be appropriate in patients who present to a clinical provider with a cat bite. Additionally, while no causative link is known to explain this association, there is growing evidence to suggest that the relationship between cats and human mental illness, such as depression, warrants further investigation.
Collapse
Affiliation(s)
- David A Hanauer
- Department of Pediatrics, University of Michigan, Ann Arbor, Michigan, USA.
| | | | | |
Collapse
|
12
|
Zampi JD, Donohue JE, Charpie JR, Yu S, Hanauer DA, Hirsch JC. Retrospective database research in pediatric cardiology and congenital heart surgery: an illustrative example of limitations and possible solutions. World J Pediatr Congenit Heart Surg 2013; 3:283-7. [PMID: 23804858 DOI: 10.1177/2150135112440462] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND Secondary use of data, whether from clinical information systems or registries, for carrying out clinical research in rare diseases is a common practice but is fraught with potential errors. We sought to elucidate some of the limitations of database research and describe possible solutions to overcome these limitations. METHODS Using a disease model of a rare postsurgical outcome, we evaluated the ability of four different data sources to correctly identify patients who had that outcome both as individual databases and also when used in conjunction with each other. These results were compared with manual chart review. RESULTS The sensitivity of the various databases to pick up a rare and specific outcome was poor (9.9%-37%), while the specificities were fairly good (91%-96.7%). By combining the databases, the sensitivity was increased to as much as 56.8% without a large decrease in the specificity (85.2%-91.6%). The electronic medical record (EMR) search engine had the highest sensitivity (96.9%) and a high specificity (89.3%) with a very high negative predictive value (99.4%). CONCLUSION For rare and specific diseases or outcomes, a single data source search methodology can miss large numbers of patients and potentially bias study results. Combining overlapping databases can improve the ability to capture these rare diseases or outcomes. While chart review remains the most accurate way to obtain complete case capture, new tools like EMR search engines can facilitate the efficiency of this process without sacrificing search quality.
Collapse
Affiliation(s)
- Jeffrey D Zampi
- Division of Pediatric Cardiology, Department of Pediatrics, University of Michigan, C.S. Mott Children's Hospital, Ann Arbor, MI, USA
| | | | | | | | | | | |
Collapse
|
13
|
Abstract
Along with the increasing adoption of electronic health records (EHRs) are expectations that data collected within EHRs will be readily available for outcomes and comparative effectiveness research. Yet the ability to effectively share and reuse data depends on implementing and configuring EHRs with these goals in mind from the beginning. Data sharing and integration must be planned both locally as well as nationally. The rich data transmission and semantic infrastructure developed by the National Cancer Institute (NCI) for research provides an excellent example of moving beyond paper-based paradigms and exploiting the power of semantically robust, network-based systems, and engaging both domain and informatics expertise. Similar efforts are required to address current challenges in sharing EHR data.
Collapse
|
14
|
Hanauer D, Aberdeen J, Bayer S, Wellner B, Clark C, Zheng K, Hirschman L. Bootstrapping a de-identification system for narrative patient records: cost-performance tradeoffs. Int J Med Inform 2013; 82:821-31. [PMID: 23643147 DOI: 10.1016/j.ijmedinf.2013.03.005] [Citation(s) in RCA: 20] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2012] [Revised: 02/12/2013] [Accepted: 03/25/2013] [Indexed: 11/19/2022]
Abstract
PURPOSE We describe an experiment to build a de-identification system for clinical records using the open source MITRE Identification Scrubber Toolkit (MIST). We quantify the human annotation effort needed to produce a system that de-identifies at high accuracy. METHODS Using two types of clinical records (history and physical notes, and social work notes), we iteratively built statistical de-identification models by annotating 10 notes, training a model, applying the model to another 10 notes, correcting the model's output, and training from the resulting larger set of annotated notes. This was repeated for 20 rounds of 10 notes each, and then an additional 6 rounds of 20 notes each, and a final round of 40 notes. At each stage, we measured precision, recall, and F-score, and compared these to the amount of annotation time needed to complete the round. RESULTS After the initial 10-note round (33min of annotation time) we achieved an F-score of 0.89. After just over 8h of annotation time (round 21) we achieved an F-score of 0.95. Number of annotation actions needed, as well as time needed, decreased in later rounds as model performance improved. Accuracy on history and physical notes exceeded that of social work notes, suggesting that the wider variety and contexts for protected health information (PHI) in social work notes is more difficult to model. CONCLUSIONS It is possible, with modest effort, to build a functioning de-identification system de novo using the MIST framework. The resulting system achieved performance comparable to other high-performing de-identification systems.
Collapse
Affiliation(s)
- David Hanauer
- Department of Pediatrics, University of Michigan, Ann Arbor, MI, USA.
| | | | | | | | | | | | | |
Collapse
|
15
|
Abstract
Clinical research informatics is the rapidly evolving sub-discipline within biomedical informatics that focuses on developing new informatics theories, tools, and solutions to accelerate the full translational continuum: basic research to clinical trials (T1), clinical trials to academic health center practice (T2), diffusion and implementation to community practice (T3), and ‘real world’ outcomes (T4). We present a conceptual model based on an informatics-enabled clinical research workflow, integration across heterogeneous data sources, and core informatics tools and platforms. We use this conceptual model to highlight 18 new articles in the JAMIA special issue on clinical research informatics.
Collapse
Affiliation(s)
- Michael G Kahn
- Department of Pediatrics, University of Colorado, Aurora, Colorado 80045, USA.
| | | |
Collapse
|
16
|
Zheng K, Mei Q, Yang L, Manion FJ, Balis UJ, Hanauer DA. Voice-dictated versus typed-in clinician notes: linguistic properties and the potential implications on natural language processing. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2011; 2011:1630-1638. [PMID: 22195229 PMCID: PMC3243272] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
In this study, we comparatively examined the linguistic properties of narrative clinician notes created through voice dictation versus those directly entered by clinicians via a computer keyboard. Intuitively, the nature of voice-dictated notes would resemble that of natural language, while typed-in notes may demonstrate distinctive language features for reasons such as intensive usage of acronyms. The study analyses were based on an empirical dataset retrieved from our institutional electronic health records system. The dataset contains 30,000 voice-dictated notes and 30,000 notes that were entered manually; both were encounter notes generated in ambulatory care settings. The results suggest that between the narrative clinician notes created via these two different methods, there exists a considerable amount of lexical and distributional differences. Such differences could have a significant impact on the performance of natural language processing tools, necessitating these two different types of documents being differentially treated.
Collapse
Affiliation(s)
- Kai Zheng
- School of Public Health Department of Health Management and Policy, University of Michigan, Ann Arbor, MI, USA
| | | | | | | | | | | |
Collapse
|
17
|
Yang L, Mei Q, Zheng K, Hanauer DA. Query log analysis of an electronic health record search engine. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2011; 2011:915-924. [PMID: 22195150 PMCID: PMC3243246] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 05/31/2023]
Abstract
We analyzed a longitudinal collection of query logs of a full-text search engine designed to facilitate information retrieval in electronic health records (EHR). The collection, 202,905 queries and 35,928 user sessions recorded over a course of 4 years, represents the information-seeking behavior of 533 medical professionals, including frontline practitioners, coding personnel, patient safety officers, and biomedical researchers for patient data stored in EHR systems. In this paper, we present descriptive statistics of the queries, a categorization of information needs manifested through the queries, as well as temporal patterns of the users' information-seeking behavior. The results suggest that information needs in medical domain are substantially more sophisticated than those that general-purpose web search engines need to accommodate. Therefore, we envision there exists a significant challenge, along with significant opportunities, to provide intelligent query recommendations to facilitate information retrieval in EHR.
Collapse
Affiliation(s)
- Lei Yang
- School of Information, University of Michigan, Ann Arbor, MI, USA
| | | | | | | |
Collapse
|