1
|
Iscoe M, Socrates V, Gilson A, Chi L, Li H, Huang T, Kearns T, Perkins R, Khandjian L, Taylor RA. Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models. Acad Emerg Med 2024; 31:599-610. [PMID: 38567658 DOI: 10.1111/acem.14883] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/12/2023] [Revised: 01/24/2024] [Accepted: 01/24/2024] [Indexed: 04/04/2024]
Abstract
BACKGROUND Natural language processing (NLP) tools including recently developed large language models (LLMs) have myriad potential applications in medical care and research, including the efficient labeling and classification of unstructured text such as electronic health record (EHR) notes. This opens the door to large-scale projects that rely on variables that are not typically recorded in a structured form, such as patient signs and symptoms. OBJECTIVES This study is designed to acquaint the emergency medicine research community with the foundational elements of NLP, highlighting essential terminology, annotation methodologies, and the intricacies involved in training and evaluating NLP models. Symptom characterization is critical to urinary tract infection (UTI) diagnosis, but identification of symptoms from the EHR has historically been challenging, limiting large-scale research, public health surveillance, and EHR-based clinical decision support. We therefore developed and compared two NLP models to identify UTI symptoms from unstructured emergency department (ED) notes. METHODS The study population consisted of patients aged ≥ 18 who presented to an ED in a northeastern U.S. health system between June 2013 and August 2021 and had a urinalysis performed. We annotated a random subset of 1250 ED clinician notes from these visits for a list of 17 UTI symptoms. We then developed two task-specific LLMs to perform the task of named entity recognition: a convolutional neural network-based model (SpaCy) and a transformer-based model designed to process longer documents (Clinical Longformer). Models were trained on 1000 notes and tested on a holdout set of 250 notes. We compared model performance (precision, recall, F1 measure) at identifying the presence or absence of UTI symptoms at the note level. RESULTS A total of 8135 entities were identified in 1250 notes; 83.6% of notes included at least one entity. Overall F1 measure for note-level symptom identification weighted by entity frequency was 0.84 for the SpaCy model and 0.88 for the Longformer model. F1 measure for identifying presence or absence of any UTI symptom in a clinical note was 0.96 (232/250 correctly classified) for the SpaCy model and 0.98 (240/250 correctly classified) for the Longformer model. CONCLUSIONS The study demonstrated the utility of LLMs and transformer-based models in particular for extracting UTI symptoms from unstructured ED clinical notes; models were highly accurate for detecting the presence or absence of any UTI symptom on the note level, with variable performance for individual symptoms.
Collapse
Affiliation(s)
- Mark Iscoe
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
- Section for Biomedical Informatics and Data Science, Yale University School of Medicine, New Haven, Connecticut, USA
| | - Vimig Socrates
- Section for Biomedical Informatics and Data Science, Yale University School of Medicine, New Haven, Connecticut, USA
- Program of Computational Biology and Bioinformatics, Yale University, New Haven, Connecticut, USA
| | - Aidan Gilson
- Yale School of Medicine, New Haven, Connecticut, USA
| | - Ling Chi
- Department of Biostatistics, Yale School of Public Health, New Haven, Connecticut, USA
| | - Huan Li
- Program of Computational Biology and Bioinformatics, Yale University, New Haven, Connecticut, USA
| | - Thomas Huang
- Yale School of Medicine, New Haven, Connecticut, USA
| | - Thomas Kearns
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - Rachelle Perkins
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - Laura Khandjian
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - R Andrew Taylor
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
- Section for Biomedical Informatics and Data Science, Yale University School of Medicine, New Haven, Connecticut, USA
| |
Collapse
|
2
|
Yu CW, Nanji K, Hatamnejad A, Gemae M, Joarder I, Achunair A, Devji T, Phillips M, Zeraatkar D, Steel DH, Guymer RH, Sivaprasad S, Wykoff CC, Chaudhary V. Patient-Reported Outcome Measure Use in Guidelines Published by the American Academy of Ophthalmology: A Review. Ophthalmology 2023; 130:1201-1211. [PMID: 37429499 DOI: 10.1016/j.ophtha.2023.07.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2023] [Revised: 06/13/2023] [Accepted: 07/03/2023] [Indexed: 07/12/2023] Open
Abstract
TOPIC We reviewed the use of patient-reported outcome measures (PROMs) in the treatment of ophthalmologic conditions as recommended by the Clinical Practice Guidelines (CPGs) published by the American Academy of Ophthalmology (AAO). CLINICAL RELEVANCE Patient-reported outcome measures are standardized instruments that provide information regarding a patient's health status or health-related quality of life. Patient-reported outcome measures are increasingly used to inform study end points in ophthalmology studies. However, the extent to which PROMs are ultimately informing patient management recommendations in ophthalmology as part of CPGs remains an area of evidence gap. METHODS We included all CPGs published by the AAO from inception to June 2022. We also included all primary studies and systematic reviews cited in the treatment sections of the CPGs evaluating treatment of an ophthalmic condition. The primary outcome was the frequency of PROMs discussed in CPGs and in cited studies evaluating treatment. Secondary outcomes included frequency of minimal important difference (MID) use to contextualize PROM results and percentage of strong and discretionary recommendations supported by PROMs. We published a study protocol a priori on PROSPERO (CRD42022307427). Reporting followed the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines. We assessed risk of bias using the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument. RESULTS We identified 24 eligible CPGs, providing 2458 cited studies (2191 primary, 267 secondary) evaluating treatment of eye conditions. Ten CPGs (41.7%) reported consideration of PROMs. Of these, 31 of 94 (33%) recommendations were informed by studies evaluating a PROM as an outcome. Across all studies cited in the development of CPGs, 221 (9.0%) used PROMs as a primary or secondary outcome, of which 4 PROM results (1.8%) were interpreted using an empirically determined MID. Overall, the risk of bias was low for all CPGs. CONCLUSIONS Overall, outcomes of PROMs are seldom used in ophthalmology CPGs published by the AAO and in cited primary and secondary research on treatments. When PROMs were considered, their interpretation was seldom based on an MID. To improve patient care, guideline developers may consider incorporating PROMs and applicable MIDs to inform key outcomes when formulating treatment recommendations. FINANCIAL DISCLOSURE(S) Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Collapse
Affiliation(s)
- Caberry W Yu
- Department of Surgery, McMaster University, Hamilton, Ontario, Canada
| | - Keean Nanji
- Department of Surgery, McMaster University, Hamilton, Ontario, Canada; Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Amin Hatamnejad
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Ontario, Canada
| | - Mohamed Gemae
- School of Medicine, Queen's University, Kingston, Ontario, Canada
| | - Ishraq Joarder
- Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada
| | | | - Tahira Devji
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Mark Phillips
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada
| | - Dena Zeraatkar
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada; Department of Anesthesia, McMaster University, Hamilton, Ontario, Canada
| | - David H Steel
- Bioscience Institute, Newcastle University, Newcastle Upon Tyne, United Kingdom; Sunderland Eye Infirmary, Queen Alexandra Road, Sunderland, United Kingdom
| | - Robyn H Guymer
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Australia; Department of Surgery, The University of Melbourne, Melbourne, Australia
| | - Sobha Sivaprasad
- NIHR Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, United Kingdom
| | - Charles C Wykoff
- Retina Consultants of Texas, Retina Consultants of America, Houston, Texas; Blanton Eye Institute, Houston Methodist Hospital, Houston, Texas
| | - Varun Chaudhary
- Department of Surgery, McMaster University, Hamilton, Ontario, Canada; Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada.
| |
Collapse
|
3
|
Lund-Jacobsen T, Bentsen L, Schwarz P, Knop AS, Pappot H, Piil K. Aromatase Inhibitor-Related Symptoms Reported by Postmenopausal Women with Nonmetastatic, Estrogen Receptor-Positive Breast Cancer: A Systematic Review. Semin Oncol Nurs 2023; 39:151487. [PMID: 37612223 DOI: 10.1016/j.soncn.2023.151487] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2023] [Revised: 07/13/2023] [Accepted: 07/19/2023] [Indexed: 08/25/2023]
Abstract
PURPOSE The objective of this systematic review was to establish an overview of aromatase inhibitor-related symptoms reported by postmenopausal women with nonmetastatic, estrogen receptor-positive breast cancer. DATA SOURCES Eight databases (PubMed, Cochrane, Cumulative Index to Nursing and Allied Health Literature [CINAHL], Ovid EMBASE, Ovid MEDLINE, PsycINFO, Scopus, and Web of Science) were searched for trials published between January 2004 and November 2021. Inclusion criteria were studies exploring patient-reported aromatase inhibitor-related symptoms in postmenopausal women with nonmetastatic estrogen receptor-positive breast cancer. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and the Mixed Method Appraisal Tool were used to rate the quality of the trials included. Of 325 full-text papers, 10 were included. Patient-reported symptoms were clustered by using the European Organization for Research and Treatment of Cancer Quality of Life C30 questionnaire domains. Additional domains were used to cluster other symptoms mentioned: menopausal, sex-related, body alteration, and eye-related. The following clusters were the most frequently presented: sex-related (14 symptoms), pain (9 symptoms), insomnia (5 symptoms), and menopausal (5 symptoms). CONCLUSION The target group reported a variety of symptoms related to aromatase inhibitors. No tools are currently available to measure all the symptoms reported, indicating a need to revise the tools to acknowledge additional symptoms. Prospective studies are needed to investigate the prevalence of aromatase inhibitor-related symptoms in women with breast cancer. IMPLICATION FOR NURSING PRACTICE Identification of patient-reported clinically relevant symptoms can enable targeted symptom assessment and management strategies for women with breast cancer undergoing aromatase inhibitor treatment.
Collapse
Affiliation(s)
- Trine Lund-Jacobsen
- BSN, MSc, PhD student, Department of Endocrinology and Metabolism, Centre for Cancer and Organ Diseases, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark.
| | - Line Bentsen
- MD, PhD student, Department of Oncology, Centre for Cancer and Organ Diseases, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Peter Schwarz
- Professor, MD, DMSc, Department of Endocrinology and Metabolism, Centre for Cancer and Organ Diseases, Copenhagen University Hospital, Denmark, and Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Ann Søegaard Knop
- MD, PhD, Department of Oncology, Centre for Cancer and Organ Diseases, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
| | - Helle Pappot
- Professor, MD, DMSc, Department of Oncology, Centre for Cancer and Organ Diseases, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark and Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Karin Piil
- MHScN, associate professor, Department of Oncology, Centre for Cancer and Organ Diseases, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark and Department of Publich Health, Faculty of Health, University of Aarhus, Aarhus C, Denmark
| |
Collapse
|
4
|
Woodward MA, Niziol LM, Ballouz D, Lu MC, Kang L, Thibodeau A, Singh K. Prediction of Visual Acuity in Patients With Microbial Keratitis. Cornea 2023; 42:217-223. [PMID: 36256452 PMCID: PMC9805475 DOI: 10.1097/ico.0000000000003129] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2022] [Accepted: 07/16/2022] [Indexed: 01/03/2023]
Abstract
PURPOSE The purpose of this study was to predict visual acuity (VA) 90 days after presentation for patients with microbial keratitis (MK) from data at the initial clinical ophthalmic encounter. METHODS Patients with MK were identified in the electronic health record between August 2012 and February 2021. Random forest (RF) models were used to predict 90-day VA < 20/40 [visual impairment (VI)]. Predictors evaluated included age, sex, initial VA, and information documented in notes at presentation. Model diagnostics are reported with 95% confidence intervals (CIs) for area under the curve (AUC), misclassification rate, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). RESULTS One thousand seven hundred ninety-one patients were identified. The presenting logMAR VA was on average 0.86 (Snellen equivalent and standard deviation = 20/144 ± 12.6 lines) in the affected or worse eye, and 43.6% with VI. VI at 90-day follow-up was present in the affected eye or worse eye for 26.9% of patients. The RF model for predicting 90-day VI had an AUC of 95% (CI: 93%-97%) and a misclassification rate of 9% (7%-12%). The percent sensitivity, specificity, PPV, and NPV were 86% (80%-91%), 92% (89%-95%), 81% (74%-86%), and 95% (92%-97%), respectively. Older age, worse presenting VA, and more mentions of "penetrating keratoplasty" and "bandage contact lens" were associated with increased probability of 90-day VI, whereas more mentions of "quiet" were associated with decreased probability of 90-day VI. CONCLUSIONS RF modeling yielded good sensitivity and specificity to predict VI at 90 days which could guide clinicians about the risk of poor vision outcomes for patients with MK.
Collapse
Affiliation(s)
- Maria A. Woodward
- Department of Ophthalmology and Visual Sciences, W.K. Kellogg Eye Center, University of Michigan, Ann Arbor, MI, USA
- Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MI, USA
| | - Leslie M. Niziol
- Department of Ophthalmology and Visual Sciences, W.K. Kellogg Eye Center, University of Michigan, Ann Arbor, MI, USA
| | - Dena Ballouz
- Department of Ophthalmology and Visual Sciences, W.K. Kellogg Eye Center, University of Michigan, Ann Arbor, MI, USA
| | - Ming-Chen Lu
- Department of Ophthalmology and Visual Sciences, W.K. Kellogg Eye Center, University of Michigan, Ann Arbor, MI, USA
| | - Linda Kang
- Department of Ophthalmology and Visual Sciences, W.K. Kellogg Eye Center, University of Michigan, Ann Arbor, MI, USA
| | - Alexa Thibodeau
- Department of Ophthalmology and Visual Sciences, W.K. Kellogg Eye Center, University of Michigan, Ann Arbor, MI, USA
| | - Karandeep Singh
- Department of Learning Health Sciences, University of Michigan, Ann Arbor, MI, USA
- Department of Internal Medicine, University of Michigan, Ann Arbor, MI, USA
| |
Collapse
|
5
|
Reynolds ME, Gu W. Trend of Incidence of Ocular Injury in Service Members of the U.S. Armed Forces in 2016–2019. Ophthalmic Epidemiol 2022:1-7. [DOI: 10.1080/09286586.2022.2129697] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Affiliation(s)
- Mark E. Reynolds
- Vision Center of Excellence, Defense Health Agency, Falls Church, Maryland, USA
| | - Weidong Gu
- Vision Center of Excellence, Defense Health Agency, Falls Church, Maryland, USA
| |
Collapse
|
6
|
Calanzani N, Pannebakker MM, Tagg MJ, Walford H, Holloway P, de Wit N, Hamilton W, Walter FM. Who are the patients being offered the faecal immunochemical test in routine English general practice, and for what symptoms? A prospective descriptive study. BMJ Open 2022; 12:e066051. [PMID: 36123111 PMCID: PMC9486301 DOI: 10.1136/bmjopen-2022-066051] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/03/2022] Open
Abstract
OBJECTIVES The faecal immunochemical test (FIT) was introduced to triage patients with lower-risk symptoms of colorectal cancer (CRC) in English primary care in 2018. While there is growing evidence on its utility to triage patients in this setting, evidence is still limited on how official FIT guidance is being used, for which patients and for what symptoms. We aimed to investigate the use of FIT in primary care practice for lower-risk patients who did not immediately meet criteria for urgent referral. DESIGN A prospective, descriptive study of symptomatic patients offered a FIT in primary care between January and June 2020. SETTING East of England general practices. PARTICIPANTS Consenting patients (aged ≥40 years) who were seen by their general practitioners (GPs) with symptoms of possible CRC for whom a FIT was requested. We excluded patients receiving a FIT for asymptomatic screening purposes, or patients deemed by GPs as lacking capacity for informed consent. Data were obtained via patient questionnaire, medical and laboratory records. PRIMARY AND SECONDARY OUTCOME MEASURES FIT results (10 µg Hb/g faeces defined a positive result); patient sociodemographic and clinical characteristics; patient-reported and GP-recorded symptoms, symptom severity and symptom agreement between patient and GP (% and kappa statistics). RESULTS Complete data were available for 310 patients, median age 70 (IQR 61-77) years, 53% female and 23% FIT positive. Patients most commonly reported change in bowel habit (69%) and fatigue (57%), while GPs most commonly recorded abdominal pain (25%) and change in bowel habit (24%). Symptom agreement ranged from 44% (fatigue) to 80% (unexplained weight loss). Kappa agreement was universally low across symptoms. CONCLUSION Almost a quarter of this primary care cohort of symptomatic patients with FIT testing were found to be positive. However, there was low agreement between patient-reported and GP-recorded symptoms. This may impact cancer risk assessment and optimal patient management in primary care.
Collapse
Affiliation(s)
- Natalia Calanzani
- The Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
| | - Merel M Pannebakker
- The Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
| | - Max J Tagg
- School of Clinical Medicine, Addenbrooke's Hospital, University of Cambridge, Cambridge, UK
| | - Hugo Walford
- School of Clinical Medicine, University College London, London, UK
| | | | - Niek de Wit
- Julius Center for Health Sciences and Primary Care, University Medical Center, Utrecht University, Utrecht, The Netherlands
| | - Willie Hamilton
- University of Exeter Medical School, University of Exeter, Exeter, UK
| | - Fiona M Walter
- The Primary Care Unit, Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
- Wolfson Institute of Population Health, Queen Mary University of London, London, UK
| |
Collapse
|
7
|
Mockler GL, Novotny SP, Hou W, Liu Y, Schoenfeld ER. Patient Self-Report Superior to Electronic Medical Record Abstraction for Identifying Positive COVID-19 Symptoms at Illness Onset. AJPM FOCUS 2022; 1:100005. [PMID: 36942014 PMCID: PMC9095498 DOI: 10.1016/j.focus.2022.100005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Introduction Most initial COVID-19 research focused on hospitalized patients. Presenting symptomatology in the outpatient setting was poorly characterized, making it difficult for primary care physicians to predict which patients would require hospitalization. The purpose of this study was to characterize the presenting symptoms of COVID-19 infection and baseline patient characteristics and evaluate for correlation with disease severity, duration, and chronicity in the outpatient setting. Methods A total of 107 adult, English-speaking patients with suspected and confirmed COVID-19 cases at the 3 primary care practices of Stony Brook University Hospital were studied between March and December 2020. Survey data were collected from patient telephone interviews and electronic medical record abstraction. The potential risk factors assessed included participant demographics, medical comorbidities, and the number and type of symptoms at illness onset. Outcome measures included symptom duration, hospitalizations, and persistence of symptoms at 12 weeks from study enrollment. Results Patient self-report survey elicited nearly twice as many symptoms described at illness onset as those recorded in the electronic medical record (p<0.0001). A higher number of symptoms at illness onset was positively associated with symptom duration and chronicity. The presence of fever and hypoxia at the onset of illness were each positively associated with eventual hospitalization for COVID-19 disease. Conclusions Early in the setting of newly emerging infectious diseases, particularly those such as COVID-19 that involve multiple organ systems, patient self-report of symptoms using a complete review of systems rather than electronic medical record abstraction alone may be key for accurate disease identification and characterization as well as prediction of eventual disease severity, duration, and chronicity.
Collapse
Affiliation(s)
- Gretchen L Mockler
- Department of Family, Population & Preventive Medicine, Renaissance School of Medicine, Stony Brook University Hospital, Stony Brook, New York
| | - Samantha P Novotny
- Renaissance School of Medicine, Stony Brook University Hospital, Stony Brook, New York
| | - Wei Hou
- Department of Family, Population & Preventive Medicine, Renaissance School of Medicine, Stony Brook University Hospital, Stony Brook, New York
| | - Yuhang Liu
- Department of Applied Mathematics & Statistics, College of Engineering and Applied Sciences, Stony Brook University, Stony Brook, New York
| | - Elinor R Schoenfeld
- Department of Family, Population & Preventive Medicine, Renaissance School of Medicine, Stony Brook University Hospital, Stony Brook, New York
| |
Collapse
|
8
|
Goldstein JE, Guo X, Boland MV, Smith KE. Visual Acuity – Assessment of Data Quality and Usability in an Electronic Health Record System. OPHTHALMOLOGY SCIENCE 2022; 3:100215. [PMID: 36275199 PMCID: PMC9574716 DOI: 10.1016/j.xops.2022.100215] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/27/2022] [Revised: 08/19/2022] [Accepted: 08/25/2022] [Indexed: 11/30/2022]
Abstract
Objective To examine the data quality and usability of visual acuity (VA) data extracted from an electronic health record (EHR) system during ophthalmology encounters and provide recommendations for consideration of relevant VA end points in retrospective analyses. Design Retrospective, EHR data analysis. Participants All patients with eyecare office encounters at any 1 of the 9 locations of a large academic medical center between August 1, 2013, and December 31, 2015. Methods Data from 13 of the 21 VA fields (accounting for 93% VA data) in EHR encounters were extracted, categorized, recoded, and assessed for conformance and plausibility using an internal data dictionary, a 38-item listing of VA line measurements and observations including 28 line measurements (e.g., 20/30, 20/400) and 10 observations (e.g., no light perception). Entries were classified into usable and unusable data. Usable data were further categorized based on conformance to the internal data dictionary: (1) exact match; (2) conditional conformance, letter count (e.g., 20/30+2-3); (3) convertible conformance (e.g., 5/200 to 20/800); (4) plausible but cannot be conformed (e.g., 5/400). Data were deemed unusable when they were not plausible. Main Outcome Measures Proportions of usable and unusable VA entries at the overall and subspecialty levels. Results All VA data from 513 036 encounters representing 166 212 patients were included. Of the 1 573 643 VA entries, 1 438 661 (91.4%) contained usable data. There were 1 196 720 (76.0%) exact match (category 1), 185 692 (11.8%) conditional conformance (category 2), 40 270 (2.6%) convertible conformance (category 3), and 15 979 (1.0%) plausible but not conformed entries (category 4). Visual acuity entries during visits with providers from retina (17.5%), glaucoma (14.0%), neuro-ophthalmology (8.9%), and low vision (8.8%) had the highest rates of unusable data. Documented VA entries with providers from comprehensive eyecare (86.7%), oculoplastics (81.5%), and pediatrics/strabismus (78.6%) yielded the highest proportions of exact match with the data dictionary. Conclusions Electronic health record VA data quality and usability vary across documented VA measures, observations, and eyecare subspecialty. We proposed a checklist of considerations and recommendations for planning, extracting, analyzing, and reporting retrospective study outcomes using EHR VA data. These are important first steps to standardize analyses enabling comparative research.
Collapse
Affiliation(s)
- Judith E. Goldstein
- Wilmer Eye Institute, Johns Hopkins University, Baltimore, Maryland
- Correspondence: Judith E. Goldstein, OD, 600 N Wolfe Street, Baltimore, MD 21287.
| | - Xinxing Guo
- Wilmer Eye Institute, Johns Hopkins University, Baltimore, Maryland
| | - Michael V. Boland
- Wilmer Eye Institute, Johns Hopkins University, Baltimore, Maryland
- Department of Ophthalmology, Massachusetts Eye and Ear Infirmary, Harvard Medical School, Boston, Massachusetts
| | - Kerry E. Smith
- Wilmer Eye Institute, Johns Hopkins University, Baltimore, Maryland
| |
Collapse
|
9
|
Zhou H, Yao M, Gu X, Liu M, Zeng R, Li Q, Chen T, He W, Chen X, Yuan G. Application of Patient-Reported Outcome Measurements in Clinical Trials in China. JAMA Netw Open 2022; 5:e2211644. [PMID: 35544134 PMCID: PMC9096600 DOI: 10.1001/jamanetworkopen.2022.11644] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
IMPORTANCE Regulatory authorities, industry peers, and international health policies have emphasized the value of assessing patient-reported outcomes (PROs) in clinical studies. Despite the increase in the number of clinical studies in the last decade in China, little is known about the extent of the use of PROs. OBJECTIVE To evaluate the application and characteristics of PRO instruments as primary and secondary outcomes in randomized clinical trials in China. DESIGN, SETTING, AND PARTICIPANTS A cross-sectional study of interventional clinical trials conducted in China from January 1, 2010, to December 31, 2020, was performed. Data obtained from the Chinese Clinical Trial Registry and ClinicalTrials.gov databases were evaluated. MAIN OUTCOMES AND MEASURES Trials were categorized according to those that (1) precisely listed PRO tools as outcomes, (2) mentioned patient subjective feelings in outcomes but did not clarify which tools were used for assessment, and (3) did not mention any PRO measurements. Data on study phase, setting, participant age, and sex were extracted from trials that considered patient feelings, along with the target diseases and names of the PRO tools. RESULTS Among a total of 34 033 trials, 6915 (20.3%) listed the explicit PRO instruments used and 3178 (9.3%) included PRO in their outcomes but did not include the names of the assessment tools. From more than 32 million people included in the registered trials, data on 1.5 million (4.7%) patients were scientifically collected by PRO instruments, and subjective feelings were assessed for 693 867 (2.1%) participants. Pain (16.8%), cancer (15.6%), and musculoskeletal symptoms (13.3%) were the most common conditions for which PROs were precisely collected by tools. The most common tools for PRO measurements were the visual analog scale, Short-Form 36, and Hamilton Depression Scale. CONCLUSIONS AND RELEVANCE In this cross-sectional study, the use of PROs increased during the study period in clinical trials conducted in China. However, patient opinion appears to still be rarely measured. The application of PRO is geographically unevenly distributed. Development of PRO instruments, especially those suitable for the Chinese population, may be useful. Further expansion of PROs with respect to the scope of diseases is needed to avoid missing important data.
Collapse
Affiliation(s)
- Hui Zhou
- Phase I Clinical Trial Center, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
- Department of Pharmacy, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Mi Yao
- Institute of Applied Health Research, University of Birmingham, Birmingham, United Kingdom
| | - Xiaodan Gu
- Phase I Clinical Trial Center, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Mingrui Liu
- School of Pharmaceutical Sciences, Sun Yat-sen University, Guangzhou, China
| | - Ruifeng Zeng
- Phase I Clinical Trial Center, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Qin Li
- Department of Geriatrics, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Tingjia Chen
- Healthcare Center, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Wen He
- Department of Geriatrics, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Xiao Chen
- Department of Pharmacy, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| | - Gang Yuan
- Phase I Clinical Trial Center, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
- Department of Geriatrics, the First Affiliated Hospital, Sun Yat-sen University, Guangzhou, China
| |
Collapse
|
10
|
Influencing factors of treatment and prognosis perceptions among advanced cancer patients: a cross-sectional study. Support Care Cancer 2021; 30:1209-1220. [PMID: 34455482 DOI: 10.1007/s00520-021-06516-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2021] [Accepted: 08/21/2021] [Indexed: 02/06/2023]
Abstract
PURPOSE The present study aimed to investigate the status and significantly influencing factors of treatment and prognosis perceptions among advanced cancer patients based on patient-reported outcome. METHODS A cross-sectional study was conducted at two tertiary A general hospitals. From June to September 2019, 300 patients were invited and 292 of them participated in this study. Except for 9 invalid questionnaires, 283 pen-paper questionnaires including sociodemographic and clinical characteristics, Chinese Version of Prognosis and Treatment Perception Questionnaire, Herth Hope Index, and Hospital Anxiety and Depression Scale were well completed. Descriptive analysis, Pearson's correlation test, logistic regression analysis, and multiple linear regression analysis were applied for analysis. RESULTS One hundred seventy-five (61.8%) advanced cancer patients reported inaccurate treatment perception. Prognosis perception scored 87.9 ± 13.72 indicating a middle level of prognosis perception. Fourteen (4.9%), 138 (48.8%), and 131 (46.3%) patients presented low, middle, and high prognosis perception levels, respectively. In patients, without spouse and religion beliefs, received chemoradiotherapy, diagnosed as cancer equal to or less than 1 year, and higher hope level were inaccurate treatment perception's risk factors. Younger age, longer diagnosis time, higher educational level, less support for medical expenses payment, receiving chemoradiotherapy, and lower hope level but more anxiety and depression symptoms were positive predictors of prognosis perception. CONCLUSIONS A majority of advanced cancer patients in this study reported inaccurate treatment and middle level of prognosis perception influencing by objective and subjective factors. Clinical interventions could be developed referring these impacting factors originating from patient-reported outcome.
Collapse
|
11
|
Gleason K, Dahm MR. How patients describe their diagnosis compared to clinical documentation. ACTA ACUST UNITED AC 2021; 9:250-254. [PMID: 34391215 DOI: 10.1515/dx-2021-0070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2021] [Accepted: 07/28/2021] [Indexed: 11/15/2022]
Abstract
OBJECTIVES To explore how patients describe their diagnoses following Emergency Department (ED) discharge, and how this compares to electronic medical record (EMR) documentation. METHODS We conducted a cohort study of patients discharged from three EDs. Patients completed questionnaires regarding their understanding of their diagnosis. Inclusion criteria: adult ED patients aged 18 and older seen within the last seven days. We independently compared patient-reported new diagnoses following discharge to EMR-documented diagnoses regarding diagnostic content (identical, insignificantly different, different, not enough detail) and the level of technical language in diagnostic description (technical, semi-technical, lay). RESULTS The majority of participants (n=95 out of 137) reported receiving a diagnosis and stated the given diagnosis. Of those who reported their diagnosis, 66%, were females (n=62), the average age was 43 (SD 16), and a fourth (n=24) were Black and 66% (n=63) were white. The majority (84%) described either the same or an insignificantly different diagnosis. For 11% the patient-reported diagnosis differed from the one documented. More than half reported their diagnosis using semi-technical (34%) or technical language (26%), and over a third (40%) described their diagnosis in lay language. CONCLUSIONS Patient-reported diagnoses following ED discharge had moderate agreement with EMR-documented diagnoses. Findings suggest that patients might reproduce verbatim semi-technical or technical diagnoses they received from clinicians, but not fully understood what the diagnosis means for them.
Collapse
Affiliation(s)
- Kelly Gleason
- School of Nursing, Johns Hopkins University, Baltimore, MD, USA
| | - Maria R Dahm
- Institute for Communication in Health Care, Australian National University, Canberra, ACT, Australia
| |
Collapse
|
12
|
Singh K, Woodward MA. The Rigorous Work of Evaluating Consistency and Accuracy in Electronic Health Record Data. JAMA Ophthalmol 2021; 139:894-895. [PMID: 34196690 DOI: 10.1001/jamaophthalmol.2021.2042] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Affiliation(s)
- Karandeep Singh
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor.,Department of Internal Medicine, University of Michigan Medical School, Ann Arbor.,Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor
| | - Maria A Woodward
- Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor.,Department of Ophthalmology and Visual Sciences, W. K. Kellogg Eye Center, University of Michigan Medical School, Ann Arbor
| |
Collapse
|
13
|
Hatt SR, Leske DA, Iezzi R, Holmes JM. Binocular Interference vs Diplopia in Patients With Epiretinal Membrane. JAMA Ophthalmol 2021; 138:1121-1127. [PMID: 32910144 DOI: 10.1001/jamaophthalmol.2020.3328] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Importance Patients with epiretinal membrane (ERM) sometimes close 1 eye for improved vision, but associations have not been rigorously studied. Objective To evaluate associations with monocular eye closure in patients with ERM, and to report binocular interference (closing 1 eye to improve visual quality). Design, Setting, and Participants Retrospective medical record review of an adult strabismus clinic at a tertiary referral center. Patients with ERM referred from retina clinicians between June 2010 and October 2019 who completed the Adult Strabismus (AS)-20 questionnaire, including the question: "I cover or close one eye to see things better." Two groups were identified: (1) patients reporting eye closure sometimes or more, and (2) patients reporting no eye closure (as control patients). Main Outcomes and Measures Frequencies of (1) central-peripheral rivalry (CPR)-type diplopia (dragged fovea diplopia); (2) binocular interference (monocular eye closure but no diplopia or strabismus); and (3) other, associated with monocular eye closure. Visual acuity, metamorphopsia, aniseikonia, and AS-20 quality of life domain scores (self-perception, interactions, reading function, and general function) compared between binocular interference, CPR-type diplopia, and control patients. Results A total of 124 patients with ERM (58 of 124 were women [47%]; mean [SD] age, 70 [9] years) reported monocular eye closure. Associations were binocular interference in 36 (29%; 95% CI, 21%-38%), CPR-type diplopia in 34 (27%; 95% CI, 20%-36%), and other (primarily strabismus) in 54 (44%). Compared with control patients with ERM (n = 11), patients with ERM and binocular interference had worse quality of life on AS-20 reading function (95 vs 62; mean difference, 22 points; 95% CI, 7-27 points; P = .007) and general function (89 vs 68; mean difference, 23 points; 95% CI, 13-34 points; P = .01) domains. Compared with CPR-type diplopia, patients with binocular interference had poorer worst-eye visual acuity (median 0.50 vs 0.30 logMAR [20/63 vs 20/40]; mean difference, 0.13 logMAR; 95% CI, 0.00-0.25 logMAR [20/20 to 20/35]; P = .03), and a larger interocular difference (0.46 vs 0.19 logMAR [20/58 vs 20/30]; mean difference, 0.15 logMAR; 95% CI, 0.03-0.28 logMAR [20/21 to 20/38]; P = .004). Conclusions and Relevance Study findings suggest that binocular interference, manifesting as monocular eye closure (without diplopia or strabismus), is a distinct entity affecting quality of life in patients with epiretinal membrane.
Collapse
Affiliation(s)
- Sarah R Hatt
- Department of Ophthalmology, Mayo Clinic, Rochester, Minnesota
| | - David A Leske
- Department of Ophthalmology, Mayo Clinic, Rochester, Minnesota
| | - Raymond Iezzi
- Department of Ophthalmology, Mayo Clinic, Rochester, Minnesota
| | - Jonathan M Holmes
- Department of Ophthalmology, Mayo Clinic, Rochester, Minnesota.,Department of Ophthalmology and Vision Science, University of Arizona-Tucson
| |
Collapse
|
14
|
Wilson MR, Sandberg KA, Foutch BK. Macular pigment optical density and visual quality of life. JOURNAL OF OPTOMETRY 2021; 14:92-99. [PMID: 32868244 PMCID: PMC7753046 DOI: 10.1016/j.optom.2020.07.008] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 04/21/2020] [Revised: 07/16/2020] [Accepted: 07/30/2020] [Indexed: 06/11/2023]
Abstract
PURPOSE There is robust evidence that higher macular pigment concentrations help reduce both veiling and discomfort glare in patients with or without ocular disease. We investigated whether there was also a relationship between macular pigment optical density (MPOD) and patient surveys about glare or ocular discomfort. METHODS We measured MPOD psychophysically in 23 healthy subjects and administered the National Eye Institute Visual Functioning Questionnaire (VFQ-25). Responses for each survey question were sorted from low (no limitation) to high (very severe limitation). The median response for each question was determined, and independent t-tests were performed on the mean MPOD values for survey responses above and below the median. We also performed a non-parametric correlation analysis between MPOD and survey responses. RESULTS While the median response was "no limitation" for most (22 of 25) survey questions, responses were slightly higher for two questions concerning ocular discomfort and one question related to driving at night. MPOD levels were significantly higher in subjects that reported no discomfort in or around their eyes than in those that reported mild discomfort. There was also a trend toward higher MPOD levels in subjects who reported that pain in or around their eyes never limited their activity as well as in subjects who reported no difficulty driving at night. CONCLUSION These preliminary findings are consistent with the well-established discomfort and glare hypotheses for MPOD. The current findings on subjective ocular discomfort in the absence of glare deserve further study.
Collapse
Affiliation(s)
- Molly R Wilson
- University of the Incarnate Word, Rosenberg School of Optometry, United States
| | - Kyle A Sandberg
- University of the Incarnate Word, Rosenberg School of Optometry, United States
| | - Brian K Foutch
- University of the Incarnate Word, Rosenberg School of Optometry, United States.
| |
Collapse
|
15
|
Rousseau JF, Ip IK, Raja AS, Schuur JD, Khorasani R. Can emergency department provider notes help to achieve more dynamic clinical decision support? J Am Coll Emerg Physicians Open 2020; 1:1269-1277. [PMID: 33392531 PMCID: PMC7771753 DOI: 10.1002/emp2.12232] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2020] [Revised: 07/31/2020] [Accepted: 08/05/2020] [Indexed: 11/19/2022] Open
Abstract
OBJECTIVE Assess whether clinical data were present in emergency department (ED) provider notes at time of order entry for cervical spine (c-spine) imaging that could be used to augment or pre-populate clinical decision support (CDS) attributes. METHODS This Institutional Review Board-approved retrospective study, performed in a quaternary hospital, included all encounters for adult ED patients seen April 1, 2013-September 30, 2014 for a chief complaint of trauma who received c-spine computed tomography (CT) or x-ray. We assessed proportion of ED encounters with at least 1 c-spine-specific CDS rule attribute in clinical notes available at the time of imaging order and agreement between attributes in clinical notes and data entered into CDS. RESULTS A portion of the clinical note was submitted before imaging order in 42% (184/438) of encounters reviewed; 59.2% (109/184) of encounters with note portions submitted before imaging order had at least 1 positive CDS attribute identified supporting imaging study appropriateness; 34.8% (64/184) identified exclusion criteria where CDS appropriateness recommendations would not be applicable. 65.8% (121/184) of encounters had either a positive CDS attribute or an exclusion criterion. Concordance of c-spine CDS attributes when present in both notes and CDS was 68.4% (κ = 0.35 95% CI: 0.15-0.56; McNemar P = 0.23). CONCLUSIONS Clinical notes are an underutilized source of clinical attributes needed for CDS, available in a substantial percentage of encounters at the time of imaging order. Automated pre-population of imaging order requisitions with relevant clinical information extracted from electronic health record provider notes may: (1) improve ordering efficiency by reducing redundant data entry, (2) help improve clinical relevance of CDS alerts, and (3) potentially reduce provider burnout from extraneous alerts.
Collapse
Affiliation(s)
- Justin F. Rousseau
- Center for Evidence‐Based ImagingBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
- Department of RadiologyBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
- Department of Population HealthDell Medical School, The University of Texas at AustinAustinTexasUSA
- Department of NeurologyDell Medical School, The University of Texas at AustinAustinTexasUSA
| | - Ivan K. Ip
- Center for Evidence‐Based ImagingBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
- Department of RadiologyBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
- Department of MedicineBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
| | - Ali S. Raja
- Center for Evidence‐Based ImagingBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
- Department of RadiologyBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
| | - Jeremiah D. Schuur
- Department of Emergency MedicineBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
| | - Ramin Khorasani
- Center for Evidence‐Based ImagingBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
- Department of RadiologyBrigham and Women's HospitalHarvard Medical SchoolBostonMassachusettsUSA
| |
Collapse
|
16
|
Gocuk SA, Lee J, Keller PR, Ayton LN, Guymer RH, McKendrick AM, Downie LE. Clinical audit as an educative tool for optometrists: an intervention study in age‐related macular degeneration. Ophthalmic Physiol Opt 2020; 41:53-72. [DOI: 10.1111/opo.12754] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2020] [Revised: 09/13/2020] [Accepted: 09/21/2020] [Indexed: 11/29/2022]
Affiliation(s)
- Sena A Gocuk
- Department of Optometry and Vision Sciences The University of Melbourne Parkville Australia
| | - Ji‐hyun Lee
- Department of Optometry and Vision Sciences The University of Melbourne Parkville Australia
| | - Peter R Keller
- Department of Optometry and Vision Sciences The University of Melbourne Parkville Australia
| | - Lauren N Ayton
- Department of Optometry and Vision Sciences The University of Melbourne Parkville Australia
- Department of Surgery (Ophthalmology) The University of Melbourne Parkville Australia
- Centre for Eye Research Australia Royal Victorian Eye and Ear Hospital Melbourne Australia
| | - Robyn H Guymer
- Department of Surgery (Ophthalmology) The University of Melbourne Parkville Australia
- Centre for Eye Research Australia Royal Victorian Eye and Ear Hospital Melbourne Australia
| | - Allison M McKendrick
- Department of Optometry and Vision Sciences The University of Melbourne Parkville Australia
| | - Laura E Downie
- Department of Optometry and Vision Sciences The University of Melbourne Parkville Australia
| |
Collapse
|
17
|
Ensari I, Pichon A, Lipsky-Gorman S, Bakken S, Elhadad N. Augmenting the Clinical Data Sources for Enigmatic Diseases: A Cross-Sectional Study of Self-Tracking Data and Clinical Documentation in Endometriosis. Appl Clin Inform 2020; 11:769-784. [PMID: 33207385 PMCID: PMC7673957 DOI: 10.1055/s-0040-1718755] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2020] [Accepted: 07/14/2020] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND Self-tracking through mobile health technology can augment the electronic health record (EHR) as an additional data source by providing direct patient input. This can be particularly useful in the context of enigmatic diseases and further promote patient engagement. OBJECTIVES This study aimed to investigate the additional information that can be gained through direct patient input on poorly understood diseases, beyond what is already documented in the EHR. METHODS This was an observational study including two samples with a clinically confirmed endometriosis diagnosis. We analyzed data from 6,925 women with endometriosis using a research app for tracking endometriosis to assess prevalence of self-reported pain problems, between- and within-person variability in pain over time, endometriosis-affected tasks of daily function, and self-management strategies. We analyzed data from 4,389 patients identified through a large metropolitan hospital EHR to compare pain problems with the self-tracking app and to identify unique data elements that can be contributed via patient self-tracking. RESULTS Pelvic pain was the most prevalent problem in the self-tracking sample (57.3%), followed by gastrointestinal-related (55.9%) and lower back (49.2%) pain. Unique problems that were captured by self-tracking included pain in ovaries (43.7%) and uterus (37.2%). Pain experience was highly variable both across and within participants over time. Within-person variation accounted for 58% of the total variance in pain scores, and was large in magnitude, based on the ratio of within- to between-person variability (0.92) and the intraclass correlation (0.42). Work was the most affected daily function task (49%), and there was significant within- and between-person variability in self-management effectiveness. Prevalence rates in the EHR were significantly lower, with abdominal pain being the most prevalent (36.5%). CONCLUSION For enigmatic diseases, patient self-tracking as an additional data source complementary to EHR can enable learning from the patient to more accurately and comprehensively evaluate patient health history and status.
Collapse
Affiliation(s)
- Ipek Ensari
- Data Science Institute, Columbia University, New York, New York, United States
| | - Adrienne Pichon
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
| | - Sharon Lipsky-Gorman
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
| | - Suzanne Bakken
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
- Columbia School of Nursing, Columbia University, New York, New York, United States
| | - Noémie Elhadad
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, New York, United States
| |
Collapse
|
18
|
Ko YA, Hu Y, Quyyumi AA, Waller LA, Voit EO, Ziegler TR, Lampl M, Martin GS. Comparison of physical examination and laboratory data between a clinical study and electronic health records. PLoS One 2020; 15:e0236189. [PMID: 32697795 PMCID: PMC7375598 DOI: 10.1371/journal.pone.0236189] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/03/2020] [Accepted: 06/30/2020] [Indexed: 11/18/2022] Open
Abstract
Research based on secondary analysis of data stored in electronic health records (EHR) has gained popularity, but whether the data are consistent with those collected under a study setting is unknown. The objective is to assess the agreement between data obtained in a prospective study and routine-care data extracted retrospectively from the EHR. We compared the data collected in a longitudinal lifestyle intervention study with those recorded in the EHR system over 5 years. A total of 225 working adults were recruited at an academic institution between 2008–2012, whose EHR data were also available during the same time period. After aligning the participants’ study visit dates with their hospital encounter dates, data on blood pressure, body mass index (BMI), and laboratory measurements (including high-density lipoprotein (HDL), low-density lipoprotein (LDL), triglycerides, and total cholesterol) were compared via a paired t-test for equivalence with pre-specified margins. Summary statistics were used to compare smoking status and medication prescriptions. Overall, data were consistent between the two sources (i.e., BMI, smoking status, medication prescriptions), whereas some differences were found in cholesterol measurements (i.e., HDL and total cholesterol), possibly due to different lab assays and subject’s fasting status. In conclusion, some EHR data are fairly consistent with those collected in a clinical study, whereas others may require further examination. Researchers should evaluate the consistency and quality of EHR data and compare them with other sources of data when possible.
Collapse
Affiliation(s)
- Yi-An Ko
- Department of Biostatistics and Bioinformatics, Rollins School of Public Health, Emory University, Atlanta, Georgia, United States of America
| | - Yingtian Hu
- Department of Biostatistics and Bioinformatics, Rollins School of Public Health, Emory University, Atlanta, Georgia, United States of America
| | - Arshed A. Quyyumi
- Division of Cardiology, School of Medicine, Emory University, Atlanta, Georgia, United States of America
| | - Lance A. Waller
- Department of Biostatistics and Bioinformatics, Rollins School of Public Health, Emory University, Atlanta, Georgia, United States of America
| | - Eberhard O. Voit
- Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, Georgia, United States of America
| | - Thomas R. Ziegler
- Division of Endocrinology, School of Medicine, Emory University, Atlanta, Georgia, United States of America
| | - Michelle Lampl
- Department of Anthropology, Emory University, Atlanta, Georgia, United States of America
| | - Greg S. Martin
- Division of Pulmonary, Allergy, Critical Care, and Sleep Medicine, School of Medicine, Emory University, Atlanta, Georgia, United States of America
- * E-mail:
| |
Collapse
|
19
|
Berman AL, Carter G. Technological Advances and the Future of Suicide Prevention: Ethical, Legal, and Empirical Challenges. Suicide Life Threat Behav 2020; 50:643-651. [PMID: 31803971 DOI: 10.1111/sltb.12610] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/20/2019] [Accepted: 11/05/2019] [Indexed: 11/26/2022]
Abstract
Technological advancements have brought multiple and diverse benefits to our human existence. In suicide prevention, new technologies have spurred great interest in and reports of the applicability to assessing, monitoring, and intervening in various community and clinical populations. We argue in this article that we need to better understand the complexities of implementation of technological advances; especially the accuracy, effectiveness, safety, ethical, and legal issues, even as implementation occurs at individual, clinical, and population levels, in order to achieve that measure of public health impact we all desire (i.e., greater benefit than harm).
Collapse
Affiliation(s)
- Alan L Berman
- Department of Psychiatry and Behavioral Sciences, Johns Hopkins School of Medicine, Baltimore, MD, USA
| | - Gregory Carter
- Faculty of Health and Medicine, Centre for Brain and Mental Health Research, University of Newcastle, Callaghan, NSW, Australia
| |
Collapse
|
20
|
Salgado TM, Liu J, Reed HL, Quinn CS, Syverson JG, Le-Rademacher J, Lopez CL, Beutler AS, Loprinzi CL, Vangipuram K, Smith EML, Henry NL, Farris KB, Hertz DL. Patient factors associated with discrepancies between patient-reported and clinician-documented peripheral neuropathy in women with breast cancer receiving paclitaxel: A pilot study. Breast 2020; 51:21-28. [PMID: 32193049 PMCID: PMC7198332 DOI: 10.1016/j.breast.2020.02.011] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2019] [Revised: 01/27/2020] [Accepted: 02/28/2020] [Indexed: 12/14/2022] Open
Abstract
Purpose Discrepancies between clinicians’ assessment of chemotherapy-induced peripheral neuropathy (CIPN) and patient-reported outcomes (PRO) have been described, though the underlying reasons are unknown. Our objective was to identify potential patient-specific factors associated with under-describing of CIPN to clinicians in women with non-metastatic breast cancer treated with paclitaxel. Methods Patients enrolled in an observational study (n = 60) completed weekly CIPN PRO using the EORTC CIPN20. Clinician-documented CIPN using the NCI CTCAE were abstracted from the electronic medical record and paired with CIPN20 data at weeks 7 and 10. Patients were classified as under-describers if their CIPN20 was above the 80th percentile of the CIPN20 distribution for that CTCAE grade from an independent clinical trial (N08CA). Demographics, Assessment of Survivor Concerns (ASC), Trust in Oncologist Scale (TiOS), and health literacy assessment were collected post-treatment via survey. Repeated measures cumulative logistic regression models were used to identify factors associated with under-describing CIPN. Results Forty-two women completed the survey (response rate 70%). Three and 9 patients were categorized as under-describers at weeks 7 and 10, respectively. Women who were not working (OR = 9.00, 95%CI 1.06–76.15), had lower income (OR = 7.04, 95%CI 1.5–32.99), and displayed higher trust in their oncologist’s competence (OR = 1.29, 95%CI 1.03–1.62 for a 0.1-unit increase in score) were more likely to under-describe CIPN symptoms. Conclusions This preliminary study identified non-working status, low income and trust in oncologist’s competence as potential factors influencing under-description of CIPN to the clinical team. Further work is needed to clarify these relationships and test additional factors. This pilot study examined factors associated with under-describing of neuropathy. Patient-reported and clinician-documented neuropathy severity were compared. Non-working status and low income were associated with neuropathy under-describing. Trust in oncologist’s competence was associated with neuropathy under-describing. Recording patient-clinician interactions would confirm under-describing behavior.
Collapse
Affiliation(s)
- Teresa M Salgado
- Department of Pharmacotherapy & Outcomes Science, School of Pharmacy, Virginia Commonwealth University, 410 N. 12th Street PO Box 98053, 23298, Richmond, VA, USA.
| | - Jin Liu
- Department of Biostatistics, School of Medicine, Virginia Commonwealth University, 830 East Main Street, 7th Floor, 23298, Richmond, VA, USA
| | - Holly L Reed
- Department of Pharmaceutical and Nutrition Care, Nebraska Medicine, 981090 Nebraska Medical Center, 68198-1090, Omaha, NE, USA
| | - Caroline S Quinn
- Department of Pharmacy, University of Wisconsin Health, 600 Highland Avenue, 53792, Madison, WI, USA
| | | | - Jennifer Le-Rademacher
- Alliance Statistics and Data Center, Mayo Clinic, 200 First Street SW, 55905, Rochester, MN, USA; Mayo Clinic Cancer Center, Mayo Clinic, 200 First Street SW, 55905, Rochester, MN, USA
| | - Camden L Lopez
- Division of Biomedical Statistics and Informatics, Mayo Clinic, 200 First Street SW, 55905, Rochester, MN, USA
| | - Andreas S Beutler
- Mayo Clinic Cancer Center, Mayo Clinic, 200 First Street SW, 55905, Rochester, MN, USA; Department of Oncology, Mayo Clinic, 200 First Street SW, 55905, Rochester, MN, USA
| | - Charles L Loprinzi
- Mayo Clinic Cancer Center, Mayo Clinic, 200 First Street SW, 55905, Rochester, MN, USA; Department of Oncology, Mayo Clinic, 200 First Street SW, 55905, Rochester, MN, USA
| | - Kiran Vangipuram
- Takeda Pharmaceuticals Inc., 40 Landsdowne St, 02139, Cambridge, MA, USA
| | - Ellen M Lavoie Smith
- Department of Health Behavior and Biological Sciences, School of Nursing, University of Michigan, 426 N Ingalls St, 48104, Ann Arbor, MI, USA
| | - N Lynn Henry
- Rogel Cancer Center, University of Michigan, 1500 E Medical Center Dr, 48109, Ann Arbor, MI, USA
| | - Karen B Farris
- Department of Clinical Pharmacy, College of Pharmacy, University of Michigan, 428 Church St, 48109, Ann Arbor, MI, USA
| | - Daniel L Hertz
- Department of Clinical Pharmacy, College of Pharmacy, University of Michigan, 428 Church St, 48109, Ann Arbor, MI, USA
| |
Collapse
|
21
|
Meisha DE. Evaluation of Accuracy and Completeness of Electronic Dental Records in a Dental School Setting. Open Dent J 2019. [DOI: 10.2174/1874210601913010520] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Background:High-quality data in Electronic Dental Records (EDR) is essential for dental schools to provide high-quality patient care, improve dental students’ professionalism, and support a platform for research. Therefore, ensuring data quality in EDRs is extremely important.Objective:To perform a quality appraisal of EDRs by evaluating their accuracy and completeness.Methods:This was a Cross-sectional Observational Study conducted over four consecutive years in a dental school setting. Manual chart reviews were performed on an annual basis. EDR data were audited for accuracy and completeness. Accuracy was evaluated by comparing the entered data with an external source, where possible. An EDR data field was considered complete if it was not missing.Results:A total of 1,720 de-identified chart reviews were studied. The accuracy of the data to identify the patient was 93%. The completeness of the essential components of EDRs was 48-94%. Completeness was highest for documenting the patient’s chief complaint (94%) and the lowest for recall plan (48%). Completeness of data documenting social and behavioral determinants of health in EDRs was 36-77%, with the highest proportion of completeness in this domain being for oral hygiene habits, smoking habits, and social history.Conclusion:The quality appraisal of EDRs varied according to the data field. Understanding patterns of accuracy and completeness in EDRs will guide training and quality enforcement activities.
Collapse
|
22
|
Wang JY, Leske DA, Hatt SR, Holmes JM. Diplopia after strabismus surgery for adults with nondiplopic childhood-onset strabismus. J AAPOS 2019; 23:313.e1-313.e5. [PMID: 31586582 PMCID: PMC6925318 DOI: 10.1016/j.jaapos.2019.07.005] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/17/2019] [Revised: 06/26/2019] [Accepted: 07/03/2019] [Indexed: 11/24/2022]
Abstract
PURPOSE To describe frequency of postoperative diplopia after strabismus surgery in nondiplopic adults with childhood-onset strabismus and to report health-related quality-of-life (HRQOL) outcomes. METHODS We prospectively enrolled 79 adults with no diplopia in any gaze who had childhood-onset strabismus and were scheduled for strabismus surgery. Diplopia was assessed preoperatively and at 6 weeks and 1 year postoperatively using a standardized diplopia questionnaire with 5 response options in 7 gaze positions. HRQOL was assessed using the Adult Strabismus-20 (AS-20) questionnaire, with self-perception, interactions, reading function, and general function domains. RESULTS Constant diplopia in straight-ahead distance and reading gaze occurred in 1 patient (1% [95% CI, 0%-7%] at 6 weeks and 2% [95% CI, 0%-10%] at 1 year). Regarding the rate of any diplopia (including rarely) in any gaze, 15 of 78 patients (19%) reported diplopia at 6 weeks, of whom 13 had diplopia in straight-ahead distance gaze; 8 (10%), in reading gaze. At 1 year, 8 of 51 patients (16%) reported any diplopia (including rarely) in any gaze, of whom 7 had diplopia in straight-ahead distance gaze and 4 (8%) in reading gaze. Mean AS-20 scores improved at 1 year overall (by 32, 19, 14, and 15 points, resp., per domain) and for the 8 diplopic patients (by 21, 13, 16, and 11 points). CONCLUSIONS In adults with nondiplopic strabismus, constant postoperative diplopia is rare, although the rate of intermittent diplopia is higher. Even when postoperative diplopia occurs, HRQOL often improves.
Collapse
Affiliation(s)
- Jenny Y Wang
- Department of Ophthalmology, Mayo Clinic, Rochester, Minnesota
| | - David A Leske
- Department of Ophthalmology, Mayo Clinic, Rochester, Minnesota
| | - Sarah R Hatt
- Department of Ophthalmology, Mayo Clinic, Rochester, Minnesota
| | | |
Collapse
|
23
|
Variation in Physicians' Electronic Health Record Documentation and Potential Patient Harm from That Variation. J Gen Intern Med 2019; 34:2355-2367. [PMID: 31183688 PMCID: PMC6848521 DOI: 10.1007/s11606-019-05025-3] [Citation(s) in RCA: 52] [Impact Index Per Article: 10.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/19/2018] [Revised: 12/14/2018] [Accepted: 03/21/2019] [Indexed: 10/26/2022]
Abstract
BACKGROUND Physician-to-physician variation in electronic health record (EHR) documentation not driven by patients' clinical status could be harmful. OBJECTIVE Measure variation in completion of common clinical documentation domains. Identify perceived causes and effects of variation and strategies to mitigate negative effects. DESIGN Sequential, explanatory, mixed methods using log data from a commercial EHR vendor and semi-structured interviews with outpatient primary care practices. PARTICIPANTS Quantitative: 170,332 encounters led by 809 physicians in 237 practices. Qualitative: 40 interviewees in 10 practices. MAIN MEASURES Interquartile range (IQR) of the proportion of encounters in which a physician completed documentation, for each documentation category. Multilevel linear regression measured the proportion of variation at the physician level. KEY RESULTS Five clinical documentation categories had substantial and statistically significant (p < 0.001) variation at the physician level after accounting for state, organization, and practice levels: (1) discussing results (IQR = 50.8%, proportion of variation explained by physician level = 78.1%); (2) assessment and diagnosis (IQR = 60.4%, physician-level variation = 76.0%); (3) problem list (IQR = 73.1%, physician-level variation = 70.1%); (4) review of systems (IQR = 62.3%, physician-level variation = 67.7%); and (5) social history (IQR = 53.3%, physician-level variation = 62.2%). Drivers of variation from interviews included user preferences and EHR designs with multiple places to record similar information. Variation was perceived to create documentation inefficiencies and risk patient harm due to missed or misinterpreted information. Mitigation strategies included targeted user training during EHR implementation and practice meetings focused on documentation standardization. CONCLUSIONS Physician-to-physician variation in EHR documentation impedes effective and safe use of EHRs, but there are potential strategies to mitigate negative consequences.
Collapse
|
24
|
Navarro-Millán I, Zinski A, Shurbaji S, Johnson B, Fraenkel L, Willig J, Danila MI, Yun H, Curtis JR, Safford MM. Perspectives of Rheumatoid Arthritis Patients on Electronic Communication and Patient-Reported Outcome Data Collection: A Qualitative Study. Arthritis Care Res (Hoboken) 2019; 71:80-87. [PMID: 29669191 DOI: 10.1002/acr.23580] [Citation(s) in RCA: 30] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2017] [Accepted: 04/10/2018] [Indexed: 01/22/2023]
Abstract
OBJECTIVE To identify the perspectives of patients with rheumatoid arthritis (RA) on electronic recording of between-visit disease activity and other patient-reported outcomes (PROs) and on sharing this information with health care providers or peers. METHODS Patients with RA were recruited to participate in focus groups from December 2014 to April 2015. The topic guide and analysis were based on the Andersen-Newman framework. Sessions were audiorecorded, transcribed, independently coded, and analyzed for themes. RESULTS Thirty-one patients participated in 7 focus groups. Their mean ± SD age was 51 ± 13.1 years, 94% were women, 52% were African American, 11% were Hispanic, and 37% were white. Three themes emerged: provider communication, information-seeking about RA, and social and peer support. Participants expressed a willingness to track disease activity data to share with health care providers electronically if providers would act on the information. Participants envisioned symptom tracking and information sharing as a mechanism to relay and obtain reliable information about RA. Participants were also interested in electronic communication between visits if it facilitated learning about symptom management and enhanced opportunities for social support among patients with RA. CONCLUSION Patients with RA may be amenable to electronic collection and sharing of PRO-type data between clinical encounters if it facilitates communication with health care providers and provides access to reliable information about RA. Providing patients with social support was important for enhancing PROs collection by helping them overcome barriers by using electronic devices and overcome reservations about the value of these data.
Collapse
Affiliation(s)
- Iris Navarro-Millán
- Weill Cornell Medical College and the Hospital for Special Surgery, New York, New York
| | | | | | | | | | | | | | | | | | | |
Collapse
|
25
|
Khanji C, Schnitzer ME, Bareil C, Perreault S, Lalonde L. Concordance of care processes between medical records and patient self-administered questionnaires. BMC FAMILY PRACTICE 2019; 20:92. [PMID: 31269902 PMCID: PMC6607524 DOI: 10.1186/s12875-019-0979-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 09/14/2018] [Accepted: 06/13/2019] [Indexed: 11/10/2022]
Abstract
Background Despite the increasing use of medical records to measure quality of care, studies have shown that their validity is suboptimal. The objective of this study is to assess the concordance of cardiovascular care processes evaluated through medical record review and patient self-administered questionnaires (SAQs) using ten quality indicators (TRANSIT indicators). These indicators were developed as part of a participatory research program (TRANSIT study) dedicated to TRANSforming InTerprofessional clinical practices to improve cardiovascular disease (CVD) prevention in primary care. Methods For every patient participating in the TRANSIT study, the compliance to each indicator (individual scores) as well as the mean compliance to all indicators of a category (subscale scores) and to the complete set of ten indicators (overall scale score) were established. Concordance between results obtained using medical records and patient SAQs was assessed by prevalence-adjusted bias-adjusted kappa (PABAK) coefficients as well as intraclass correlation coefficients (ICCs) and 95% confidence intervals (95% CI). Generalized linear mixed models (GLMM) were used to identify patients’ sociodemographic and clinical characteristics associated with agreement between the two data sources. Results The TRANSIT study was conducted in a primary care setting among patients (n = 759) with multimorbidity, at moderate (16%) and high risk (83%) of cardiovascular diseases. Quality of care, as measured by the TRANSIT indicators, varied substantially between medical records and patient SAQ. Concordance between the two data sources, as measured by ICCs (95% CI), was poor for the subscale (0.18 [0.08–0.27] to 0.46 [0.40–0.52]) and overall (0.46 [0.40–0.53]) compliance scale scores. GLMM showed that agreement was not affected by patients’ characteristics. Conclusions In quality improvement strategies, researchers must acknowledge that care processes may not be consistently recorded in medical records. They must also be aware that the evaluation of the quality of care may vary depending on the source of information, the clinician responsible of documenting the interventions, and the domain of care. Electronic supplementary material The online version of this article (10.1186/s12875-019-0979-7) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Cynthia Khanji
- Faculty of pharmacy, University of Montreal, 2940 Polytechnique Road, Montreal, Quebec, H3T1J4, Canada
| | - Mireille E Schnitzer
- Faculty of pharmacy, University of Montreal, 2940 Polytechnique Road, Montreal, Quebec, H3T1J4, Canada
| | - Céline Bareil
- HEC Montréal, University of Montreal, 3000 Côte-Sainte-Catherine Road, Montreal, Quebec, H3T2A7, Canada
| | - Sylvie Perreault
- Faculty of pharmacy, University of Montreal, 2940 Polytechnique Road, Montreal, Quebec, H3T1J4, Canada.,Sanofi Aventis Endowment Chair in Drug Utilization, Montreal, Canada
| | - Lyne Lalonde
- Faculty of pharmacy, University of Montreal, 2940 Polytechnique Road, Montreal, Quebec, H3T1J4, Canada. .,Sanofi Aventis Endowment Chair in Ambulatory Pharmaceutical Care, Montreal, Canada.
| |
Collapse
|
26
|
Goldstein IH, Hwang T, Gowrisankaran S, Bales R, Chiang MF, Hribar MR. Changes in Electronic Health Record Use Time and Documentation over the Course of a Decade. Ophthalmology 2019; 126:783-791. [PMID: 30664893 PMCID: PMC6534421 DOI: 10.1016/j.ophtha.2019.01.011] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2018] [Revised: 01/07/2019] [Accepted: 01/09/2019] [Indexed: 11/13/2022] Open
Abstract
PURPOSE With the current wide adoption of electronic health records (EHRs) by ophthalmologists, there are widespread concerns about the amount of time spent using the EHR. The goal of this study was to examine how the amount of time spent using EHRs as well as related documentation behaviors changed 1 decade after EHR adoption. DESIGN Single-center cohort study. PARTICIPANTS Six hundred eighty-five thousand three hundred sixty-one office visits with 70 ophthalmology providers. METHODS We calculated time spent using the EHR associated with each individual office visit using EHR audit logs and determined chart closure times and progress note length from secondary EHR data. We tracked and modeled how these metrics changed from 2006 to 2016 with linear mixed models. MAIN OUTCOME MEASURES Minutes spent using the EHR associated with an office visit, chart closure time in hours from the office visit check-in time, and progress note length in characters. RESULTS Median EHR time per office visit in 2006 was 4.2 minutes (interquartile range [IQR], 3.5 minutes), and increased to 6.4 minutes (IQR, 4.5 minutes) in 2016. Median chart closure time was 2.8 hours (IQR, 21.3 hours) in 2006 and decreased to 2.3 hours (IQR, 18.5 hours) in 2016. In 2006, median note length was 1530 characters (IQR, 1435 characters) and increased to 3838 characters (IQR, 2668.3 characters) in 2016. Linear mixed models found EHR time per office visit was 31.9±0.2% (P < 0.001) greater from 2014 through 2016 than from 2006 through 2010, chart closure time was 6.7±0.3 hours (P < 0.001) shorter from 2014 through 2016 versus 2006 through 2010, and note length was 1807.4±6.5 characters (P < 0.001) longer from 2014 through 2016 versus 2006 through 2010. CONCLUSIONS After 1 decade of use, providers spend more time using the EHR for an office visit, generate longer notes, and close the chart faster. These changes are likely to represent increased time and documentation pressure for providers. Electronic health record redesign and new documentation regulations may help to address these issues.
Collapse
Affiliation(s)
- Isaac H Goldstein
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Thomas Hwang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Sowjanya Gowrisankaran
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon
| | - Ryan Bales
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon
| | - Michael F Chiang
- Department of Ophthalmology, Casey Eye Institute, Oregon Health & Science University, Portland, Oregon; Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon
| | - Michelle R Hribar
- Department of Medical Informatics and Clinical Epidemiology, Oregon Health & Science University, Portland, Oregon.
| |
Collapse
|
27
|
Nachar VR, Farris K, Beekman K, Griggs J, Hough S, Mackler E. Clinician Report of Oral Oncolytic Symptoms and Adherence Obtained via a Patient-Reported Outcome Measure (PROM). JCO Clin Cancer Inform 2019; 3:1-6. [PMID: 31100014 DOI: 10.1200/cci.18.00128] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022] Open
Abstract
PURPOSE Patient-reported outcome measures (PROMs) for symptom monitoring during cancer therapy have been shown to have a positive impact on outcomes. These findings have primarily been shown for patients receiving intravenous chemotherapy. In addition, there is known discordance between physician reporting of symptoms and patient self-report. This initiative sought to describe patient-reported symptom burden and medication adherence and to indicate the degree of PROM results being discussed with the provider as indicated by documentation in the medical record for patients taking oral oncolytic therapy. METHODS The Michigan Oncology Quality Consortium (MOQC) PROM, which included symptom ratings, medication adherence, and patient confidence in self-management, was completed during outpatient visits and compared with corresponding data documented in the electronic medical record (EMR). RESULTS There were 82 completed PROMs. Approximately half included at least one symptom rated as severe (46%). Sixty-five percent of reported severe symptoms were documented in the EMR. Patient-reported moderate-to-severe pain was most likely to be documented in the EMR (100%), whereas patient-reported moderate-to-severe depression and anxiety were least likely to be documented (21%). Of the total symptoms documented, grading of symptom severity matched that of the patients' own report for 11% of severe symptoms. Adherence to oral oncolytics was excellent for 63% of patients, and patient adherence was documented in 7% of provider notes. CONCLUSION Patients frequently reported moderate-to-severe symptoms, and approximately 40% of patients reported nonadherence. Clinician report (documented in the EMR) of the patient symptom burden, symptom severity, and adherence to oral oncolytic therapy was not consistent with the patients' self-report. Use of a PROM for patients taking oral oncolytics has the opportunity to improve symptom management and medication adherence.
Collapse
Affiliation(s)
- Victoria R Nachar
- University of Michigan, Ann Arbor, MI.,University of Michigan College of Pharmacy, Ann Arbor, MI
| | - Karen Farris
- University of Michigan College of Pharmacy, Ann Arbor, MI.,Michigan Oncology Quality Consortium, Ann Arbor, MI
| | | | - Jennifer Griggs
- University of Michigan, Ann Arbor, MI.,Michigan Oncology Quality Consortium, Ann Arbor, MI
| | - Shannon Hough
- University of Michigan, Ann Arbor, MI.,Michigan Oncology Quality Consortium, Ann Arbor, MI
| | | |
Collapse
|
28
|
Levitin SA, Grbic JT, Finkelstein J. Completeness of Electronic Dental Records in a Student Clinic: Retrospective Analysis. JMIR Med Inform 2019; 7:e13008. [PMID: 30896435 PMCID: PMC6447991 DOI: 10.2196/13008] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2018] [Revised: 02/11/2019] [Accepted: 03/13/2019] [Indexed: 12/12/2022] Open
Abstract
BACKGROUND A well-designed, adequately documented, and properly maintained patient record is an important tool for quality assurance and care continuity. Good clinical documentation skills are supposed to be a fundamental part of dental student training. OBJECTIVE The goal of this study was to assess the completeness of electronic patient records in a student clinic. METHODS Completeness of patient records was assessed using comparative review of validated cases of alveolar osteitis treated between August 2011 and May 2017 in a student clinic at Columbia University College of Dental Medicine, New York, USA. Based on a literature review, population-based prevalence of nine most frequently mentioned symptoms, signs, and treatment procedures of alveolar osteitis was identified. Completeness of alveolar osteitis records was assessed by comparison of population-based prevalence and frequency of corresponding items in the student documentation. To obtain all alveolar osteitis cases, we ran a query on the electronic dental record, which included all cases with diagnostic code Z1820 or any variation of the phrases "dry socket" and "alveolar osteitis" in the notes. The resulting records were manually reviewed to definitively confirm alveolar osteitis and to extract all index items. RESULTS Overall, 296 definitive cases of alveolar osteitis were identified. Only 22% (64/296) of cases contained a diagnostic code. Comparison of the frequency of the nine index categories in the validated alveolar osteitis cases between the student clinic and the population showed the following results: severe pain: 94% (279/296) vs 100% (430/430); bare bone/missing blood clot: 27% (80/296) vs 74% (35/47) to 100% (329/329); malodor: 7% (22/296) vs 33%-50% (18/54); radiating pain to the ear: 8% (24/296) vs 56% (30/54); lymphadenopathy: 1% (3/296) vs 9% (5/54); inflammation: 14% (42/296) vs 50% (27/54); debris: 12% (36/296) vs 87% (47/54); alveolar osteitis site noted: 96% (283/296) vs 100% (430/430; accepted documentation requirement); and anesthesia during debridement: 77% (20/24) vs 100% (430/430; standard of anesthetization prior to debridement). CONCLUSIONS There was a significant discrepancy between the index category frequency in alveolar osteitis cases documented by dental students and in the population (reported in peer-reviewed literature). More attention to clinical documentation skills is warranted in dental student training.
Collapse
Affiliation(s)
- Seth Aaron Levitin
- Division of Foundational Sciences, Columbia University College of Dental Medicine, New York, NY, United States
| | - John T Grbic
- Division of Foundational Sciences, Columbia University College of Dental Medicine, New York, NY, United States
| | - Joseph Finkelstein
- Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, United States
| |
Collapse
|
29
|
McDonald L, Malcolm B, Ramagopalan S, Syrad H. Real-world data and the patient perspective: the PROmise of social media? BMC Med 2019; 17:11. [PMID: 30646913 PMCID: PMC6334434 DOI: 10.1186/s12916-018-1247-8] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/11/2018] [Accepted: 12/21/2018] [Indexed: 12/30/2022] Open
Abstract
Understanding the patient perspective is fundamental to delivering patient-centred care. In most healthcare systems, however, patient-reported outcomes are not regularly collected or recorded as part of routine clinical care, despite evidence that doing so can have tangible clinical benefit. In the absence of the routine collection of these data, research is beginning to turn to social media as a novel means to capture the patient voice. Publicly available social media data can now be analysed with relative ease, bypassing many logistical hurdles associated with traditional approaches and allowing for accelerated and cost-effective data collection. Existing work has shown these data can offer credible insight into the patient experience, although more work is needed to understand limitations with respect to patient representativeness and nuances of captured experience. Nevertheless, linking social media to electronic medical records offers a significant opportunity for patient views to be systematically collected for health services research and ultimately to improve patient care.
Collapse
Affiliation(s)
- Laura McDonald
- Centre for Observational Research and Data Sciences, Bristol-Myers Squibb, Uxbridge, UK
| | | | - Sreeram Ramagopalan
- Centre for Observational Research and Data Sciences, Bristol-Myers Squibb, Uxbridge, UK.
| | | |
Collapse
|
30
|
Abstract
OBJECTIVES Ophthalmologists assess eye complaints with a careful history and eye examination; however, other types of physicians have limited tools to evaluate anterior segment (AS) eye diseases. We identified the eye symptom questions that providers should ask to help determine the presence and urgency of AS eye diseases. METHODS Persons with and without AS disease completed a self-report eye symptom questionnaire (ESQ) based on the National Institutes of Health Toolbox symptom items in an academic center's corneal and comprehensive eye clinics. Gold standard ophthalmic examination determined the presence and urgency of AS disease. The association between reported symptom severity and the probability of AS disease, or urgent AS disease, was evaluated using logistic regression models, and sensitivity and specificity of the ESQ were also calculated. RESULTS A total of 324 eyes of 162 subjects were included in the study. Of these, AS disease was present in 255 eyes (79%); of which, 111 eyes showed urgent disease. Increasing symptom severity for eye pain (odds ratio [OR]=2.58; P<0.001), glare (OR=2.61; P=0.001), and blurry vision (OR=1.98; P<0.001) were associated with increased odds of AS disease. Increasing symptom severity for eye pain (OR=2.02; P<0.001), eye redness (OR=1.69; P=0.02), and blurry vision (OR=1.41, P=0.01) were associated with increased odds of urgent AS disease. For the primary analysis with mild symptoms considered relevant, the sensitivity of the ESQ to detect AS disease was 83% and to detect urgent AS disease was 92%. CONCLUSION Symptoms of eye pain, glare, redness, and blurry vision indicate the presence and urgency of AS disease.
Collapse
|
31
|
Goldstein IH, Hribar MR, Reznick LG, Chiang MF. Analysis of Total Time Requirements of Electronic Health Record Use by Ophthalmologists Using Secondary EHR Data. AMIA ... ANNUAL SYMPOSIUM PROCEEDINGS. AMIA SYMPOSIUM 2018; 2018:490-497. [PMID: 30815089 PMCID: PMC6371357] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 06/09/2023]
Abstract
Electronic Health Records (EHRs) are widely used in the United States for clinical care and billing activities. Their widespread adoption has raised a variety of concerns about their effects on providers and medical care. As researchers address these concerns, they will need to understand how much time providers actually spend on the EHR. This study develops and validates methods for calculating total time requirements for EHR use by ophthalmologists using secondary EHR data from audit logs. Key findings from this study are that (1) Secondary EHR data can be used to estimate lower bounds on provider EHR use, (2) Providers spend a large amount of time using the EHR, (3) Most time spent on the EHR is spent reviewing information. These findings have important implications for practicing clinicians, and for EHR system design in the future.
Collapse
Affiliation(s)
- Isaac H Goldstein
- Medical Informatics & Clinical Epidemiology
- Ophthalmology Oregon Health & Science University, Portland, OR
| | | | - Leah G Reznick
- Ophthalmology Oregon Health & Science University, Portland, OR
| | - Michael F Chiang
- Medical Informatics & Clinical Epidemiology
- Ophthalmology Oregon Health & Science University, Portland, OR
| |
Collapse
|
32
|
Fares CM, Williamson TJ, Theisen MK, Cummings A, Bornazyan K, Carroll J, Spiegel ML, Stanton AL, Garon EB. Low Concordance of Patient-Reported Outcomes With Clinical and Clinical Trial Documentation. JCO Clin Cancer Inform 2018; 2:1-12. [PMID: 30652613 PMCID: PMC6724714 DOI: 10.1200/cci.18.00059] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022] Open
Abstract
PURPOSE Health care research increasingly relies on assessment of data extracted from electronic medical records (EMRs). Clinical trial adverse event (AE) logs and patient-reported outcomes (PROs) are sources of data often available in the context of specific research projects. The aim of this study was to evaluate the extent of data concordance from these sources. PATIENTS AND METHODS Patients enrolled in clinical trials or receiving standard treatment for lung cancer (n = 62) completed validated questionnaires on physical and psychological symptoms at up to three assessment points. Temporally matched documentation was extracted from EMR notes and, for clinical trial participants (n = 41), AE logs. Evaluated data included symptom assessment, vital signs, medication logs, and laboratory values. Agreement (positive, negative) and Cohen's κ coefficients were calculated to assess concordance of symptoms among sources, with PROs considered the gold standard. RESULTS Patient-reported weight loss correlated significantly with clinical measurements ( t = 2.90; P = .02), and average number of PROs correlated negatively with albumin concentration, supporting PROs as the gold standard. Comparisons of PROs versus EMR yielded poor concordance across 11 physical symptoms, anxiety, and depressive symptoms (all κ < 0.40). Providers under-reported the presence of each symptom in the EMR compared with PROs. AE logs showed similarly poor concordance with PROs (all κ < 0.40, except shortness of breath). Negative agreement among sources was higher than positive agreement for all symptoms except pain. CONCLUSION There was poor concordance between EMR notes and AE logs with PROs. Findings suggest that EMR notes and AE logs may not be reliable sources for capturing physical and psychological symptoms experienced by patients with lung cancer, supporting use of PRO assessments in oncology practices.
Collapse
Affiliation(s)
| | | | | | - Amy Cummings
- All authors: University of California, Los Angeles, Los Angeles, CA
| | - Krikor Bornazyan
- All authors: University of California, Los Angeles, Los Angeles, CA
| | - James Carroll
- All authors: University of California, Los Angeles, Los Angeles, CA
| | | | | | - Edward B. Garon
- All authors: University of California, Los Angeles, Los Angeles, CA
| |
Collapse
|
33
|
The challenges of assessing effectiveness of lacosamide using electronic medical record databases. Epilepsy Behav 2018; 85:195-199. [PMID: 30032807 DOI: 10.1016/j.yebeh.2018.06.031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/24/2018] [Revised: 06/15/2018] [Accepted: 06/15/2018] [Indexed: 11/21/2022]
Abstract
PURPOSE Electronic health record (EHR) databases are a potential source for conducting research to generate real world evidence on patient outcomes. The objective of the study was to evaluate the feasibility of using EHR data to assess seizure outcomes in patients treated with lacosamide (LCM) monotherapy. METHODS This was a retrospective cohort study conducted using the Optum clinical EHR database. The study sample comprised patients ≥17 years of age with epilepsy or seizures and treated with LCM monotherapy between 1 January 2009 and 31 December 2013. Structured and unstructured data from prescribed medication and abstracted physician note records were used to identify patients with epilepsy treated with LCM monotherapy and measure seizure frequency outcomes. The index date was the first date of LCM monotherapy, with a 6-month baseline period. Patients were observed for up to 12 months beginning on the index date (follow-up period). The EHR data were not sufficient to compute days supply and explicit duration of LCM and other antiepileptic drug (AED) therapies; therefore, LCM monotherapy was estimated from prescription dates of AEDs. Outcomes were change in seizures per month or change in seizure frequency category from baseline to follow-up. Descriptive statistics were used to describe baseline characteristics and study outcomes. RESULTS A total of 10,988 patients with at least one LCM prescription were identified during the study period, 470 of whom met all the selection criteria and were included in the study sample. Although many patients had abstracted physician note records that referred to their seizures, only 3.2% of the patients had seizure frequency information that could be used to quantify the number of seizures per month in both the baseline and follow-up periods; thus, this information could not be used to assess the effectiveness of LCM monotherapy on seizure outcomes. CONCLUSION Lacosamide monotherapy effectiveness was not estimated because the EHR prescription record data did not have sufficient information on days supply. Additionally, most patients' records did not contain adequate information to allow for evaluation of quantitative changes in seizure frequency based on the number of seizures per month. More studies are needed to validate these study findings.
Collapse
|
34
|
Warner MJ. Response. J Osteopath Med 2018; 118:362-364. [DOI: 10.7556/jaoa.2018.078] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
|
35
|
Adler-Milstein J, Embi PJ, Middleton B, Sarkar IN, Smith J. Crossing the health IT chasm: considerations and policy recommendations to overcome current challenges and enable value-based care. J Am Med Inform Assoc 2017; 24:1036-1043. [PMID: 28340128 PMCID: PMC7651968 DOI: 10.1093/jamia/ocx017] [Citation(s) in RCA: 36] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2016] [Revised: 02/05/2017] [Accepted: 02/13/2017] [Indexed: 01/05/2023] Open
Abstract
While great progress has been made in digitizing the US health care system, today's health information technology (IT) infrastructure remains largely a collection of systems that are not designed to support a transition to value-based care. In addition, the pursuit of value-based care, in which we deliver better care with better outcomes at lower cost, places new demands on the health care system that our IT infrastructure needs to be able to support. Provider organizations pursuing new models of health care delivery and payment are finding that their electronic systems lack the capabilities needed to succeed. The result is a chasm between the current health IT ecosystem and the health IT ecosystem that is desperately needed.In this paper, we identify a set of focal goals and associated near-term achievable actions that are critical to pursue in order to enable the health IT ecosystem to meet the acute needs of modern health care delivery. These ideas emerged from discussions that occurred during the 2015 American Medical Informatics Association Policy Invitational Meeting. To illustrate the chasm and motivate our recommendations, we created a vignette from the multistakeholder perspectives of a patient, his provider, and researchers/innovators. It describes an idealized scenario in which each stakeholder's needs are supported by an integrated health IT environment. We identify the gaps preventing such a reality today and present associated policy recommendations that serve as a blueprint for critical actions that would enable us to cross the current health IT chasm by leveraging systems and information to routinely deliver high-value care.
Collapse
Affiliation(s)
- Julia Adler-Milstein
- School of Information, Department of Health Management and Policy, School of Public Health, University of Michigan, Ann Arbor, MI, USA
| | | | | | - Indra Neil Sarkar
- Center for Biomedical Informatics, Brown University, Providence, RI, USA
| | - Jeff Smith
- American Medical Informatics Association, Bethesda, MD, USA
| |
Collapse
|
36
|
Lau M, Prenner JL, Brucker AJ, VanderBeek BL. Accuracy of Billing Codes Used in the Therapeutic Care of Diabetic Retinopathy. JAMA Ophthalmol 2017; 135:791-794. [PMID: 28570735 DOI: 10.1001/jamaophthalmol.2017.1595] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022]
Abstract
Importance Insurance billing claim databases represent a growing field of scientific inquiry within ophthalmology. Validating the accuracy of billing claim codes used during the care of diabetic retinopathy is a necessary precursor to fully understanding the underlying data and subsequent results of these types of studies. Objective To determine the accuracy of diagnostic, procedural, and therapeutic billing codes used in the treatment of diabetic retinopathy. Design, Setting, and Participants This retrospective medical record review was conducted at 3 clinical practices (1 academic and 2 private). Insured patients with diabetic retinopathy were seen by the practices between 2011 and 2013. Each patient then had every visit for 2 years reviewed twice, once for billing data and the second for data from the medical record. Data were collected and analyzed from October 2015 to July 2016. Main Outcomes and Measures The positive predictive value (PPV) and negative predictive value (NPV) for each code of interest. Sensitivity and specificity were secondary outcomes. Results A total of 146 patients (mean [SD] age, 60.3 [12.5] years) from 11 physicians had 1072 encounters reviewed over 2 calendar years. Among the included patients, 49.3% were female (n = 72), 48.6% were white (n = 71), 37.0% were black (n = 54), and 18.5% had type 1 diabetes and a mean (SD) hemoglobin A1C level of 7.7% (1.8) (n = 27). Nearly all codes of interest that were used frequently also had a high PPV (range, 89.5%-100%) and NPV (88.6%-100%) including billing codes for intravitreal injection, focal laser, panretinal photocoagulation, laterality of procedure, ranibizumab, bevacizumab, fundus photographs, fluorescein angiography, and optical coherence tomography. Codes that were used infrequently (<20 instances) but still had a high PPV (all 100%) and NPV (99.7%-100%) were codes for aflibercept, triamcinolone, and the dexamethasone implant. Only the codes for infrequently used B-scan ultrasonography (PPV, 69.6%) and subtenon injection (PPV, 100%; NPV, 99.7%, but sensitivity of only 40%) were found to be of questionable accuracy. Other than subtenon injection (40%), all codes were also found to have a high sensitivity (range, 87.6%-100%) and a high specificity (range, 97.2%-100%). Conclusions and Relevance These data suggest diagnostic, procedure, and therapeutic codes derived from insurance billing claims accurately reflect the medical record for patients with diabetic retinopathy.
Collapse
Affiliation(s)
- Marisa Lau
- Scheie Eye Institute, Department of Ophthalmology, University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Jonathan L Prenner
- Department of Ophthalmology, Rutgers Robert Wood Johnson Medical School, New Brunswick, New Jersey
| | - Alexander J Brucker
- Scheie Eye Institute, Department of Ophthalmology, University of Pennsylvania Perelman School of Medicine, Philadelphia
| | - Brian L VanderBeek
- Scheie Eye Institute, Department of Ophthalmology, University of Pennsylvania Perelman School of Medicine, Philadelphia3Center for Pharmacoepidemiology Research and Training, University of Pennsylvania Perelman School of Medicine, Philadelphia4Leonard Davis Institute, University of Pennsylvania Perelman School of Medicine, Philadelphia
| |
Collapse
|