Agreement of Ocular Symptom Reporting Between Patient-Reported Outcomes and Medical Records

Identifying signs and symptoms of urinary tract infection from emergency department clinical notes using large language models

BACKGROUND

Natural language processing (NLP) tools including recently developed large language models (LLMs) have myriad potential applications in medical care and research, including the efficient labeling and classification of unstructured text such as electronic health record (EHR) notes. This opens the door to large-scale projects that rely on variables that are not typically recorded in a structured form, such as patient signs and symptoms.

OBJECTIVES

This study is designed to acquaint the emergency medicine research community with the foundational elements of NLP, highlighting essential terminology, annotation methodologies, and the intricacies involved in training and evaluating NLP models. Symptom characterization is critical to urinary tract infection (UTI) diagnosis, but identification of symptoms from the EHR has historically been challenging, limiting large-scale research, public health surveillance, and EHR-based clinical decision support. We therefore developed and compared two NLP models to identify UTI symptoms from unstructured emergency department (ED) notes.

METHODS

The study population consisted of patients aged ≥ 18 who presented to an ED in a northeastern U.S. health system between June 2013 and August 2021 and had a urinalysis performed. We annotated a random subset of 1250 ED clinician notes from these visits for a list of 17 UTI symptoms. We then developed two task-specific LLMs to perform the task of named entity recognition: a convolutional neural network-based model (SpaCy) and a transformer-based model designed to process longer documents (Clinical Longformer). Models were trained on 1000 notes and tested on a holdout set of 250 notes. We compared model performance (precision, recall, F1 measure) at identifying the presence or absence of UTI symptoms at the note level.

RESULTS

A total of 8135 entities were identified in 1250 notes; 83.6% of notes included at least one entity. Overall F1 measure for note-level symptom identification weighted by entity frequency was 0.84 for the SpaCy model and 0.88 for the Longformer model. F1 measure for identifying presence or absence of any UTI symptom in a clinical note was 0.96 (232/250 correctly classified) for the SpaCy model and 0.98 (240/250 correctly classified) for the Longformer model.

CONCLUSIONS

The study demonstrated the utility of LLMs and transformer-based models in particular for extracting UTI symptoms from unstructured ED clinical notes; models were highly accurate for detecting the presence or absence of any UTI symptom on the note level, with variable performance for individual symptoms.

Patient-Reported Outcome Measure Use in Guidelines Published by the American Academy of Ophthalmology: A Review

TOPIC

We reviewed the use of patient-reported outcome measures (PROMs) in the treatment of ophthalmologic conditions as recommended by the Clinical Practice Guidelines (CPGs) published by the American Academy of Ophthalmology (AAO).

CLINICAL RELEVANCE

Patient-reported outcome measures are standardized instruments that provide information regarding a patient's health status or health-related quality of life. Patient-reported outcome measures are increasingly used to inform study end points in ophthalmology studies. However, the extent to which PROMs are ultimately informing patient management recommendations in ophthalmology as part of CPGs remains an area of evidence gap.

METHODS

We included all CPGs published by the AAO from inception to June 2022. We also included all primary studies and systematic reviews cited in the treatment sections of the CPGs evaluating treatment of an ophthalmic condition. The primary outcome was the frequency of PROMs discussed in CPGs and in cited studies evaluating treatment. Secondary outcomes included frequency of minimal important difference (MID) use to contextualize PROM results and percentage of strong and discretionary recommendations supported by PROMs. We published a study protocol a priori on PROSPERO (CRD42022307427). Reporting followed the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines. We assessed risk of bias using the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument.

RESULTS

We identified 24 eligible CPGs, providing 2458 cited studies (2191 primary, 267 secondary) evaluating treatment of eye conditions. Ten CPGs (41.7%) reported consideration of PROMs. Of these, 31 of 94 (33%) recommendations were informed by studies evaluating a PROM as an outcome. Across all studies cited in the development of CPGs, 221 (9.0%) used PROMs as a primary or secondary outcome, of which 4 PROM results (1.8%) were interpreted using an empirically determined MID. Overall, the risk of bias was low for all CPGs.

CONCLUSIONS

Overall, outcomes of PROMs are seldom used in ophthalmology CPGs published by the AAO and in cited primary and secondary research on treatments. When PROMs were considered, their interpretation was seldom based on an MID. To improve patient care, guideline developers may consider incorporating PROMs and applicable MIDs to inform key outcomes when formulating treatment recommendations.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Aromatase Inhibitor-Related Symptoms Reported by Postmenopausal Women with Nonmetastatic, Estrogen Receptor-Positive Breast Cancer: A Systematic Review

PURPOSE

The objective of this systematic review was to establish an overview of aromatase inhibitor-related symptoms reported by postmenopausal women with nonmetastatic, estrogen receptor-positive breast cancer.

DATA SOURCES

Eight databases (PubMed, Cochrane, Cumulative Index to Nursing and Allied Health Literature [CINAHL], Ovid EMBASE, Ovid MEDLINE, PsycINFO, Scopus, and Web of Science) were searched for trials published between January 2004 and November 2021. Inclusion criteria were studies exploring patient-reported aromatase inhibitor-related symptoms in postmenopausal women with nonmetastatic estrogen receptor-positive breast cancer. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines and the Mixed Method Appraisal Tool were used to rate the quality of the trials included. Of 325 full-text papers, 10 were included. Patient-reported symptoms were clustered by using the European Organization for Research and Treatment of Cancer Quality of Life C30 questionnaire domains. Additional domains were used to cluster other symptoms mentioned: menopausal, sex-related, body alteration, and eye-related. The following clusters were the most frequently presented: sex-related (14 symptoms), pain (9 symptoms), insomnia (5 symptoms), and menopausal (5 symptoms).

CONCLUSION

The target group reported a variety of symptoms related to aromatase inhibitors. No tools are currently available to measure all the symptoms reported, indicating a need to revise the tools to acknowledge additional symptoms. Prospective studies are needed to investigate the prevalence of aromatase inhibitor-related symptoms in women with breast cancer.

IMPLICATION FOR NURSING PRACTICE

Identification of patient-reported clinically relevant symptoms can enable targeted symptom assessment and management strategies for women with breast cancer undergoing aromatase inhibitor treatment.

Prediction of Visual Acuity in Patients With Microbial Keratitis

PURPOSE

The purpose of this study was to predict visual acuity (VA) 90 days after presentation for patients with microbial keratitis (MK) from data at the initial clinical ophthalmic encounter.

METHODS

Patients with MK were identified in the electronic health record between August 2012 and February 2021. Random forest (RF) models were used to predict 90-day VA < 20/40 [visual impairment (VI)]. Predictors evaluated included age, sex, initial VA, and information documented in notes at presentation. Model diagnostics are reported with 95% confidence intervals (CIs) for area under the curve (AUC), misclassification rate, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS

One thousand seven hundred ninety-one patients were identified. The presenting logMAR VA was on average 0.86 (Snellen equivalent and standard deviation = 20/144 ± 12.6 lines) in the affected or worse eye, and 43.6% with VI. VI at 90-day follow-up was present in the affected eye or worse eye for 26.9% of patients. The RF model for predicting 90-day VI had an AUC of 95% (CI: 93%-97%) and a misclassification rate of 9% (7%-12%). The percent sensitivity, specificity, PPV, and NPV were 86% (80%-91%), 92% (89%-95%), 81% (74%-86%), and 95% (92%-97%), respectively. Older age, worse presenting VA, and more mentions of "penetrating keratoplasty" and "bandage contact lens" were associated with increased probability of 90-day VI, whereas more mentions of "quiet" were associated with decreased probability of 90-day VI.

CONCLUSIONS

RF modeling yielded good sensitivity and specificity to predict VI at 90 days which could guide clinicians about the risk of poor vision outcomes for patients with MK.

Who are the patients being offered the faecal immunochemical test in routine English general practice, and for what symptoms? A prospective descriptive study

OBJECTIVES

The faecal immunochemical test (FIT) was introduced to triage patients with lower-risk symptoms of colorectal cancer (CRC) in English primary care in 2018. While there is growing evidence on its utility to triage patients in this setting, evidence is still limited on how official FIT guidance is being used, for which patients and for what symptoms. We aimed to investigate the use of FIT in primary care practice for lower-risk patients who did not immediately meet criteria for urgent referral.

DESIGN

A prospective, descriptive study of symptomatic patients offered a FIT in primary care between January and June 2020.

SETTING

East of England general practices.

PARTICIPANTS

Consenting patients (aged ≥40 years) who were seen by their general practitioners (GPs) with symptoms of possible CRC for whom a FIT was requested. We excluded patients receiving a FIT for asymptomatic screening purposes, or patients deemed by GPs as lacking capacity for informed consent. Data were obtained via patient questionnaire, medical and laboratory records.

PRIMARY AND SECONDARY OUTCOME MEASURES

FIT results (10 µg Hb/g faeces defined a positive result); patient sociodemographic and clinical characteristics; patient-reported and GP-recorded symptoms, symptom severity and symptom agreement between patient and GP (% and kappa statistics).

RESULTS

Complete data were available for 310 patients, median age 70 (IQR 61-77) years, 53% female and 23% FIT positive. Patients most commonly reported change in bowel habit (69%) and fatigue (57%), while GPs most commonly recorded abdominal pain (25%) and change in bowel habit (24%). Symptom agreement ranged from 44% (fatigue) to 80% (unexplained weight loss). Kappa agreement was universally low across symptoms.

CONCLUSION

Almost a quarter of this primary care cohort of symptomatic patients with FIT testing were found to be positive. However, there was low agreement between patient-reported and GP-recorded symptoms. This may impact cancer risk assessment and optimal patient management in primary care.

Patient Self-Report Superior to Electronic Medical Record Abstraction for Identifying Positive COVID-19 Symptoms at Illness Onset

Introduction

Most initial COVID-19 research focused on hospitalized patients. Presenting symptomatology in the outpatient setting was poorly characterized, making it difficult for primary care physicians to predict which patients would require hospitalization. The purpose of this study was to characterize the presenting symptoms of COVID-19 infection and baseline patient characteristics and evaluate for correlation with disease severity, duration, and chronicity in the outpatient setting.

Methods

A total of 107 adult, English-speaking patients with suspected and confirmed COVID-19 cases at the 3 primary care practices of Stony Brook University Hospital were studied between March and December 2020. Survey data were collected from patient telephone interviews and electronic medical record abstraction. The potential risk factors assessed included participant demographics, medical comorbidities, and the number and type of symptoms at illness onset. Outcome measures included symptom duration, hospitalizations, and persistence of symptoms at 12 weeks from study enrollment.

Results

Patient self-report survey elicited nearly twice as many symptoms described at illness onset as those recorded in the electronic medical record (p<0.0001). A higher number of symptoms at illness onset was positively associated with symptom duration and chronicity. The presence of fever and hypoxia at the onset of illness were each positively associated with eventual hospitalization for COVID-19 disease.

Conclusions

Early in the setting of newly emerging infectious diseases, particularly those such as COVID-19 that involve multiple organ systems, patient self-report of symptoms using a complete review of systems rather than electronic medical record abstraction alone may be key for accurate disease identification and characterization as well as prediction of eventual disease severity, duration, and chronicity.

Visual Acuity – Assessment of Data Quality and Usability in an Electronic Health Record System

Objective

To examine the data quality and usability of visual acuity (VA) data extracted from an electronic health record (EHR) system during ophthalmology encounters and provide recommendations for consideration of relevant VA end points in retrospective analyses.

Design

Retrospective, EHR data analysis.

Participants

All patients with eyecare office encounters at any 1 of the 9 locations of a large academic medical center between August 1, 2013, and December 31, 2015.

Methods

Data from 13 of the 21 VA fields (accounting for 93% VA data) in EHR encounters were extracted, categorized, recoded, and assessed for conformance and plausibility using an internal data dictionary, a 38-item listing of VA line measurements and observations including 28 line measurements (e.g., 20/30, 20/400) and 10 observations (e.g., no light perception). Entries were classified into usable and unusable data. Usable data were further categorized based on conformance to the internal data dictionary: (1) exact match; (2) conditional conformance, letter count (e.g., 20/30^+2-3); (3) convertible conformance (e.g., 5/200 to 20/800); (4) plausible but cannot be conformed (e.g., 5/400). Data were deemed unusable when they were not plausible.

Main Outcome Measures

Proportions of usable and unusable VA entries at the overall and subspecialty levels.

Results

All VA data from 513 036 encounters representing 166 212 patients were included. Of the 1 573 643 VA entries, 1 438 661 (91.4%) contained usable data. There were 1 196 720 (76.0%) exact match (category 1), 185 692 (11.8%) conditional conformance (category 2), 40 270 (2.6%) convertible conformance (category 3), and 15 979 (1.0%) plausible but not conformed entries (category 4). Visual acuity entries during visits with providers from retina (17.5%), glaucoma (14.0%), neuro-ophthalmology (8.9%), and low vision (8.8%) had the highest rates of unusable data. Documented VA entries with providers from comprehensive eyecare (86.7%), oculoplastics (81.5%), and pediatrics/strabismus (78.6%) yielded the highest proportions of exact match with the data dictionary.

Conclusions

Electronic health record VA data quality and usability vary across documented VA measures, observations, and eyecare subspecialty. We proposed a checklist of considerations and recommendations for planning, extracting, analyzing, and reporting retrospective study outcomes using EHR VA data. These are important first steps to standardize analyses enabling comparative research.

Application of Patient-Reported Outcome Measurements in Clinical Trials in China

IMPORTANCE

Regulatory authorities, industry peers, and international health policies have emphasized the value of assessing patient-reported outcomes (PROs) in clinical studies. Despite the increase in the number of clinical studies in the last decade in China, little is known about the extent of the use of PROs.

OBJECTIVE

To evaluate the application and characteristics of PRO instruments as primary and secondary outcomes in randomized clinical trials in China.

DESIGN, SETTING, AND PARTICIPANTS

A cross-sectional study of interventional clinical trials conducted in China from January 1, 2010, to December 31, 2020, was performed. Data obtained from the Chinese Clinical Trial Registry and ClinicalTrials.gov databases were evaluated.

MAIN OUTCOMES AND MEASURES

Trials were categorized according to those that (1) precisely listed PRO tools as outcomes, (2) mentioned patient subjective feelings in outcomes but did not clarify which tools were used for assessment, and (3) did not mention any PRO measurements. Data on study phase, setting, participant age, and sex were extracted from trials that considered patient feelings, along with the target diseases and names of the PRO tools.

RESULTS

Among a total of 34 033 trials, 6915 (20.3%) listed the explicit PRO instruments used and 3178 (9.3%) included PRO in their outcomes but did not include the names of the assessment tools. From more than 32 million people included in the registered trials, data on 1.5 million (4.7%) patients were scientifically collected by PRO instruments, and subjective feelings were assessed for 693 867 (2.1%) participants. Pain (16.8%), cancer (15.6%), and musculoskeletal symptoms (13.3%) were the most common conditions for which PROs were precisely collected by tools. The most common tools for PRO measurements were the visual analog scale, Short-Form 36, and Hamilton Depression Scale.

CONCLUSIONS AND RELEVANCE

In this cross-sectional study, the use of PROs increased during the study period in clinical trials conducted in China. However, patient opinion appears to still be rarely measured. The application of PRO is geographically unevenly distributed. Development of PRO instruments, especially those suitable for the Chinese population, may be useful. Further expansion of PROs with respect to the scope of diseases is needed to avoid missing important data.

Influencing factors of treatment and prognosis perceptions among advanced cancer patients: a cross-sectional study

PURPOSE

The present study aimed to investigate the status and significantly influencing factors of treatment and prognosis perceptions among advanced cancer patients based on patient-reported outcome.

METHODS

A cross-sectional study was conducted at two tertiary A general hospitals. From June to September 2019, 300 patients were invited and 292 of them participated in this study. Except for 9 invalid questionnaires, 283 pen-paper questionnaires including sociodemographic and clinical characteristics, Chinese Version of Prognosis and Treatment Perception Questionnaire, Herth Hope Index, and Hospital Anxiety and Depression Scale were well completed. Descriptive analysis, Pearson's correlation test, logistic regression analysis, and multiple linear regression analysis were applied for analysis.

RESULTS

One hundred seventy-five (61.8%) advanced cancer patients reported inaccurate treatment perception. Prognosis perception scored 87.9 ± 13.72 indicating a middle level of prognosis perception. Fourteen (4.9%), 138 (48.8%), and 131 (46.3%) patients presented low, middle, and high prognosis perception levels, respectively. In patients, without spouse and religion beliefs, received chemoradiotherapy, diagnosed as cancer equal to or less than 1 year, and higher hope level were inaccurate treatment perception's risk factors. Younger age, longer diagnosis time, higher educational level, less support for medical expenses payment, receiving chemoradiotherapy, and lower hope level but more anxiety and depression symptoms were positive predictors of prognosis perception.

CONCLUSIONS

A majority of advanced cancer patients in this study reported inaccurate treatment and middle level of prognosis perception influencing by objective and subjective factors. Clinical interventions could be developed referring these impacting factors originating from patient-reported outcome.

How patients describe their diagnosis compared to clinical documentation

OBJECTIVES

To explore how patients describe their diagnoses following Emergency Department (ED) discharge, and how this compares to electronic medical record (EMR) documentation.

METHODS

We conducted a cohort study of patients discharged from three EDs. Patients completed questionnaires regarding their understanding of their diagnosis. Inclusion criteria: adult ED patients aged 18 and older seen within the last seven days. We independently compared patient-reported new diagnoses following discharge to EMR-documented diagnoses regarding diagnostic content (identical, insignificantly different, different, not enough detail) and the level of technical language in diagnostic description (technical, semi-technical, lay).

RESULTS

The majority of participants (n=95 out of 137) reported receiving a diagnosis and stated the given diagnosis. Of those who reported their diagnosis, 66%, were females (n=62), the average age was 43 (SD 16), and a fourth (n=24) were Black and 66% (n=63) were white. The majority (84%) described either the same or an insignificantly different diagnosis. For 11% the patient-reported diagnosis differed from the one documented. More than half reported their diagnosis using semi-technical (34%) or technical language (26%), and over a third (40%) described their diagnosis in lay language.

CONCLUSIONS

Patient-reported diagnoses following ED discharge had moderate agreement with EMR-documented diagnoses. Findings suggest that patients might reproduce verbatim semi-technical or technical diagnoses they received from clinicians, but not fully understood what the diagnosis means for them.

Binocular Interference vs Diplopia in Patients With Epiretinal Membrane

Importance

Patients with epiretinal membrane (ERM) sometimes close 1 eye for improved vision, but associations have not been rigorously studied.

Objective

To evaluate associations with monocular eye closure in patients with ERM, and to report binocular interference (closing 1 eye to improve visual quality).

Design, Setting, and Participants

Retrospective medical record review of an adult strabismus clinic at a tertiary referral center. Patients with ERM referred from retina clinicians between June 2010 and October 2019 who completed the Adult Strabismus (AS)-20 questionnaire, including the question: "I cover or close one eye to see things better." Two groups were identified: (1) patients reporting eye closure sometimes or more, and (2) patients reporting no eye closure (as control patients).

Main Outcomes and Measures

Frequencies of (1) central-peripheral rivalry (CPR)-type diplopia (dragged fovea diplopia); (2) binocular interference (monocular eye closure but no diplopia or strabismus); and (3) other, associated with monocular eye closure. Visual acuity, metamorphopsia, aniseikonia, and AS-20 quality of life domain scores (self-perception, interactions, reading function, and general function) compared between binocular interference, CPR-type diplopia, and control patients.

Results

A total of 124 patients with ERM (58 of 124 were women [47%]; mean [SD] age, 70 [9] years) reported monocular eye closure. Associations were binocular interference in 36 (29%; 95% CI, 21%-38%), CPR-type diplopia in 34 (27%; 95% CI, 20%-36%), and other (primarily strabismus) in 54 (44%). Compared with control patients with ERM (n = 11), patients with ERM and binocular interference had worse quality of life on AS-20 reading function (95 vs 62; mean difference, 22 points; 95% CI, 7-27 points; P = .007) and general function (89 vs 68; mean difference, 23 points; 95% CI, 13-34 points; P = .01) domains. Compared with CPR-type diplopia, patients with binocular interference had poorer worst-eye visual acuity (median 0.50 vs 0.30 logMAR [20/63 vs 20/40]; mean difference, 0.13 logMAR; 95% CI, 0.00-0.25 logMAR [20/20 to 20/35]; P = .03), and a larger interocular difference (0.46 vs 0.19 logMAR [20/58 vs 20/30]; mean difference, 0.15 logMAR; 95% CI, 0.03-0.28 logMAR [20/21 to 20/38]; P = .004).

Conclusions and Relevance

Study findings suggest that binocular interference, manifesting as monocular eye closure (without diplopia or strabismus), is a distinct entity affecting quality of life in patients with epiretinal membrane.

Macular pigment optical density and visual quality of life

PURPOSE

There is robust evidence that higher macular pigment concentrations help reduce both veiling and discomfort glare in patients with or without ocular disease. We investigated whether there was also a relationship between macular pigment optical density (MPOD) and patient surveys about glare or ocular discomfort.

METHODS

We measured MPOD psychophysically in 23 healthy subjects and administered the National Eye Institute Visual Functioning Questionnaire (VFQ-25). Responses for each survey question were sorted from low (no limitation) to high (very severe limitation). The median response for each question was determined, and independent t-tests were performed on the mean MPOD values for survey responses above and below the median. We also performed a non-parametric correlation analysis between MPOD and survey responses.

RESULTS

While the median response was "no limitation" for most (22 of 25) survey questions, responses were slightly higher for two questions concerning ocular discomfort and one question related to driving at night. MPOD levels were significantly higher in subjects that reported no discomfort in or around their eyes than in those that reported mild discomfort. There was also a trend toward higher MPOD levels in subjects who reported that pain in or around their eyes never limited their activity as well as in subjects who reported no difficulty driving at night.

CONCLUSION

These preliminary findings are consistent with the well-established discomfort and glare hypotheses for MPOD. The current findings on subjective ocular discomfort in the absence of glare deserve further study.

Can emergency department provider notes help to achieve more dynamic clinical decision support?

OBJECTIVE

Assess whether clinical data were present in emergency department (ED) provider notes at time of order entry for cervical spine (c-spine) imaging that could be used to augment or pre-populate clinical decision support (CDS) attributes.

METHODS

This Institutional Review Board-approved retrospective study, performed in a quaternary hospital, included all encounters for adult ED patients seen April 1, 2013-September 30, 2014 for a chief complaint of trauma who received c-spine computed tomography (CT) or x-ray. We assessed proportion of ED encounters with at least 1 c-spine-specific CDS rule attribute in clinical notes available at the time of imaging order and agreement between attributes in clinical notes and data entered into CDS.

RESULTS

A portion of the clinical note was submitted before imaging order in 42% (184/438) of encounters reviewed; 59.2% (109/184) of encounters with note portions submitted before imaging order had at least 1 positive CDS attribute identified supporting imaging study appropriateness; 34.8% (64/184) identified exclusion criteria where CDS appropriateness recommendations would not be applicable. 65.8% (121/184) of encounters had either a positive CDS attribute or an exclusion criterion. Concordance of c-spine CDS attributes when present in both notes and CDS was 68.4% (κ = 0.35 95% CI: 0.15-0.56; McNemar P = 0.23).

CONCLUSIONS

Clinical notes are an underutilized source of clinical attributes needed for CDS, available in a substantial percentage of encounters at the time of imaging order. Automated pre-population of imaging order requisitions with relevant clinical information extracted from electronic health record provider notes may: (1) improve ordering efficiency by reducing redundant data entry, (2) help improve clinical relevance of CDS alerts, and (3) potentially reduce provider burnout from extraneous alerts.

Augmenting the Clinical Data Sources for Enigmatic Diseases: A Cross-Sectional Study of Self-Tracking Data and Clinical Documentation in Endometriosis

BACKGROUND

Self-tracking through mobile health technology can augment the electronic health record (EHR) as an additional data source by providing direct patient input. This can be particularly useful in the context of enigmatic diseases and further promote patient engagement.

OBJECTIVES

This study aimed to investigate the additional information that can be gained through direct patient input on poorly understood diseases, beyond what is already documented in the EHR.

METHODS

This was an observational study including two samples with a clinically confirmed endometriosis diagnosis. We analyzed data from 6,925 women with endometriosis using a research app for tracking endometriosis to assess prevalence of self-reported pain problems, between- and within-person variability in pain over time, endometriosis-affected tasks of daily function, and self-management strategies. We analyzed data from 4,389 patients identified through a large metropolitan hospital EHR to compare pain problems with the self-tracking app and to identify unique data elements that can be contributed via patient self-tracking.

RESULTS

Pelvic pain was the most prevalent problem in the self-tracking sample (57.3%), followed by gastrointestinal-related (55.9%) and lower back (49.2%) pain. Unique problems that were captured by self-tracking included pain in ovaries (43.7%) and uterus (37.2%). Pain experience was highly variable both across and within participants over time. Within-person variation accounted for 58% of the total variance in pain scores, and was large in magnitude, based on the ratio of within- to between-person variability (0.92) and the intraclass correlation (0.42). Work was the most affected daily function task (49%), and there was significant within- and between-person variability in self-management effectiveness. Prevalence rates in the EHR were significantly lower, with abdominal pain being the most prevalent (36.5%).

CONCLUSION

For enigmatic diseases, patient self-tracking as an additional data source complementary to EHR can enable learning from the patient to more accurately and comprehensively evaluate patient health history and status.

Comparison of physical examination and laboratory data between a clinical study and electronic health records

Research based on secondary analysis of data stored in electronic health records (EHR) has gained popularity, but whether the data are consistent with those collected under a study setting is unknown. The objective is to assess the agreement between data obtained in a prospective study and routine-care data extracted retrospectively from the EHR. We compared the data collected in a longitudinal lifestyle intervention study with those recorded in the EHR system over 5 years. A total of 225 working adults were recruited at an academic institution between 2008–2012, whose EHR data were also available during the same time period. After aligning the participants’ study visit dates with their hospital encounter dates, data on blood pressure, body mass index (BMI), and laboratory measurements (including high-density lipoprotein (HDL), low-density lipoprotein (LDL), triglycerides, and total cholesterol) were compared via a paired t-test for equivalence with pre-specified margins. Summary statistics were used to compare smoking status and medication prescriptions. Overall, data were consistent between the two sources (i.e., BMI, smoking status, medication prescriptions), whereas some differences were found in cholesterol measurements (i.e., HDL and total cholesterol), possibly due to different lab assays and subject’s fasting status. In conclusion, some EHR data are fairly consistent with those collected in a clinical study, whereas others may require further examination. Researchers should evaluate the consistency and quality of EHR data and compare them with other sources of data when possible.

Technological Advances and the Future of Suicide Prevention: Ethical, Legal, and Empirical Challenges

Technological advancements have brought multiple and diverse benefits to our human existence. In suicide prevention, new technologies have spurred great interest in and reports of the applicability to assessing, monitoring, and intervening in various community and clinical populations. We argue in this article that we need to better understand the complexities of implementation of technological advances; especially the accuracy, effectiveness, safety, ethical, and legal issues, even as implementation occurs at individual, clinical, and population levels, in order to achieve that measure of public health impact we all desire (i.e., greater benefit than harm).

Patient factors associated with discrepancies between patient-reported and clinician-documented peripheral neuropathy in women with breast cancer receiving paclitaxel: A pilot study

Purpose

Discrepancies between clinicians’ assessment of chemotherapy-induced peripheral neuropathy (CIPN) and patient-reported outcomes (PRO) have been described, though the underlying reasons are unknown. Our objective was to identify potential patient-specific factors associated with under-describing of CIPN to clinicians in women with non-metastatic breast cancer treated with paclitaxel.

Methods

Patients enrolled in an observational study (n = 60) completed weekly CIPN PRO using the EORTC CIPN20. Clinician-documented CIPN using the NCI CTCAE were abstracted from the electronic medical record and paired with CIPN20 data at weeks 7 and 10. Patients were classified as under-describers if their CIPN20 was above the 80th percentile of the CIPN20 distribution for that CTCAE grade from an independent clinical trial (N08CA). Demographics, Assessment of Survivor Concerns (ASC), Trust in Oncologist Scale (TiOS), and health literacy assessment were collected post-treatment via survey. Repeated measures cumulative logistic regression models were used to identify factors associated with under-describing CIPN.

Results

Forty-two women completed the survey (response rate 70%). Three and 9 patients were categorized as under-describers at weeks 7 and 10, respectively. Women who were not working (OR = 9.00, 95%CI 1.06–76.15), had lower income (OR = 7.04, 95%CI 1.5–32.99), and displayed higher trust in their oncologist’s competence (OR = 1.29, 95%CI 1.03–1.62 for a 0.1-unit increase in score) were more likely to under-describe CIPN symptoms.

Conclusions

This preliminary study identified non-working status, low income and trust in oncologist’s competence as potential factors influencing under-description of CIPN to the clinical team. Further work is needed to clarify these relationships and test additional factors.

•

This pilot study examined factors associated with under-describing of neuropathy.

•

Patient-reported and clinician-documented neuropathy severity were compared.

•

Non-working status and low income were associated with neuropathy under-describing.

•

Trust in oncologist’s competence was associated with neuropathy under-describing.

•

Recording patient-clinician interactions would confirm under-describing behavior.

Evaluation of Accuracy and Completeness of Electronic Dental Records in a Dental School Setting

Background:

High-quality data in Electronic Dental Records (EDR) is essential for dental schools to provide high-quality patient care, improve dental students’ professionalism, and support a platform for research. Therefore, ensuring data quality in EDRs is extremely important.

Objective:

To perform a quality appraisal of EDRs by evaluating their accuracy and completeness.

Methods:

This was a Cross-sectional Observational Study conducted over four consecutive years in a dental school setting. Manual chart reviews were performed on an annual basis. EDR data were audited for accuracy and completeness. Accuracy was evaluated by comparing the entered data with an external source, where possible. An EDR data field was considered complete if it was not missing.

Results:

A total of 1,720 de-identified chart reviews were studied. The accuracy of the data to identify the patient was 93%. The completeness of the essential components of EDRs was 48-94%. Completeness was highest for documenting the patient’s chief complaint (94%) and the lowest for recall plan (48%). Completeness of data documenting social and behavioral determinants of health in EDRs was 36-77%, with the highest proportion of completeness in this domain being for oral hygiene habits, smoking habits, and social history.

Conclusion:

The quality appraisal of EDRs varied according to the data field. Understanding patterns of accuracy and completeness in EDRs will guide training and quality enforcement activities.

Diplopia after strabismus surgery for adults with nondiplopic childhood-onset strabismus

PURPOSE

To describe frequency of postoperative diplopia after strabismus surgery in nondiplopic adults with childhood-onset strabismus and to report health-related quality-of-life (HRQOL) outcomes.

METHODS

We prospectively enrolled 79 adults with no diplopia in any gaze who had childhood-onset strabismus and were scheduled for strabismus surgery. Diplopia was assessed preoperatively and at 6 weeks and 1 year postoperatively using a standardized diplopia questionnaire with 5 response options in 7 gaze positions. HRQOL was assessed using the Adult Strabismus-20 (AS-20) questionnaire, with self-perception, interactions, reading function, and general function domains.

RESULTS

Constant diplopia in straight-ahead distance and reading gaze occurred in 1 patient (1% [95% CI, 0%-7%] at 6 weeks and 2% [95% CI, 0%-10%] at 1 year). Regarding the rate of any diplopia (including rarely) in any gaze, 15 of 78 patients (19%) reported diplopia at 6 weeks, of whom 13 had diplopia in straight-ahead distance gaze; 8 (10%), in reading gaze. At 1 year, 8 of 51 patients (16%) reported any diplopia (including rarely) in any gaze, of whom 7 had diplopia in straight-ahead distance gaze and 4 (8%) in reading gaze. Mean AS-20 scores improved at 1 year overall (by 32, 19, 14, and 15 points, resp., per domain) and for the 8 diplopic patients (by 21, 13, 16, and 11 points).

CONCLUSIONS

In adults with nondiplopic strabismus, constant postoperative diplopia is rare, although the rate of intermittent diplopia is higher. Even when postoperative diplopia occurs, HRQOL often improves.

Variation in Physicians' Electronic Health Record Documentation and Potential Patient Harm from That Variation

BACKGROUND

Physician-to-physician variation in electronic health record (EHR) documentation not driven by patients' clinical status could be harmful.

OBJECTIVE

Measure variation in completion of common clinical documentation domains. Identify perceived causes and effects of variation and strategies to mitigate negative effects.

DESIGN

Sequential, explanatory, mixed methods using log data from a commercial EHR vendor and semi-structured interviews with outpatient primary care practices.

PARTICIPANTS

Quantitative: 170,332 encounters led by 809 physicians in 237 practices. Qualitative: 40 interviewees in 10 practices.

MAIN MEASURES

Interquartile range (IQR) of the proportion of encounters in which a physician completed documentation, for each documentation category. Multilevel linear regression measured the proportion of variation at the physician level.

KEY RESULTS

Five clinical documentation categories had substantial and statistically significant (p < 0.001) variation at the physician level after accounting for state, organization, and practice levels: (1) discussing results (IQR = 50.8%, proportion of variation explained by physician level = 78.1%); (2) assessment and diagnosis (IQR = 60.4%, physician-level variation = 76.0%); (3) problem list (IQR = 73.1%, physician-level variation = 70.1%); (4) review of systems (IQR = 62.3%, physician-level variation = 67.7%); and (5) social history (IQR = 53.3%, physician-level variation = 62.2%). Drivers of variation from interviews included user preferences and EHR designs with multiple places to record similar information. Variation was perceived to create documentation inefficiencies and risk patient harm due to missed or misinterpreted information. Mitigation strategies included targeted user training during EHR implementation and practice meetings focused on documentation standardization.

CONCLUSIONS

Physician-to-physician variation in EHR documentation impedes effective and safe use of EHRs, but there are potential strategies to mitigate negative consequences.

Perspectives of Rheumatoid Arthritis Patients on Electronic Communication and Patient-Reported Outcome Data Collection: A Qualitative Study

OBJECTIVE

To identify the perspectives of patients with rheumatoid arthritis (RA) on electronic recording of between-visit disease activity and other patient-reported outcomes (PROs) and on sharing this information with health care providers or peers.

METHODS

Patients with RA were recruited to participate in focus groups from December 2014 to April 2015. The topic guide and analysis were based on the Andersen-Newman framework. Sessions were audiorecorded, transcribed, independently coded, and analyzed for themes.

RESULTS

Thirty-one patients participated in 7 focus groups. Their mean ± SD age was 51 ± 13.1 years, 94% were women, 52% were African American, 11% were Hispanic, and 37% were white. Three themes emerged: provider communication, information-seeking about RA, and social and peer support. Participants expressed a willingness to track disease activity data to share with health care providers electronically if providers would act on the information. Participants envisioned symptom tracking and information sharing as a mechanism to relay and obtain reliable information about RA. Participants were also interested in electronic communication between visits if it facilitated learning about symptom management and enhanced opportunities for social support among patients with RA.

CONCLUSION

Patients with RA may be amenable to electronic collection and sharing of PRO-type data between clinical encounters if it facilitates communication with health care providers and provides access to reliable information about RA. Providing patients with social support was important for enhancing PROs collection by helping them overcome barriers by using electronic devices and overcome reservations about the value of these data.

Concordance of care processes between medical records and patient self-administered questionnaires

Background

Despite the increasing use of medical records to measure quality of care, studies have shown that their validity is suboptimal. The objective of this study is to assess the concordance of cardiovascular care processes evaluated through medical record review and patient self-administered questionnaires (SAQs) using ten quality indicators (TRANSIT indicators). These indicators were developed as part of a participatory research program (TRANSIT study) dedicated to TRANSforming InTerprofessional clinical practices to improve cardiovascular disease (CVD) prevention in primary care.

Methods

For every patient participating in the TRANSIT study, the compliance to each indicator (individual scores) as well as the mean compliance to all indicators of a category (subscale scores) and to the complete set of ten indicators (overall scale score) were established. Concordance between results obtained using medical records and patient SAQs was assessed by prevalence-adjusted bias-adjusted kappa (PABAK) coefficients as well as intraclass correlation coefficients (ICCs) and 95% confidence intervals (95% CI). Generalized linear mixed models (GLMM) were used to identify patients’ sociodemographic and clinical characteristics associated with agreement between the two data sources.

Results

The TRANSIT study was conducted in a primary care setting among patients (n = 759) with multimorbidity, at moderate (16%) and high risk (83%) of cardiovascular diseases. Quality of care, as measured by the TRANSIT indicators, varied substantially between medical records and patient SAQ. Concordance between the two data sources, as measured by ICCs (95% CI), was poor for the subscale (0.18 [0.08–0.27] to 0.46 [0.40–0.52]) and overall (0.46 [0.40–0.53]) compliance scale scores. GLMM showed that agreement was not affected by patients’ characteristics.

Conclusions

In quality improvement strategies, researchers must acknowledge that care processes may not be consistently recorded in medical records. They must also be aware that the evaluation of the quality of care may vary depending on the source of information, the clinician responsible of documenting the interventions, and the domain of care.

Electronic supplementary material

The online version of this article (10.1186/s12875-019-0979-7) contains supplementary material, which is available to authorized users.

Changes in Electronic Health Record Use Time and Documentation over the Course of a Decade

PURPOSE

With the current wide adoption of electronic health records (EHRs) by ophthalmologists, there are widespread concerns about the amount of time spent using the EHR. The goal of this study was to examine how the amount of time spent using EHRs as well as related documentation behaviors changed 1 decade after EHR adoption.

DESIGN

Single-center cohort study.

PARTICIPANTS

Six hundred eighty-five thousand three hundred sixty-one office visits with 70 ophthalmology providers.

METHODS

We calculated time spent using the EHR associated with each individual office visit using EHR audit logs and determined chart closure times and progress note length from secondary EHR data. We tracked and modeled how these metrics changed from 2006 to 2016 with linear mixed models.

MAIN OUTCOME MEASURES

Minutes spent using the EHR associated with an office visit, chart closure time in hours from the office visit check-in time, and progress note length in characters.

RESULTS

Median EHR time per office visit in 2006 was 4.2 minutes (interquartile range [IQR], 3.5 minutes), and increased to 6.4 minutes (IQR, 4.5 minutes) in 2016. Median chart closure time was 2.8 hours (IQR, 21.3 hours) in 2006 and decreased to 2.3 hours (IQR, 18.5 hours) in 2016. In 2006, median note length was 1530 characters (IQR, 1435 characters) and increased to 3838 characters (IQR, 2668.3 characters) in 2016. Linear mixed models found EHR time per office visit was 31.9±0.2% (P < 0.001) greater from 2014 through 2016 than from 2006 through 2010, chart closure time was 6.7±0.3 hours (P < 0.001) shorter from 2014 through 2016 versus 2006 through 2010, and note length was 1807.4±6.5 characters (P < 0.001) longer from 2014 through 2016 versus 2006 through 2010.

CONCLUSIONS

After 1 decade of use, providers spend more time using the EHR for an office visit, generate longer notes, and close the chart faster. These changes are likely to represent increased time and documentation pressure for providers. Electronic health record redesign and new documentation regulations may help to address these issues.

Clinician Report of Oral Oncolytic Symptoms and Adherence Obtained via a Patient-Reported Outcome Measure (PROM)

PURPOSE

Patient-reported outcome measures (PROMs) for symptom monitoring during cancer therapy have been shown to have a positive impact on outcomes. These findings have primarily been shown for patients receiving intravenous chemotherapy. In addition, there is known discordance between physician reporting of symptoms and patient self-report. This initiative sought to describe patient-reported symptom burden and medication adherence and to indicate the degree of PROM results being discussed with the provider as indicated by documentation in the medical record for patients taking oral oncolytic therapy.

METHODS

The Michigan Oncology Quality Consortium (MOQC) PROM, which included symptom ratings, medication adherence, and patient confidence in self-management, was completed during outpatient visits and compared with corresponding data documented in the electronic medical record (EMR).

RESULTS

There were 82 completed PROMs. Approximately half included at least one symptom rated as severe (46%). Sixty-five percent of reported severe symptoms were documented in the EMR. Patient-reported moderate-to-severe pain was most likely to be documented in the EMR (100%), whereas patient-reported moderate-to-severe depression and anxiety were least likely to be documented (21%). Of the total symptoms documented, grading of symptom severity matched that of the patients' own report for 11% of severe symptoms. Adherence to oral oncolytics was excellent for 63% of patients, and patient adherence was documented in 7% of provider notes.

CONCLUSION

Patients frequently reported moderate-to-severe symptoms, and approximately 40% of patients reported nonadherence. Clinician report (documented in the EMR) of the patient symptom burden, symptom severity, and adherence to oral oncolytic therapy was not consistent with the patients' self-report. Use of a PROM for patients taking oral oncolytics has the opportunity to improve symptom management and medication adherence.

Completeness of Electronic Dental Records in a Student Clinic: Retrospective Analysis

BACKGROUND

A well-designed, adequately documented, and properly maintained patient record is an important tool for quality assurance and care continuity. Good clinical documentation skills are supposed to be a fundamental part of dental student training.

OBJECTIVE

The goal of this study was to assess the completeness of electronic patient records in a student clinic.

METHODS

Completeness of patient records was assessed using comparative review of validated cases of alveolar osteitis treated between August 2011 and May 2017 in a student clinic at Columbia University College of Dental Medicine, New York, USA. Based on a literature review, population-based prevalence of nine most frequently mentioned symptoms, signs, and treatment procedures of alveolar osteitis was identified. Completeness of alveolar osteitis records was assessed by comparison of population-based prevalence and frequency of corresponding items in the student documentation. To obtain all alveolar osteitis cases, we ran a query on the electronic dental record, which included all cases with diagnostic code Z1820 or any variation of the phrases "dry socket" and "alveolar osteitis" in the notes. The resulting records were manually reviewed to definitively confirm alveolar osteitis and to extract all index items.

RESULTS

Overall, 296 definitive cases of alveolar osteitis were identified. Only 22% (64/296) of cases contained a diagnostic code. Comparison of the frequency of the nine index categories in the validated alveolar osteitis cases between the student clinic and the population showed the following results: severe pain: 94% (279/296) vs 100% (430/430); bare bone/missing blood clot: 27% (80/296) vs 74% (35/47) to 100% (329/329); malodor: 7% (22/296) vs 33%-50% (18/54); radiating pain to the ear: 8% (24/296) vs 56% (30/54); lymphadenopathy: 1% (3/296) vs 9% (5/54); inflammation: 14% (42/296) vs 50% (27/54); debris: 12% (36/296) vs 87% (47/54); alveolar osteitis site noted: 96% (283/296) vs 100% (430/430; accepted documentation requirement); and anesthesia during debridement: 77% (20/24) vs 100% (430/430; standard of anesthetization prior to debridement).

CONCLUSIONS

There was a significant discrepancy between the index category frequency in alveolar osteitis cases documented by dental students and in the population (reported in peer-reviewed literature). More attention to clinical documentation skills is warranted in dental student training.

Real-world data and the patient perspective: the PROmise of social media?

Understanding the patient perspective is fundamental to delivering patient-centred care. In most healthcare systems, however, patient-reported outcomes are not regularly collected or recorded as part of routine clinical care, despite evidence that doing so can have tangible clinical benefit. In the absence of the routine collection of these data, research is beginning to turn to social media as a novel means to capture the patient voice. Publicly available social media data can now be analysed with relative ease, bypassing many logistical hurdles associated with traditional approaches and allowing for accelerated and cost-effective data collection. Existing work has shown these data can offer credible insight into the patient experience, although more work is needed to understand limitations with respect to patient representativeness and nuances of captured experience. Nevertheless, linking social media to electronic medical records offers a significant opportunity for patient views to be systematically collected for health services research and ultimately to improve patient care.

Eye Symptom Questionnaire to Evaluate Anterior Eye Health

OBJECTIVES

Ophthalmologists assess eye complaints with a careful history and eye examination; however, other types of physicians have limited tools to evaluate anterior segment (AS) eye diseases. We identified the eye symptom questions that providers should ask to help determine the presence and urgency of AS eye diseases.

METHODS

Persons with and without AS disease completed a self-report eye symptom questionnaire (ESQ) based on the National Institutes of Health Toolbox symptom items in an academic center's corneal and comprehensive eye clinics. Gold standard ophthalmic examination determined the presence and urgency of AS disease. The association between reported symptom severity and the probability of AS disease, or urgent AS disease, was evaluated using logistic regression models, and sensitivity and specificity of the ESQ were also calculated.

RESULTS

A total of 324 eyes of 162 subjects were included in the study. Of these, AS disease was present in 255 eyes (79%); of which, 111 eyes showed urgent disease. Increasing symptom severity for eye pain (odds ratio [OR]=2.58; P<0.001), glare (OR=2.61; P=0.001), and blurry vision (OR=1.98; P<0.001) were associated with increased odds of AS disease. Increasing symptom severity for eye pain (OR=2.02; P<0.001), eye redness (OR=1.69; P=0.02), and blurry vision (OR=1.41, P=0.01) were associated with increased odds of urgent AS disease. For the primary analysis with mild symptoms considered relevant, the sensitivity of the ESQ to detect AS disease was 83% and to detect urgent AS disease was 92%.

CONCLUSION

Symptoms of eye pain, glare, redness, and blurry vision indicate the presence and urgency of AS disease.

Analysis of Total Time Requirements of Electronic Health Record Use by Ophthalmologists Using Secondary EHR Data

Electronic Health Records (EHRs) are widely used in the United States for clinical care and billing activities. Their widespread adoption has raised a variety of concerns about their effects on providers and medical care. As researchers address these concerns, they will need to understand how much time providers actually spend on the EHR. This study develops and validates methods for calculating total time requirements for EHR use by ophthalmologists using secondary EHR data from audit logs. Key findings from this study are that (1) Secondary EHR data can be used to estimate lower bounds on provider EHR use, (2) Providers spend a large amount of time using the EHR, (3) Most time spent on the EHR is spent reviewing information. These findings have important implications for practicing clinicians, and for EHR system design in the future.

Low Concordance of Patient-Reported Outcomes With Clinical and Clinical Trial Documentation

PURPOSE

Health care research increasingly relies on assessment of data extracted from electronic medical records (EMRs). Clinical trial adverse event (AE) logs and patient-reported outcomes (PROs) are sources of data often available in the context of specific research projects. The aim of this study was to evaluate the extent of data concordance from these sources.

PATIENTS AND METHODS

Patients enrolled in clinical trials or receiving standard treatment for lung cancer (n = 62) completed validated questionnaires on physical and psychological symptoms at up to three assessment points. Temporally matched documentation was extracted from EMR notes and, for clinical trial participants (n = 41), AE logs. Evaluated data included symptom assessment, vital signs, medication logs, and laboratory values. Agreement (positive, negative) and Cohen's κ coefficients were calculated to assess concordance of symptoms among sources, with PROs considered the gold standard.

RESULTS

Patient-reported weight loss correlated significantly with clinical measurements ( t = 2.90; P = .02), and average number of PROs correlated negatively with albumin concentration, supporting PROs as the gold standard. Comparisons of PROs versus EMR yielded poor concordance across 11 physical symptoms, anxiety, and depressive symptoms (all κ < 0.40). Providers under-reported the presence of each symptom in the EMR compared with PROs. AE logs showed similarly poor concordance with PROs (all κ < 0.40, except shortness of breath). Negative agreement among sources was higher than positive agreement for all symptoms except pain.

CONCLUSION

There was poor concordance between EMR notes and AE logs with PROs. Findings suggest that EMR notes and AE logs may not be reliable sources for capturing physical and psychological symptoms experienced by patients with lung cancer, supporting use of PRO assessments in oncology practices.

The challenges of assessing effectiveness of lacosamide using electronic medical record databases

PURPOSE

Electronic health record (EHR) databases are a potential source for conducting research to generate real world evidence on patient outcomes. The objective of the study was to evaluate the feasibility of using EHR data to assess seizure outcomes in patients treated with lacosamide (LCM) monotherapy.

METHODS

This was a retrospective cohort study conducted using the Optum clinical EHR database. The study sample comprised patients ≥17 years of age with epilepsy or seizures and treated with LCM monotherapy between 1 January 2009 and 31 December 2013. Structured and unstructured data from prescribed medication and abstracted physician note records were used to identify patients with epilepsy treated with LCM monotherapy and measure seizure frequency outcomes. The index date was the first date of LCM monotherapy, with a 6-month baseline period. Patients were observed for up to 12 months beginning on the index date (follow-up period). The EHR data were not sufficient to compute days supply and explicit duration of LCM and other antiepileptic drug (AED) therapies; therefore, LCM monotherapy was estimated from prescription dates of AEDs. Outcomes were change in seizures per month or change in seizure frequency category from baseline to follow-up. Descriptive statistics were used to describe baseline characteristics and study outcomes.

RESULTS

A total of 10,988 patients with at least one LCM prescription were identified during the study period, 470 of whom met all the selection criteria and were included in the study sample. Although many patients had abstracted physician note records that referred to their seizures, only 3.2% of the patients had seizure frequency information that could be used to quantify the number of seizures per month in both the baseline and follow-up periods; thus, this information could not be used to assess the effectiveness of LCM monotherapy on seizure outcomes.

CONCLUSION

Lacosamide monotherapy effectiveness was not estimated because the EHR prescription record data did not have sufficient information on days supply. Additionally, most patients' records did not contain adequate information to allow for evaluation of quantitative changes in seizure frequency based on the number of seizures per month. More studies are needed to validate these study findings.

Crossing the health IT chasm: considerations and policy recommendations to overcome current challenges and enable value-based care

While great progress has been made in digitizing the US health care system, today's health information technology (IT) infrastructure remains largely a collection of systems that are not designed to support a transition to value-based care. In addition, the pursuit of value-based care, in which we deliver better care with better outcomes at lower cost, places new demands on the health care system that our IT infrastructure needs to be able to support. Provider organizations pursuing new models of health care delivery and payment are finding that their electronic systems lack the capabilities needed to succeed. The result is a chasm between the current health IT ecosystem and the health IT ecosystem that is desperately needed.In this paper, we identify a set of focal goals and associated near-term achievable actions that are critical to pursue in order to enable the health IT ecosystem to meet the acute needs of modern health care delivery. These ideas emerged from discussions that occurred during the 2015 American Medical Informatics Association Policy Invitational Meeting. To illustrate the chasm and motivate our recommendations, we created a vignette from the multistakeholder perspectives of a patient, his provider, and researchers/innovators. It describes an idealized scenario in which each stakeholder's needs are supported by an integrated health IT environment. We identify the gaps preventing such a reality today and present associated policy recommendations that serve as a blueprint for critical actions that would enable us to cross the current health IT chasm by leveraging systems and information to routinely deliver high-value care.

Accuracy of Billing Codes Used in the Therapeutic Care of Diabetic Retinopathy

Importance

Insurance billing claim databases represent a growing field of scientific inquiry within ophthalmology. Validating the accuracy of billing claim codes used during the care of diabetic retinopathy is a necessary precursor to fully understanding the underlying data and subsequent results of these types of studies.

Objective

To determine the accuracy of diagnostic, procedural, and therapeutic billing codes used in the treatment of diabetic retinopathy.

Design, Setting, and Participants

This retrospective medical record review was conducted at 3 clinical practices (1 academic and 2 private). Insured patients with diabetic retinopathy were seen by the practices between 2011 and 2013. Each patient then had every visit for 2 years reviewed twice, once for billing data and the second for data from the medical record. Data were collected and analyzed from October 2015 to July 2016.

Main Outcomes and Measures

The positive predictive value (PPV) and negative predictive value (NPV) for each code of interest. Sensitivity and specificity were secondary outcomes.

Results

A total of 146 patients (mean [SD] age, 60.3 [12.5] years) from 11 physicians had 1072 encounters reviewed over 2 calendar years. Among the included patients, 49.3% were female (n = 72), 48.6% were white (n = 71), 37.0% were black (n = 54), and 18.5% had type 1 diabetes and a mean (SD) hemoglobin A1C level of 7.7% (1.8) (n = 27). Nearly all codes of interest that were used frequently also had a high PPV (range, 89.5%-100%) and NPV (88.6%-100%) including billing codes for intravitreal injection, focal laser, panretinal photocoagulation, laterality of procedure, ranibizumab, bevacizumab, fundus photographs, fluorescein angiography, and optical coherence tomography. Codes that were used infrequently (<20 instances) but still had a high PPV (all 100%) and NPV (99.7%-100%) were codes for aflibercept, triamcinolone, and the dexamethasone implant. Only the codes for infrequently used B-scan ultrasonography (PPV, 69.6%) and subtenon injection (PPV, 100%; NPV, 99.7%, but sensitivity of only 40%) were found to be of questionable accuracy. Other than subtenon injection (40%), all codes were also found to have a high sensitivity (range, 87.6%-100%) and a high specificity (range, 97.2%-100%).

Conclusions and Relevance

These data suggest diagnostic, procedure, and therapeutic codes derived from insurance billing claims accurately reflect the medical record for patients with diabetic retinopathy.