Comparison of methodologies for calculating quality measures based on administrative data versus clinical data from an electronic health record system: implications for performance measures

The Development of the Municipal Registry of People with Diabetes in Porto Alegre, Brazil

Background/Objective: Diabetes registries that enhance surveillance and improve medical care are uncommon in low- and middle-income countries, where most of the diabetes burden lies. We aimed to describe the methodological and technical aspects adopted in the development of a municipal registry of people with diabetes using local and national Brazilian National Health System databases. Methods: We obtained data between July 2018 and June 2021 based on eight databases covering primary care, specialty and emergency consultations, medication dispensing, outpatient exam management, hospitalizations, and deaths. We identified diabetes using the International Classification of Disease (ICD), International Classification of Primary Care (ICPC), medications for diabetes, hospital codes for the treatment of diabetes complications, and exams for diabetes management. Results: After data processing and database merging using deterministic and probabilistic linkage, we identified 73,185 people with diabetes. Considering that 1.33 million people live in Porto Alegre, the registry captured 5.5% of the population. Conclusions: With additional data processing, the registry can reveal information on the treatment and outcomes of people with diabetes who are receiving publicly financed care in Porto Alegre. It will provide metrics for epidemiologic surveillance, such as the incidence, prevalence, rates, and trends of complications and causes of mortality; identify inadequacies; and provide information. It will enable healthcare providers to monitor the quality of care, identify inadequacies, and provide feedback as needed.

The shaky foundations of large language models and foundation models for electronic health records

The success of foundation models such as ChatGPT and AlphaFold has spurred significant interest in building similar models for electronic medical records (EMRs) to improve patient care and hospital operations. However, recent hype has obscured critical gaps in our understanding of these models' capabilities. In this narrative review, we examine 84 foundation models trained on non-imaging EMR data (i.e., clinical text and/or structured data) and create a taxonomy delineating their architectures, training data, and potential use cases. We find that most models are trained on small, narrowly-scoped clinical datasets (e.g., MIMIC-III) or broad, public biomedical corpora (e.g., PubMed) and are evaluated on tasks that do not provide meaningful insights on their usefulness to health systems. Considering these findings, we propose an improved evaluation framework for measuring the benefits of clinical foundation models that is more closely grounded to metrics that matter in healthcare.

The relationship of hospital and surgeon volume indicators and post-operative outcomes in pancreatic surgery: a systematic literature review, meta-analysis and guidance for valid outcome assessment

BACKGROUND

Available evidence on the volume-outcome relationship after pancreatic surgery is limited due to the narrow focus of interventions, volume indicators and outcomes considered as well as due to methodological differences of the included studies. Therefore, we aim to evaluate the volume-outcome relationship following pancreatic surgery following strict study selection and quality criteria, to identify aspects of methodological variation and to define a set of key methodological indicators to consider when aiming for comparable and valid outcome assessment.

METHODS

Four electronic databases were searched to identify studies on the volume-outcome relationship in pancreatic surgery published between the years 2000-2018. Following a double-screening process, data extraction, quality appraisal, and subgroup analysis, results of included studies were stratified and pooled using random effects meta-analysis.

RESULTS

Consistent associations were found between high hospital volume and both postoperative mortality (OR 0.35, 95% CI: 0.29-0.44) and major complications (OR 0.87, 95% CI: 0.80-0.94). A significant decrease in the odds ratio was also found for high surgeon volume and postoperative mortality (OR 0.29, 95%CI: 0.22-0.37).

DISCUSSION

Our meta-analysis confirms a positive effect for both hospital and surgeon volume indicators for pancreatic surgery. Further harmonization (e.g. surgery types, volume cut-offs/definition, case-mix adjustment, reported outcomes) are recommended for future empirical studies.

Optimization of a Quality Improvement Tool for Cancer Diagnosis in Primary Care: Qualitative Study

Background

The most common route to a diagnosis of cancer is through primary care. Delays in diagnosing cancer occur when an opportunity to make a timely diagnosis is missed and is evidenced by patients visiting the general practitioner (GP) on multiple occasions before referral to a specialist. Tools that minimize prolonged diagnostic intervals and reduce missed opportunities to investigate patients for cancer are therefore a priority.

Objective

This study aims to explore the usefulness and feasibility of a novel quality improvement (QI) tool in which algorithms flag abnormal test results that may be indicative of undiagnosed cancer. This study allows for the optimization of the cancer recommendations before testing the efficacy in a randomized controlled trial.

Methods

GPs, practice nurses, practice managers, and consumers were recruited to participate in individual interviews or focus groups. Participants were purposively sampled as part of a pilot and feasibility study, in which primary care practices were receiving recommendations relating to the follow-up of abnormal test results for prostate-specific antigen, thrombocytosis, and iron-deficiency anemia. The Clinical Performance Feedback Intervention Theory (CP-FIT) was applied to the analysis using a thematic approach.

Results

A total of 17 interviews and 3 focus groups (n=18) were completed. Participant themes were mapped to CP-FIT across the constructs of context, recipient, and feedback variables. The key facilitators to use were alignment with workflow, recognized need, the perceived importance of the clinical topic, and the GPs’ perception that the recommendations were within their control. Barriers to use included competing priorities, usability and complexity of the recommendations, and knowledge of the clinical topic. There was consistency between consumer and practitioner perspectives, reporting language concerns associated with the word cancer, the need for more patient-facing resources, and time constraints of the consultation to address patients’ worries.

Conclusions

There was a recognized need for the QI tool to support the diagnosis of cancer in primary care, but barriers were identified that hindered the usability and actionability of the recommendations in practice. In response, the tool has been refined and is currently being evaluated as part of a randomized controlled trial. Successful and effective implementation of this QI tool could support the detection of patients at risk of undiagnosed cancer in primary care and assist in preventing unnecessary delays.

Data Quality of Automated Comorbidity Lists in Patients With Mental Health and Substance Use Disorders

EHRs provide an opportunity to conduct research on underrepresented oncology populations with mental health and substance use disorders. However, a lack of data quality may introduce unintended bias into EHR data. The objective of this article is describe our analysis of data quality within automated comorbidity lists commonly found in EHRs. Investigators conducted a retrospective chart review of 395 oncology patients from a safety-net integrated healthcare system. Statistical analysis included κ coefficients and a condition logistic regression. Subjects were racially and ethnically diverse and predominantly used Medicaid insurance. Weak κ coefficients ( κ = 0.2-0.39, P < .01) were noted for drug and alcohol use disorders indicating deficiencies in comorbidity documentation within the automated comorbidity list. Further, conditional logistic regression analyses revealed deficiencies in comorbidity documentation in patients with drug use disorders (odds ratio, 11.03; 95% confidence interval, 2.71-44.9; P = .01) and psychoses (odds ratio, 0.04; confidence interval, 0.02-0.10; P < .01). Findings suggest deficiencies in automatic comorbidity lists as compared with a review of provider narrative notes when identifying comorbidities. As healthcare systems increasingly use EHR data in clinical studies and decision making, the quality of healthcare delivery and clinical research may be affected by discrepancies in the documentation of comorbidities.

Uncovering interpretable potential confounders in electronic medical records

Randomized clinical trials (RCT) are the gold standard for informing treatment decisions. Observational studies are often plagued by selection bias, and expert-selected covariates may insufficiently adjust for confounding. We explore how unstructured clinical text can be used to reduce selection bias and improve medical practice. We develop a framework based on natural language processing to uncover interpretable potential confounders from text. We validate our method by comparing the estimated hazard ratio (HR) with and without the confounders against established RCTs. We apply our method to four cohorts built from localized prostate and lung cancer datasets from the Stanford Cancer Institute and show that our method shifts the HR estimate towards the RCT results. The uncovered terms can also be interpreted by oncologists for clinical insights. We present this proof-of-concept study to enable more credible causal inference using observational data, uncover meaningful insights from clinical text, and inform high-stakes medical decisions.

Expert Consensus on Currently Accepted Measures of Harm

BACKGROUND

Twenty-five years after the seminal work of the Harvard Medical Practice Study, the numbers and specific types of health care measures of harm have evolved and expanded. Using the World Café method to derive expert consensus, we sought to generate a contemporary list of triggers and adverse event measures that could be used for chart review to determine the current incidence of inpatient and outpatient adverse events.

METHODS

We held a modified World Café event in March 2018, during which content experts were divided into 10 tables by clinical domain. After a focused discussion of a prepopulated list of literature-based triggers and measures relevant to that domain, they were asked to rate each measure on clinical importance and suitability for chart review and electronic extraction (very low, low, medium, high, very high).

RESULTS

Seventy-one experts from 9 diverse institutions attended (primary acceptance rate, 72%). Of 525 total triggers and measures, 67% of 391 measures and 46% of 134 triggers were deemed to have high or very high clinical importance. For those triggers and measures with high or very high clinical importance, 218 overall were deemed to be highly amenable to chart review and 198 overall were deemed to be suitable for electronic surveillance.

CONCLUSIONS

The World Café method effectively prioritized measures/triggers of high clinical importance including those that can be used in chart review, which is considered the gold standard. A future goal is to validate these measures using electronic surveillance mechanisms to decrease the need for chart review.

Clinical data sharing improves quality measurement and patient safety

OBJECTIVE

Accurate and robust quality measurement is critical to the future of value-based care. Having incomplete information when calculating quality measures can cause inaccuracies in reported patient outcomes. This research examines how quality calculations vary when using data from an individual electronic health record (EHR) and longitudinal data from a health information exchange (HIE) operating as a multisource registry for quality measurement.

MATERIALS AND METHODS

Data were sampled from 53 healthcare organizations in 2018. Organizations represented both ambulatory care practices and health systems participating in the state of Kansas HIE. Fourteen ambulatory quality measures for 5300 patients were calculated using the data from an individual EHR source and contrasted to calculations when HIE data were added to locally recorded data.

RESULTS

A total of 79% of patients received care at more than 1 facility during the 2018 calendar year. A total of 12 994 applicable quality measure calculations were compared using data from the originating organization vs longitudinal data from the HIE. A total of 15% of all quality measure calculations changed (P < .001) when including HIE data sources, affecting 19% of patients. Changes in quality measure calculations were observed across measures and organizations.

DISCUSSION

These results demonstrate that quality measures calculated using single-site EHR data may be limited by incomplete information. Effective data sharing significantly changes quality calculations, which affect healthcare payments, patient safety, and care quality.

CONCLUSIONS

Federal, state, and commercial programs that use quality measurement as part of reimbursement could promote more accurate and representative quality measurement through methods that increase clinical data sharing.

Role of social deprivation on asthma care quality among a cohort of children in US community health centres

OBJECTIVE

Social deprivation is associated with worse asthma outcomes. The Social Deprivation Index is a composite measure of social determinants of health used to identify neighbourhood-level disadvantage in healthcare. Our objective was to determine if higher neighbourhood-level social deprivation is associated with documented asthma care quality measures among children treated at community health centres (CHCs).

METHODS SETTING, PARTICIPANTS, OUTCOME MEASURES

We used data from CHCs in 15 states in the Accelerating Data Value Across a National Community Health Center Network (ADVANCE). The sample included 34 266 children with asthma from 2008 to 2017, aged 3-17 living in neighbourhoods with differing levels of social deprivation measured using quartiles of the Social Deprivation Index score. We conducted logistic regression to examine the odds of problem list documentation of asthma and asthma severity, and negative binomial regression for rates of albuterol, inhaled steroid and oral steroid prescription adjusted for patient-level covariates.

RESULTS

Children from the most deprived neighbourhoods had increased rates of albuterol (rate ratio (RR)=1.22, 95% CI 1.13 to 1.32) compared with those in the least deprived neighbourhoods, while the point estimate for inhaled steroids was higher, but fell just short of significance at the alpha=0.05 level (RR=1.16, 95% CI 0.99 to 1.34). We did not observe community-level differences in problem list documentation of asthma or asthma severity.

CONCLUSIONS

Higher neighbourhood-level social deprivation was associated with more albuterol and inhaled steroid prescriptions among children with asthma, while problem list documentation of asthma and asthma severity varied little across neighbourhoods with differing deprivation scores. While the homogeneity of the CHC safety net setting studied may mitigate variation in diagnosis and documentation of asthma, enhanced clinician awareness of differences in community risk could help target paediatric patients at risk of lower quality asthma care.

Comparing the Predictive Effects of Patient Medication Adherence Indices in Electronic Health Record and Claims-Based Risk Stratification Models

Multiple indices are available to measure medication adherence behaviors. Medication adherence measures, however, have rarely been extracted from electronic health records (EHRs) for population-level risk predictions. This study assessed the value of medication adherence indices in improving predictive models of cost and hospitalization. This study included a 2-year retrospective cohort of patients younger than age 65 years with linked EHR and insurance claims data. Three medication adherence measures were calculated: medication regimen complexity index (MRCI), medication possession ratio (MPR), and prescription fill rate (PFR). The authors examined the effects of adding these measures to 3 predictive models of utilization: a demographics model, a conventional model (Charlson index), and an advanced diagnosis-based model. Models were trained using EHR and claims data. The study population had an overall MRCI, MPR, and PFR of 14.6 ± 17.8, .624 ± .310, and .810 ± .270, respectively. Adding MRCI and MPR to the demographic and the morbidity models using claims data improved forecasting of next-year hospitalization substantially (eg, AUC of the demographic model increased from .605 to .656 using MRCI). Nonetheless, such boosting effects were attenuated for the advanced diagnosis-based models. Although EHR models performed inferior to claims models, adding adherence indices improved EHR model performances at a larger scale (eg, adding MRCI increased AUC by 4.4% for the Charlson model using EHR data compared to 3.8% using claims). This study shows that medication adherence measures can modestly improve EHR- and claims-derived predictive models of cost and hospitalization in non-elderly patients; however, the improvements are minimal for advanced diagnosis-based models.

Indicators of guideline-concordant care in lung cancer defined with a modified Delphi method and piloted in a cohort of over 5,800 cases

BACKGROUND

To identify indicators of guideline-concordant care in lung cancer, to implement such indicators with cancer registry data linked to health databases, and to pilot them in a cohort of patients from the cancer registry of the Milan Province.

METHODS

Thirty-four indicators were selected by revision of main guidelines by cancer epidemiologists, and then evaluated by a multidisciplinary panel of clinicians involved in lung cancer care and working on the pathway of lung cancer diagnosis and treatment in the Lombardy region, Italy. With a modified Delphi method, they assessed for each indicator the content validity as a quality measure of the care pathway, the degree of modifiability from the health professional, and the relevance to the health professional. Feasibility was assessed using the cancer registry and the routine health records of the Lombardy region. Feasible indicators were then calculated in the cohort of lung cancer patients diagnosed in 2007-2012 derived from the cancer registry of the Milan Province. Criterion validity was assessed reviewing clinical records of a random sample of 114 patients (threshold for acceptable discordance ≤20%). Finally, reliability was evaluated at the provider level.

RESULTS

Initially, 34 indicators were proposed for evaluation in the first Delphi round. Of the finally 22 selected indicators, 3 were not feasible because the required information was actually not available. The remaining 19 were calculated on the pilot cohort. After assessment of criterion validity (3 eliminated), 16 indicators were retained in the final set and evaluated for reliability.

CONCLUSION

The developed and piloted set of indicators is now available to implement and monitor, over time, quality initiatives for lung cancer care in the studied health system.

Personalized treatment options for chronic diseases using precision cohort analytics

To support point-of-care decision making by presenting outcomes of past treatment choices for cohorts of similar patients based on observational data from electronic health records (EHRs), a machine-learning precision cohort treatment option (PCTO) workflow consisting of (1) data extraction, (2) similarity model training, (3) precision cohort identification, and (4) treatment options analysis was developed. The similarity model is used to dynamically create a cohort of similar patients, to inform clinical decisions about an individual patient. The workflow was implemented using EHR data from a large health care provider for three different highly prevalent chronic diseases: hypertension (HTN), type 2 diabetes mellitus (T2DM), and hyperlipidemia (HL). A retrospective analysis demonstrated that treatment options with better outcomes were available for a majority of cases (75%, 74%, 85% for HTN, T2DM, HL, respectively). The models for HTN and T2DM were deployed in a pilot study with primary care physicians using it during clinic visits. A novel data-analytic workflow was developed to create patient-similarity models that dynamically generate personalized treatment insights at the point-of-care. By leveraging both knowledge-driven treatment guidelines and data-driven EHR data, physicians can incorporate real-world evidence in their medical decision-making process when considering treatment options for individual patients.

Validation of an Optimized Algorithm for Identifying Persons Living With Diagnosed HIV From New York State Medicaid Data, 2006-2014

Algorithms are regularly used to identify persons living with diagnosed human immunodeficiency virus (HIV) (PLWDH) in Medicaid data. To our knowledge, there are no published reports of an HIV algorithm from Medicaid claims codes that have been compared with an HIV surveillance system to assess its sensitivity, specificity, positive predictive value, and negative predictive value in identifying PLWDH. Therefore, our aims in this study were to 1) develop an algorithm that could identify PLWDH in New York State Medicaid data from 2006-2014 and 2) validate this algorithm using the New York State HIV surveillance system. Classification and regression tree analysis identified 16 nodes that we combined to create a case-finding algorithm with 5 criteria. This algorithm identified 86,930 presumed PLWDH, 88.0% of which were verified by matching to the surveillance system. The algorithm yielded a sensitivity of 94.5%, a specificity of 94.4%, a positive predictive value of 88.0%, and a negative predictive value of 97.6%. This validated algorithm has the potential to improve the utility of Medicaid data for assessing health outcomes and programmatic interventions.

Impact of the diabetes Canada guideline dissemination strategy on dispensed vascular protective medications for older patients in Ontario, Canada: a linked EMR and administrative data study

BACKGROUND

The 2013 Diabetes Canada guidelines recommended routinely using vascular protective medications for most patients with diabetes. These medications included statins and angiotensin-converting enzyme inhibitors (ACEIs) or angiotensin receptor blockers (ARBs). Antiplatelet agents were only recommended for secondary prevention of cardiovascular disease. Using Electronic Medical Record (EMR) data, we previously found that guideline dissemination efforts were not associated with an increase in the rate of primary care prescriptions of these medications. However, this needs confirmation: patients can receive prescriptions from different sources including specialists and they may not always fill these prescriptions. Using both EMR and administrative health data, we examined whether guideline dissemination impacted the dispensing of vascular protective medications to patients.

METHODS

The study population included patients with diabetes aged 66 or over in Ontario, Canada. We created two cohorts using two different approaches: an Electronic Medical Record (EMR) algorithm for diabetes using linked EMR-administrative data and an administrative algorithm using population level administrative data. We examined data from January 2010 to December 2016. Patients with diabetes were deemed to be likely taking a medication (or covered) during a quarter if the daily amount for a dispensed medication would last for at least 75% of days in any given quarter. An interrupted time series analysis was used to assess the proportion of patients covered by each medication class. Proton pump inhibitors (PPIs) were used as a reference.

RESULTS

There was no increase in the rate of change for medication coverage following guideline release in either the EMR or the administrative diabetes cohorts. For statins, the change in trend was - 0.03, p = 0.7 (EMR) and - 0.12, p = 0.04(administrative). For ACEI/ARBs, this was 0.03, p = 0.6 (EMR) and 0, p = 1(administrative). For antiplatelets, this was 0.001, P = .97 (EMR) and - 0.03, p = 0.03 (administrative). The comparator PPI was - 0.07, p = 0.4 (EMR) and - 0.11, p = 0.002 (administrative).

CONCLUSIONS

Using both EMR and administrative health data, we confirmed that the Diabetes Canada 2013 guideline dissemination strategy did not lead to an increased rate of coverage for vascular protective medications. Alternative strategies are needed to effect change in practice.

Veterans Health Administration Primary Care Provider Adherence to Prescribing Guidelines for Systemic Hormone Therapy in Menopausal Women

BACKGROUND

Systemic hormone therapy (HT) is effective for treating menopausal symptoms but also confers risks. Therefore, experts have developed clinical guidelines for its use.

PURPOSE

We assessed primary care guideline adherence in prescribing systemic HT, and associations between adherence and provider characteristics, in four Veterans Health Administration (VA) facilities.

METHODS

We abstracted medical records associated with new and renewal systemic HT prescriptions examining adherence to guidelines for documenting indications and contraindications; prescribing appropriate dosages; and prescribing progesterone.

RESULTS

Average guideline adherence was 58%. Among new prescriptions, 74% documented a guideline-adherent indication and 28% documented absence of contraindications. Among renewals, 39% documented a guideline-adherent indication. In prescribing an appropriate dose, 45% of new prescriptions were guideline-adherent. Among renewal prescriptions with conjugated equine estrogen doses ≥0.625 mg or equivalent, 16% documented the dosing rationale. Among 116 prescriptions for systemic estrogen in women with a uterus, progesterone was not prescribed in 8.

CONCLUSIONS

Guideline adherence in prescribing systemic HT was low among VA primary care providers. Failures to coprescribe progesterone put women at increased risk for endometrial cancer.

IMPLICATIONS

Intervention development is urgently needed to improve guideline adherence among primary care prescribers of systemic HT for menopause. Similar assessments should be conducted in community settings.

Estimating misclassification error in a binary performance indicator: case study of low value care in Australian hospitals

OBJECTIVE

Indicators based on hospital administrative data have potential for misclassification error, especially if they rely on clinical detail that may not be well recorded in the data. We applied an approach using modified logistic regression models to assess the misclassification (false-positive and false-negative) rates of low-value care indicators.

DESIGN AND SETTING

We applied indicators involving 19 procedures to an extract from the New South Wales Admitted Patient Data Collection (1 January 2012 to 30 June 2015) to label episodes as low value. We fit four models (no misclassification, false-positive only, false-negative only, both false-positive and false-negative) for each indicator to estimate misclassification rates and used the posterior probabilities of the models to assess which model fit best.

RESULTS

False-positive rates were low for most indicators-if the indicator labels care as low value, the care is most likely truly low value according to the relevant recommendation. False-negative rates were much higher but were poorly estimated (wide credible intervals). For most indicators, the models allowing no misclassification or allowing false-negatives but no false-positives had the highest posterior probability. The overall low-value care rate from the indicators was 12%. After adjusting for the estimated misclassification rates from the highest probability models, this increased to 35%.

CONCLUSION

Binary performance indicators have a potential for misclassification error, especially if they depend on clinical information extracted from administrative data. Indicators should be validated by chart review, but this is resource-intensive and costly. The modelling approach presented here can be used as an initial validation step to identify and revise indicators that may have issues before continuing to a full chart review validation.

Combining population-based administrative health records and electronic medical records for disease surveillance

BACKGROUND

Administrative health records (AHRs) and electronic medical records (EMRs) are two key sources of population-based data for disease surveillance, but misclassification errors in the data can bias disease estimates. Methods that combine information from error-prone data sources can build on the strengths of AHRs and EMRs. We compared bias and error for four data-combining methods and applied them to estimate hypertension prevalence.

METHODS

Our study included rule-based OR and AND methods that identify disease cases from either or both data sources, respectively, rule-based sensitivity-specificity adjusted (RSSA) method that corrects for inaccuracies using a deterministic rule, and probabilistic-based sensitivity-specificity adjusted (PSSA) method that corrects for error using a statistical model. Computer simulation was used to estimate relative bias (RB) and mean square error (MSE) under varying conditions of population disease prevalence, correlation amongst data sources, and amount of misclassification error. AHRs and EMRs for Manitoba, Canada were used to estimate hypertension prevalence using validated case definitions and multiple disease markers.

RESULTS

The OR method had the lowest RB and MSE when population disease prevalence was 10%, and the RSSA method had the lowest RB and MSE when population prevalence increased to 20%. As the correlation between data sources increased, the OR method resulted in the lowest RB and MSE. Estimates of hypertension prevalence for AHRs and EMRs alone were 30.9% (95% CI: 30.6-31.2) and 24.9% (95% CI: 24.6-25.2), respectively. The estimates were 21.4% (95% CI: 21.1-21.7), for the AND method, 34.4% (95% CI: 34.1-34.8) for the OR method, 32.2% (95% CI: 31.8-32.6) for the RSSA method, and ranged from 34.3% (95% CI: 34.1-34.5) to 35.9% (95% CI, 35.7-36.1) for the PSSA method, depending on the statistical model.

CONCLUSIONS

The OR and AND methods are influenced by correlation amongst the data sources, while the RSSA method is dependent on the accuracy of prior sensitivity and specificity estimates. The PSSA method performed well when population prevalence was high and average correlations amongst disease markers was low. This study will guide researchers to select a data-combining method that best suits their data characteristics.

Three Different Databases, Three Different Complication Rates for Knee and Hip Arthroplasty: Comparing the National Inpatient Sample, National Hospital Discharge Survey, and National Surgical Quality Improvement Program, 2006 to 2010

BACKGROUND

National databases are increasingly used to research complication rates, risk factors, and the role of comorbidities. Three commonly used databases are the Healthcare Cost and Utilization Program's National Inpatient Sample (NIS), the National Hospital Discharge Survey (NHDS), and the American College of Surgeons National Surgical Quality Improvement Program (NSQIP). Despite many publications, the accuracy of results from these databases remains unclear.

METHODS

We compared demographics and complication rates of primary total hip arthroplasty (THA) and primary total knee arthroplasty (TKA) across three national databases from 2006 to 2010. Using International Classification of Diseases, Ninth Revision, Clinical Modification and Current Procedural Terminology codes to identify cases, we calculated postoperative inpatient complication rates in all three databases and 30-day complication rates in the NSQIP.

RESULTS

We identified a total of 607,322 TKAs and 279,428 THAs. Overall complication rates varied greatly between the databases. For TKA, the overall complication rates were the highest in the NIS (17.3% [16.6 to 18.0]), followed by the NHDS (14.9% [14.0 to 15.8]), and then the NSQIP 30 days (10.20% [9.73 to 10.70]) and the NSQIP until discharge (7.34% [6.95 to 7.75]). Similarly, for THA, the NIS was the highest (24.09% [23.05 to 25.16]), and then the NHDS (21.5% [19.8 to 23.2]), followed by the NSQIP 30 days (12.00% [11.31 to 12.72]), and the NSQIP until discharge (9.25% [8.64 to 9.90]). Breakdown and comparison of individual adverse events further revealed different complication rates.

CONCLUSION

The estimated complication rates from THA and TKA depend on which data source is used because of differences in data collection and sampling methodology. Similar differences may exist in other publications that use such secondary data sources.

The qualculative dimension of healthcare data interoperability

Research on interoperability and information exchange between information technology systems touts the use of secondary data for a variety of purposes, including research, management, quality improvement, and accountability. However, many studies have pointed out that this is difficult to achieve in practice. Hence, this article aims to examine the causes for this by reporting an ethnographic study of the data work performed by medical records coders and birth certificate clerks working in a hospital system to uncover the practices of creating administrative data (e.g. secondary data). The article illustrates that clerks and coders use situated qualitative judgments of the accuracy and authority of different primary medical accounts. Coders and clerks also employ their understandings of the importance of different future uses of data as they make crucial decisions about how much discretion to exercise in producing accurate data and how much effort to put toward clarifying problematic medical data. These findings suggest that information technology systems designed for interoperability and secondary data also need to be designed in ways that support the qualculative practices of data workers in order to succeed, including making future uses of data clear to data workers and finding ways to minimize conflicting data before data workers encounter it.

The Development of a Brief Version of the Posttraumatic Cognitions Inventory (PTCI-9)

Negative posttraumatic cognitions lead to the development and maintenance of posttraumatic stress disorder symptoms. There is a need for a brief measure to assess these cognitions. Participants were administered the Posttraumatic Cognitions Inventory (PTCI) and measures of mental health symptomatology. These data were used to develop a brief version of the PTCI (PTCI-9) in 223 male and female veterans, which was then examined in a sample of 117 female civilians. Confirmatory factor analyses demonstrated an acceptable fit in both samples. The PTCI-9 total and subscale scores showed strong internal consistencies (Cronbach's αs = .80-.87) and strong correlations with the PTCI in veterans ( rs = .90-.96) and civilians ( rs = .91-.96). Measurement invariance testing demonstrated partial invariance between the two samples. The PTCI-9 significantly correlated with measures of PTSD, depression, and quality of life. These findings demonstrate that the PTCI-9 is a reliable and valid measure of posttraumatic cognitions that can reduce patient and provider burden.

Guideline Adherence of Veterans Health Administration Primary Care for Abnormal Uterine Bleeding

BACKGROUND

Abnormal uterine bleeding (AUB) is common among primary care patients. We assessed the extent to which Veterans Health Administration (VA) primary care patients with AUB are receiving guideline-adherent primary care.

METHODS

We identified women with AUB presenting to primary care providers across four VA health care systems from June 2013 to September 2015. We performed a structured abstraction of electronic medical record data for 15 indicators of guideline-adherent AUB care. We determined whether documented care was guideline-adherent and compared adherence of care by primary care providers by VA Designated Women's Health Provider status and by volume of clinical encounters with women veterans.

RESULTS

Across 305 episodes of AUB, 53% of the care was guideline adherent. There was high adherence with documenting menopausal status (98%), ordering diagnostic studies and referrals for postmenopausal women (92%), and documenting bleeding patterns (87%). There was lower adherence with documenting whether there was active bleeding (55%), performing thyroid testing (47%), performing a pelvic examination (42%), ordering diagnostic studies and referrals in younger women with increased endometrial cancer risk (40%), assessing for pregnancy (32%), assessing for cervical motion, uterine, or adnexal tenderness in patients with intrauterine devices (30%), and assessing for elevated endometrial cancer risk (6%). There were no significant differences in overall guideline adherence between primary care providers who were, versus were not, VA Designated Women's Health Providers, or by provider volume of encounters with women veterans.

CONCLUSIONS

VA primary care has high guideline adherence when caring for postmenopausal women with AUB. Quality improvement and educational initiatives are needed to improve primary care for AUB in younger women veterans.

An Organization-Specific and Modifiable Inpatient Safety Composite Measure

In early 2013, seeking to apply the principles of value-based purchasing to all Kaiser Permanente hospitals as part of an existing organizationwide value-based performance incentive plan, Kaiser Permanente developed an inpatient safety composite measure that tracks hospital-level performance improvement related to 10 key inpatient safety events. The elements of the composite are weighted equally, and the tool draws on scoring methodologies used by the National Committee for Quality Assurance and the Centers for Medicare & Medicaid Services Hospital Inpatient Value-Based Purchasing Program. Two years after implementation of the composite measure, hospitals experienced improvement across 9 of the 10 adverse events assessed, though only one improvement achieved statistical significance. The measure successfully distinguishes four levels of improvement and is broadly applicable to hospitals and hospital systems.

Detecting moderate or complex congenital heart defects in adults from an electronic health records system

Background

The prevalence of moderate or complex (moderate-complex) congenital heart defects (CHDs) among adults is increasing due to improved survival, but many patients experience lapses in specialty care or their CHDs are undocumented in the medical system. There is, to date, no efficient approach to identify this population.

Objective

To develop and assess the performance of a risk score to identify adults aged 20-60 years with undocumented specific moderate-complex CHDs from electronic health records (EHR).

Methods

We used a case-control study (596 adults with specific moderate-complex CHDs and 2384 controls). We extracted age, race/ethnicity, electrocardiogram (EKG), and blood tests from routine outpatient visits (1/2009 through 12/2012). We used multivariable logistic regression models and a split-sample (4: 1 ratio) approach to develop and internally validate the risk score, respectively. We generated receiver operating characteristic (ROC) c-statistics and Brier scores to assess the ability of models to predict the presence of specific moderate-complex CHDs.

Results

Out of six models, the non-blood biomarker model that included age, sex, and EKG parameters offered a high ROC c-statistic of 0.96 [95% confidence interval: 0.95, 0.97] and low Brier score (0.05) relative to the other models. The adult moderate-complex congenital heart defect risk score demonstrated good accuracy with 96.4% sensitivity and 80.0% specificity at a threshold score of 10.

Conclusions

A simple risk score based on age, sex, and EKG parameters offers early proof of concept and may help accurately identify adults with specific moderate-complex CHDs from routine EHR systems who may benefit from specialty care.

Comparing Quality of Care in Veterans Affairs and Non-Veterans Affairs Settings

BACKGROUND

Congress, veterans' groups, and the press have expressed concerns that access to care and quality of care in Department of Veterans Affairs (VA) settings are inferior to access and quality in non-VA settings.

OBJECTIVE

To assess quality of outpatient and inpatient care in VA at the national level and facility level and to compare performance between VA and non-VA settings using recent performance measure data.

MAIN MEASURES

We assessed Patient Safety Indicators (PSIs), 30-day risk-standardized mortality and readmission measures, and ORYX measures for inpatient safety and effectiveness; Healthcare Effectiveness Data and Information Set (HEDIS®) measures for outpatient effectiveness; and Consumer Assessment of Healthcare Providers and Systems Hospital Survey (HCAHPS) and Survey of Healthcare Experiences of Patients (SHEP) survey measures for inpatient patient-centeredness. For inpatient care, we used propensity score matching to identify a subset of non-VA hospitals that were comparable to VA hospitals.

KEY RESULTS

VA hospitals performed on average the same as or significantly better than non-VA hospitals on all six measures of inpatient safety, all three inpatient mortality measures, and 12 inpatient effectiveness measures, but significantly worse than non-VA hospitals on three readmission measures and two effectiveness measures. The performance of VA facilities was significantly better than commercial HMOs and Medicaid HMOs for all 16 outpatient effectiveness measures and for Medicare HMOs, it was significantly better for 14 measures and did not differ for two measures. High variation across VA facilities in the performance of some quality measures was observed, although variation was even greater among non-VA facilities.

CONCLUSIONS

The VA system performed similarly or better than the non-VA system on most of the nationally recognized measures of inpatient and outpatient care quality, but high variation across VA facilities indicates a need for targeted quality improvement.

Performance on Electronic Clinical Quality Measures in the Comprehensive Primary Care Initiative

Electronic clinical quality measures (eCQMs) that capture data from electronic health records promise accurate and timely measurement, but their use has been limited in payment. The Comprehensive Primary Care initiative sponsored by the Centers for Medicare & Medicaid Services used eCQMs as part of a shared savings incentive. To assess performance, the authors developed benchmarks for 11 measures and compared performance of initiative and benchmark practices. Initiative practices outperformed benchmark practices on 8 measures in 2015 (between 6.3 and 17.7 percentage points) and 9 measures in 2016 (between 1.7 and 20 percentage points). Initiative practices improved significantly on 7 measures from 2015 to 2016 (between 3.3 and 8.6 percentage points). For 3 measures, the improvement was greater than benchmark practices that reported the same measures in a 2-year period (between 1 and 8.9 percentage points). The authors conclude that eCQMs can be used for payment.

Assessing the validity of existing dental sealant quality measures

BACKGROUND

Although sealants are highly effective in preventing caries in children, placement rates continue to be low. The authors' goals were to implement and assess the performance of 2 existing sealant quality measures against a manual audit of charts at 4 dental institutions and to identify measurement gaps that may be filled by using data from electronic health records.

METHODS

The authors evaluated the performance of 2 quality measures designed for claims-based data: the Dental Quality Alliance (DQA) sealant measure, which includes patients at risk of developing elevated caries, and the Oregon Health Authority (OHA) sealant measure (irrespective of caries risk). The authors adapted and validated these measures at 4 sites: 3 dental schools and 1 large dental accountable care organization.

RESULTS

The overall modified DQA and modified OHA measure scores in the 6- through 9-year-old age group were 37.0% and 31.6% and in the 10- through 14-year-old age group were 15.8% and 6.6%, respectively. Results from the manual review of charts showed that 67.6% of children who did not receive sealants did not have any teeth to seal because their molars had not yet erupted, had been extracted, had been sealed previously, or had existing caries or restorations.

CONCLUSIONS

Both the DQA and OHA measures, which rely mainly on Current Dental Terminology procedure codes, led to underestimation of the care delivered from a practice perspective. Future sealant quality measures should exclude patients whose teeth cannot be sealed.

PRACTICAL IMPLICATIONS

This study's results support the suitability of using electronic health record data for assessing the quality of oral health care, particularly for measuring sealant placement in children.

Mapping the Gaps: Gender Differences in Preventive Cardiovascular Care among Managed Care Members in Four Metropolitan Areas

BACKGROUND

Prior research documents gender gaps in cardiovascular risk management, with women receiving poorer quality routine care on average, even in managed care systems. Although population health management tools and quality improvement efforts have led to better overall care quality and narrowing of racial/ethnic gaps for a variety of measures, we sought to quantify persistent gender gaps in cardiovascular risk management and to assess the performance of routinely used commercial population health management tools in helping systems narrow gender gaps.

METHODS

Using 2013 through 2014 claims and enrollment data from more than 1 million members of a large national health insurance plan, we assessed performance on seven evidence-based quality measures for the management of coronary artery disease and diabetes mellitus, a cardiac risk factor, across and within four metropolitan areas. We used logistic regression to adjust for region, demographics, and risk factors commonly tracked in population health management tools.

FINDINGS

Low-density lipoprotein (LDL) cholesterol control (LDL < 100 mg/dL) rates were 5 and 15 percentage points lower for women than men with diabetes mellitus (p < .0001), and coronary artery disease (p < .0001), respectively. Adjusted analyses showed women were more likely to have gaps in LDL control, with an odds ratio of 1.31 (95% confidence interval, 1.27-1.38) in diabetes mellitus and 1.88 (95% confidence interval, 1.65-2.10) in coronary artery disease.

CONCLUSIONS

Given our findings that gender gaps persist across both clinical and geographic variation, we identified additional steps health plans can take to reduce disparities. For measures where gaps have been consistently identified, we recommend that gender-stratified quality reporting and analysis be used to complement widely used algorithms to identify individuals with unmet needs for referral to population health and wellness behavior support programs.

Mining Electronic Health Records to Extract Patient-Centered Outcomes Following Prostate Cancer Treatment

The clinical, granular data in electronic health record (EHR) systems provide opportunities to improve patient care using informatics retrieval methods. However, it is well known that many methodological obstacles exist in accessing data within EHRs. In particular, clinical notes routinely stored in EHR are composed from narrative, highly unstructured and heterogeneous biomedical text. This inherent complexity hinders the ability to perform automated large-scale medical knowledge extraction tasks without the use of computational linguistics methods. The aim of this work was to develop and validate a Natural Language Processing (NLP) pipeline to detect important patient-centered outcomes (PCOs) as interpreted and documented by clinicians in their dictated notes for male patients receiving treatment for localized prostate cancer at an academic medical center.

Comparison of Electronic Health Record-Based and Claims-Based Diabetes Care Quality Measures: Causes of Discrepancies

OBJECTIVE

To investigate magnitude and sources of discrepancy in quality metrics using claims versus electronic health record (EHR) data.

STUDY DESIGN

Assessment of proportions of HbA1c and LDL testing for people ascertained as diabetic from the respective sources. Qualitative interviews and review of EHRs of discrepant cases.

DATA COLLECTION/EXTRACTION

Claims submitted to Rhode Island Medicaid by three practice sites in 2013; program-coded EHR extraction; manual review of selected EHRs.

PRINCIPAL FINDINGS

Of 21,030 adult Medicaid beneficiaries attributed to a primary care patient at a site by claims or EHR data, concordance on assignment ranged from 0.30 to 0.41. Of patients with concordant assignment, the ratio of patients ascertained as diabetic by EHR versus claims ranged from 1.06 to 1.14. For patients with concordant assignment and diagnosis, the ratio based on EHR versus claims ranged from 1.08 to 18.34 for HbA1c testing, and from 1.29 to 14.18 for lipid testing. Manual record review of 264 patients discrepant on diagnosis or testing identified problems such as misuse of ICD-9 codes, failure to submit claims, and others.

CONCLUSIONS

Claims data underestimate performance on these metrics compared to EHR documentation, by varying amounts. Use of claims data for these metrics is problematic.

Validation of asthma recording in electronic health records: a systematic review

Objective

To describe the methods used to validate asthma diagnoses in electronic health records and summarize the results of the validation studies.

Background

Electronic health records are increasingly being used for research on asthma to inform health services and health policy. Validation of the recording of asthma diagnoses in electronic health records is essential to use these databases for credible epidemiological asthma research.

Methods

We searched EMBASE and MEDLINE databases for studies that validated asthma diagnoses detected in electronic health records up to October 2016. Two reviewers independently assessed the full text against the predetermined inclusion criteria. Key data including author, year, data source, case definitions, reference standard, and validation statistics (including sensitivity, specificity, positive predictive value [PPV], and negative predictive value [NPV]) were summarized in two tables.

Results

Thirteen studies met the inclusion criteria. Most studies demonstrated a high validity using at least one case definition (PPV >80%). Ten studies used a manual validation as the reference standard; each had at least one case definition with a PPV of at least 63%, up to 100%. We also found two studies using a second independent database to validate asthma diagnoses. The PPVs of the best performing case definitions ranged from 46% to 58%. We found one study which used a questionnaire as the reference standard to validate a database case definition; the PPV of the case definition algorithm in this study was 89%.

Conclusion

Attaining high PPVs (>80%) is possible using each of the discussed validation methods. Identifying asthma cases in electronic health records is possible with high sensitivity, specificity or PPV, by combining multiple data sources, or by focusing on specific test measures. Studies testing a range of case definitions show wide variation in the validity of each definition, suggesting this may be important for obtaining asthma definitions with optimal validity.

Improving Performance on Preventive Health Quality Measures Using Clinical Decision Support to Capture Care Done Elsewhere and Patient Exceptions

Preventive services required for performance measurement often are completed in outside health systems and not captured in electronic medical records (EMRs). A before-after study was conducted to examine the ability of clinical decision support (CDS) to improve performance on preventive quality measures, capture clinician-reported services completed elsewhere, and patient/medical exceptions and to describe their impact on quality measurement. CDS improved performance on colorectal cancer screening, osteoporosis screening, and pneumococcal vaccination measures ( P < .05) but not breast or cervical cancer screening. CDS captured clinician-reported services completed elsewhere (2% to 10%) and patient/medical exceptions (<3%). Compared to measures using only within-system data, including services completed elsewhere in the numerator improved performance: pneumococcal vaccine (73% vs 82%); breast (69% vs 75%), colorectal (58% vs 70%), and cervical cancer (53% vs 62%); and osteoporosis (72% vs 75%) screening ( P < .05). Visit-based CDS can capture clinician-reported preventive services, and accounting for services completed elsewhere improves performance on quality measures.

Mortality Risk Factors Among Patients With Cirrhosis and a Low Model for End-Stage Liver Disease Sodium Score (≤15): An Analysis of Liver Transplant Allocation Policy Using Aggregated Electronic Health Record Data

Although the Model for End-Stage Liver Disease sodium (MELD Na) score is now used for liver transplant allocation in the United States, mortality prediction may be underestimated by the score. Using aggregated electronic health record data from 7834 adult patients with cirrhosis, we determined whether the cause of cirrhosis or cirrhosis complications was associated with an increased risk of death among patients with a MELD Na score ≤15 and whether patients with the greatest risk of death could benefit from liver transplantation (LT). Over median follow-up of 2.3 years, 3715 patients had a maximum MELD Na score ≤15. Overall, 3.4% were waitlisted for LT. Severe hypoalbuminemia, hepatorenal syndrome, and hepatic hydrothorax conferred the greatest risk of death independent of MELD Na score with 1-year predicted mortality >14%. Approximately 10% possessed these risk factors. Of these high-risk patients, only 4% were waitlisted for LT, despite no difference in nonliver comorbidities between waitlisted patients and those not listed. In addition, risk factors for death among waitlisted patients were the same as those for patients not waitlisted, although the effect of malnutrition was significantly greater for waitlisted patients (hazard ratio 8.65 [95% CI 2.57-29.11] vs. 1.47 [95% CI 1.08-1.98]). Using the MELD Na score for allocation may continue to limit access to LT.

Transitioning From Paper to Digital: State Statutory and Regulatory Frameworks for Health Information Technology

OBJECTIVES

In all health system sectors, electronic health information (EHI) is created, used, released, and reused. We examined states' efforts to address EHI uses in law to provide an understanding of the EHI legal environment.

METHODS

Attorney researchers used WestlawNext to search for EHI-related statutes and regulations of the US states, US territories, and the District of Columbia in effect as of January 2014. The researchers independently catalogued provisions by the EHI use described in the law. Researchers resolved discrepancies through peer review meetings and recorded the consensus codes for each law.

RESULTS

This study identified 2364 EHI-related laws representing 49 EHI uses in 54 jurisdictions. A total of 18 EHI uses were regulated by ≥10 jurisdictions. More than 750 laws addressed 2 or more EHI uses. Jurisdictions varied by the number of EHI laws in effect, with a mean of 44 laws. Texas had the most EHI laws (n = 145). Hawaii and South Carolina had the fewest (n = 14 each).

CONCLUSIONS

The EHI legal landscape is complex. The large quantity and diversity of laws complicate legal analysis, likely delay implementation of public health solutions, and might be detrimental to the development of emerging health information technology. Research is needed to understand the effect of EHI-related laws.

Evaluating quality of dental care among patients with diabetes: Adaptation and testing of a dental quality measure in electronic health records

BACKGROUND

Patients with diabetes are at increased risk of developing oral complications, and annual dental examinations are an endorsed preventive strategy. The authors evaluated the feasibility and validity of implementing an automated electronic health record (EHR)-based dental quality measure to determine whether patients with diabetes received such evaluations.

METHODS

The authors selected a Dental Quality Alliance measure developed for claims data and adapted the specifications for EHRs. Automated queries identified patients with diabetes across 4 dental institutions, and the authors manually reviewed a subsample of charts to evaluate query performance. After assessing the initial EHR measure, the authors defined and tested a revised EHR measure to capture better the oral care received by patients with diabetes.

RESULTS

In the initial and revised measures, the authors used EHR automated queries to identify 12,960 and 13,221 patients with diabetes, respectively, in the reporting year. Variations in the measure scores across sites were greater with the initial measure (range, 36.4-71.3%) than with the revised measure (range, 78.8-88.1%). The automated query performed well (93% or higher) for sensitivity, specificity, and positive and negative predictive values for both measures.

CONCLUSIONS

The results suggest that an automated EHR-based query can be used successfully to measure the quality of oral health care delivered to patients with diabetes. The authors also found that using the rich data available in EHRs may help estimate the quality of care better than can relying on claims data.

PRACTICAL IMPLICATIONS

Detailed clinical patient-level data in dental EHRs may be useful to dentists in evaluating the quality of dental care provided to patients with diabetes.

Enhanced Quality Measurement Event Detection: An Application to Physician Reporting

The wide-scale adoption of electronic health records (EHR)s has increased the availability of routinely collected clinical data in electronic form that can be used to improve the reporting of quality of care. However, the bulk of information in the EHR is in unstructured form (e.g., free-text clinical notes) and not amenable to automated reporting. Traditional methods are based on structured diagnostic and billing data that provide efficient, but inaccurate or incomplete summaries of actual or relevant care processes and patient outcomes. To assess the feasibility and benefit of implementing enhanced EHR- based physician quality measurement and reporting, which includes the analysis of unstructured free- text clinical notes, we conducted a retrospective study to compare traditional and enhanced approaches for reporting ten physician quality measures from multiple National Quality Strategy domains. We found that our enhanced approach enabled the calculation of five Physician Quality and Performance System measures not measureable in billing or diagnostic codes and resulted in over a five-fold increase in event at an average precision of 88 percent (95 percent CI: 83–93 percent). Our work suggests that enhanced EHR-based quality measurement can increase event detection for establishing value-based payment arrangements and can expedite quality reporting for physician practices, which are increasingly burdened by the process of manual chart review for quality reporting.

Validation of asthma recording in electronic health records: protocol for a systematic review

BACKGROUND

Asthma is a common, heterogeneous disease with significant morbidity and mortality worldwide. It can be difficult to define in epidemiological studies using electronic health records as the diagnosis is based on non-specific respiratory symptoms and spirometry, neither of which are routinely registered. Electronic health records can nonetheless be valuable to study the epidemiology, management, healthcare use and control of asthma. For health databases to be useful sources of information, asthma diagnoses should ideally be validated. The primary objectives are to provide an overview of the methods used to validate asthma diagnoses in electronic health records and summarise the results of the validation studies.

METHODS

EMBASE and MEDLINE will be systematically searched for appropriate search terms. The searches will cover all studies in these databases up to October 2016 with no start date and will yield studies that have validated algorithms or codes for the diagnosis of asthma in electronic health records. At least one test validation measure (sensitivity, specificity, positive predictive value, negative predictive value or other) is necessary for inclusion. In addition, we require the validated algorithms to be compared with an external golden standard, such as a manual review, a questionnaire or an independent second database. We will summarise key data including author, year of publication, country, time period, date, data source, population, case characteristics, clinical events, algorithms, gold standard and validation statistics in a uniform table.

ETHICS AND DISSEMINATION

This study is a synthesis of previously published studies and, therefore, no ethical approval is required. The results will be submitted to a peer-reviewed journal for publication. Results from this systematic review can be used to study outcome research on asthma and can be used to identify case definitions for asthma.

PROSPERO REGISTRATION NUMBER

CRD42016041798.

Making electronic health records support quality management: A narrative review

BACKGROUND

Since the 1990s many hospitals in the OECD countries have introduced electronic health record (EHR) systems. A number of studies have examined the factors impinging on EHR implementation. Others have studied the clinical efficacy of EHR. However, only few studies have explored the (intermediary) factors that make EHR systems conducive to quality management (QM).

OBJECTIVE

Undertake a narrative review of existing studies in order to identify and discuss the factors conducive to making EHR support three dimensions of QM: clinical outcomes, managerial monitoring and cost-effectiveness.

METHOD

A narrative review of Web of Science, Cochrane, EBSCO, ProQuest, Scopus and three Nordic research databases.

LIMITATION

most studies do not specify the type of EHR examined.

RESULTS

39 studies were identified for analysis. 10 factors were found to be conducive to make EHR support QM. However, the contribution of EHR to the three specific dimensions of QM varied substantially. Most studies (29) included clinical outcomes. However, only half of these reported EHR to have a positive impact. Almost all the studies (36) dealt with the ability of EHR to enhance managerial monitoring of clinical activities, the far majority of which showed a positive relationship. Finally, only five dealt with cost-effectiveness of which two found positive effects.

DISCUSSION AND CONCLUSION

The findings resonates well with previous reviews, though two factors making EHR support QM seem new, namely: political goals and strategies, and integration of guidelines for clinical conduct. Lacking EHR type specification and diversity in study method imply that there is a strong need for further research on the factors that may make EHR may support QM.

Eliciting the Intension of Drug Value Sets - Principles and Quality Assurance Applications

Value sets (VSs) used in electronic clinical quality measures are lists of codes from standard terminologies ("extensional" VSs), whose purpose ("intension") is not always explicitly stated. We elicited the intension for the 09/01/2014 release of extensional medication value sets by comparison to drug classes from the October 2014 release of RxClass. Value sets matched drug classes if they shared common ingredients, as evidenced by Jaccard similarity score. We elicited the intension of 80 extensional value sets. The average Jaccard similarity was 0.65 for single classes and 0.80 for combination classes, with 34% (27/80) of the value sets having high similarity scores. Manual review by a pharmacist indicated 51% (41/80) of the drug classes selected as the best mapping for a value set matched the intension reflected in that value set name. This approach has the potential for facilitating the development and maintenance of medication value sets.

Leveraging Real-World Evidence in Disease-Management Decision-Making with a Total Cost of Care Estimator

BACKGROUND

Health management is becoming increasingly complex, given a range of care options and the need to balance costs and quality. The ability to measure and understand drivers of costs is critical for healthcare organizations to effectively manage their patient populations. Healthcare decision makers can leverage real-world evidence to explore the value of disease-management interventions in shifting total cost trends.

OBJECTIVE

To develop a real-world, evidence-based estimator that examines the impact of disease-management interventions on the total cost of care (TCoC) for a patient population with nonvalvular atrial fibrillation (NVAF).

METHODS

Data were collected from a patient-level real-world evidence data set that uses the IMS PharMetrics Health Plan Claims Database. Pharmacy and medical claims for patients meeting the inclusion or exclusion criteria were combined in longitudinal cohorts with a 180-day preindex and 360-day follow-up period. Descriptive statistics, such as mean and median patient costs and event rates, were derived from a real-world evidence analysis and were used to populate the base-case estimates within the TCoC estimator, an exploratory economic model that was designed to estimate the potential impact of several disease-management activities on the TCoC for a patient population with NVAF. Using Microsoft Excel, the estimator is designed to compare current direct costs of medical care to projected costs by varying assumptions on the impact of disease-management activities and applying the associated changes in cost trends to the affected populations. Disease-management levers are derived from literature-based concepts affecting costs along the NVAF disease continuum. The use of the estimator supports analyses across 4 US geographic regions, age, cost types, and care settings during 1 year.

RESULTS

All patients included in the study were continuously enrolled in their health plan (within the IMS PharMetrics Health Plan Claims Database) between July 1, 2010, and June 30, 2012. Patients were included in the final analytic file and were indexed based on (1) the service date of the first claim within the selection window (December 28, 2010-July 11, 2011) with a diagnosis of NVAF, or (2) the service date of the second claim for an NVAF medication of interest during the same selection window. The model estimates the current trends in national benchmark data for a hypothetical health plan with 1 million covered lives. The annual total direct healthcare costs (allowable and patient out-of-pocket costs) of managing patients with NVAF in this hypothetical plan are estimated at $184,981,245 ($25,754 per patient, for 7183 patients). A potential 25% improvement from the base-case disease burden and disease management could translate into TCoC savings from reducing the excess costs related to hypertension (-5.3%) and supporting the use of an appropriate antithrombotic treatment that prevents ischemic stroke (-0.7%) and reduces bleeding events (-0.1%).

CONCLUSIONS

The use of the TCoC estimator supports population health management by providing real-world evidence benchmark data on NVAF disease burden and by quantifying the potential value of disease-management activities in shifting cost trends.

Data extraction from electronic health records (EHRs) for quality measurement of the physical therapy process: comparison between EHR data and survey data

BACKGROUND

With the emergence of the electronic health records (EHRs) as a pervasive healthcare information technology, new opportunities and challenges for use of clinical data for quality measurements arise with respect to data quality, data availability and comparability. The objective of this study is to test whether data extracted from electronic health records (EHRs) was of comparable quality as survey data for the calculation of quality indicators.

METHODS

Data from surveys describing patient cases and filled out by physiotherapists in 2009-2010 were used to calculate scores on eight quality indicators (QIs) to measure the quality of physiotherapy care. In 2011, data was extracted directly from EHRs. The data collection methods were evaluated for comparability. EHR data was compared to survey data on completeness and correctness.

RESULTS

Five of the eight QIs could be extracted from the EHRs. Three were omitted from the indicator set, as they proved too difficult to be extracted from the EHRs. Another QI proved incomparable due to errors in the extraction software of some of the EHRs. Three out of four comparable QIs performed better (p < 0.001) in EHR data on completeness. EHR data also proved to be correct; the relative change in indicator scores between EHR and survey data were small (<5 %) in three out of four QIs.

CONCLUSION

Data quality of EHRs was sufficient to be used for the calculation of QIs, although comparability to survey data was problematic. Standardization is needed, not only to be able to compare different data collection methods properly, but also to compare between practices with different EHRs. EHRs have the option to administrate narrative data, but natural language processing tools are needed to quantify these text boxes. Such development, can narrow the comparability gap between scoring QIs based on EHR data and based on survey data. EHRs have the potential to provide real time feedback to professionals and quality measurements for research, but more effort is needed to create unambiguous and uniform information and to unlock written text in a standardized manner.

Cognitive Bias and the Creation and Translation of Evidence Into Clinical Practice

The optimal translation of evidence into the clinical practice of nephrology follows validated evidence-based medicine (EBM) principles. Most importantly, the evidence-based medicine practitioner requires that the evidence, as much as possible, addresses in an unbiased manner clinical questions of importance to patients and reflects the truth. In this chapter, we evaluate how cognitive biases that affect medical decision making might systematically bias the overall management of patients with kidney disease and, thus, distort the observations about disease causation, prognosis, diagnosis, and management that are derived from analysis of administrative databases or electronic medical records of health care systems. These cognitive biases influence how questions are framed, how risk factors and health status are measured, how decisions are made, and what actions are implemented. We explore the nature of these biases and the possible places where such biases are most likely to distort clinical care and the collection of the data that is used to inform clinical practice in nephrology. To the extent that cognitive biases selectively modify diagnostic interrogations and therapeutic interventions in some patients, the resultant databases may provide a skewed picture of the health of the nephrology patient population and should be considered with some skepticism when exploited to achieve best practice.

Measuring Preventive Care Delivery: Comparing Rates Across Three Data Sources

INTRODUCTION

Preventive care delivery is an important quality outcome, and electronic data reports are being used increasingly to track these services. It is highly informative when electronic data sources are compared to information manually extracted from medical charts to assess validity and completeness.

METHODS

This cross-sectional study used a random sample of Medicaid-insured patients seen at 43 community health centers in 2011 to calculate standard measures of correspondence between manual chart review and two automated sources (electronic health records [EHRs] and Medicaid claims), comparing documentation of orders for and receipt of ten preventive services (n=150 patients/service). Data were analyzed in 2015.

RESULTS

Using manual chart review as the gold standard, automated EHR extraction showed near-perfect to perfect agreement (κ=0.96-1.0) for services received within the primary care setting (e.g., BMI, blood pressure). Receipt of breast and colorectal cancer screenings, services commonly referred out, showed moderate (κ=0.42) to substantial (κ=0.62) agreement, respectively. Automated EHR extraction showed near-perfect agreement (κ=0.83-0.97) for documentation of ordered services. Medicaid claims showed near-perfect agreement (κ=0.87) for hyperlipidemia and diabetes screening, and substantial agreement (κ=0.67-0.80) for receipt of breast, cervical, and colorectal cancer screenings, and influenza vaccination. Claims showed moderate agreement (κ=0.59) for chlamydia screening receipt. Medicaid claims did not capture ordered or unbilled services.

CONCLUSIONS

Findings suggest that automated EHR and claims data provide valid sources for measuring receipt of most preventive services; however, ordered and unbilled services were primarily captured via EHR data and completed referrals were more often documented in claims data.

Constructing Episodes of Inpatient Care: How to Define Hospital Transfer in Hospital Administrative Health Data?

BACKGROUND

Hospital administrative health data create separate records for each hospital stay of patients. Treating a hospital transfer as a readmission could lead to biased results in health service research.

METHODS

This is a cross-sectional study. We used the hospital discharge abstract database in 2013 from Alberta, Canada. Transfer cases were defined by transfer institution code and were used as the reference standard. Four time gaps between 2 hospitalizations (6, 9, 12, and 24 h) and 2 day gaps between hospitalizations [same day (up to 24 h), ≤1 d (up to 48 h)] were used to identify transfer cases. We compared the sensitivity and positive predictive value (PPV) of 6 definitions across different categories of sex, age, and location of residence. Readmission rates within 30 days were compared after episodes of care were defined at the different time gaps.

RESULTS

Among the 6 definitions, sensitivity ranged from 93.3% to 98.7% and PPV ranged from 86.4% to 96%. The time gap of 9 hours had the optimal balance of sensitivity and PPV. The time gaps of same day (up to 24 h) and 9 hours had comparable 30-day readmission rates as the transfer indicator after defining episode of care.

CONCLUSIONS

We recommend the use of a time gap of 9 hours between 2 hospitalizations to define hospital transfer in inpatient databases. When admission or discharge time is not available in the database, a time gap of same day (up to 24 h) can be used to define hospital transfer.

Challenges in patient safety improvement research in the era of electronic health records

Electronic health record (EHR) data repositories contain large volumes of aggregated, longitudinal clinical data that could allow patient safety researchers to identify important safety issues and conduct comprehensive evaluations of health care delivery outcomes. However, few health systems have successfully converted this abundance of data into useful information or knowledge for safety improvement. In this paper, we use a case study involving a project on missed/delayed follow-up of test results to discuss real-world challenges in using EHR data for patient safety research. We identify three types of challenges that pose as barriers to advance patient safety improvement research: 1) gaining approval to access/review EHR data; 2) interpreting EHR data; 3) working with local IT/EHR personnel. We discuss the complexity of these challenges, all of which are unlikely to be unique to this project, and outline some key next steps that must be taken to support research that uses EHR data to improve safety. We recognize that all organizations face competing priorities between clinical operations and research. However, to leverage EHRs and their abundant data for patient safety improvement research, many current data access and security policies and procedures must be rewritten and standardized across health care organizations. These efforts are essential to help make EHRs and EHR data useful for progress in our journey to safer health care.