Development of a social and environmental determinants of health informatics maturity model

Introduction

Integrating social and environmental determinants of health (SEDoH) into enterprise-wide clinical workflows and decision-making is one of the most important and challenging aspects of improving health equity. We engaged domain experts to develop a SEDoH informatics maturity model (SIMM) to help guide organizations to address technical, operational, and policy gaps.

Methods

We established a core expert group consisting of developers, informaticists, and subject matter experts to identify different SIMM domains and define maturity levels. The candidate model (v0.9) was evaluated by 15 informaticists at a Center for Data to Health community meeting. After incorporating feedback, a second evaluation round for v1.0 collected feedback and self-assessments from 35 respondents from the National COVID Cohort Collaborative, the Center for Leading Innovation and Collaboration's Informatics Enterprise Committee, and a publicly available online self-assessment tool.

Results

We developed a SIMM comprising seven maturity levels across five domains: data collection policies, data collection methods and technologies, technology platforms for analysis and visualization, analytics capacity, and operational and strategic impact. The evaluation demonstrated relatively high maturity in analytics and technological capacity, but more moderate maturity in operational and strategic impact among academic medical centers. Changes made to the tool in between rounds improved its ability to discriminate between intermediate maturity levels.

Conclusion

The SIMM can help organizations identify current gaps and next steps in improving SEDoH informatics. Improving the collection and use of SEDoH data is one important component of addressing health inequities.

Assessing the Predictive and Analytics Capability of Electronic Clinical Data for High-Cost Patients

Hotspotting may prevent high healthcare costs surrounding a minority of patients when void of issues such as availability, completeness, and accessibility of information in electronic health records (EHRs). We performed a descriptive study using Barnes-Jewish Hospital patients to assess the availability and accessibility of information that can predict negative outcomes. Manual electronic chart review produced descriptive statistics for a sample of 100 High Resource and 100 Control patient records. The majority of cases were not predictive. Predictive information and their sources were inconsistent. Certain types of patients were more predictive than others, albeit a small percentage of the total. Among the largest and most predictive groups was the most difficult to classify, "Other." These findings were expected and consistent with previous studies but contrast with approaches for attempting prediction such as hotspotting. Further studies may provide solutions to the problems and limitations identified in this study.

Issues with Variability in EHR Data About Race and Ethnicity: A Descriptive Analysis of the National COVID Cohort Collaborative Data Enclave (Preprint)

Background

The adverse impact of COVID-19 on marginalized and under-resourced communities of color has highlighted the need for accurate, comprehensive race and ethnicity data. However, a significant technical challenge related to integrating race and ethnicity data in large, consolidated databases is the lack of consistency in how data about race and ethnicity are collected and structured by health care organizations.

Objective

This study aims to evaluate and describe variations in how health care systems collect and report information about the race and ethnicity of their patients and to assess how well these data are integrated when aggregated into a large clinical database.

Methods

At the time of our analysis, the National COVID Cohort Collaborative (N3C) Data Enclave contained records from 6.5 million patients contributed by 56 health care institutions. We quantified the variability in the harmonized race and ethnicity data in the N3C Data Enclave by analyzing the conformance to health care standards for such data. We conducted a descriptive analysis by comparing the harmonized data available for research purposes in the database to the original source data contributed by health care institutions. To make the comparison, we tabulated the original source codes, enumerating how many patients had been reported with each encoded value and how many distinct ways each category was reported. The nonconforming data were also cross tabulated by 3 factors: patient ethnicity, the number of data partners using each code, and which data models utilized those particular encodings. For the nonconforming data, we used an inductive approach to sort the source encodings into categories. For example, values such as “Declined” were grouped with “Refused,” and “Multiple Race” was grouped with “Two or more races” and “Multiracial.”

Results

“No matching concept” was the second largest harmonized concept used by the N3C to describe the race of patients in their database. In addition, 20.7% of the race data did not conform to the standard; the largest category was data that were missing. Hispanic or Latino patients were overrepresented in the nonconforming racial data, and data from American Indian or Alaska Native patients were obscured. Although only a small proportion of the source data had not been mapped to the correct concepts (0.6%), Black or African American and Hispanic/Latino patients were overrepresented in this category.

Conclusions

Differences in how race and ethnicity data are conceptualized and encoded by health care institutions can affect the quality of the data in aggregated clinical databases. The impact of data quality issues in the N3C Data Enclave was not equal across all races and ethnicities, which has the potential to introduce bias in analyses and conclusions drawn from these data. Transparency about how data have been transformed can help users make accurate analyses and inferences and eventually better guide clinical care and public policy.

Controlled evaLuation of Angiotensin Receptor Blockers for COVID-19 respIraTorY disease (CLARITY): statistical analysis plan for a randomised controlled Bayesian adaptive sample size trial

The CLARITY trial (Controlled evaLuation of Angiotensin Receptor Blockers for COVID-19 respIraTorY disease) is a two-arm, multi-centre, randomised controlled trial being run in India and Australia that investigates the effectiveness of angiotensin receptor blockers in addition to standard care compared to placebo (in Indian sites) with standard care in reducing the duration and severity of lung failure in patients with COVID-19. The trial was designed as a Bayesian adaptive sample size trial with regular planned analyses where pre-specified decision rules will be assessed to determine whether the trial should be stopped due to sufficient evidence of treatment effectiveness or futility. Here, we describe the statistical analysis plan for the trial and define the pre-specified decision rules, including those that could lead to the trial being halted. The primary outcome is clinical status on a 7-point ordinal scale adapted from the WHO Clinical Progression scale assessed at day 14. The primary analysis will follow the intention-to-treat principle. A Bayesian adaptive trial design was selected because there is considerable uncertainty about the extent of potential benefit of this treatment.

Trial registration

ClinicalTrials.gov NCT04394117. Registered on 19 May 2020Clinical Trial Registry of India CTRI/2020/07/026831

Version and revisions

Version 1.0. No revisions.

Extracting Patient-level Social Determinants of Health into the OMOP Common Data Model

Deficiencies in data sharing capabilities limit Social Determinants of Health (SDoH) analysis as part of COVID-19 research. The National COVID Cohort Collaborative (N3C) is an example of an Electronic Health Record (EHR) database of patients tested for COVID-19 that could benefit from a SDoH elements framework that captures various screening instruments in EHR data warehouse systems. This paper uses the University of Washington Enterprise Data Warehouse (a data contributor to N3C) to demonstrate how SDoH can be represented and managed to be made available within an OMOP common data model. We found that these data varied by type of social determinants data and where it was collected, in the time period that it was collected, and in how it was represented.

Simplified data science approach to extract social and behavioural determinants: a retrospective chart review

OBJECTIVES

We aim to extract a subset of social factors from clinical notes using common text classification methods.

DESIGN

Retrospective chart review.

SETTING

We collaborated with a local level I trauma hospital located in an underserved area that has a housing unstable patient population of about 6.5% and extracted text notes related to various social determinants for acute care patients.

PARTICIPANTS

Notes were retrospectively extracted from 43 798 acute care patients.

METHODS

We solely use open source Python packages to test simple text classification methods that can potentially be easily generalisable and implemented. We extracted social history text from various sources, such as admission and emergency department notes, over a 5-year timeframe and performed manual chart reviews to ensure data quality. We manually labelled the sentiment of the notes, treating each text entry independently. Four different models with two different feature selection methods (bag of words and bigrams) were used to classify and predict housing stability, tobacco use and alcohol use status for the extracted clinical text.

RESULTS

From our analysis, we found overall positive results and metrics in applying open-source classification techniques; the accuracy scores were 91.2%, 84.7%, 82.8% for housing stability, tobacco use and alcohol use, respectively. There were many limitations in our analysis including social factors not present due to patient condition, multiple copy-forward entries and shorthand. Additionally, it was difficult to translate usage degrees for tobacco and alcohol use. However, when compared with structured data sources, our classification approach on unstructured notes yielded more results for housing and alcohol use; tobacco use proved less fruitful for unstructured notes.

The National COVID Cohort Collaborative: Analyses of Original and Computationally Derived Electronic Health Record Data

BACKGROUND

Computationally derived ("synthetic") data can enable the creation and analysis of clinical, laboratory, and diagnostic data as if they were the original electronic health record data. Synthetic data can support data sharing to answer critical research questions to address the COVID-19 pandemic.

OBJECTIVE

We aim to compare the results from analyses of synthetic data to those from original data and assess the strengths and limitations of leveraging computationally derived data for research purposes.

METHODS

We used the National COVID Cohort Collaborative's instance of MDClone, a big data platform with data-synthesizing capabilities (MDClone Ltd). We downloaded electronic health record data from 34 National COVID Cohort Collaborative institutional partners and tested three use cases, including (1) exploring the distributions of key features of the COVID-19-positive cohort; (2) training and testing predictive models for assessing the risk of admission among these patients; and (3) determining geospatial and temporal COVID-19-related measures and outcomes, and constructing their epidemic curves. We compared the results from synthetic data to those from original data using traditional statistics, machine learning approaches, and temporal and spatial representations of the data.

RESULTS

For each use case, the results of the synthetic data analyses successfully mimicked those of the original data such that the distributions of the data were similar and the predictive models demonstrated comparable performance. Although the synthetic and original data yielded overall nearly the same results, there were exceptions that included an odds ratio on either side of the null in multivariable analyses (0.97 vs 1.01) and differences in the magnitude of epidemic curves constructed for zip codes with low population counts.

CONCLUSIONS

This paper presents the results of each use case and outlines key considerations for the use of synthetic data, examining their role in collaborative research for faster insights.

A Continuously Benchmarked and Crowdsourced Challenge for Rapid Development and Evaluation of Models to Predict COVID-19 Diagnosis and Hospitalization

Importance

Machine learning could be used to predict the likelihood of diagnosis and severity of illness. Lack of COVID-19 patient data has hindered the data science community in developing models to aid in the response to the pandemic.

Objectives

To describe the rapid development and evaluation of clinical algorithms to predict COVID-19 diagnosis and hospitalization using patient data by citizen scientists, provide an unbiased assessment of model performance, and benchmark model performance on subgroups.

Design, Setting, and Participants

This diagnostic and prognostic study operated a continuous, crowdsourced challenge using a model-to-data approach to securely enable the use of regularly updated COVID-19 patient data from the University of Washington by participants from May 6 to December 23, 2020. A postchallenge analysis was conducted from December 24, 2020, to April 7, 2021, to assess the generalizability of models on the cumulative data set as well as subgroups stratified by age, sex, race, and time of COVID-19 test. By December 23, 2020, this challenge engaged 482 participants from 90 teams and 7 countries.

Main Outcomes and Measures

Machine learning algorithms used patient data and output a score that represented the probability of patients receiving a positive COVID-19 test result or being hospitalized within 21 days after receiving a positive COVID-19 test result. Algorithms were evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC) scores. Ensemble models aggregating models from the top challenge teams were developed and evaluated.

Results

In the analysis using the cumulative data set, the best performance for COVID-19 diagnosis prediction was an AUROC of 0.776 (95% CI, 0.775-0.777) and an AUPRC of 0.297, and for hospitalization prediction, an AUROC of 0.796 (95% CI, 0.794-0.798) and an AUPRC of 0.188. Analysis on top models submitting to the challenge showed consistently better model performance on the female group than the male group. Among all age groups, the best performance was obtained for the 25- to 49-year age group, and the worst performance was obtained for the group aged 17 years or younger.

Conclusions and Relevance

In this diagnostic and prognostic study, models submitted by citizen scientists achieved high performance for the prediction of COVID-19 testing and hospitalization outcomes. Evaluation of challenge models on demographic subgroups and prospective data revealed performance discrepancies, providing insights into the potential bias and limitations in the models.

Research informatics and the COVID-19 pandemic: Challenges, innovations, lessons learned, and recommendations

The recipients of NIH's Clinical and Translational Science Awards (CTSA) have worked for over a decade to build informatics infrastructure in support of clinical and translational research. This infrastructure has proved invaluable for supporting responses to the current COVID-19 pandemic through direct patient care, clinical decision support, training researchers and practitioners, as well as public health surveillance and clinical research to levels that could not have been accomplished without the years of ground-laying work by the CTSAs. In this paper, we provide a perspective on our COVID-19 work and present relevant results of a survey of CTSA sites to broaden our understanding of the key features of their informatics programs, the informatics-related challenges they have experienced under COVID-19, and some of the innovations and solutions they developed in response to the pandemic. Responses demonstrated increased reliance by healthcare providers and researchers on access to electronic health record (EHR) data, both for local needs and for sharing with other institutions and national consortia. The initial work of the CTSAs on data capture, standards, interchange, and sharing policies all contributed to solutions, best illustrated by the creation, in record time, of a national clinical data repository in the National COVID-19 Cohort Collaborative (N3C). The survey data support seven recommendations for areas of informatics and public health investment and further study to support clinical and translational research in the post-COVID-19 era.

Unraveling COVID-19: a large-scale characterization of 4.5 million COVID-19 cases using CHARYBDIS

Background: Routinely collected real world data (RWD) have great utility in aiding the novel coronavirus disease (COVID-19) pandemic response [1,2]. Here we present the international Observational Health Data Sciences and Informatics (OHDSI) [3] Characterizing Health Associated Risks, and Your Baseline Disease In SARS-COV-2 (CHARYBDIS) framework for standardisation and analysis of COVID-19 RWD. Methods: We conducted a descriptive cohort study using a federated network of data partners in the United States, Europe (the Netherlands, Spain, the UK, Germany, France and Italy) and Asia (South Korea and China). The study protocol and analytical package were released on 11 th June 2020 and are iteratively updated via GitHub [4]. Findings: We identified three non-mutually exclusive cohorts of 4,537,153 individuals with a clinical COVID-19 diagnosis or positive test, 886,193 hospitalized with COVID-19 , and 113,627 hospitalized with COVID-19 requiring intensive services . All comorbidities, symptoms, medications, and outcomes are described by cohort in aggregate counts, and are available in an interactive website: https://data.ohdsi.org/Covid19CharacterizationCharybdis/. Interpretation: CHARYBDIS findings provide benchmarks that contribute to our understanding of COVID-19 progression, management and evolution over time. This can enable timely assessment of real-world outcomes of preventative and therapeutic options as they are introduced in clinical practice.

SCOR: A secure international informatics infrastructure to investigate COVID-19

Global pandemics call for large and diverse healthcare data to study various risk factors, treatment options, and disease progression patterns. Despite the enormous efforts of many large data consortium initiatives, scientific community still lacks a secure and privacy-preserving infrastructure to support auditable data sharing and facilitate automated and legally compliant federated analysis on an international scale. Existing health informatics systems do not incorporate the latest progress in modern security and federated machine learning algorithms, which are poised to offer solutions. An international group of passionate researchers came together with a joint mission to solve the problem with our finest models and tools. The SCOR Consortium has developed a ready-to-deploy secure infrastructure using world-class privacy and security technologies to reconcile the privacy/utility conflicts. We hope our effort will make a change and accelerate research in future pandemics with broad and diverse samples on an international scale.

Expert Perspectives on Definitions, Drivers and Informatics Contributions to Learning Health Systems

Learning Health System (LHS) proposes a new paradigm in scientific enterprise to facilitate the rapid movement of data to knowledge (D2K) and knowledge to practice (K2P). Informatics can play a pivotal role in facilitating feedback loops and rapid cycles of learning across D2K and K2P. Though informatics has been acknowledged as a critical component of LHS, it remains unclear how leaders in informatics are conceptualizing its role in promoting LHS. This study sought to gain insights from informatics leaders and experts on their perspectives around role of informatics in LHS. We conducted semi-structured interviews with fourteen informatics leaders across different informatics domains and leadership positions. Our results revealed areas of agreement around key concepts related to LHS as well as opportunities to improve messaging and add clarity to the role of informatics in promoting LHS.

Association of Integrated Team-Based Care With Health Care Quality, Utilization, and Cost

IMPORTANCE

The value of integrated team delivery models is not firmly established.

OBJECTIVE

To evaluate the association of receiving primary care in integrated team-based care (TBC) practices vs traditional practice management (TPM) practices (usual care) with patient outcomes, health care utilization, and costs.

DESIGN

A retrospective, longitudinal, cohort study to assess the association of integrating physical and mental health over time in TBC practices with patient outcomes and costs.

SETTING AND PARTICIPANTS

Adult patients (aged ≥18 years) who received primary care at 113 unique Intermountain Healthcare Medical Group primary care practices from 2003 through 2005 and had yearly encounters with Intermountain Healthcare through 2013, including some patients who received care in both TBC and TPM practices.

EXPOSURES

Receipt of primary care in TBC practices compared with TPM practices for patients treated in internal medicine, family practice, and geriatrics practices.

MAIN OUTCOMES AND MEASURES

Outcomes included 7 quality measures, 6 health care utilization measures, payments to the delivery system, and program investment costs.

RESULTS

During the study period (January 2010-December 2013), 113,452 unique patients (mean age, 56.1 years; women, 58.9%) accounted for 163,226 person-years of exposure in 27 TBC practices and 171,915 person-years in 75 TPM practices. Patients treated in TBC practices compared with those treated in TPM practices had higher rates of active depression screening (46.1% for TBC vs 24.1% for TPM; odds ratio [OR], 1.91 [95% CI, 1.75 to 2.08), adherence to a diabetes care bundle (24.6% for TBC vs 19.5% for TPM; OR, 1.26 [95% CI, 1.11 to 1.42]), and documentation of self-care plans (48.4% for TBC vs 8.7% for TPM; OR, 5.59 [95% CI, 4.27 to 7.33]), lower proportion of patients with controlled hypertension (<140/90 mm Hg) (85.0% for TBC vs 97.7% for TPM; OR, 0.87 [95% CI, 0.80 to 0.95]), and no significant differences in documentation of advanced directives (9.6% for TBC vs 9.9% for TPM; OR, 0.97 [95% CI, 0.91 to 1.03]). Per 100 person-years, rates of health care utilization were lower for TBC patients compared with TPM patients for emergency department visits (18.1 for TBC vs 23.5 for TPM; incidence rate ratio [IRR], 0.77 [95% CI, 0.74 to 0.80]), hospital admissions (9.5 for TBC vs 10.6 for TPM; IRR, 0.89 [95% CI, 0.85 to 0.94]), ambulatory care sensitive visits and admissions (3.3 for TBC vs 4.3 for TPM; IRR, 0.77 [95% CI, 0.70 to 0.85]), and primary care physician encounters (232.8 for TBC vs 250.4 for TPM; IRR, 0.93 [95% CI, 0.92 to 0.94]), with no significant difference in visits to urgent care facilities (55.7 for TBC vs 56.2 for TPM; IRR, 0.99 [95% CI, 0.97 to 1.02]) and visits to specialty care physicians (213.5 for TBC vs 217.9 for TPM; IRR, 0.98 [95% CI, 0.97 to 0.99], P > .008). Payments to the delivery system were lower in the TBC group vs the TPM group ($3400.62 for TBC vs $3515.71 for TPM; β, -$115.09 [95% CI, -$199.64 to -$30.54]) and were less than investment costs of the TBC program.

CONCLUSIONS AND RELEVANCE

Among adults enrolled in an integrated health care system, receipt of primary care at TBC practices compared with TPM practices was associated with higher rates of some measures of quality of care, lower rates for some measures of acute care utilization, and lower actual payments received by the delivery system.

Characterization of a Cardiorenal-like Syndrome in Aged Chimpanzees (Pan troglodytes)

Cardiorenal syndrome involves disease and dysfunction of the heart that leads to progressive renal dysfunction. This study investigated the relationship between cardiac and renal disease in 91 aged chimpanzees at the Alamogordo Primate Facility by evaluation of the medical histories, metabolic parameters, functional measurements of the cardiovascular system, clinical pathology, and histopathology focused on the heart and kidney. Cardiac fibrosis was the most frequent microscopic finding in 82 of 91 animals (90%), followed by glomerulosclerosis with tubulointerstitial fibrosis in 63 of 91 (69%). Cardiac fibrosis with attendant glomerulosclerosis and tubulointerstitial fibrosis was observed in 58 of 91 animals (63%); there was a statistically significant association between the 2 conditions. As the severity of cardiac fibrosis increased, there was corresponding increase in severity of glomerulosclerosis with tubulointerstitial fibrosis. Altered metabolic, cardiovascular, and clinical pathology parameters indicative of heart and kidney failure were commonly associated with the moderate to severe microscopic changes, and concurrent heart and kidney failure were considered the cause of death. The constellation of findings in the chimpanzees were similar to cardiorenal syndrome in humans.

The Association Between Online Health Information-Seeking Behaviors and Health Behaviors Among Hispanics in New York City: A Community-Based Cross-Sectional Study

Background

Hispanics are the fastest-growing minority group in the United States and they suffer from a disproportionate burden of chronic diseases. Studies have shown that online health information has the potential to affect health behaviors and influence management of chronic disease for a significant proportion of the population, but little research has focused on Hispanics.

Objective

The specific aim of this descriptive, cross-sectional study was to examine the association between online health information–seeking behaviors and health behaviors (physical activity, fruit and vegetable consumption, alcohol use, and hypertension medication adherence) among Hispanics.

Methods

Data were collected from a convenience sample (N=2680) of Hispanics living in northern Manhattan by bilingual community health workers in a face-to-face interview and analyzed using linear and ordinal logistic regression. Variable selection and statistical analyses were guided by the Integrative Model of eHealth Use.

Results

Only 7.38% (198/2680) of the sample reported online health information–seeking behaviors. Levels of moderate physical activity and fruit, vegetable, and alcohol consumption were low. Among individuals taking hypertension medication (n=825), adherence was reported as high by approximately one-third (30.9%, 255/825) of the sample. Controlling for demographic, situational, and literacy variables, online health information–seeking behaviors were significantly associated with fruit (β=0.35, 95% CI 0.08-0.62, P=.01) and vegetable (β=0.36, 95% CI 0.06-0.65, P=.02) consumption and physical activity (β=3.73, 95% CI 1.99-5.46, P<.001), but not alcohol consumption or hypertension medication adherence. In the regression models, literacy factors, which were used as control variables, were associated with 3 health behaviors: social networking site membership (used to measure one dimension of computer literacy) was associated with fruit consumption (β=0.23, 95% CI 0.05-0.42, P=.02), health literacy was associated with alcohol consumption (β=0.44, 95% CI 0.24-0.63, P<.001), and hypertension medication adherence (β=–0.32, 95% CI –0.62 to –0.03, P=.03). Models explained only a small amount of the variance in health behaviors.

Conclusions

Given the promising, although modest, associations between online health information–seeking behaviors and some health behaviors, efforts are needed to improve Hispanics’ ability to access and understand health information and to enhance the availability of online health information that is suitable in terms of language, readability level, and cultural relevance.

Research Data Explorer: Lessons Learned in Design and Development of Context-based Cohort Definition and Selection

Research Data eXplorer (RedX) was designed to support self-service research data queries and cohort identification from clinical research databases. The primary innovation of RedX was the electronic health record view of patient data, to provide better contextual understanding for non-technical users in building complex data queries. The design of RedX around this need identified multiple functions that would use individual patient views to better understand population-based data, and vice-versa. During development, the more necessary and valuable components of RedX were refined, leading to a functional self-service query and cohort identification tool. However, with the improved capabilities and extensibility of other applications for data querying and navigation, our long-term implementation and dissemination plans have moved towards consolidation and alignment of RedX functions as enhancements in these other initiatives.

Sustaining the effective use of health care data: a message from the editors

Introduction:

Over the past decade, several initiatives have funded large projects to develop clinical research data infrastructures totaling several hundred million dollars. While most of this funding has ended or is expected to end soon, the projects themselves must struggle to continue operations beyond the initial funding. Examples of sustained research-data infrastructures are lacking, and recommended approaches to improve sustainability of developing infrastructures are even rarer. Early on, the Electronic Data Methods (EDM) Forum—and the Agency for Healthcare Research and Quality (AHRQ) as its sponsor—recognized the need to study strategies for sustainability.

Themes:

Three prominent themes relating to sustainability arise among the articles in this special issue: the importance of project maturity, commercialization activities, and stakeholder support. Maturity was relevant to all the papers since a project’s maturity directly influences the opportunities that are available, while commercialization and stakeholder support emerged from comparisons among subsets of articles.

Next Steps:

The papers in this issue create a useful initial set of case studies to help in understanding sustainability issues for data infrastructures needed for research and QI. Each paper includes important lessons learned from the authors’ experience with the different projects that should resonate with the broader fields of clinical research and clinical research informatics. There is an ongoing need for greater understanding of sustainability beyond what this issue provides. As more case studies of sustainability are accumulated, it is expected even more important themes will emerge from qualitative reviews that can eventually be demonstrated quantitatively.

Sustainability considerations for health research and analytic data infrastructures

Introduction:

The United States has made recent large investments in creating data infrastructures to support the important goals of patient-centered outcomes research (PCOR) and comparative effectiveness research (CER), with still more investment planned. These initial investments, while critical to the creation of the infrastructures, are not expected to sustain them much beyond the initial development. To provide the maximum benefit, the infrastructures need to be sustained through innovative financing models while providing value to PCOR and CER researchers.

Sustainability Factors:

Based on our experience with creating flexible sustainability strategies (i.e., strategies that are adaptive to the different characteristics and opportunities of a resource or infrastructure), we define specific factors that are important considerations in developing a sustainability strategy. These factors include assets, expansion, complexity, and stakeholders. Each factor is described, with examples of how it is applied. These factors are dimensions of variation in different resources, to which a sustainability strategy should adapt.

Summary Observations:

We also identify specific important considerations for maintaining an infrastructure, so that the long-term intended benefits can be realized. These observations are presented as lessons learned, to be applied to other sustainability efforts. We define the lessons learned, relating them to the defined sustainability factors as interactions between factors.

Conclusion and Next Steps:

Using perspectives and experiences from a diverse group of experts, we define broad characteristics of sustainability strategies and important observations, which can vary for different projects. Other descriptions of adaptive, flexible, and successful models of collaboration between stakeholders and data infrastructures can expand this framework by identifying other factors for sustainability, and give more concrete directions on how sustainability can be best achieved.

Mission and Sustainability of Informatics for Integrating Biology and the Bedside (i2b2)

INTRODUCTION

A visible example of a successfully disseminated research project in the healthcare space is Informatics for Integrating Biology and the Bedside, or i2b2. The project serves to provide the software that can allow a researcher to do direct, self-serve queries against the electronic healthcare data form a hospital. The goals of these queries are to find cohorts of patients that fit specific profiles, while providing for patient privacy and discretion. Sustaining this resource and keeping its direction has always been a challenge, but ever more so as the ten year National Centers for Biomedical Computing (NCBCs) sunset their funding.

FINDINGS

Building on the i2b2 structures has helped the dissemination plans for grants leveraging it because it is a disseminated national resource. While this has not directly increased the support of i2b2 internally, it has increased the ability of institutions to leverage the resource and generally leads to increased institutional support.

DISCUSSION

The successful development, use, and dissemination i2b2 has been significant in clinical research and informatics. Its evolution has been from a local research data infrastructure to one disseminated more broadly than any other product of the National Centers for Biomedical Computing, and an infrastructure spawning larger investments than were originally used to create it. Throughout this, there were two main lessons about the benefits of dissemination: that people have great creativity in utilizing a resource in different ways and that broader system use can make the system more robust. One option for long-term sustainability of the central authority would be to translate the function to an industry partner. Another option currently being pursued is to create a foundation that would be a central authority for the project.

CONCLUSION

Over the past 10 years, i2b2 has risen to be an important staple in the toolkit of health care researchers. There are now over 110 hospitals that use i2b2 for research. This open-source platform has a community of developers that are continuously enhancing the analytic capacities of the platform and inventing new functionality. By understanding how i2b2 has been sustained, we hope that other research infrastructure projects may better navigate options in making those initiatives sustainable over time.

Online health information seeking behaviors of Hispanics in New York City: a community-based cross-sectional study

BACKGROUND

The emergence of the Internet has increased access to health information and can facilitate active individual engagement in health care decision making. Hispanics are the fastest-growing minority group in the United States and are also the most underserved in terms of access to online health information. A growing body of literature has examined correlates of online health information seeking behaviors (HISBs), but few studies have included Hispanics.

OBJECTIVE

The specific aim of this descriptive, correlational study was to examine factors associated with HISBs of Hispanics.

METHODS

The study sample (N=4070) was recruited from five postal zip codes in northern Manhattan for the Washington Heights Inwood Informatics Infrastructure for Comparative Effectiveness Research project. Survey data were collected via interview by bilingual community health workers in a community center, households, and other community settings. Data were analyzed using bivariate analyses and logistic regression.

RESULTS

Among individual respondents, online HISBs were significantly associated with higher education (OR 3.03, 95% CI 2.15-4.29, P<.001), worse health status (OR 0.42, 95% CI 0.31-0.57, P<.001), and having no hypertension (OR 0.60, 95% CI 0.43-0.84, P=.003). Online HISBs of other household members were significantly associated with respondent factors: female gender (OR 1.60, 95% CI 1.22-2.10, P=.001), being younger (OR 0.75, 95% CI 0.62-0.90, P=.002), being married (OR 1.36, 95% CI 1.09-1.71, P=.007), having higher education (OR 1.80, 95% CI 1.404-2.316, P<.001), being in worse health (OR 0.59, 95% CI 0.46-0.77, P<.001), and having serious health problems increased the odds of their household members' online HISBs (OR 1.83, 95% CI 1.29-2.60, P=.001).

CONCLUSIONS

This large-scale community survey identified factors associated with online HISBs among Hispanics that merit closer examination. To enhance online HISBs among Hispanics, health care providers and policy makers need to understand the cultural context of the Hispanic population. Results of this study can provide a foundation for the development of informatics-based interventions to improve the health of Hispanics in the United States.

Which components of health information technology will drive financial value?

OBJECTIVES

The financial effects of electronic health records (EHRs) and health information exchange (HIE) are largely unknown, despite unprecedented federal incentives for their use. We sought to understand which components of EHRs and HIE are most likely to drive financial savings in the ambulatory, inpatient, and emergency department settings.

STUDY DESIGN

Framework development and a national expert panel.

METHODS

We searched the literature to identify functionalities enabled by EHRs and HIE across the 3 healthcare settings. We rated each of 233 functionality-setting combinations on their likelihood of having a positive financial effect. We validated the top-scoring functionalities with a panel of 28 national experts, and we compared the high-scoring functionalities with Stage 1 meaningful use criteria.

RESULTS

We identified 54 high-scoring functionality- setting combinations, 27 for EHRs and 27 for HIE. Examples of high-scoring functionalities included providing alerts for expensive medications, providing alerts for redundant lab orders, sending and receiving imaging reports, and enabling structured medication reconciliation. Of the 54 high-scoring functionalities, 25 (46%) are represented in Stage 1 meaningful use. Many of the functionalities not yet represented in meaningful use correspond with functionalities that focus directly on healthcare utilization and costs rather than on healthcare quality per se.

CONCLUSIONS

This work can inform the development and selection of future meaningful use measures; inform implementation efforts, as clinicians and hospitals choose from among a "menu" of measures for meaningful use; and inform evaluation efforts, as investigators seek to measure the actual financial impact of EHRs and HIE.

Predictors of health information-seeking behaviors in hispanics

The objective of this study was to examine factors predicting use of the Internet to seek health information among Hispanics in the Washington Heights and Inwood areas of New York City. Data were collected by community health workers through the Washington Heights/Inwood Informatics Infrastructure for Community-Centered Comparative Effectiveness Research (WICER) community survey and a random sample of 100 surveys was selected for analysis. Binary logistic regression (N=100) was used to examine predictors of online health information-seeking behaviors (HISBs) of respondent and household members (dependent variables). Younger age, better health status, and higher education level significantly predicted respondents' HISBs. Respondents' health status and education level also significantly predicted household members' HISBs.

Using the LOINC Semantic Structure to Integrate Community-based Survey Items into a Concept-based Enterprise Data Dictionary to Support Comparative Effectiveness Research

In designing informatics infrastructure to support comparative effectiveness research (CER), it is necessary to implement approaches for integrating heterogeneous data sources such as clinical data typically stored in clinical data warehouses and those that are normally stored in separate research databases. One strategy to support this integration is the use of a concept-oriented data dictionary with a set of semantic terminology models. The aim of this paper is to illustrate the use of the semantic structure of Clinical LOINC (Logical Observation Identifiers, Names, and Codes) in integrating community-based survey items into the Medical Entities Dictionary (MED) to support the integration of survey data with clinical data for CER studies.

Comparing the effectiveness of a clinical registry and a clinical data warehouse for supporting clinical trial recruitment: a case study

This paper reports a case study comparing the relative efficiency of using a Diabetes Registry or a Clinical Data Warehouse to recruit participants for a diabetes clinical trial, TECOS. The Clinical Data Warehouse generated higher positive predictive accuracy (31% vs. 6.6%) and higher participant recruitment than the Registry (30 vs. 14 participants) in a shorter time period (59 vs. 74 working days). We identify important factors that increase clinical trial recruitment efficiency and lower cost.

Influence of muscle training on resting blood flow and forearm vessel diameter in patients with chronic renal failure

Background

Blood flow and vessel diameter are predictors of the success of vascular access procedures. This study investigated whether a simple exercise programme could influence these variables.

Methods

Twenty-three patients with chronic kidney disease were prescribed a simple exercise programme for one arm only; the investigators were blinded to the patients' choice. All underwent arterial and venous duplex imaging, handgrip strength and blood pressure measurements before and 1 month after the exercise programme.

Results

Twelve patients exercised their dominant and 11 their non-dominant arm. In the trained arm, the exercise programme resulted in a significant increase in handgrip strength, by a median (interquartile range) of 4 (0–8) kg (P &lt; 0·001), and in the diameter of the brachial artery (0·2 (0·1–0·3) mm; P &lt; 0·001), radial artery (0·3 (0·2–0·4) mm; P &lt; 0·001), and cephalic vein (0·6 (0·4–1·2) mm in the forearm and 1·1 (0·4–1·2) mm above the elbow; P &lt; 0·001). There was an increase in brachial artery mean velocity (3 (1–7) cm/s; P = 0·009) and peak systolic velocity (8 (1–15) cm/s; P = 0·020), despite a marginally lower systolic blood pressure (−8 (−16 to 0) mmHg; P = 0·007). There was no change in any of these parameters in the non-exercised arm.

Conclusion

In patients with chronic kidney disease, forearm exercise increased blood flow and vessel diameters. This may be beneficial before vascular access formation.

Measuring the effects of health information technology on quality of care: a novel set of proposed metrics for electronic quality reporting

BACKGROUND

Electronic health records (EHRs), in combination with health information exchange, are being promoted in the United States as a strategy for improving quality of care. No single metric set exists for measuring the effectiveness of these interventions. A set of quality metrics was sought that could be retrieved electronically and would be sensitive to the changes in quality that EHRs with health information exchange may contribute to ambulatory care.

METHODS

A literature search identified quality metric sets for ambulatory care. Two rounds of quantitative rating of individual metrics were conducted. Metrics were developed de novo to capture additional expected effects of EHRs with health information exchange. A 36-member national expert panel validated the rating process and final metric set.

RESULTS

Seventeen metric sets containing 1,064 individual metrics were identified; 510 metrics met inclusion criteria. Two rounds of rating narrowed these to 59 metrics and then to 18. The final 18 consisted of metrics for asthma, cardiovascular disease, congestive heart failure, diabetes, medication and allergy documentation, mental health, osteoporosis, and prevention. Fourteen metrics were developed de novo to address test ordering, medication management, referrals, follow-up after discharge, and revisits.

DISCUSSION

The novel set of 32 metrics is proposed as suitable for electronic reporting to capture the potential quality effects of EHRs with health information exchange. This metric set may have broad utility as health information technology becomes increasingly common with funding from the federal stimulus package and other sources. This work may also stimulate discussion on improving how data are entered and extracted from clinically rich, electronic sources, with the goal of more accurately measuring and improving care.

Using Personal Health Records for Automated Clinical Trials Recruitment: the ePaIRing Model

We describe the development of a model describing the use of patient information to improve patient recruitment in clinical trials. This model, named ePaIRing (electronic Participant Identification and Recruitment Model) describes variations in how information flows between stakeholders, and how personal health records can specifically facilitate patient recruitment.

Sociotechnical analysis of a neonatal ICU

Sociotechnical theory has been used to inform the development of computer systems in the complex and dynamic environment of healthcare. The key components of the sociotechnical system are the workers, their practices, their mental models, their interactions, and the tools used in the work process. We conducted a sociotechnical analysis of a neonatal intensive care unit towards the development of decision support for antimicrobial prescribing. We found that the core task was to save the baby in the face of complex and often incomplete information. Organizational climate characteristics were pride in clinical and educational practice. In addition, the structure of work identified interdisciplinary teamwork with some communication breakdown and interruptive work environment. Overall, sociotechnical analysis provided a solid method to understand work environment during the decision support development process.

Content and structure of clinical problem lists: a corpus analysis

In the interest of designing an automated high-level, longitudinal clinical summary of a patient record, we analyze traditional ways in which medical problems pertaining to the patient are summarized in the electronic health record. The patient problem list has become a commonly used proxy for a summary of patient history and automated methods have been proposed to generate it. However, little research has been conducted on how to structure the problem list in a manner most effective for supporting clinical care. This study analyzes the structure and content of the Past Medical History (PMH) sections of a large corpus of clinical notes, as a proxy for problem lists. Findings show that when listing patients history, physicians convey several semantic types of information, not only problems. Furthermore, they often group related concepts in a single line of the PMH. In contrast, traditional problem lists allow only a simple enumeration of coded terms. Content analysis goes on to reiterate the value of more complex representations as well as provide valuable data and guidelines for automated generation of a clinical summary.

Physician use of outpatient electronic health records to improve care

We applied a model of usage categories of electronic health records for outpatient physicians to a large population of physicians, using an established electronic health record. This model categorizes physician users according to how extensively they adopt the various capabilities of electronic health records. We identified representative indicators from usage statistics for outpatient physician use of the HELP-2 outpatient electronic medical record, in use at Intermountain Healthcare. Using these indicators, we calculated the relative proportion of users in each category. These proportions are useful for predicting the expected benefits of electronic health record adoption.

Sociotechnical analysis of a neonatal ICU in the context of CPOE

Computerized provider order entry (CPOE) with decision support is an important tool for addressing preventable medication errors. However, reports of poorly designed systems have shown an increase in adverse events. As part of a project aimed at designing a decision support system for antibiotic prescribing, a sociotechnical approach was used to understand the environment where CPOE is used in a neonatal intensive care unit (NICU). Themes identified included pride in practice, teamwork and collaboration, information integration, and a constantly changing environment.

Architectural design of a data warehouse to support operational and analytical queries across disparate clinical databases

As the clinical data warehouse of the New York Presbyterian Hospital has evolved innovative methods of integrating new data sources and providing more effective and efficient data reporting and analysis need to be explored. We designed and implemented a new clinical data warehouse architecture to handle the integration of disparate clinical databases in the institution. By examining the way downstream systems are populated and streamlining the way data is stored we create a virtual clinical data warehouse that is adaptable to future needs of the organization.

Life expectancy and death from cardiomyopathy amongst carriers of Duchenne and Becker muscular dystrophy in Scotland

OBJECTIVES

To assess life expectancy and cardiovascular mortality in carriers of Duchenne and Becker muscular dystrophy.

DESIGN

Family pedigrees of individuals affected with these conditions, held by the four genetics centres in Scotland, were examined to identify a cohort of definite carriers. Electronic death registration data, held by the General Register Office for Scotland, were used to identify death certificates of carriers who had died, to obtain age at death and cause of death. Survival and mortality data were obtained for the general population for comparison.

PATIENTS

397 definite carriers in 202 pedigrees were identified from which 94 deaths were identified by record linkage to death certificates.

MAIN OUTCOME MEASURES

Observed numbers surviving to certain ages and numbers dying of cardiac causes were compared with expected numbers calculated from general population data.

RESULTS

There were no significant differences between observed and expected numbers surviving to ages 40-90. The standardised mortality ratio for the 371 carriers alive in 1974 was 0.53 (95% confidence interval 0.32 to 0.82).

CONCLUSIONS

Whereas female carriers may have clinical features of cardiomyopathy, this study does not suggest that this is associated with reduced life expectancy or increased risk of cardiac death. Routine cardiac surveillance of obligate carriers is therefore probably unnecessary.

Care management dosage

BACKGROUND

The care of patients with complex illnesses requires careful management, but systems of care management (CM) vary in their structure and effectiveness.

OBJECTIVE

To create a framework identifying components of broad-based CM interventions and validate the framework, including using this framework to evaluate the contribution of varying components on outcomes of patients with chronic illness.

DESIGN

We create the framework using retrospective information about CM activities and services over 12 months and categorize it using cluster and factor analysis. We then validate this framework through content and criterion techniques. Content validity is assessed through a Delphi study and criterion validity through relationship of the dosage measures and patterns of care to process and outcomes measures.

PARTICIPANTS

Patients with diabetes and/or cardiovascular disease receiving CM services in a model known as Care Management Plus implemented in primary care.

RESULTS

Six factors of CM activity were identified, including a single dosage summary measure and 5 separate patterns of care. Of these, the overall dosage summary measure, face-to-face time, duration of follow-up, and breadth of services were all related to improved processes for hemoglobin A1c and LDL testing and control. Brief intense patterns of care and high face-to-face care manager time were also related to improved outcomes.

CONCLUSIONS

Using this framework, we isolate components of a CM intervention directly related to improved process of care or patient outcomes. Current efforts to structure CM to include face-to-face time and multiple diseases are discussed.

Emergency department access to a longitudinal medical record

Our goal is to assess how clinical information from previous visits is used in the emergency department. We used detailed user audit logs to measure access to different data types. We found that clinician-authored notes and laboratory and radiology data were used most often (common data types were used up to 5% to 20% of the time). Data were accessed less than half the time (up to 20% to 50%) even when the user was alerted to the presence of data. Our access rate indicates that health information exchange projects should be conservative in estimating how often shared data will be used and the wide breadth of data accessed indicates that although a clinical summary is likely to be useful, an ideal solution will supply a broad variety of data.

Productivity enhancement for primary care providers using multicondition care management

OBJECTIVE

To assess the impact of a multicondition care management system on primary care physician efficiency and productivity.

STUDY DESIGN

Retrospective controlled repeated-measures design comparing physician productivity with the proportion of patients in the care management system.

METHODS

The setting was primary care clinics in Intermountain Healthcare, a large integrated delivery network. The care management system consisted of a trained team with nurses as care managers and specialized information technology. We defined the use of the care management system as a proportion of referrals by the physician to the care manager. Clinic, physician, and patient panel demographics were used to adjust expected visit productivity and were included in a multivariate mixed model with repeated measures comprising work relative value units and system use.

RESULTS

The productivity of 120 physicians in 7 intervention clinics and 14 control clinics was compared during 24 months. Clinic, physician, and patient panel characteristics exhibited similar characteristics, although patients in intervention clinics were less likely to be married. Adjusted work relative value units were 8% (range, 5%-12%) higher for intervention clinics vs control clinics. Additional annual revenue was estimated at 99,986 dollars per clinic. These additional revenues outweighed the estimated cost of the program of 92,077 dollars.

CONCLUSIONS

Physician productivity increased when more than 2% of patients were seen by a care management team; the increased revenue in our market exceeded the cost of the program. Implications for the creation, structure, and reimbursement of such teams are discussed.

A framework for information system usage in collaborative care

Clinical information systems (CIS) can affect the quality of patient care. In this paper, we focus on CIS use in the collaborative treatment of chronic diseases. We have developed a framework to determine which CIS functions have general usefulness for improving patient outcomes.

METHODS

We reviewed the use of clinical information systems within a collaborative care environment, identifying CIS functions important in chronic disease care. We grouped the functions into categories of access, best practices, and communication (ABC). Three independent raters selected the most important collaborative care related functions from the HL7 Electronic Health Record Systems functional model, and mapped the HL7 functions against the ABC categories. We then built a model of CIS use and tested it on data from a cohort of patients with chronic illnesses.

RESULTS

Of the 133 HL7 elements in the ABC model, 60 (45%) were ranked as important for collaborative care by two reviewers. Agreement was moderate for importance (kappa=.20) but high for ABC categorization (kappa=.67). In our data tests, for the 1105 patients, access 4.4+/-6.5, best practices 0.8+/-1.6, and communication 2.9+/-4.5. CIS functions were used per episode of care. We were able to identify several key functions that may affect patient care. For example, certain CIS functions related to best practices were associated with higher clinician adherence to testing guidelines.

DISCUSSION

This framework may be useful to assess and compare CIS systems for collaborative care. Future refinements of the model are discussed.

Using discordance to improve classification in narrative clinical databases: an application to community-acquired pneumonia

Data mining in electronic medical records may facilitate clinical research, but much of the structured data may be miscoded, incomplete, or non-specific. The exploitation of narrative data using natural language processing may help, although nesting, varying granularity, and repetition remain challenges. In a study of community-acquired pneumonia using electronic records, these issues led to poor classification. Limiting queries to accurate, complete records led to vastly reduced, possibly biased samples. We exploited knowledge latent in the electronic records to improve classification. A similarity metric was used to cluster cases. We defined discordance as the degree to which cases within a cluster give different answers for some query that addresses a classification task of interest. Cases with higher discordance are more likely to be incorrectly classified, and can be reviewed manually to adjust the classification, improve the query, or estimate the likely accuracy of the query. In a study of pneumonia--in which the ICD9-CM coding was found to be very poor--the discordance measure was statistically significantly correlated with classification correctness (.45; 95% CI .15-.62).

Use of Health-Related, Quality-of-Life Metrics to Predict Mortality and Hospitalizations in Community-Dwelling Seniors

OBJECTIVES

To investigate whether health-related quality-of-life (HRQoL) scores in a primary care population can be used as a predictor of future hospital utilization and mortality.

DESIGN

Prospective cohort study measuring Short Form 12 (SF-12) scores obtained using a mailed survey. SF-12 scores, age, and a comorbidity score were used to predict hospitalization and mortality rate using multivariable logistic regression and Cox proportional hazards during the ensuing 28-month period for elderly patients.

SETTING

Intermountain Health Care, a large integrated-delivery network serving a population of more than 150,000 seniors.

PARTICIPANTS

Participants were senior patients who had one or more chronic diseases, were community dwelling, and were initially treated in primary care clinics.

MEASUREMENTS

SF-12 survey Version 1.

RESULTS

Seven thousand seventy-six surveys were sent to eligible participants; 3,042 (43%) were returned. Of the returned surveys, 2,166 (71%) were complete and scoreable. For the respondent group, a multivariable analysis demonstrated that older age, male sex, higher comorbidity score, and lower mental and physical summary measures of SF-12 predicted higher mortality and hospitalization. On average, nonresponders were older and had higher comorbidity scores and mortality rates than responders.

CONCLUSION

The SF-12 survey provided additional predictive ability for future hospitalizations and mortality. Such predictive ability might facilitate preemptive interventions that would change the course of disease in this segment of the population. However, nonresponder bias may limit the utility of mailed SF-12 surveys in certain populations.

Implementing a Multidisease Chronic Care Model in Primary Care Using People and Technology

Management of chronic disease is performed inadequately in the United States in spite of the availability of beneficial, effective therapies. Successful programs to manage patients with these diseases must overcome multiple challenges, including the recognized fragmentation and complexity of the healthcare system, misaligned incentives, a focus on acute problems, and a lack of team-based care. In many successful programs, care is provided in settings or episodes that focus on a single disease. While these programs may allow for streamlined, focused provision of care, comprehensive care for multiple diseases may be more difficult. At Intermountain Healthcare (Intermountain), a generalist model of chronic disease management was formulated to overcome the limitations associated with specialization. In the Intermountain approach, which reflects elements of the Chronic Care Model (CCM), care managers located within multipayer primary care clinics collaborate with physicians, patients, and other members of a primary care team to improve patient outcomes for a variety of conditions. An important part of the intervention is widespread use of an electronic health record (EHR). This EHR provides flexible access to clinical data, individualized decision support designed to encourage best practice for patients with a variety of diseases (including co-occurring ones), and convenient communication between providers. This generalized model is used to treat diverse patients with disparate and coexisting chronic conditions. Early results from the application of this model show improved patient outcomes and improved physician productivity. Success factors, challenges, and obstacles in implementing the model are discussed.

Architectural strategies and issues with health information exchange

We present a model for architectural strategies and issues regarding health information exchange (HIE). This model is a continuum of options, between separated systems and a monolithic approach. We discuss characteristics, examples and potential impacts of each approach, based on generalized observations of practical implementations. The continuum model allows for approaches that mature from an initial implementation to a long-term strategy.

Impact of generalist care managers on patients with diabetes

OBJECTIVE

To determine how the addition of generalist care managers and collaborative information technology to an ambulatory team affects the care of patients with diabetes.

STUDY SETTING

Multiple ambulatory clinics within Intermountain Health Care (IHC), a large integrated delivery network.

STUDY DESIGN

A retrospective cohort study comparing diabetic patients treated by generalist care managers with matched controls was completed. Exposure patients had one or more contacts with a care manager; controls were matched on utilization, demographics, testing, and baseline glucose control. Using role-specific information technology to support their efforts, care managers assessed patients' readiness for change, followed guidelines, and educated and motivated patients.

DATA COLLECTION

Patient data collected as part of an electronic patient record were combined with care manager-created databases to assess timely testing of glycosylated hemoglobin (HbA1c) and low-density lipoprotein (LDL) levels and changes in LDL and HbA1c levels.

PRINCIPAL FINDINGS

In a multivariable model, the odds of being overdue for testing for HbA1c decreased by 21 percent in the exposure group (n=1,185) versus the control group (n=4,740). The odds of being tested when overdue for HbA1c or LDL increased by 49 and 26 percent, respectively, and the odds of HbA1c <7.0 percent also increased by 19 percent in the exposure group. The average HbA1c levels decreased more in the exposure group than in the controls. The effect on LDL was not significant.

CONCLUSIONS

Generalist care managers using computer-supported diabetes management helped increase adherence to guidelines for testing and control of HbA1c levels, leading to improved health status of patients with diabetes.

Drug-age alerting for outpatient geriatric prescriptions: a joint study using interoperable drug standards

For more than a decade, the Beers criteria have identified specific medications that should generally be avoided in the geriatric population. Studies that have shown high prevalence rates of these potentially inappropriate medications have used disparate methodologies to identify these medications and hence are difficult to replicate and generalize. In an effort to improve prescribing behavior, we are building a drug-age alerting system utilizing standard drug coding systems for use in our Electronic Health Record (EHR) systems.

Reference standards, judges, and comparison subjects: roles for experts in evaluating system performance

Medical informatics systems are often designed to perform at the level of human experts. Evaluation of the performance of these systems is often constrained by lack of reference standards, either because the appropriate response is not known or because no simple appropriate response exists. Even when performance can be assessed, it is not always clear whether the performance is sufficient or reasonable. These challenges can be addressed if an evaluator enlists the help of clinical domain experts. 1) The experts can carry out the same tasks as the system, and then their responses can be combined to generate a reference standard. 2)The experts can judge the appropriateness of system output directly. 3) The experts can serve as comparison subjects with which the system can be compared. These are separate roles that have different implications for study design, metrics, and issues of reliability and validity. Diagrams help delineate the roles of experts in complex study designs.