Comparison of clinical knowledge management capabilities of commercially-available and leading internally-developed electronic health records

Ontologies Applied in Clinical Decision Support System Rules: Systematic Review

BACKGROUND

Clinical decision support systems (CDSSs) are important for the quality and safety of health care delivery. Although CDSS rules guide CDSS behavior, they are not routinely shared and reused.

OBJECTIVE

Ontologies have the potential to promote the reuse of CDSS rules. Therefore, we systematically screened the literature to elaborate on the current status of ontologies applied in CDSS rules, such as rule management, which uses captured CDSS rule usage data and user feedback data to tailor CDSS services to be more accurate, and maintenance, which updates CDSS rules. Through this systematic literature review, we aim to identify the frontiers of ontologies used in CDSS rules.

METHODS

The literature search was focused on the intersection of ontologies; clinical decision support; and rules in PubMed, the Association for Computing Machinery (ACM) Digital Library, and the Nursing & Allied Health Database. Grounded theory and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines were followed. One author initiated the screening and literature review, while 2 authors validated the processes and results independently. The inclusion and exclusion criteria were developed and refined iteratively.

RESULTS

CDSSs were primarily used to manage chronic conditions, alerts for medication prescriptions, reminders for immunizations and preventive services, diagnoses, and treatment recommendations among 81 included publications. The CDSS rules were presented in Semantic Web Rule Language, Jess, or Jena formats. Despite the fact that ontologies have been used to provide medical knowledge, CDSS rules, and terminologies, they have not been used in CDSS rule management or to facilitate the reuse of CDSS rules.

CONCLUSIONS

Ontologies have been used to organize and represent medical knowledge, controlled vocabularies, and the content of CDSS rules. So far, there has been little reuse of CDSS rules. More work is needed to improve the reusability and interoperability of CDSS rules. This review identified and described the ontologies that, despite their limitations, enable Semantic Web technologies and their applications in CDSS rules.

Trends and gaps in precision health research: a scoping review

OBJECTIVE

To determine progress and gaps in global precision health research, examining whether precision health studies integrate multiple types of information for health promotion or restoration.

DESIGN

Scoping review.

DATA SOURCES

Searches in Medline (OVID), PsycINFO (OVID), Embase, Scopus, Web of Science and grey literature (Google Scholar) were carried out in June 2020.

ELIGIBILITY CRITERIA

Studies should describe original precision health research; involve human participants, datasets or samples; and collect health-related information. Reviews, editorial articles, conference abstracts or posters, dissertations and articles not published in English were excluded.

DATA EXTRACTION AND SYNTHESIS

The following data were extracted in independent duplicate: author details, study objectives, technology developed, study design, health conditions addressed, precision health focus, data collected for personalisation, participant characteristics and sentence defining 'precision health'. Quantitative and qualitative data were summarised narratively in text and presented in tables and graphs.

RESULTS

After screening 8053 articles, 225 studies were reviewed. Almost half (105/225, 46.7%) of the studies focused on developing an intervention, primarily digital health promotion tools (80/225, 35.6%). Only 28.9% (65/225) of the studies used at least four types of participant data for tailoring, with personalisation usually based on behavioural (108/225, 48%), sociodemographic (100/225, 44.4%) and/or clinical (98/225, 43.6%) information. Participant median age was 48 years old (IQR 28-61), and the top three health conditions addressed were metabolic disorders (35/225, 15.6%), cardiovascular disease (29/225, 12.9%) and cancer (26/225, 11.6%). Only 68% of the studies (153/225) reported participants' gender, 38.7% (87/225) provided participants' race/ethnicity, and 20.4% (46/225) included people from socioeconomically disadvantaged backgrounds. More than 57% of the articles (130/225) have authors from only one discipline.

CONCLUSIONS

Although there is a growing number of precision health studies that test or develop interventions, there is a significant gap in the integration of multiple data types, systematic intervention assessment using randomised controlled trials and reporting of participant gender and ethnicity. Greater interdisciplinary collaboration is needed to gather multiple data types; collectively analyse big and complex data; and provide interventions that restore, maintain and/or promote good health for all, from birth to old age.

What We Can Learn From Failure: An EHR-Based Child Protection Alert System

This study aimed to evaluate the efficacy of a newly implemented Child Protection Alert System (CPAS) that utilizes triggering diagnoses to identify children who have been confirmed/strongly suspected as maltreated. We retrospectively reviewed electronic health records (EHRs) of 666 patients evaluated by our institution's child protection team between 2009 and 2014. We examined each EHR for the presence of a pop-up alert, a persistent text-based visual alert, and diagnoses denoting child maltreatment. Diagnostic accuracy of the CPAS for child maltreatment identification was assessed. Of 323 patients for whom child maltreatment was confirmed/strongly suspected, 21.7% (70/323) had a qualifying longitudinal diagnosis listed. The pop-up alert fired in 14% of cases (45/323) with a sensitivity and specificity of 13.9% (95% CI [10.4%, 18.2%]) and 100% (95% CI [98.9%, 100.0%]), respectively. The text-based visual alert displayed in 44 of 45 cases. The CPAS is a novel simple way to support clinical decision-making to identify and protect children at risk of (re)abuse. This study highlights multiple barriers that must be overcome to effectively design and implement a CPAS to protect at-risk children.

Standards-Based Clinical Decision Support Platform to Manage Patients Who Meet Guideline-Based Criteria for Genetic Evaluation of Familial Cancer

PURPOSE

The ubiquitous adoption of electronic health records (EHRs) with family health history (FHH) data provides opportunities for tailoring cancer screening strategies to individuals. We aimed to enable a standards-based clinical decision support (CDS) platform for identifying and managing patients who meet guidelines for genetic evaluation of hereditary cancer.

METHODS

The CDS platform (www.opencds.org) was used to implement algorithms based on the 2018 National Comprehensive Cancer Network guidelines for genetic evaluation of hereditary breast/ovarian and colorectal cancer. The platform was designed to be interfaced with different EHR systems via the Health Level Seven International Fast Healthcare Interoperability Resources standard. The platform was integrated with the Epic EHR and evaluated in a pilot study at an academic health care system.

RESULTS

The CDS platform was executed against a target population of 143,012 patients; 5,245 (3.7%) met criteria for genetic evaluation based on the FHH recorded in the EHR. In a clinical pilot study, genetic counselors attempted to reach out to 71 of the patients. Of those patients, 25 (35%) scheduled an appointment, 10 (14%) declined, 2 (3%) did not need genetic counseling, 7 (10%) said they would consider it in the future, and 27 (38%) were unreachable. To date, 13 (52%) of the scheduled patients completed visits, and 2 (15%) of those were found to have pathogenic variants in cancer predisposition genes.

CONCLUSION

A standards-based CDS platform integrated with EHR systems is a promising population-based approach to identify patients who are appropriate candidates for genetic evaluation of hereditary cancers.

Availability and usage of clinical decision support systems (CDSSs) in office-based primary care settings in the USA

BACKGROUND

A clinical decision support system (CDSS) covers a broad spectrum of applications, for example, screening reminders, can reduce malpractice, improve preventive services and enable better management of chronic conditions. CDSSs have traditionally been used successfully in large hospitals. The availability (ie, whether the function is provided by the software) and usage (ie, actual use) of a CDSS in office-based primary care settings, however, are less well studied.

OBJECTIVE

To establish a benchmark of CDSS availability and usage in office-based primary care settings, particularly given the large volume of visits in such settings.

METHODS

We used the 2015 Centers for Disease Control and Prevention's National Ambulatory Medical Care Survey to conduct secondary data analysis. We selected preventive services reminders and drug interaction alerts, along with several other variables as examples of a CDSS.

RESULTS

CDSS usage rates ranged from 68.5% to 100% among solo or non-solo primary care practices owned by physicians or physician groups that have electronic medical records (EMRs)/electronic health records (EHRs) and 44.7% to 96.1%, regardless of EMR/EHR status. According to proportion tests, solo practices had significantly lower CDSS usage and availability rates on several measures if the practice is entirely EMR/EHR based and significantly lower (16.3%-28.9%) CDSS usage rates than did non-solo practices on each measure, regardless of EMR/EHR status.

CONCLUSION

In the USA, a CDSS, especially under the categories of basic preventive reminders and drug interaction alerts, is used routinely between 68% and 100% in primary care if a practice is entirely EMR/EHR based. More work is needed, however, to determine the reasons for large usage gaps between solo and non-solo practices and to reduce such gaps.

Best practices for preventing malfunctions in rule-based clinical decision support alerts and reminders: Results of a Delphi study

OBJECTIVE

Developing effective and reliable rule-based clinical decision support (CDS) alerts and reminders is challenging. Using a previously developed taxonomy for alert malfunctions, we identified best practices for developing, testing, implementing, and maintaining alerts and avoiding malfunctions.

MATERIALS AND METHODS

We identified 72 initial practices from the literature, interviews with subject matter experts, and prior research. To refine, enrich, and prioritize the list of practices, we used the Delphi method with two rounds of consensus-building and refinement. We used a larger than normal panel of experts to include a wide representation of CDS subject matter experts from various disciplines.

RESULTS

28 experts completed Round 1 and 25 completed Round 2. Round 1 narrowed the list to 47 best practices in 7 categories: knowledge management, designing and specifying, building, testing, deployment, monitoring and feedback, and people and governance. Round 2 developed consensus on the importance and feasibility of each best practice.

DISCUSSION

The Delphi panel identified a range of best practices that may help to improve implementation of rule-based CDS and avert malfunctions. Due to limitations on resources and personnel, not everyone can implement all best practices. The most robust processes require investing in a data warehouse. Experts also pointed to the issue of shared responsibility between the healthcare organization and the electronic health record vendor.

CONCLUSION

These 47 best practices represent an ideal situation. The research identifies the balance between importance and difficulty, highlights the challenges faced by organizations seeking to implement CDS, and describes several opportunities for future research to reduce alert malfunctions.

Clinical decision support alert malfunctions: analysis and empirically derived taxonomy

Objective

To develop an empirically derived taxonomy of clinical decision support (CDS) alert malfunctions.

Materials and Methods

We identified CDS alert malfunctions using a mix of qualitative and quantitative methods: (1) site visits with interviews of chief medical informatics officers, CDS developers, clinical leaders, and CDS end users; (2) surveys of chief medical informatics officers; (3) analysis of CDS firing rates; and (4) analysis of CDS overrides. We used a multi-round, manual, iterative card sort to develop a multi-axial, empirically derived taxonomy of CDS malfunctions.

Results

We analyzed 68 CDS alert malfunction cases from 14 sites across the United States with diverse electronic health record systems. Four primary axes emerged: the cause of the malfunction, its mode of discovery, when it began, and how it affected rule firing. Build errors, conceptualization errors, and the introduction of new concepts or terms were the most frequent causes. User reports were the predominant mode of discovery. Many malfunctions within our database caused rules to fire for patients for whom they should not have (false positives), but the reverse (false negatives) was also common.

Discussion

Across organizations and electronic health record systems, similar malfunction patterns recurred. Challenges included updates to code sets and values, software issues at the time of system upgrades, difficulties with migration of CDS content between computing environments, and the challenge of correctly conceptualizing and building CDS.

Conclusion

CDS alert malfunctions are frequent. The empirically derived taxonomy formalizes the common recurring issues that cause these malfunctions, helping CDS developers anticipate and prevent CDS malfunctions before they occur or detect and resolve them expediently.

Implementation of Knowledge Management in Chinese Hospitals

The implementation of knowledge management (KM) in hospitals affects efficiency and outcomes of hospitals. However, few studies explored the implementation of KM in China. Twenty-two questions were designed concerning KM implementation status in over 50 hospitals. In order to understand the KM level and attitude to KM of the hospital's managers, a random sampling survey was conducted among 138 managers from 50 different scales of hospitals in 15 provinces of China. The survey showed that overall level of KM implementation in Chinese hospitals was still low and differed among different scales of hospitals (P<0.05, or P<0.01). In all the hospitals investigated, 63.8% did not implement KM yet, among which 46% even had not planned for that. 49.8% of the hospitals investigated had no training program about KM ever and the main source of hospital staff to get knowledge was internet. It suggested that hospital managers should make much more efforts to get to know and understand theories on KM, so that hospital KM could be promoted more rapidly.

Knowledge Management Implementation and the Tools Utilized in Healthcare for Evidence-Based Decision Making: A Systematic Review

BACKGROUND

Healthcare is a knowledge driven process and thus knowledge management and the tools to manage knowledge in healthcare sector are gaining attention. The aim of this systematic review is to investigate knowledge management implementation and knowledge management tools used in healthcare for informed decision making.

METHODS

Three databases, two journals websites and Google Scholar were used as sources for the review. The key terms used to search relevant articles include: "Healthcare and Knowledge Management"; "Knowledge Management Tools in Healthcare" and "Community of Practices in healthcare".

RESULTS

It was found that utilization of knowledge management in healthcare is encouraging. There exist numbers of opportunities for knowledge management implementation, though there are some barriers as well. Some of the opportunities that can transform healthcare are advances in health information and communication technology, clinical decision support systems, electronic health record systems, communities of practice and advanced care planning.

CONCLUSION

Providing the right knowledge at the right time, i.e., at the point of decision making by implementing knowledge management in healthcare is paramount. To do so, it is very important to use appropriate tools for knowledge management and user-friendly system because it can significantly improve the quality and safety of care provided for patients both at hospital and home settings.

Comparison of Overridden Medication-related Clinical Decision Support in the Intensive Care Unit between a Commercial System and a Legacy System

BACKGROUND

Electronic health records (EHRs) with clinical decision support (CDS) have shown to be effective at improving patient safety. Despite this, alerts delivered as part of CDS are overridden frequently, which is of concern in the critical care population as this group may have an increased risk of harm. Our organization recently transitioned from an internally-developed EHR to a commercial system. Data comparing various EHR systems, especially after transitions between EHRs, are needed to identify areas for improvement.

OBJECTIVES

To compare the two systems and identify areas for potential improvement with the new commercial system at a single institution.

METHODS

Overridden medication-related CDS alerts were included from October to December of the systems' respective years (legacy, 2011; commercial, 2015), restricted to three intensive care units. The two systems were compared with regards to CDS presentation and override rates for four types of CDS: drug-allergy, drug-drug interaction (DDI), geriatric and renal alerts. A post hoc analysis to evaluate for adverse drug events (ADEs) potentially resulting from overridden alerts was performed for 'contraindicated' DDIs via chart review.

RESULTS

There was a significant increase in provider exposure to alerts and alert overrides in the commercial system (commercial: n=5,535; legacy: n=1,030). Rates of overrides were higher for the allergy and DDI alerts (p<0.001) in the commercial system. Geriatric and renal alerts were significantly different in incidence and presentation between the two systems. No ADEs were identified in an analysis of 43 overridden contraindicated DDI alerts.

CONCLUSIONS

The vendor system had much higher rates of both alerts and overrides, although we did not find evidence of harm in a review of DDIs which were overridden. We propose recommendations for improving our current system which may be helpful to other similar institutions; improving both alert presentation and the underlying knowledge base appear important.

Analysis of clinical decision support system malfunctions: a case series and survey

OBJECTIVE

To illustrate ways in which clinical decision support systems (CDSSs) malfunction and identify patterns of such malfunctions.

MATERIALS AND METHODS

We identified and investigated several CDSS malfunctions at Brigham and Women's Hospital and present them as a case series. We also conducted a preliminary survey of Chief Medical Information Officers to assess the frequency of such malfunctions.

RESULTS

We identified four CDSS malfunctions at Brigham and Women's Hospital: (1) an alert for monitoring thyroid function in patients receiving amiodarone stopped working when an internal identifier for amiodarone was changed in another system; (2) an alert for lead screening for children stopped working when the rule was inadvertently edited; (3) a software upgrade of the electronic health record software caused numerous spurious alerts to fire; and (4) a malfunction in an external drug classification system caused an alert to inappropriately suggest antiplatelet drugs, such as aspirin, for patients already taking one. We found that 93% of the Chief Medical Information Officers who responded to our survey had experienced at least one CDSS malfunction, and two-thirds experienced malfunctions at least annually.

DISCUSSION

CDSS malfunctions are widespread and often persist for long periods. The failure of alerts to fire is particularly difficult to detect. A range of causes, including changes in codes and fields, software upgrades, inadvertent disabling or editing of rules, and malfunctions of external systems commonly contribute to CDSS malfunctions, and current approaches for preventing and detecting such malfunctions are inadequate.

CONCLUSION

CDSS malfunctions occur commonly and often go undetected. Better methods are needed to prevent and detect these malfunctions.

Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision

OBJECTIVE

The objective of this review is to summarize the state of the art of clinical decision support (CDS) circa 1990, review progress in the 25 year interval from that time, and provide a vision of what CDS might look like 25 years hence, or circa 2040.

METHOD

Informal review of the medical literature with iterative review and discussion among the authors to arrive at six axes (data, knowledge, inference, architecture and technology, implementation and integration, and users) to frame the review and discussion of selected barriers and facilitators to the effective use of CDS.

RESULT

In each of the six axes, significant progress has been made. Key advances in structuring and encoding standardized data with an increased availability of data, development of knowledge bases for CDS, and improvement of capabilities to share knowledge artifacts, explosion of methods analyzing and inferring from clinical data, evolution of information technologies and architectures to facilitate the broad application of CDS, improvement of methods to implement CDS and integrate CDS into the clinical workflow, and increasing sophistication of the end-user, all have played a role in improving the effective use of CDS in healthcare delivery.

CONCLUSION

CDS has evolved dramatically over the past 25 years and will likely evolve just as dramatically or more so over the next 25 years. Increasingly, the clinical encounter between a clinician and a patient will be supported by a wide variety of cognitive aides to support diagnosis, treatment, care-coordination, surveillance and prevention, and health maintenance or wellness.

Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development

OBJECTIVE

Clinical decision support (CDS) is essential for delivery of high-quality, cost-effective, and safe healthcare. The authors sought to evaluate the CDS capabilities across electronic health record (EHR) systems.

METHODS

We evaluated the CDS implementation capabilities of 8 Office of the National Coordinator for Health Information Technology Authorized Certification Body (ONC-ACB)-certified EHRs. Within each EHR, the authors attempted to implement 3 user-defined rules that utilized the various data and logic elements expected of typical EHRs and that represented clinically important evidenced-based care. The rules were: 1) if a patient has amiodarone on his or her active medication list and does not have a thyroid-stimulating hormone (TSH) result recorded in the last 12 months, suggest ordering a TSH; 2) if a patient has a hemoglobin A1c result >7% and does not have diabetes on his or her problem list, suggest adding diabetes to the problem list; and 3) if a patient has coronary artery disease on his or her problem list and does not have aspirin on the active medication list, suggest ordering aspirin.

RESULTS

Most evaluated EHRs lacked some CDS capabilities; 5 EHRs were able to implement all 3 rules, and the remaining 3 EHRs were unable to implement any of the rules. One of these did not allow users to customize CDS rules at all. The most frequently found shortcomings included the inability to use laboratory test results in rules, limit rules by time, use advanced Boolean logic, perform actions from the alert interface, and adequately test rules.

CONCLUSION

Significant improvements in the EHR certification and implementation procedures are necessary.

Comparative outcome studies of clinical decision support software: limitations to the practice of evidence-based system acquisition

Clinical decision support systems (CDSSs) assist clinicians with patient diagnosis and treatment. However, inadequate attention has been paid to the process of selecting and buying systems. The diversity of CDSSs, coupled with research obstacles, marketplace limitations, and legal impediments, has thwarted comparative outcome studies and reduced the availability of reliable information and advice for purchasers. We review these limitations and recommend several comparative studies, which were conducted in phases; studies conducted in phases and focused on limited outcomes of safety, efficacy, and implementation in varied clinical settings. Additionally, we recommend the increased availability of guidance tools to assist purchasers with evidence-based purchases. Transparency is necessary in purchasers' reporting of system defects and vendors' disclosure of marketing conflicts of interest to support methodologically sound studies. Taken together, these measures can foster the evolution of evidence-based tools that, in turn, will enable and empower system purchasers to make wise choices and improve the care of patients.

Understanding critical barriers to implementing a clinical information system in a nursing home through the lens of a socio-technical perspective

This paper addresses key barriers to implementing a clinical information system (CIS) in a Hong Kong nursing home setting, from a healthcare specific socio-technical perspective. Data was collected through field observations (n = 12) and semi-structured individual interviews (n = 18) of CIS stakeholders in a Hong Kong nursing home, and analyzed using the immersion/crystallization approach. Complex interactions relevant to our case were contextualized and interpreted within the perspective of the Sittig-Singh Healthcare Socio-Technical Framework (HSTF). Three broad clusters of implementation barriers from the eight HSTF dimensions were identified: (a) Infrastructure-based barriers, which relate to conflict between government regulations and system functional needs of users; lack of financial support; inconsistency between workflow, work policy, and procedures; and inadequacy of hardware-software infrastructural and technical support; (b) Process-based barriers, which relate to mismatch between the technology, existing work practice and workflow, and communication; low system speed, accessibility, and stability; deficient computer literacy; more experience in health care profession; clinical content inadequacy and unavailability; as well as poor system usefulness and user interface design; and (c) Outcome-based barriers, which relate to the lack of measurement and monitoring of system effectiveness. Two additional dimensions underlining the importance of the ability of a CIS to change are proposed to extend the Sittig-Singh HSTF. First, advocacy would promote the articulation and influence of changes in the system and subsequent outcomes by CIS stakeholders, and second, adaptability would ensure the ability of the system to adjust to emerging needs. The broad set of discovered implementation shortcomings expands prior research on why CIS can fail in nursing home settings. Moreover, our investigation offers a knowledge base and recommendations that can serve as a guide for future implementation strategies and policies in CIS initiatives.

Clinical decision support for colon and rectal surgery: an overview

Clinical decision support (CDS) has been shown to improve clinical processes, promote patient safety, and reduce costs in healthcare settings, and it is now a requirement for clinicians as part of the Meaningful Use Regulation. However, most evidence for CDS has been evaluated primarily in internal medicine care settings, and colon and rectal surgery (CRS) has unique needs with CDS that are not frequently described in the literature. The authors reviewed published literature in informatics and medical journals, combined with expert opinion to define CDS, describe the evidence for CDS, outline the implementation process for CDS, and present applications of CDS in CRS.CDS functionalities such as order sets, documentation templates, and order facilitation aids are most often described in the literature and most likely to be beneficial in CRS. Further research is necessary to identify and better evaluate additional CDS systems in the setting of CRS.

Measuring agreement between decision support reminders: the cloud vs. the local expert

BACKGROUND

A cloud-based clinical decision support system (CDSS) was implemented to remotely provide evidence-based guideline reminders in support of preventative health. Following implementation, we measured the agreement between preventive care reminders generated by an existing, local CDSS and the new, cloud-based CDSS operating on the same patient visit data.

METHODS

Electronic health record data for the same set of patients seen in primary care were sent to both the cloud-based web service and local CDSS. The clinical reminders returned by both services were captured for analysis. Cohen's Kappa coefficient was calculated to compare the two sets of reminders. Kappa statistics were further adjusted for prevalence and bias due to the potential effects of bias in the CDS logic and prevalence in the relative small sample of patients.

RESULTS

The cloud-based CDSS generated 965 clinical reminders for 405 patient visits over 3 months. The local CDSS returned 889 reminders for the same patient visit data. When adjusted for prevalence and bias, observed agreement varied by reminder from 0.33 (95% CI 0.24 - 0.42) to 0.99 (95% CI 0.97 - 1.00) and demonstrated almost perfect agreement for 7 of the 11 reminders.

CONCLUSIONS

Preventive care reminders delivered by two disparate CDS systems show substantial agreement. Subtle differences in rule logic and terminology mapping appear to account for much of the discordance. Cloud-based CDSS therefore show promise, opening the door for future development and implementation in support of health care providers with limited resources for knowledge management of complex logic and rules.

A qualitative study of the activities performed by people involved in clinical decision support: recommended practices for success

Objective

To describe the activities performed by people involved in clinical decision support (CDS) at leading sites.

Materials and methods

We conducted ethnographic observations at seven diverse sites with a history of excellence in CDS using the Rapid Assessment Process and analyzed the data using a series of card sorts, informed by Linstone's Multiple Perspectives Model.

Results

We identified 18 activities and grouped them into four areas. Area 1: Fostering relationships across the organization, with activities (a) training and support, (b) visibility/presence on the floor, (c) liaising between people, (d) administration and leadership, (e) project management, (f) cheerleading/buy-in/sponsorship, (g) preparing for CDS implementation. Area 2: Assembling the system with activities (a) providing technical support, (b) CDS content development, (c) purchasing products from vendors (d) knowledge management, (e) system integration. Area 3: Using CDS to achieve the organization's goals with activities (a) reporting, (b) requirements-gathering/specifications, (c) monitoring CDS, (d) linking CDS to goals, (e) managing data. Area 4: Participation in external policy and standards activities (this area consists of only a single activity). We also identified a set of recommendations associated with these 18 activities.

Discussion

All 18 activities we identified were performed at all sites, although the way they were organized into roles differed substantially. We consider these activities critical to the success of a CDS program.

Conclusions

A series of activities are performed by sites strong in CDS, and sites adopting CDS should ensure they incorporate these activities into their efforts.

Lessons learned for collaborative clinical content development

BACKGROUND

Site-specific content configuration of vendor-based Electronic Health Records (EHRs) is a vital step in the development of standardized and interoperable content that can be used for clinical decision-support, reporting, care coordination, and information exchange. The multi-site, multi-stakeholder Acute Care Documentation (ACD) project at Partners Healthcare Systems (PHS) aimed to develop highly structured clinical content with adequate breadth and depth to meet the needs of all types of acute care clinicians at two academic medical centers. The Knowledge Management (KM) team at PHS led the informatics and knowledge management effort for the project.

OBJECTIVES

We aimed to evaluate the role, governance, and project management processes and resources for the KM team's effort as part of the standardized clinical content creation.

METHODS

We employed the Center for Disease Control's six step Program Evaluation Framework to guide our evaluation steps. We administered a forty-four question, open-ended, semi-structured voluntary survey to gather focused, credible evidence from members of the KM team. Qualitative open-coding was performed to identify themes for lessons learned and concluding recommendations.

RESULTS

Six surveys were completed. Qualitative data analysis informed five lessons learned and thirty specific recommendations associated with the lessons learned. The five lessons learned are: 1) Assess and meet knowledge needs and set expectations at the start of the project; 2) Define an accountable decision-making process; 3) Increase team meeting moderation skills; 4) Ensure adequate resources and competency training with online asynchronous collaboration tools; 5) Develop focused, goal-oriented teams and supportive, consultative service based teams.

CONCLUSIONS

Knowledge management requirements for the development of standardized clinical content within a vendor-based EHR among multi-stakeholder teams and sites include: 1) assessing and meeting informatics knowledge needs, 2) setting expectations and standardizing the process for decision-making, and 3) ensuring the availability of adequate resources and competency training.

A pilot study of distributed knowledge management and clinical decision support in the cloud

OBJECTIVE

Implement and perform pilot testing of web-based clinical decision support services using a novel framework for creating and managing clinical knowledge in a distributed fashion using the cloud. The pilot sought to (1) develop and test connectivity to an external clinical decision support (CDS) service, (2) assess the exchange of data to and knowledge from the external CDS service, and (3) capture lessons to guide expansion to more practice sites and users.

MATERIALS AND METHODS

The Clinical Decision Support Consortium created a repository of shared CDS knowledge for managing hypertension, diabetes, and coronary artery disease in a community cloud hosted by Partners HealthCare. A limited data set for primary care patients at a separate health system was securely transmitted to a CDS rules engine hosted in the cloud. Preventive care reminders triggered by the limited data set were returned for display to clinician end users for review and display. During a pilot study, we (1) monitored connectivity and system performance, (2) studied the exchange of data and decision support reminders between the two health systems, and (3) captured lessons.

RESULTS

During the six month pilot study, there were 1339 patient encounters in which information was successfully exchanged. Preventive care reminders were displayed during 57% of patient visits, most often reminding physicians to monitor blood pressure for hypertensive patients (29%) and order eye exams for patients with diabetes (28%). Lessons learned were grouped into five themes: performance, governance, semantic interoperability, ongoing adjustments, and usability.

DISCUSSION

Remote, asynchronous cloud-based decision support performed reasonably well, although issues concerning governance, semantic interoperability, and usability remain key challenges for successful adoption and use of cloud-based CDS that will require collaboration between biomedical informatics and computer science disciplines.

CONCLUSION

Decision support in the cloud is feasible and may be a reasonable path toward achieving better support of clinical decision-making across the widest range of health care providers.

Evolution in clinical knowledge management strategy at Intermountain Healthcare

In this manuscript, we present an overview of the clinical knowledge management strategy at Intermountain Healthcare in support of our electronic medical record systems. Intermountain first initiated efforts in developing a centralized enterprise knowledge repository in 2001. Applications developed, areas of emphasis served, and key areas of focus are presented. We also detail historical and current areas of emphasis, in response to business needs.

Transitioning from a legacy EHR to a commercial, vendor-supplied, EHR: one academic health system's experience

OBJECTIVE

Describe the planning, decisions, and implementation results experienced during the large-scale transition from one EHR to another throughout a large academic health system, which occurred simultaneously throughout both in-patient and all ambulatory settings.

METHODS

Review of internal decision-making documents, interviews with key participants, and data from conversion software.

RESULTS

Over 7,000 unique users caring for a population of more than 1.2 million patients in both inpatient and outpatient venues and distributed across two states were successfully transitioned to a new EHR simultaneously. Challenges in data conversion were encountered resulting in more work for end-users than desired or anticipated. Users continued to access older information (principally schedules) in the legacy EHR one year later.

CONCLUSION

Data conversion from one EHR to another can be unsuccessful due to differences in how EHR's structure data obtained from underlying feeder applications or databases. Abstraction of only the pertinent clinical content is difficult in the context of transitioning to a new EHR. Clinicians require facile access to legacy content that can be achieved by implanting CCOW compliant solutions.

Key principles for a national clinical decision support knowledge sharing framework: synthesis of insights from leading subject matter experts

OBJECTIVE

To identify key principles for establishing a national clinical decision support (CDS) knowledge sharing framework.

MATERIALS AND METHODS

As part of an initiative by the US Office of the National Coordinator for Health IT (ONC) to establish a framework for national CDS knowledge sharing, key stakeholders were identified. Stakeholders' viewpoints were obtained through surveys and in-depth interviews, and findings and relevant insights were summarized. Based on these insights, key principles were formulated for establishing a national CDS knowledge sharing framework.

RESULTS

Nineteen key stakeholders were recruited, including six executives from electronic health record system vendors, seven executives from knowledge content producers, three executives from healthcare provider organizations, and three additional experts in clinical informatics. Based on these stakeholders' insights, five key principles were identified for effectively sharing CDS knowledge nationally. These principles are (1) prioritize and support the creation and maintenance of a national CDS knowledge sharing framework; (2) facilitate the development of high-value content and tooling, preferably in an open-source manner; (3) accelerate the development or licensing of required, pragmatic standards; (4) acknowledge and address medicolegal liability concerns; and (5) establish a self-sustaining business model.

DISCUSSION

Based on the principles identified, a roadmap for national CDS knowledge sharing was developed through the ONC's Advancing CDS initiative.

CONCLUSION

The study findings may serve as a useful guide for ongoing activities by the ONC and others to establish a national framework for sharing CDS knowledge and improving clinical care.

Context-based electronic health record: toward patient specific healthcare

Due to the increasingly data-intensive clinical environment, physicians now have unprecedented access to detailed clinical information from a multitude of sources. However, applying this information to guide medical decisions for a specific patient case remains challenging. One issue is related to presenting information to the practitioner: displaying a large (irrelevant) amount of information often leads to information overload. Next-generation interfaces for the electronic health record (EHR) should not only make patient data easily searchable and accessible, but also synthesize fragments of evidence documented in the entire record to understand the etiology of a disease and its clinical manifestation in individual patients. In this paper, we describe our efforts toward creating a context-based EHR, which employs biomedical ontologies and (graphical) disease models as sources of domain knowledge to identify relevant parts of the record to display. We hypothesize that knowledge (e.g., variables, relationships) from these sources can be used to standardize, annotate, and contextualize information from the patient record, improving access to relevant parts of the record and informing medical decision making. To achieve this goal, we describe a framework that aggregates and extracts findings and attributes from free-text clinical reports, maps findings to concepts in available knowledge sources, and generates a tailored presentation of the record based on the information needs of the user. We have implemented this framework in a system called Adaptive EHR, demonstrating its capabilities to present and synthesize information from neurooncology patients. This paper highlights the challenges and potential applications of leveraging disease models to improve the access, integration, and interpretation of clinical patient data.

A multi-layered framework for disseminating knowledge for computer-based decision support

BACKGROUND

There are several challenges in encoding guideline knowledge in a form that is portable to different clinical sites, including the heterogeneity of clinical decision support (CDS) tools, of patient data representations, and of workflows.

METHODS

We have developed a multi-layered knowledge representation framework for structuring guideline recommendations for implementation in a variety of CDS contexts. In this framework, guideline recommendations are increasingly structured through four layers, successively transforming a narrative text recommendation into input for a CDS system. We have used this framework to implement rules for a CDS service based on three guidelines. We also conducted a preliminary evaluation, where we asked CDS experts at four institutions to rate the implementability of six recommendations from the three guidelines.

CONCLUSION

The experience in using the framework and the preliminary evaluation indicate that this approach has promise in creating structured knowledge, to implement in CDS systems, that is usable across organizations.