What may help or hinder the implementation of computerized decision support systems (CDSSs): a focus group study with physicians

Barriers to and Facilitators of Key Stakeholders Influencing Successful Digital Implementation of Remote Monitoring Solutions: Mixed Methods Analysis

BACKGROUND

Implementation of remote monitoring solutions and digital alerting tools in health care has historically been challenging, despite the impetus provided by the COVID-19 pandemic. To date, a health systems-based approach to systematically describe barriers and facilitators across multiple domains has not been undertaken.

OBJECTIVE

We aimed to undertake a comprehensive mixed methods analysis of barriers and facilitators for successful implementation of remote monitoring and digital alerting tools in complex health organizations.

METHODS

A mixed methods approach using a modified Technology Acceptance Model questionnaire and semistructured interviews mapped to the validated fit among humans, organizations, and technology (HOT-fit) framework was undertaken. Likert frequency responses and deductive thematic analyses were performed.

RESULTS

A total of 11 participants responded to the questionnaire and 18 participants to the interviews. Key barriers and facilitators could be mapped onto 6 dimensions, which incorporated aspects of digitization: system use (human), user satisfaction (human), environment (organization), structure (organization), information and service quality (technology), and system quality (technology).

CONCLUSIONS

The recommendations proposed can enhance the potential for future remote sensing solutions to be more successfully integrated in health care practice, resulting in more successful use of "virtual wards."

TRIAL REGISTRATION

ClinicalTrials.gov NCT05321004; https://www.clinicaltrials.gov/study/NCT05321004.

A Knowledge-based Decision Support System for recommending safe recipes to individuals with dysphagia

BACKGROUND

Dysphagia is a disorder that can be associated to several pathological conditions, including neuromuscular diseases, with significant impact on quality of life. Dysphagia often leads to malnutrition, as a consequence of the dietary changes made by patients or their caregivers, who may deliberately decide to reduce or avoid specific food consistencies (because they are not perceived as safe), and the lack of knowledge in how to process foods are critics. Such dietary changes often result in unbalanced nutrients intake, which can have significant consequences for frail patients. This paper presents the development of a prototypical novel ontology-based Decision Support System (DSS) to support neuromuscular patients with dysphagia (following a per-oral nutrition) and their caregivers in preparing nutritionally balanced and safe meals.

METHOD

After reviewing scientific literature, we developed in collaboration with Ear-Nose-Throat (ENT) specialists, neurologists, and dieticians the DSS formalizes expert knowledge to suggest recipes that are considered safe according to patient's consistency limitations and dysphagia severity and also nutritionally well-balanced.

RESULTS

The prototype can be accessed via digital applications both by physicians to generate and verify the recommendations, and by the patients and their caregivers to follow the step-by-step procedures to autonomously prepare and process one or more recipe. The system is evaluated with 9 clinicians to assess the quality of the DSS's suggested recipes and its acceptance in clinical practice.

CONCLUSIONS

Preliminary results suggest a global positive outcome for the recipes inferred by the DSS and a good usability of the system.

The Facilitators and Barriers of the Implementation of a Clinical Decision Support System for Breast Cancer Multidisciplinary Team Meetings-An Interview Study

BACKGROUND

AI-driven clinical decision support systems (CDSSs) hold promise for multidisciplinary team meetings (MDTMs). This study aimed to uncover the hurdles and aids in implementing CDSSs during breast cancer MDTMs.

METHODS

Twenty-four core team members from three hospitals engaged in semi-structured interviews, revealing a collective interest in experiencing CDSS workflows in clinical practice. All interviews were audio recorded, transcribed verbatim and analyzed anonymously. A standardized approach, 'the framework method', was used to create an analytical framework for data analysis, which was performed by two independent researchers.

RESULTS

Positive aspects included improved data visualization, time-saving features, automated trial matching, and enhanced documentation transparency. However, challenges emerged, primarily concerning data connectivity, guideline updates, the accuracy of AI-driven suggestions, and the risk of losing human involvement in decision making. Despite the complexities involved in CDSS development and integration, clinicians demonstrated enthusiasm to explore its potential benefits.

CONCLUSIONS

Acknowledging the multifaceted nature of this challenge, insights into the barriers and facilitators identified in this study offer a potential roadmap for smoother future implementations. Understanding these factors could pave the way for more effective utilization of CDSSs in breast cancer MDTMs, enhancing patient care through informed decision making.

General practitioners' user experience of the nationwide digital decision support system in primary care

Objectives

The aim of the study is to describe the user experiences of a nationwide digital decision support system (DDSS).

Summary of background data

DDSSs have the potential to improve the quality and safety of healthcare services by supporting clinical decision-making with evidence-based recommendations. Due to a lack of knowledge, it is difficult to assess whether DDSSs are fulfilling their purpose. In Estonia, a nationwide DDSS for general practitioners (GPs) was implemented in 2020. To understand the impact of DDSS on the quality of care in the Estonian context and meet the demands of healthcare, it is necessary to gather information about the experiences of the users. This is the first study that examines the experiences of GPs on the use of DDSS nationwide.

Methods

A qualitative descriptive study was conducted based on snowball sampling. Semi-structured interviews were performed in February-March 2022 with nine GPs. Data were analyzed by thematic analysis. A total of six themes and 16 subthemes emerged from the data.

Results

A total of six themes and 16 subthemes emerged from the data. The following themes were identified: user-friendliness, DDSS use in clinical practice, benefits of the DDSS, and the impact of the DDSS on GPs' work, barriers to using the DDSS, and suggestions for improving the user experience. The results of the study are important, as they address and contribute to the relevant aspects of digital health in primary care.

Conclusion

GPs shared their individual user experiences, including user-perceived barriers and enabling factors that influence the implementation and use of a decision support system in primary care settings. It is revealed that GPs have different benefits and barriers depending on the topic discussed. Future research should evaluate the functioning of the DDSS and the quality of the decisions it provides by observing and evaluating patient records. Systematic user experiences need to be collected and examined to ensure the usability and sustainability of the DDSS.

Acceptance and use of a clinical decision support system in musculoskeletal pain disorders - the SupportPrim project

BACKGROUND

We have developed a clinical decision support system (CDSS) based on methods from artificial intelligence to support physiotherapists and patients in the decision-making process of managing musculoskeletal (MSK) pain disorders in primary care. The CDSS finds the most similar successful patients from the past to give treatment recommendations for a new patient. Using previous similar patients with successful outcomes to advise treatment moves management of MSK pain patients from one-size fits all recommendations to more individually tailored treatment. This study aimed to summarise the development and explore the acceptance and use of the CDSS for MSK pain patients.

METHODS

This qualitative study was carried out in the Norwegian physiotherapy primary healthcare sector between October and November 2020, ahead of a randomised controlled trial. We included four physiotherapists and three of their patients, in total 12 patients, with musculoskeletal pain in the neck, shoulder, back, hip, knee or complex pain. We conducted semi-structured telephone interviews with all participants. The interviews were analysed using the Framework Method.

RESULTS

Overall, both the physiotherapists and patients found the system acceptable and usable. Important findings from the analysis of the interviews were that the CDSS was valued as a preparatory and exploratory tool, facilitating the therapeutic relationship. However, the physiotherapists used the system mainly to support their previous and current practice rather than involving patients to a greater extent in decisions and learning from previous successful patients.

CONCLUSIONS

The CDSS was acceptable and usable to both the patients and physiotherapists. However, the system appeared not to considerably influence the physiotherapists' clinical reasoning and choice of treatment based on information from most similar successful patients. This could be due to a smaller than optimal number of previous patients in the CDSS or insufficient clinical implementation. Extensive training of physiotherapists should not be underestimated to build understanding and trust in CDSSs.

[Factors influencing the implementation of AI-based decision support systems for antibiotic prescription in hospitals: a qualitative analysis from the perspective of health professionals]

BACKGROUND

Decision support systems based on artificial intelligence might optimize antibiotic prescribing in hospitals and prevent the development of antimicrobial resistance. The aim of this study was to identify impeding and facilitating factors for successful implementation from the perspective of health professionals.

METHODS

Problem-centered individual interviews were conducted with health professionals working in hospitals. Data evaluation was based on the structured qualitative content analysis according Kuckartz.

RESULTS

Attitudes of health professionals were presented along the Human-Organization -Technology-fit model. Technological and organizational themes were the most important factors for system implementation. Especially, compatibility with existing systems and user-friendliness were seen to play a major role in successful implementation. Additionally, the training of potential users and the technical equipment of the organization were considered essential. Finally, the importance of promoting technical skills of potential users in the long term and creating trust in the benefits of the system were highlighted.

CONCLUSION

The identified factors provide a basis for prioritizing and quantifying needs and attitudes in a next step. It becomes clear that, beside technological factors, attention to context-specific and user-related conditions are of fundamental importance to ensure successful implementation and system trust in the long term.

Physicians' Perspectives on AI in Clinical Decision Support Systems: Interview Study of the CURATE.AI Personalized Dose Optimization Platform

BACKGROUND

Physicians play a key role in integrating new clinical technology into care practices through user feedback and growth propositions to developers of the technology. As physicians are stakeholders involved through the technology iteration process, understanding their roles as users can provide nuanced insights into the workings of these technologies that are being explored. Therefore, understanding physicians' perceptions can be critical toward clinical validation, implementation, and downstream adoption. Given the increasing prevalence of clinical decision support systems (CDSSs), there remains a need to gain an in-depth understanding of physicians' perceptions and expectations toward their downstream implementation. This paper explores physicians' perceptions of integrating CURATE.AI, a novel artificial intelligence (AI)-based and clinical stage personalized dosing CDSSs, into clinical practice.

OBJECTIVE

This study aims to understand physicians' perspectives of integrating CURATE.AI for clinical work and to gather insights on considerations of the implementation of AI-based CDSS tools.

METHODS

A total of 12 participants completed semistructured interviews examining their knowledge, experience, attitudes, risks, and future course of the personalized combination therapy dosing platform, CURATE.AI. Interviews were audio recorded, transcribed verbatim, and coded manually. The data were thematically analyzed.

RESULTS

Overall, 3 broad themes and 9 subthemes were identified through thematic analysis. The themes covered considerations that physicians perceived as significant across various stages of new technology development, including trial, clinical implementation, and mass adoption.

CONCLUSIONS

The study laid out the various ways physicians interpreted an AI-based personalized dosing CDSS, CURATE.AI, for their clinical practice. The research pointed out that physicians' expectations during the different stages of technology exploration can be nuanced and layered with expectations of implementation that are relevant for technology developers and researchers.

Expectation of clinical decision support systems: a survey study among nephrologist end-users

BACKGROUND

Chronic kidney disease (CKD), a major public health problem with differing disease etiologies, leads to complications, comorbidities, polypharmacy, and mortality. Monitoring disease progression and personalized treatment efforts are crucial for long-term patient outcomes. Physicians need to integrate different data levels, e.g., clinical parameters, biomarkers, and drug information, with medical knowledge. Clinical decision support systems (CDSS) can tackle these issues and improve patient management. Knowledge about the awareness and implementation of CDSS in Germany within the field of nephrology is scarce.

PURPOSE

Nephrologists' attitude towards any CDSS and potential CDSS features of interest, like adverse event prediction algorithms, is important for a successful implementation. This survey investigates nephrologists' experiences with and expectations towards a useful CDSS for daily medical routine in the outpatient setting.

METHODS

The 38-item questionnaire survey was conducted either by telephone or as a do-it-yourself online interview amongst nephrologists across all of Germany. Answers were collected and analysed using the Electronic Data Capture System REDCap, as well as Stata SE 15.1, and Excel. The survey consisted of four modules: experiences with CDSS (M1), expectations towards a helpful CDSS (M2), evaluation of adverse event prediction algorithms (M3), and ethical aspects of CDSS (M4). Descriptive statistical analyses of all questions were conducted.

RESULTS

The study population comprised 54 physicians, with a response rate of about 80-100% per question. Most participants were aged between 51-60 years (45.1%), 64% were male, and most participants had been working in nephrology out-patient clinics for a median of 10.5 years. Overall, CDSS use was poor (81.2%), often due to lack of knowledge about existing CDSS. Most participants (79%) believed CDSS to be helpful in the management of CKD patients with a high willingness to try out a CDSS. Of all adverse event prediction algorithms, prediction of CKD progression (97.8%) and in-silico simulations of disease progression when changing, e. g., lifestyle or medication (97.7%) were rated most important. The spectrum of answers on ethical aspects of CDSS was diverse.

CONCLUSION

This survey provides insights into experience with and expectations of out-patient nephrologists on CDSS. Despite the current lack of knowledge on CDSS, the willingness to integrate CDSS into daily patient care, and the need for adverse event prediction algorithms was high.

Optimizing Best Practice Advisory alerts in electronic medical records with a multi-pronged strategy at a tertiary care hospital in Singapore

Objective

Clinical decision support (CDS) alerts can aid in improving patient care. One CDS functionality is the Best Practice Advisory (BPA) alert notification system, wherein BPA alerts are automated alerts embedded in the hospital's electronic medical records (EMR). However, excessive alerts can change clinician behavior; redundant and repetitive alerts can contribute to alert fatigue. Alerts can be optimized through a multipronged strategy. Our study aims to describe these strategies adopted and evaluate the resultant BPA alert optimization outcomes.

Materials and Methods

This retrospective single-center study was done at Jurong Health Campus. Aggregated, anonymized data on patient demographics and alert statistics were collected from January 1, 2018 to December 31, 2021. "Preintervention" period was January 1-December 31, 2018, and "postintervention" period was January 1-December 31, 2021. The intervention period was the intervening period. Categorical variables were reported as frequencies and proportions and compared using the chi-square test. Continuous data were reported as median (interquartile range, IQR) and compared using the Wilcoxon rank-sum test. Statistical significance was defined at P < .05.

Results

There was a significant reduction of 59.6% in the total number of interruptive BPA alerts, despite an increase in the number of unique BPAs from 54 to 360 from pre- to postintervention. There was a 74% reduction in the number of alerts from the 7 BPAs that were optimized from the pre- to postintervention period. There was a significant increase in percentage of overall interruptive BPA alerts with action taken (8 [IQR 7.7-8.4] to 54.7 [IQR 52.5-58.9], P-value < .05) and optimized BPAs with action taken (32.6 [IQR 32.3-32.9] to 72.6 [IQR 64.3-73.4], P-value < .05). We estimate that the reduction in alerts saved 3600 h of providers' time per year.

Conclusions

A significant reduction in interruptive alert volume, and a significant increase in action taken rates despite manifold increase in the number of unique BPAs could be achieved through concentrated efforts focusing on governance, data review, and visualization using a system-embedded tool, combined with the CDS Five Rights framework, to optimize alerts. Improved alert compliance was likely multifactorial-due to decreased repeated alert firing for the same patient; better awareness due to stakeholders' involvement; and less fatigue since unnecessary alerts were removed. Future studies should prospectively focus on patients' clinical chart reviews to assess downstream effects of various actions taken, identify any possibility of harm, and collect end-user feedback regarding the utility of alerts.

Healthcare Professionals’ Perceptions, Barriers, and Facilitators towards Adopting Computerised Clinical Decision Support Systems in Antimicrobial Stewardship in Jordanian Hospitals

Understanding healthcare professionals’ perceptions towards a computerised decision support system (CDSS) may provide a platform for the determinants of the successful adoption and implementation of CDSS. This cross-sectional study examined healthcare professionals’ perceptions, barriers, and facilitators to adopting a CDSS for antibiotic prescribing in Jordanian hospitals. This study was conducted among healthcare professionals in Jordan’s two tertiary and teaching hospitals over four weeks (June–July 2021). Data were collected in a paper-based format from senior and junior prescribers and non-prescribers (n = 254) who agreed to complete a questionnaire. The majority (n = 184, 72.4%) were aware that electronic prescribing and electronic health record systems could be used specifically to facilitate antibiotic use and prescribing. The essential facilitator made CDSS available in a portable format (n = 224, 88.2%). While insufficient training to use CDSS was the most significant barrier (n = 175, 68.9%). The female providers showed significantly lower awareness (p = 0.006), and the nurses showed significantly higher awareness (p = 0.041) about using electronic prescribing and electronic health record systems. This study examined healthcare professionals’ perceptions of adopting CDSS in antimicrobial stewardship (AMS) and shed light on the perceived barriers and facilitators to adopting CDSS in AMS, reducing antibiotic resistance, and improving patient safety. Furthermore, results would provide a framework for other hospital settings concerned with implementing CDSS in AMS and inform policy decision-makers to react by implementing the CDSS system in Jordan and globally. Future studies should concentrate on establishing policies and guidelines and a framework to examine the adoption of the CDSS for AMS.

The Reasons for Physicians and Pharmacists' Acceptance of Clinical Support Systems in Saudi Arabia

This research aims to identify the technological and non-technological factors influencing user acceptance of the CDSS in a group of healthcare facilities in Saudi Arabia. The study proposes an integrated model that indicates the factors to be considered when designing and evaluating CDSS. This model is developed by integrating factors from the "Fit between Individuals, Task, and Technology" (FITT) framework into the three domains of the human, organization, and technology-fit (HOT-fit) model. The resulting FITT-HOT-fit integrated model was tested using a quantitative approach to evaluate the currently implemented CDSS as a part of Hospital Information System BESTCare 2.0 in the Saudi Ministry of National Guard Health Affairs. For data collection, a survey questionnaire was conducted at all Ministry of National Guard Health Affairs hospitals. Then, the collected survey data were analyzed using Structural Equation Modeling (SEM). This analysis included measurement instrument reliability, discriminant validity, convergent validity, and hypothesis testing. Moreover, a CDSS usage data sample was extracted from the data warehouse to be analyzed as an additional data source. The results of the hypotheses test show that usability, availability, and medical history accessibility are critical factors influencing user acceptance of CDSS. This study provides prudence about healthcare facilities and their higher management to adopt CDSS.

Machine learning-driven clinical decision support system for concept-based searching: a field trial in a Norwegian hospital

BACKGROUND

Natural language processing (NLP) based clinical decision support systems (CDSSs) have demonstrated the ability to extract vital information from patient electronic health records (EHRs) to facilitate important decision support tasks. While obtaining accurate, medical domain interpretable results is crucial, it is demanding because real-world EHRs contain many inconsistencies and inaccuracies. Further, testing of such machine learning-based systems in clinical practice has received limited attention and are yet to be accepted by clinicians for regular use.

METHODS

We present our results from the evaluation of an NLP-driven CDSS developed and implemented in a Norwegian Hospital. The system incorporates unsupervised and supervised machine learning combined with rule-based algorithms for clinical concept-based searching to identify and classify allergies of concern for anesthesia and intensive care. The system also implements a semi-supervised machine learning approach to automatically annotate medical concepts in the narrative.

RESULTS

Evaluation of system adoption was performed by a mixed methods approach applying The Unified Theory of Acceptance and Use of Technology (UTAUT) as a theoretical lens. Most of the respondents demonstrated a high degree of system acceptance and expressed a positive attitude towards the system in general and intention to use the system in the future. Increased detection of patient allergies, and thus improved quality of practice and patient safety during surgery or ICU stays, was perceived as the most important advantage of the system.

CONCLUSIONS

Our combined machine learning and rule-based approach benefits system performance, efficiency, and interpretability. The results demonstrate that the proposed CDSS increases detection of patient allergies, and that the system received high-level acceptance by the clinicians using it. Useful recommendations for further system improvements and implementation initiatives are reducing the quantity of alarms, expansion of the system to include more clinical concepts, closer EHR system integration, and more workstations available at point of care.

Using an Integrated Framework to Investigate the Facilitators and Barriers of Health Information Technology Implementation in Noncommunicable Disease Management: Systematic Review

BACKGROUND

Noncommunicable disease (NCD) management is critical for reducing attributable health burdens. Although health information technology (HIT) is a crucial strategy to improve chronic disease management, many health care systems have failed in implementing HIT. There has been a lack of research on the implementation process of HIT for chronic disease management.

OBJECTIVE

We aimed to identify the barriers and facilitators of HIT implementation, analyze how these factors influence the implementation process, and identify key areas for future action. We will develop a framework for understanding implementation determinants to synthesize available evidence.

METHODS

We conducted a systematic review to understand the barriers and facilitators of the implementation process. We searched MEDLINE, Cochrane, Embase, Scopus, and CINAHL for studies published between database inception and May 5, 2022. Original studies involving HIT-related interventions for NCD management published in peer-reviewed journals were included. Studies that did not discuss relevant outcome measures or did not have direct contact with or observation of stakeholders were excluded. The analysis was conducted in 2 parts. In part 1, we analyzed how the intrinsic attributes of HIT interventions affect the successfulness of implementation by using the intervention domain of the Consolidated Framework for Implementation Research (CFIR). In part 2, we focused on the extrinsic factors of HIT using an integrated framework, which was developed based on the CFIR and the levels of change framework by Ferlie and Shortell.

RESULTS

We identified 51 papers with qualitative, mixed-method, and cross-sectional methodologies. Included studies were heterogeneous regarding disease populations and HIT interventions. In part 1, having a relative advantage over existing health care systems was the most prominent intrinsic facilitator (eg, convenience, improvement in quality of care, and increase in access). Poor usability was the most noted intrinsic barrier of HIT. In part 2, we mapped the various factors of implementation to the integrated framework (the coordinates are shown as level of change-CFIR). The key barriers to the extrinsic factors of HIT included health literacy and lack of digital skills (individual-characteristics of individuals). The key facilitators included physicians' suggestions, cooperation (interpersonal-process), integration into a workflow, and adequate management of data (organizational-inner setting). The importance of health data security was identified. Self-efficacy issues of patients and organizational readiness for implementation were highlighted.

CONCLUSIONS

Internal factors of HIT and external human factors of implementation interplay in HIT implementation for chronic disease management. Strategies for improvement include ensuring HIT has a relative advantage over existing health care; tackling usability issues; and addressing underlying socioeconomic, interpersonal, and organizational conditions. Further research should focus on studying various stakeholders, such as service providers and administrative workforces; various disease populations, such as those with obesity and mental diseases; and various countries, including low- and middle-income countries.

Artificial Intelligence: A Shifting Paradigm in Cardio-Cerebrovascular Medicine

The future of healthcare is an organic blend of technology, innovation, and human connection. As artificial intelligence (AI) is gradually becoming a go-to technology in healthcare to improve efficiency and outcomes, we must understand our limitations. We should realize that our goal is not only to provide faster and more efficient care, but also to deliver an integrated solution to ensure that the care is fair and not biased to a group of sub-population. In this context, the field of cardio-cerebrovascular diseases, which encompasses a wide range of conditions-from heart failure to stroke-has made some advances to provide assistive tools to care providers. This article aimed to provide an overall thematic review of recent development focusing on various AI applications in cardio-cerebrovascular diseases to identify gaps and potential areas of improvement. If well designed, technological engines have the potential to improve healthcare access and equitability while reducing overall costs, diagnostic errors, and disparity in a system that affects patients and providers and strives for efficiency.

Assessing the Views of Professionals, Patients, and Care Partners Concerning the Use of Computer Tools in Memory Clinics: International Survey Study

Background

Computer tools based on artificial intelligence could aid clinicians in memory clinics in several ways, such as by supporting diagnostic decision-making, web-based cognitive testing, and the communication of diagnosis and prognosis.

Objective

This study aims to identify the preferences as well as the main barriers and facilitators related to using computer tools in memory clinics for all end users, that is, clinicians, patients, and care partners.

Methods

Between July and October 2020, we sent out invitations to a web-based survey to clinicians using the European Alzheimer’s Disease Centers network and the Dutch Memory Clinic network, and 109 clinicians participated (mean age 45 years, SD 10; 53/109, 48.6% female). A second survey was created for patients and care partners. They were invited via Alzheimer Europe, Alzheimer’s Society United Kingdom, Amsterdam Dementia Cohort, and Amsterdam Aging Cohort. A total of 50 patients with subjective cognitive decline, mild cognitive impairment, or dementia (mean age 73 years, SD 8; 17/34, 34% female) and 46 care partners (mean age 65 years, SD 12; 25/54, 54% female) participated in this survey.

Results

Most clinicians reported a willingness to use diagnostic (88/109, 80.7%) and prognostic (83/109, 76.1%) computer tools. User-friendliness (71/109, 65.1%); Likert scale mean 4.5, SD 0.7), and increasing diagnostic accuracy (76/109, 69.7%; mean 4.3, SD 0.7) were reported as the main factors stimulating the adoption of a tool. Tools should also save time and provide clear information on reliability and validity. Inadequate integration with electronic patient records (46/109, 42.2%; mean 3.8, SD 1.0) and fear of losing important clinical information (48/109, 44%; mean 3.7, SD 1.2) were most frequently indicated as barriers. Patients and care partners were equally positive about the use of computer tools by clinicians, both for diagnosis (69/96, 72%) and prognosis (73/96, 76%). In addition, most of them thought favorably regarding the possibility of using the tools themselves.

Conclusions

This study showed that computer tools in memory clinics are positively valued by most end users. For further development and implementation, it is essential to overcome the technical and practical barriers of a tool while paying utmost attention to its reliability and validity.

The appropriateness of clinical decision support systems alerts in supporting clinical workflows: A systematic review

A CDSS generates a high number of inappropriate alerts that interrupt the clinical workflow. As a result, clinicians silence, disable, or ignore alerts, thereby undermining patient safety. Therefore, the effectiveness and appropriateness of CDSS alerts need to be evaluated. A systematic review was carried out to identify the factors that affect CDSS alert appropriateness in supporting clinical workflow. Seven electronic databases (PubMed, Scopus, ACM, Science Direct, IEEE, Ovid Medline, and Ebscohost) were searched for English language articles published between 1997 and 2018. Seventy six papers met the inclusion criteria, of which 26, 24, 15, and 11 papers are retrospective cohort, qualitative, quantitative, and mixed-method studies, respectively. The review highlights various factors influencing the appropriateness and efficiencies of CDSS alerts. These factors are categorized into technology, human, organization, and process aspects using a combination of approaches, including socio-technical framework, five rights of CDSS, and Lean. Most CDSS alerts were not properly designed based on human factor methods and principles, explaining high alert overrides in clinical practices. The identified factors and recommendations from the review may offer valuable insights into how CDSS alerts can be designed appropriately to support clinical workflow.

Hospital contextual factors affecting the implementation of health technologies: a systematic review

Background

To keep a high quality of assistance it is important for hospitals to invest in health technologies (HTs) that have the potential of improving health outcomes. Even though guidance exists on how HTs should be introduced, used and dismissed, there is a surprising gap in literature concerning the awareness of hospitals in the actual utilization of HTs.

Methods

We performed a systematic literature review of qualitative and quantitative studies aimed at investigating hospital contextual factors that influence the actual utilization of HTs. PubMed, Scopus, Web of Science, Econlit and Ovid Medline electronic databases were searched to retrieve articles published in English and Italian from January 2000 to January 2019. The quality of the included articles was assessed using the Critical Appraisal Skills Programme checklist for qualitative studies, Newcastle-Ottawa Scale for the cross-sectional studies and the Mixed Methods Appraisal Tool for mixed method studies.

Results

We included 33 articles, which were of moderate to high methodological quality. The included articles mostly addressed the contextual factors that impact the implementation of information and communication technologies (ICTs). Overall, for all HTs, the hospital contextual factors were part of four categories: hospital infrastructure, human resource management, financial resources and leadership styles.

Conclusion

Our systematic review reported that the contextual factors influencing the HTs utilization at hospital level are mainly explored for ICTs. Several factors should be considered when planning the implementation of a new HTs at hospital level. A potential publication bias might be present in our work, since we included articles published only in English and Italian Language, from January 2000 to January 2019. There remains a gap in the literature on the facilitators and barriers influencing the implementation and concrete utilization of medical and surgical HTs, suggesting the need for further studies for a better understanding.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06423-2.

Clinician acceptance of complex clinical decision support systems for treatment allocation of patients with chronic low back pain

Background

The uptake of complex clinical decision support systems (CDSS) in daily practice remains low, despite the proven potential to reduce medical errors and to improve the quality of care. To improve successful implementation of a complex CDSS this study aims to identify the factors that hinder, or alleviate the acceptance of, clinicians toward the use of a complex CDSS for treatment allocation of patients with chronic low back pain.

Methods

We tested a research model in which the intention to use a CDSS by clinicians is influenced by the perceived usefulness; this usefulness, in turn is influenced by the perceived service benefits and perceived service risks. An online survey was created to test our research model and the data was analysed using Partial Least Squares Structural Equation Modelling. The study population consisted of clinicians. The online questionnaire started with demographic questions and continued with a video animation of the complex CDSS followed by the set of measurement items. The online questionnaire ended with two open questions enquiring the reasons to use and not use, a complex CDSS.

Results

Ninety-eight participants (46% general practitioners, 25% primary care physical therapists, and 29% clinicians at a rehabilitation centre) fully completed the questionnaire. Fifty-two percent of the respondents were male. The average age was 48 years (SD ± 12.2). The causal model suggests that perceived usefulness is the main factor contributing to the intention to use a complex CDSS. Perceived service benefits and risks are both significant antecedents of perceived usefulness and perceived service risks are affected by the perceived threat to autonomy and trusting beliefs, particularly benevolence and competence.

Conclusions

To improve the acceptance of complex CDSSs it is important to address the risks, but the main focus during the implementation phase should be on the expected improvements in patient outcomes and the overall gain for clinicians. Our results will help the development of complex CDSSs that fit more into the daily clinical practice of clinicians.

Technology Acceptance of a Machine Learning Algorithm Predicting Delirium in a Clinical Setting: a Mixed-Methods Study

Early identification of patients with life-threatening risks such as delirium is crucial in order to initiate preventive actions as quickly as possible. Despite intense research on machine learning for the prediction of clinical outcomes, the acceptance of the integration of such complex models in clinical routine remains unclear. The aim of this study was to evaluate user acceptance of an already implemented machine learning-based application predicting the risk of delirium for in-patients. We applied a mixed methods design to collect opinions and concerns from health care professionals including physicians and nurses who regularly used the application. The evaluation was framed by the Technology Acceptance Model assessing perceived ease of use, perceived usefulness, actual system use and output quality of the application. Questionnaire results from 47 nurses and physicians as well as qualitative results of four expert group meetings rated the overall usefulness of the delirium prediction positively. For healthcare professionals, the visualization and presented information was understandable, the application was easy to use and the additional information for delirium management was appreciated. The application did not increase their workload, but the actual system use was still low during the pilot study. Our study provides insights into the user acceptance of a machine learning-based application supporting delirium management in hospitals. In order to improve quality and safety in healthcare, computerized decision support should predict actionable events and be highly accepted by users.

Clinical decision support systems-based interventions to improve medication outcomes: A systematic literature review on features and effects

Background: Clinical decision support systems (CDSSs) interventions were used to improve the life quality and safety in patients and also to improve practitioner performance, especially in the field of medication. Therefore, the aim of the paper was to summarize the available evidence on the impact, outcomes and significant factors on the implementation of CDSS in the field of medicine. Methods: This study is a systematic literature review. PubMed, Cochrane Library, Web of Science, Scopus, EMBASE, and ProQuest were investigated by 15 February 2017. The inclusion requirements were met by 98 papers, from which 13 had described important factors in the implementation of CDSS, and 86 were medicated-related. We categorized the system in terms of its correlation with medication in which a system was implemented, and our intended results were examined. In this study, the process outcomes (such as; prescription, drug-drug interaction, drug adherence, etc.), patient outcomes, and significant factors affecting the implementation of CDSS were reviewed. Results: We found evidence that the use of medication-related CDSS improves clinical outcomes. Also, significant results were obtained regarding the reduction of prescription errors, and the improvement in quality and safety of medication prescribed. Conclusion: The results of this study show that, although computer systems such as CDSS may cause errors, in most cases, it has helped to improve prescribing, reduce side effects and drug interactions, and improve patient safety. Although these systems have improved the performance of practitioners and processes, there has not been much research on the impact of these systems on patient outcomes.

Factors That Impact the Adoption of Clinical Decision Support Systems (CDSS) for Antibiotic Management

The study evaluated individual and setting-specific factors that moderate clinicians’ perception regarding use of clinical decision support systems (CDSS) for antibiotic management. A cross-sectional online survey examined clinicians’ perceptions about CDSS implementation for antibiotic management in Australia. Multivariable logistic regression determined the association between drivers of CDSS adoption and different moderators. Clinical experience, CDSS use and care setting were important predictors of clinicians’ perception concerning CDSS adoption. Compared to nonusers, CDSS users were less likely to lack confidence in CDSS (OR = 0.63, 95%, CI = 0.32, 0.94) and consider it a threat to professional autonomy (OR = 0.47, 95%, CI = 0.08, 0.83). Conversely, there was higher likelihood in experienced clinicians (>20 years) to distrust CDSS (OR = 1.58, 95%, CI = 1.08, 2.23) due to fear of comprising their clinical judgement (OR = 1.68, 95%, CI = 1.27, 2.85). In primary care, clinicians were more likely to perceive time constraints (OR = 1.96, 95%, CI = 1.04, 3.70) and patient preference (OR = 1.84, 95%, CI = 1.19, 2.78) as barriers to CDSS adoption for antibiotic prescribing. Our findings provide differentiated understanding of the CDSS implementation landscape by identifying different individual, organisational and system-level factors that influence system adoption. The individual and setting characteristics can help understand the variability in CDSS adoption for antibiotic management in different clinicians.

Optimizing clinical decision support alerts in electronic medical records: a systematic review of reported strategies adopted by hospitals

OBJECTIVE

To identify and summarize the current internal governance processes adopted by hospitals, as reported in the literature, for selecting, optimizing, and evaluating clinical decision support (CDS) alerts in order to identify effective approaches.

MATERIALS AND METHODS

Databases (Medline, Embase, CINAHL, Scopus, Web of Science, IEEE Xplore Digital Library, CADTH, and WorldCat) were searched to identify relevant papers published from January 2010 to April 2020. All paper types published in English that reported governance processes for selecting and/or optimizing CDS alerts in hospitals were included.

RESULTS

Eight papers were included in the review. Seven papers focused specifically on medication-related CDS alerts. All papers described the use of a multidisciplinary committee to optimize alerts. Other strategies included the use of clinician feedback, alert data, literature and drug references, and a visual dashboard. Six of the 8 papers reported evaluations of their CDS alert modifications following the adoption of optimization strategies, and of these, 5 reported a reduction in alert rate.

CONCLUSIONS

A multidisciplinary committee, often in combination with other approaches, was the most frequent strategy reported by hospitals to optimize their CDS alerts. Due to the limited number of published processes, variation in system changes, and evaluation results, we were unable to compare the effectiveness of different strategies, although employing multiple strategies appears to be an effective approach for reducing CDS alert numbers. We recommend hospitals report on descriptions and evaluations of governance processes to enable identification of effective strategies for optimization of CDS alerts in hospitals.

Using artificial intelligence for improving stroke diagnosis in emergency departments: a practical framework

Stroke is the fifth leading cause of death in the United States and a major cause of severe disability worldwide. Yet, recognizing the signs of stroke in an acute setting is still challenging and leads to loss of opportunity to intervene, given the narrow therapeutic window. A decision support system using artificial intelligence (AI) and clinical data from electronic health records combined with patients' presenting symptoms can be designed to support emergency department providers in stroke diagnosis and subsequently reduce the treatment delay. In this article, we present a practical framework to develop a decision support system using AI by reflecting on the various stages, which could eventually improve patient care and outcome. We also discuss the technical, operational, and ethical challenges of the process.

Clinical Decision Support Systems in Breast Cancer: A Systematic Review

Breast cancer is the most frequently diagnosed cancer in women, with more than 2.1 million new diagnoses worldwide every year. Personalised treatment is critical to optimising outcomes for patients with breast cancer. A major advance in medical practice is the incorporation of Clinical Decision Support Systems (CDSSs) to assist and support healthcare staff in clinical decision-making, thus improving the quality of decisions and overall patient care whilst minimising costs. The usage and availability of CDSSs in breast cancer care in healthcare settings is increasing. However, there may be differences in how particular CDSSs are developed, the information they include, the decisions they recommend, and how they are used in practice. This systematic review examines various CDSSs to determine their availability, intended use, medical characteristics, and expected outputs concerning breast cancer therapeutic decisions, an area that is known to have varying degrees of subjectivity in clinical practice. Utilising the methodology of Kitchenham and Charter, a systematic search of the literature was performed in Springer, Science Direct, Google Scholar, PubMed, ACM, IEEE, and Scopus. An overview of CDSS which supports decision-making in breast cancer treatment is provided along with a critical appraisal of their benefits, limitations, and opportunities for improvement.

Computerized clinical decision support system for diabetes in primary care does not improve quality of care: a cluster-randomized controlled trial

Background

The EBMeDS system is the computerized clinical decision support (CCDS) system of EBPNet, a national computerized point-of-care information service in Belgium. There is no clear evidence of more complex CCDS systems to manage chronic diseases in primary care practices (PCPs). The objective of this study was to assess the effectiveness of EBMeDS use in improving diabetes care.

Methods

A cluster-randomized trial with before-and-after measurements was performed in Belgian PCPs over 1 year, from May 2017 to May 2018. We randomly assigned 51 practices to either the intervention group (IG), to receive the EBMeDS system, or to the control group (CG), to receive usual care. Primary and secondary outcomes were the 1-year pre- to post-implementation change in HbA1c, LDL cholesterol, and systolic and diastolic blood pressure. Composite patient and process scores were calculated. A process evaluation was added to the analysis. Results were analyzed at 6 and 12 months. Linear mixed models and logistic regression models based on generalized estimating equations were used where appropriate.

Results

Of the 51 PCPs that were enrolled and randomly assigned (26 PCPs in the CG and 25 in the IG), 29 practices (3815 patients) were analyzed in the study: 2464 patients in the CG and 1351 patients in the IG. No change differences existed between groups in primary or secondary outcomes. Change difference between CG and IG after 1-year follow-up was − 0.09 (95% CI − 0.18; 0.01, p-value = 0.06) for HbA1c; 1.76 (95% CI − 0.46; 3.98, p-value = 0.12) for LDL cholesterol; and 0.13 (95% CI − 0.91; 1.16, p-value = 0.81) and 0.12 (95% CI − 1.25;1.49, p-value = 0.86) for systolic and diastolic blood pressure respectively. The odds ratio of the IG versus the CG for the probability of no worsening and improvement was 1.09 (95% CI 0.73; 1.63, p-value = 0.67) for the process composite score and 0.74 (95% CI 0.49; 1.12, p-value = 0.16) for the composite patient score. All but one physician was satisfied with the EBMeDS system.

Conclusions

The CCDS system EBMeDS did not improve diabetes care in Belgian primary care. The lack of improvement was mainly caused by imperfections in the organizational context of Belgian primary care for chronic disease management and shortcomings in the system requirements for the correct use of the EBMeDS system (e.g., complete structured records). These shortcomings probably caused low-use rates of the system.

Trial registration

ClinicalTrials.gov, NCT01830569, Registered 12 April 2013.

Evaluating the Feasibility of a Software Prototype Supporting the Management of Multimorbid Seniors: Mixed Methods Study in General Practices

Background

Longitudinal, patient-centered care represents a challenge for general practices. Decision support and reminder systems can offer targeted support.

Objective

The objective of this study was to follow a user-oriented, stepwise approach to develop an add-on for German electronic health record (EHR) systems, which aims to support longitudinal care management of multimorbid seniors, using a flag system displaying patient-centered information relevant for comprehensive health care management. This study evaluated the prototype’s feasibility from both a technical and users’ perspective.

Methods

The study was conducted with 18 general practitioners (GPs) and practice assistants (PAs) from 9 general practices using a mixed methods approach. In all practices, 1 GP and 1 PA tested the software each for 4 multimorbid seniors selected from the practice patient data. Technical feasibility was evaluated by documenting all technical problems. To evaluate the feasibility from the users’ perspective, participants’ responses during the software test were documented. In addition, they completed a self-administered questionnaire, including the validated System Usability Scale (SUS). Data were merged by transforming qualitative data into quantitative data. Analyses were performed using univariate statistics in IBM SPSS statistics.

Results

From a technical perspective, the new software was easy to install and worked without problems. Difficulties during the installation occurred in practices lacking a 64-bit system or a current version of Microsoft .NET. As EHRs used in German practices do not provide an interface to extract the data needed, additional software was required. Incomplete flags for some laboratory data occurred, although this function was implemented in our software as shown in previous tests. From the users’ perspective, the new add-on provided a better overview of relevant patient information, reminded more comprehensively about upcoming examinations, and better supported guideline-based care when compared with their individual practice strategies. A total of 14 out of 18 participants (78%) were interested in using the software long-term. Furthermore, 8 of 9 GPs were willing to pay 5 to 25 Euros (mean 14.75, SD 5.93) monthly for its use. The usability was rated as 75% (43%-95%).

Conclusions

The new EHR add-on was well accepted and achieved a good usability rating measured by the validated SUS. In perspective, the legally consolidated, standardized interface to German EHRs will facilitate the technical integration. In view of the high feasibility, we plan to study the software’s effectiveness in everyday primary care.

Trial Registration

German Clinical Trials Register DRKS00008777; https://www.drks.de/drks_web/navigate.do? navigationId=trial.HTML&TRIAL_ID=DRKS00008777

Health care worker perceptions toward computerized clinical decision support tools for Clostridium difficile infection reduction: A qualitative study at 2 hospitals

BACKGROUND

Clostridium difficile infection (CDI) is associated with significant morbidity and mortality. Computerized clinical decision support (CCDS) tools can aid process improvement in infection prevention and antibiotic stewardship, but implementation and health care workers (HCWs) uptake of these tools is often variable. The objective of this study was to describe HCWs' perceptions of barriers and facilitators related to uptake of CCDS tools as part of a CDI reduction bundle.

METHODS

We conducted a qualitative study among HCWs at 2 acute care hospitals in Maryland. Semi-structured interviews and structured surveys were completed by HCWs to evaluate their perception to CCDS tools at 2 different stages: predevelopment and preimplementation. Emergent themes and patterns in the data were identified and condensed.

RESULTS

Gaps in CDI-related knowledge and in communication between HCWs were identified throughout the evaluation. HCWs agreed on the potential of the tools to improve CDI diagnosis, prevention, and control. An important barrier for uptake was the perceived loss of autonomy and clinical judgment, whereas standardization and error reduction were perceived advantages.

CONCLUSIONS

These observations shaped the development and implementation of the CDI reduction bundle. Qualitative findings can provide valuable contextual information during the development stages of CCDS tools in infection prevention and antibiotic stewardship.

Reduction of inappropriate medication in older populations by electronic decision support (the PRIMA-eDS study): a qualitative study of practical implementation in primary care

BACKGROUND

Within the EU-funded project PRIMA-eDS (Polypharmacy in chronic diseases: Reduction of Inappropriate Medication and Adverse drug events in older populations by electronic Decision Support) an electronic decision support tool (the "PRIMA-eDS-tool") was developed for general practitioners (GPs) to reduce inappropriate medication in their older polypharmacy patients. After entering patient data relevant to prescribing in an electronic case report form the physician received a comprehensive medication review (CMR) on his/her screen displaying recommendations regarding missing indications, necessary laboratory tests, evidence-base of current medication, dose adjustments for renal malfunction, potentially harmful drug-drug interactions, contra-indications, and possible adverse drug events. We set out to explore the usage of the PRIMA-eDS tool and the adoption of the recommendations provided by the CMR to optimise the tool and prepare it for its future implementation.

METHODS

In a qualitative study carried out in North Rhine-Westphalia, Germany, 21 GPs using the PRIMA-eDS tool within the PRIMA-eDS study were interviewed. Interviews encompassed the GPs' attitudes regarding use of the electronic case report form and the CMR, their response to the recommendations, and the implementation of the tool into daily practice routine. The collected data were analysed applying thematic qualitative text analysis.

RESULTS

GPs found the patient data entry into the electronic case report form to be inconvenient and time-consuming. The CMR was conducted often outside practice hours and without the patient present. GPs found that the PRIMA-eDS CMR provided relevant information for and had several positive effects on the caring process. However, they encountered several barriers when wanting to change medication.

CONCLUSIONS

It is unlikely that the PRIMA-eDS CMR will be used in the future as it is now as patient data entry is too time-consuming. Several barriers towards deprescribing medications were found which are common in deprescribing studies. Given the positive attitude towards the CMR, a new way of entering patient data into the PRIMA-eDS tool to create the CMR needs to be developed.

Development of a Tailored Intervention With Computerized Clinical Decision Support to Improve Quality of Care for Patients With Knee Osteoarthritis: Multi-Method Study

BACKGROUND

Clinical practice patterns greatly diverge from evidence-based recommendations to manage knee osteoarthritis conservatively before resorting to surgery.

OBJECTIVE

This study aimed to tailor a guideline-based computerized decision support (CDS) intervention that facilitates the conservative management of knee osteoarthritis.

METHODS

Experts with backgrounds in clinical medicine, research, implementation, or health informatics suggested the most important recommendations for implementation, how to develop an implementation strategy, and how to form the CDS algorithms. In 6 focus group sessions, 8 general practitioners and 22 patients from Norway, Belgium, and Finland discussed the suggested CDS intervention and identified factors that would be most critical for the success of the intervention. The focus group moderators used the GUideline Implementation with DEcision Support checklist, which we developed to support consideration of CDS success factors.

RESULTS

The experts prioritized 9 out of 22 recommendations for implementation. We formed the concept for 6 CDS algorithms to support implementation of these recommendations. The focus group suggested 59 unique factors that could affect the success of the presented CDS intervention. Five factors (out of the 59) were prioritized by focus group participants in every country, including the perceived potential to address the information needs of both patients and general practitioners; the credibility of CDS information; the timing of CDS for patients; and the need for personal dialogue about CDS between the general practitioner and the patient.

CONCLUSIONS

The focus group participants supported the CDS intervention as a tool to improve the quality of care for patients with knee osteoarthritis through shared, evidence-based decision making. We aim to develop and implement the CDS based on these study results. Future research should address optimal ways to (1) provide patient-directed CDS, (2) enable more patient-specific CDS within the context of patient complexity, and (3) maintain user engagement with CDS over time.

For which clinical rules do doctors want decision support, and why? A survey of Dutch general practitioners

Despite the promise of decision support for improving care, alerts are often overridden or ignored. We evaluated Dutch general practitioners’ intention to accept decision support in a proposed implementation based on clinical rules regarding care for elderly patients, and their reasons for wanting or not wanting support. We developed a survey based on literature and structured interviews and distributed it to all doctors who would receive support in the proposed implementation (n = 43), of which 65 percent responded. The survey consisted of six questions for each of 20 clinical rules. Despite concerns about interruption, doctors tended to choose more interruptive forms of support. Doctors wanted support when they felt the rule represented minimal care, perceived a need to improve care, and felt responsible for the action and that they might forget to perform the action; doctors declined support due to feeling that it was unnecessary and due to concerns about interruption.

What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation

BACKGROUND

Advanced Computerized Decision Support Systems (CDSSs) assist clinicians in their decision-making process, generating recommendations based on up-to-date scientific evidence. Although this technology has the potential to improve the quality of patient care, its mere provision does not guarantee uptake: even where CDSSs are available, clinicians often fail to adopt their recommendations. This study examines the barriers and facilitators to the uptake of an evidence-based CDSS as perceived by diverse health professionals in hospitals at different stages of CDSS adoption.

METHODS

Qualitative study conducted as part of a series of randomized controlled trials of CDSSs. The sample includes two hospitals using a CDSS and two hospitals that aim to adopt a CDSS in the future. We interviewed physicians, nurses, information technology staff, and members of the boards of directors (n = 30). We used a constant comparative approach to develop a framework for guiding implementation.

RESULTS

We identified six clusters of experiences of, and attitudes towards CDSSs, which we label as "positions." The six positions represent a gradient of acquisition of control over CDSSs (from low to high) and are characterized by different types of barriers to CDSS uptake. The most severe barriers (prevalent in the first positions) include clinicians' perception that the CDSSs may reduce their professional autonomy or may be used against them in the event of medical-legal controversies. Moving towards the last positions, these barriers are substituted by technical and usability problems related to the technology interface. When all barriers are overcome, CDSSs are perceived as a working tool at the service of its users, integrating clinicians' reasoning and fostering organizational learning.

CONCLUSIONS

Barriers and facilitators to the use of CDSSs are dynamic and may exist prior to their introduction in clinical contexts; providing a static list of obstacles and facilitators, irrespective of the specific implementation phase and context, may not be sufficient or useful to facilitate uptake. Factors such as clinicians' attitudes towards scientific evidences and guidelines, the quality of inter-disciplinary relationships, and an organizational ethos of transparency and accountability need to be considered when exploring the readiness of a hospital to adopt CDSSs.

The impact of a diagnostic decision support system on the consultation: perceptions of GPs and patients

Background

Clinical decision support systems (DSS) aimed at supporting diagnosis are not widely used. This is mainly due to usability issues and lack of integration into clinical work and the electronic health record (EHR). In this study we examined the usability and acceptability of a diagnostic DSS prototype integrated with the EHR and in comparison with the EHR alone.

Methods

Thirty-four General Practitioners (GPs) consulted with 6 standardised patients (SPs) using only their EHR system (baseline session); on another day, they consulted with 6 different but matched for difficulty SPs, using the EHR with the integrated DSS prototype (DSS session). GPs were interviewed twice (at the end of each session), and completed the Post-Study System Usability Questionnaire at the end of the DSS session. The SPs completed the Consultation Satisfaction Questionnaire after each consultation.

Results

The majority of GPs (74%) found the DSS useful: it helped them consider more diagnoses and ask more targeted questions. They considered three user interface features to be the most useful: (1) integration with the EHR; (2) suggested diagnoses to consider at the start of the consultation and; (3) the checklist of symptoms and signs in relation to each suggested diagnosis. There were also criticisms: half of the GPs felt that the DSS changed their consultation style, by requiring them to code symptoms and signs while interacting with the patient. SPs sometimes commented that GPs were looking at their computer more than at them; this comment was made more often in the DSS session (15%) than in the baseline session (3%). Nevertheless, SP ratings on the satisfaction questionnaire did not differ between the two sessions.

Conclusions

To use the DSS effectively, GPs would need to adapt their consultation style, so that they code more information during rather than at the end of the consultation. This presents a potential barrier to adoption. Training GPs to use the system in a patient-centred way, as well as improvement of the DSS interface itself, could facilitate coding. To enhance patient acceptability, patients should be informed about the potential of the DSS to improve diagnostic accuracy.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-017-0477-6) contains supplementary material, which is available to authorized users.

Evaluating the Effect of Unstructured Clinical Information on Clinical Decision Support Appropriateness Ratings

OBJECTIVE

To determine the appropriateness rating (AR) of advanced inpatient imaging requests that were not rated by prospective, point-of-care clinical decision support (CDS) using computerized provider order entry.

MATERIALS AND METHODS

During 30-day baseline and intervention periods, CDS generated an AR for advanced inpatient imaging requests (nuclear medicine, CT, and MRI) using provider-selected structured indications from pull-down menus in the computerized provider order entry portal. The AR was only displayed during the intervention, and providers were required to acknowledge the AR to finalize the request. Subsequently, the unstructured free text information accompanying all requests was reviewed, and the AR was revised when possible. The percentage of unrated requests and the overall AR, before and after radiologist review, were compared between periods and by provider type.

RESULTS

CDS software prospectively generated an AR for only 25.4% and 28.4% of baseline and intervention imaging requests, respectively; however, radiologist review generated an AR for 82.4% and 93.6% of the same requests. During the respective periods, the percentage of baseline and intervention imaging requests considered appropriate was 18.7% and 22.9% by prospective CDS software rating and increased to 82.4% and 88.7% with radiologist review.

CONCLUSION

Despite limited effective use of CDS software, the percentage of requests containing additional, relevant clinical information increased, and the majority of requests had overall high appropriateness when reviewed by a radiologist. Additional work is needed to improve the amount and quality of clinical information available to CDS software and to facilitate the entry of this information by appropriate end users.

Improving Care for Patients With or at Risk for Chronic Kidney Disease Using Electronic Medical Record Interventions: A Pragmatic Cluster-Randomized Trial Protocol

BACKGROUND

Many patients with or at risk for chronic kidney disease (CKD) in the primary care setting are not receiving recommended care.

OBJECTIVE

The objective of this study is to determine whether a multifaceted, low-cost intervention compared with usual care improves the care of patients with or at risk for CKD in the primary care setting.

DESIGN

A pragmatic cluster-randomized trial, with an embedded qualitative process evaluation, will be conducted.

SETTING

The study population comes from the Electronic Medical Record Administrative data Linked Database®, which includes clinical data for more than 140 000 rostered adults cared for by 194 family physicians in 34 clinics across Ontario, Canada. The 34 primary care clinics will be randomized to the intervention or control group.

INTERVENTION

The intervention group will receive resources from the "CKD toolkit" to help improve care including practice audit and feedback, printed educational materials for physicians and patients, electronic decision support and reminders, and implementation support.

MEASUREMENTS

Patients with or at risk for CKD within participating clinics will be identified using laboratory data in the electronic medical records. Outcomes will be assessed after dissemination of the CKD tools and after 2 rounds of feedback on performance on quality indicators have been sent to the physicians using information from the electronic medical records. The primary outcome is the proportion of patients aged 50 to 80 years with nondialysis-dependent CKD who are on a statin. Secondary outcomes include process of care measures such as screening tests, CKD recognition, monitoring tests, angiotensin-converting enzyme inhibitor or angiotensin receptor blocker prescriptions, blood pressure targets met, and nephrologist referral. Hierarchical analytic modeling will be performed to account for clustering. Semistructured interviews will be conducted with a random purposeful sample of physicians in the intervention group to understand why the intervention achieved the observed effects.

CONCLUSIONS

If our intervention improves care, then the CKD toolkit can be adapted and scaled for use in other primary care clinics which use electronic medical records.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02274298.

Development of a clinical decision support system for diabetes care: A pilot study

Management of complex chronic diseases such as diabetes requires the assimilation and interpretation of multiple laboratory test results. Traditional electronic health records tend to display laboratory results in a piecemeal and segregated fashion. This makes the assembly and interpretation of results related to diabetes care challenging. We developed a diabetes-specific clinical decision support system (Diabetes Dashboard) interface for displaying glycemic, lipid and renal function results, in an integrated form with decision support capabilities, based on local clinical practice guidelines. The clinical decision support system included a dashboard feature that graphically summarized all relevant laboratory results and displayed them in a color-coded system that allowed quick interpretation of the metabolic control of the patients. An alert module informs the user of tests that are due for repeat testing. An interactive graph module was also developed for better visual appreciation of the trends of the laboratory results of the patient. In a pilot study involving case scenarios administered via an electronic questionnaire, the Diabetes Dashboard, compared to the existing laboratory reporting interface, significantly improved the identification of abnormal laboratory results, of the long-term trend of the laboratory tests and of tests due for repeat testing. However, the Diabetes Dashboard did not significantly improve the identification of patients requiring treatment adjustment or the amount of time spent on each case scenario. In conclusion, we have developed and shown that the use of the Diabetes Dashboard, which incorporates several decision support features, can improve the management of diabetes. It is anticipated that this dashboard will be most helpful when deployed in an outpatient setting, where physicians can quickly make clinical decisions based on summarized information and be alerted to pertinent areas of care that require additional attention.

Attitudes of pediatric intensive care unit physicians towards the use of cognitive aids: a qualitative study

Background

Cognitive aids are increasingly recommended in clinical practice, yet little is known about the attitudes of physicians towards these tools.

Methods

We employed a qualitative, descriptive design to explore physician attitudes towards cognitive aids in pediatric intensive care units (PICUs). Semi-structured interviews elicited the opinions of a convenience sample of practicing PICU physicians towards the use of cognitive aids. We analyzed interview data for thematic content to examine the three factors of intention to use cognitive aids as defined by the Theory of Planned Behavior (TPB), attitudes, social norms, and perceived control.

Results

Analysis of 14 interviews suggested that in the PICU setting, cognitive aids are widely used. Discovered themes related to their use touched on all three TPB factors of intention and included: aids are perceived to improve team communication; aids may improve patient safety; aids may hinder clinician judgment; physicians may resist implementation if it occurs prior to demonstration of benefit; effective adoption requires cognitive aids to be integrated into local workplace culture; and implementation should take physician concerns into account.

Conclusions

Our sample of PICU physicians were open to cognitive aids in their practice, as long as such aids preserve the primacy of clinical judgment, focus on team communication, demonstrate effectiveness through preliminary testing, and are designed and implemented with the local culture and work environment in mind. Future knowledge translation efforts to implement cognitive aids would benefit from consideration of these issues.

Feasibility study of a clinical decision support system for the management of multimorbid seniors in primary care: study protocol

BACKGROUND

Care for seniors is complex because patients often have more than one disease, one medication, and one physician. It is a key challenge for primary care physicians to structure the various aspects of each patient's care, to integrate each patient's preferences, and to maintain a long-term overview. This article describes the design for the development and feasibility testing of the clinical decision support system (CDSS) eCare*Seniors© which is electronic health record (EHR)-based allowing for a long-term, comprehensive, evidence-based, and patient preference-oriented management of multimorbid seniors.

METHODS/DESIGN

This mixed-methods study is designed in three steps. First, focus groups and practice observations will be conducted to develop criteria for software design from a physicians' and practice assistants' perspective. Second, based on these criteria, a CDSS prototype will be developed. Third, the prototype's feasibility will be tested by five primary care practices in the care of 30 multimorbid seniors. Primary outcome is the usability of the software measured by the validated system usability scale (SUS) after 3 months. Secondary outcomes are the (a) willingness to routinely use the CDSS, (b) degree of utilization of the CDSS, (c) acceptance of the CDSS, (d) willingness of the physicians to purchase the CDSS, and (e) willingness of the practice assistants to use the CDSS in the long term. These outcomes will be measured using semi-structured interviews and software usage data. If the SUS score reaches ≥70 %, feasibility testing will be judged successful. Otherwise, the CDSS prototype will be refined according to the users' needs and retested by the physicians and practice assistants until it is fully adapted to their requirements and reaches a usability score ≥70 %.

DISCUSSION

The study will support the development of a CDSS which is primary care-defined, user-friendly, easy-to-comprehend, workflow-oriented, and comprehensive. The software will assist physicians and practices in their long-term, individualized care for multimorbid seniors.

TRIAL REGISTRATION

German Clinical Trials Register, DRKS00008777.

Tailoring implementation strategies for evidence-based recommendations using computerised clinical decision support systems: protocol for the development of the GUIDES tools

BACKGROUND

A computerised clinical decision support system (CCDSS) is a technology that uses patient-specific data to provide relevant medical knowledge at the point of care. It is considered to be an important quality improvement intervention, and the implementation of CCDSS is growing substantially. However, the significant investments do not consistently result in value for money due to content, context, system and implementation issues. The Guideline Implementation with Decision Support (GUIDES) project aims to improve the impact of CCDSS through optimised implementation based on high-quality evidence-based recommendations. To achieve this, we will develop tools that address the factors that determine successful CCDSS implementation.

METHODS/DESIGN

We will develop the GUIDES tools in four steps, using the methods and results of the Tailored Implementation for Chronic Diseases (TICD) project as a starting point: (1) a review of research evidence and frameworks on the determinants of implementing recommendations using CCDSS; (2) a synthesis of a comprehensive framework for the identified determinants; (3) the development of tools for use of the framework and (4) pilot testing the utility of the tools through the development of a tailored CCDSS intervention in Norway, Belgium and Finland. We selected the conservative management of knee osteoarthritis as a prototype condition for the pilot. During the process, the authors will collaborate with an international expert group to provide input and feedback on the tools.

DISCUSSION

This project will provide guidance and tools on methods of identifying implementation determinants and selecting strategies to implement evidence-based recommendations through CCDSS. We will make the GUIDES tools available to CCDSS developers, implementers, researchers, funders, clinicians, managers, educators, and policymakers internationally. The tools and recommendations will be generic, which makes them scalable to a large spectrum of conditions. Ultimately, the better implementation of CCDSS may lead to better-informed decisions and improved care and patient outcomes for a wide range of conditions.

PROTOCOL REGISTRATION

PROSPERO, CRD42016033738.

Examining perceptions of the usefulness and usability of a mobile-based system for pharmacogenomics clinical decision support: a mixed methods study

Background. Pharmacogenomic testing has the potential to improve the safety and efficacy of pharmacotherapy, but clinical application of pharmacogenetic knowledge has remained uncommon. Clinical Decision Support (CDS) systems could help overcome some of the barriers to clinical implementation. The aim of this study was to evaluate the perception and usability of a web- and mobile-enabled CDS system for pharmacogenetics-guided drug therapy–the Medication Safety Code (MSC) system–among potential users (i.e., physicians and pharmacists). Furthermore, this study sought to collect data on the practicability and comprehensibility of potential layouts of a proposed personalized pocket card that is intended to not only contain the machine-readable data for use with the MSC system but also human-readable data on the patient’s pharmacogenomic profile. Methods. We deployed an emergent mixed methods design encompassing (1) qualitative interviews with pharmacists and pharmacy students, (2) a survey among pharmacogenomics experts that included both qualitative and quantitative elements and (3) a quantitative survey among physicians and pharmacists. The interviews followed a semi-structured guide including a hypothetical patient scenario that had to be solved by using the MSC system. The survey among pharmacogenomics experts focused on what information should be printed on the card and how this information should be arranged. Furthermore, the MSC system was evaluated based on two hypothetical patient scenarios and four follow-up questions on the perceived usability. The second survey assessed physicians’ and pharmacists’ attitude towards the MSC system. Results. In total, 101 physicians, pharmacists and PGx experts coming from various relevant fields evaluated the MSC system. Overall, the reaction to the MSC system was positive across all investigated parameters and among all user groups. The majority of participants were able to solve the patient scenarios based on the recommendations displayed on the MSC interface. A frequent request among participants was to provide specific listings of alternative drugs and concrete dosage instructions. Negligence of other patient-specific factors for choosing the right treatment such as renal function and co-medication was a common concern related to the MSC system, while data privacy and cost-benefit considerations emerged as the participants’ major concerns regarding pharmacogenetic testing in general. The results of the card layout evaluation indicate that a gene-centered and tabulated presentation of the patient’s pharmacogenomic profile is helpful and well-accepted. Conclusions. We found that the MSC system was well-received among the physicians and pharmacists included in this study. A personalized pocket card that lists a patient’s metabolizer status along with critically affected drugs can alert physicians and pharmacists to the availability of essential therapy modifications.

Physicians' use of computerized clinical decision supports to improve medication management in the elderly - the Seniors Medication Alert and Review Technology intervention

Background

Elderly people (aged 65 years or more) are at increased risk of polypharmacy (five or more medications), inappropriate medication use, and associated increased health care costs. The use of clinical decision support (CDS) within an electronic medical record (EMR) could improve medication safety.

Methods

Participatory action research methods were applied to preproduction design and development and postproduction optimization of an EMR-embedded CDS implementation of the Beers’ Criteria for medication management and the Cockcroft–Gault formula for estimating glomerular filtration rates (GFR). The “Seniors Medication Alert and Review Technologies” (SMART) intervention was used in primary care and geriatrics specialty clinics. Passive (chart messages) and active (order-entry alerts) prompts exposed potentially inappropriate medications, decreased GFR, and the possible need for medication adjustments. Physician reactions were assessed using surveys, EMR simulations, focus groups, and semi-structured interviews. EMR audit data were used to identify eligible patient encounters, the frequency of CDS events, how alerts were managed, and when evidence links were followed.

Results

Analysis of subjective data revealed that most clinicians agreed that CDS appeared at appropriate times during patient care. Although managing alerts incurred a modest time burden, most also agreed that workflow was not disrupted. Prevalent concerns related to clinician accountability and potential liability. Approximately 36% of eligible encounters triggered at least one SMART alert, with GFR alert, and most frequent medication warnings were with hypnotics and anticholinergics. Approximately 25% of alerts were overridden and ~15% elicited an evidence check.

Conclusion

While most SMART alerts validated clinician choices, they were received as valuable reminders for evidence-informed care and education. Data from this study may aid other attempts to implement Beers’ Criteria in ambulatory care EMRs.

User perspectives on an electronic decision-support tool performing comprehensive medication reviews - a focus group study with physicians and nurses

Background

Although a number of studies have evaluated the effectiveness of computerized decision-support systems (CDSS), there is lack of data on user perspectives, barriers, and facilitators to the implementation of CDSSs in real-life surroundings. The aim of this study was to assess individually perceived barriers, facilitators and ideas influencing the CDSS implementation and usability.

Methods

In this qualitative study, five focus groups were carried out with physicians and nurses separately at the Tampere City Health Center, Finland. The participants were end-users of the EBMeDS computerized decision support system. An explorative data content analysis was applied.

Results

The most important barrier to benefitting from CDSS was the lack of structured and coded diagnosis documentation and outdated medication information in the electronic health records. This led to false alerts and distrust towards the system. Among the major facilitators found were e.g. the beneficial reminders that helped practitioners take into account matters otherwise ignored; automatic glomerular filtration rate (GFR) calculations; medication safety checks; and the summaries in the single medication review at a glance.

Conclusions

Physicians’ and nurses’ are keen to use the CDSS and it may enhance their inter-professional collaboration. Documenting patient information in a structured, uniform and easy manner is the essential starting point for electronic decision support. When implementing CDSS, managers need to focus on common practices in documenting structured data in their organizations in order to prevent undermining trust in the system.

Clinical decision support systems at the Vienna General Hospital using Arden Syntax: Design, implementation, and integration

INTRODUCTION

The Allgemeines Krankenhaus Informations Management (AKIM) project was started at the Vienna General Hospital (VGH) several years ago. This led to the introduction of a new hospital information system (HIS), and the installation of the expert system platform (EXP) for the integration of Arden-Syntax-based clinical decision support systems (CDSSs). In this report we take a look at the milestones achieved and the challenges faced in the creation and modification of CDSSs, and their integration into the HIS over the last three years.

MATERIALS AND METHODS

We introduce a three-stage development method, which is followed in nearly all CDSS projects at the Medical University of Vienna and the VGH. Stage one comprises requirements engineering and system conception. Stage two focuses on the implementation and testing of the system. Finally, stage three describes the deployment and integration of the system in the VGH HIS. The HIS provides a clinical work environment for healthcare specialists using customizable graphical interfaces known as parametric medical documents. Multiple Arden Syntax servers are employed to host and execute the CDSS knowledge bases: two embedded in the EXP for production and development, and a further three in clinical routine for production, development, and quality assurance.

RESULTS

Three systems are discussed; the systems serve different purposes in different clinical areas, but are all implemented with Arden Syntax. MONI-ICU is an automated surveillance system for monitoring healthcare-associated infections in the intensive care setting. TSM-CDS is a CDSS used for risk prediction in the formation of cutaneous melanoma metastases. Finally, TacroDS is a CDSS for the manipulation of dosages for tacrolimus, an immunosuppressive agent used after kidney transplantation. Problems in development and integration were related to data quality or availability, although organizational difficulties also caused delays in development and integration.

DISCUSSION AND CONCLUSION

Since the inception of the AKIM project at the VGH and its ability to support standards such as Arden Syntax and integrate CDSSs into clinical routine, the clinicians' interest in, and demand for, decision support has increased substantially. The use of Arden Syntax as a standard for CDSSs played a substantial role in the ability to rapidly create high-quality CDSS systems, whereas the ability to integrate these systems into the HIS made CDSSs more popular among physicians. Despite these successes, challenges such as lack of (consistent and high-quality) electronic data, social acceptance among healthcare personnel, and legislative issues remain. These have to be addressed effectively before CDSSs can be more widely accepted and adopted.

Exposure to and experiences with a computerized decision support intervention in primary care: results from a process evaluation

Background

Trials evaluating the effects of interventions usually provide little insight into the factors responsible for (lack of) changes in desired outcomes. A process evaluation alongside a trial can shed light on the mechanisms responsible for the outcomes of a trial. The aim of this study was to investigate exposure to and experiences with a computerized decision support system (CDSS) intervention, in order to gain insight into the intervention’s impact and to provide suggestions for improvement.

Methods

A process evaluation was conducted as part of a large-scale cluster-randomized controlled trial investigating the effects of the CDSS NHGDoc on quality of care. Data on exposure to and experiences with the intervention were collected during the trial period among participants in both the intervention and control group - whenever applicable - by means of the NHGDoc server and an electronic questionnaire. Multiple data were analyzed using descriptive statistics.

Results

Ninety-nine percent (n = 229) of the included practices generated data for the NHGDoc server and 50 % (n = 116) responded to the questionnaire: both general practitioners (GPs; n = 112; 49 %) and practice nurses (PNs; n = 52; 37 %) participated. The actual exposure to the NHGDoc system and specific heart failure module was limited with 52 % of the GPs and 42 % of the PNs reporting to either never or rarely use the system. Overall, users had a positive attitude towards CDSSs. The most perceived barriers to using NHGDoc were a lack of learning capacity of the system, the additional time and work it requires to use the CDSS, irrelevant alerts, too high intensity of alerts and insufficient knowledge regarding the system.

Conclusions

Several types of barriers may have negatively affected the impact of the intervention. Although users are generally positive about CDSSs, a large share of them is insufficiently aware of the functions of NHGDoc and, finds the decision support not always useful or relevant and difficult to integrate into daily practice. In designing CDSS interventions we suggest to more intensely involve the end-users and increase the system’s flexibility and learning capacity. To improve implementation a proper introduction of a CDSS among its target group including adequate training is advocated.

Trial registration

Clinical trials NCT01773057.

Electronic supplementary material

The online version of this article (doi:10.1186/s12875-015-0364-0) contains supplementary material, which is available to authorized users.