Computerized clinical decision support systems for primary preventive care: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes

Development and validation of a novel stemness-related prognostic model for neuroblastoma using integrated machine learning and bioinformatics analyses

Background

Neuroblastoma (NB) is a common solid tumor in children, with a dismal prognosis in high-risk cases. Despite advancements in NB treatment, the clinical need for precise prognostic models remains critical, particularly to address the heterogeneity of cancer stemness which plays a pivotal role in tumor aggressiveness and patient outcomes. By utilizing machine learning (ML) techniques, we aimed to explore the cancer stemness features in NB and identify stemness-related hub genes for future investigation and potential targeted therapy.

Methods

The public dataset GSE49710 was employed as the training set for acquire gene expression data and NB sample information, including age, stage, and MYCN amplification status and survival. The messenger RNA (mRNA) expression-based stemness index (mRNAsi) was calculated and patients were grouped according to their mRNAsi value. Stemness-related hub genes were identified from the differentially expressed genes (DEGs) to construct a gene signature. This was followed by evaluating the relationship between cancer stemness and the NB immune microenvironment, and the development of a predictive nomogram. We assessed the prognostic outcomes including overall survival (OS) and event-free survival, employing machine learning methods to measure predictive accuracy through concordance indices and validation in an independent cohort E-MTAB-8248.

Results

Based on mRNAsi, we categorized NB patients into two groups to explore the association between varying levels of stemness and their clinical outcomes. High mRNAsi was linked to the advanced International Neuroblastoma Staging System (INSS) stage, amplified MYCN, and elder age. High mRNAsi patients had a significantly poorer prognosis than low mRNAsi cases. According to the multivariate Cox analysis, the mRNAsi was an independent risk factor of prognosis in NB patients. After least absolute shrinkage and selection operator (LASSO) regression analysis, four key genes (ERCC6L, DUXAP10, NCAN, DIRAS3) most related to mRNAsi scores were discovered and a risk model was built. Our model demonstrated a significant prognostic capacity with hazard ratios (HR) ranging from 18.96 to 41.20, P values below 0.0001, and area under the receiver operating characteristic curve (AUC) values of 0.918 in the training set, suggesting high predictive accuracy which was further confirmed by external verification. Individuals with a low four-gene signature score had a favorable outcome and better immune responses. Finally, a nomogram for clinical practice was constructed by integrating the four-gene signature and INSS stage.

Conclusions

Our findings confirm the influence of CSC features in NB prognosis. The newly developed NB stemness-related four-gene signature prognostic signature could facilitate the prognostic prediction, and the identified hub genes may serve as promising targets for individualized treatments.

The SIMPLE Architectural Pattern for Integrating Patient-Facing Apps into Clinical Workflows: Desiderata and Application for Lung Cancer Screening

In December 2022, regulations from the U.S. Office of the National Coordinator for Health IT came into effect that require electronic health record (EHR) systems to accept the connection of any patient-facing digital health app using the SMART on FHIR standard. However, little has been reported with regard to architectural patterns that can be reused to take advantage of this industry development and integrate patient-facing apps into clinical workflows. To address this need, we propose SIMPLE, short for Standards-based Implementation Maximizing Portability Leveraging the EHR. The SIMPLE architectural pattern was designed to meet several key desiderata: do not require patients to install new software; do not retain patient data outside of the EHR; leverage EHRs' existing personal health record (PHR) capabilities to optimize user experience; and maximize portability. Using this pattern, an application for lung cancer screening known as MyLungHealth has been designed and is undergoing iterative user-centered enhancement.

Does Clinical Decision Support Increase Appropriate Medication Prescribing for Cardiovascular Risk Reduction?

PURPOSE

To assess the impact of a clinical decision support (CDS) system's recommendations on prescribing patterns targeting cardiovascular disease (CVD) when the recommendations are prioritized in order from greatest to least benefit toward overall CVD risk reduction.

METHODS

Secondary analysis of trial data from September 20, 2018, to March 15, 2020, where 70 community health center clinics were cluster-randomized to the CDS intervention (42 clinics; 8 organizations) or control group (28 clinics; 7 organizations). Included patients were medication-naïve and aged 40 to 75 years with ≥1 uncontrolled cardiovascular disease risk factor, with known diabetes or cardiovascular disease, or ≥10% 10-year reversible CVD risk.

RESULTS

Among eligible encounters with 29,771 patients, the probability of prescribing a medication targeting hypertension was greater at intervention clinic encounters when CDS was used (34.9% [95% CI, 31.5 to 38.3]) versus dismissed (29.6% [95% CI, 26.7 to 32.6]; P < .001), but not when compared with control clinic encounters (34.9% [95% CI, 31.1 to 38.7]; P = .998). Prescribing for dyslipidemia was significantly higher at intervention encounters where the CDS system was used (11.3% [95% CI, 9.3 to 13.3]) compared with dismissed (7.7% [95% CI, 6.1 to 9.3]; P = .003) and to control encounters (8.7% [95% CI, 7.0 to 10.4]; P = .044); smoking cessation medication showed a similar pattern. Except for dyslipidemia, prescribing rates increased according to their prioritization.

CONCLUSIONS

Use of this CDS system was associated with significantly higher prescribing targeting most cardiovascular risk factors. These results highlight how displaying prioritized actions to reduce reversible CVD risk could improve risk management.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03001713, https://clinicaltrials.gov/.

Practice facilitation for scale up of clinical decision support for hypertension management: study protocol for a cluster randomized control trial

BACKGROUND

Only half of patients with hypertension have adequately controlled blood pressure. Clinical decision support (CDS) has the potential to overcome barriers to delivering guideline-recommended care and improve hypertension management. However, optimal strategies for scaling CDS have not been well established, particularly in small, independent primary care practices which often lack the resources to effectively change practice routines. Practice facilitation is an implementation strategy that has been shown to support process changes. Our objective is to evaluate whether practice facilitation provided with hypertension-focused CDS can lead to improvements in blood pressure control for patients seen in small primary care practices.

METHODS/DESIGN

We will conduct a cluster randomized control trial to compare the effect of hypertension-focused CDS plus practice facilitation on BP control, as compared to CDS alone. The practice facilitation intervention will include an initial training in the CDS and a review of current guidelines along with follow-up for coaching and integration support. We will randomize 46 small primary care practices in New York City who use the same electronic health record vendor to intervention or control. All patients with hypertension seen at these practices will be included in the evaluation. We will also assess implementation of CDS in all practices and practice facilitation in the intervention group.

DISCUSSION

The results of this study will inform optimal implementation of CDS into small primary care practices, where much of care delivery occurs in the U.S. Additionally, our assessment of barriers and facilitators to implementation will support future scaling of the intervention.

CLINICALTRIALS

gov Identifier: NCT05588466.

Partner-developed electronic health record tools to facilitate social risk-informed care planning

OBJECTIVE

Increased social risk data collection in health care settings presents new opportunities to apply this information to improve patient outcomes. Clinical decision support (CDS) tools can support these applications. We conducted a participatory engagement process to develop electronic health record (EHR)-based CDS tools to facilitate social risk-informed care plan adjustments in community health centers (CHCs).

MATERIALS AND METHODS

We identified potential care plan adaptations through systematic reviews of hypertension and diabetes clinical guidelines. The results were used to inform an engagement process in which CHC staff and patients provided feedback on potential adjustments identified in the guideline reviews and on tool form and functions that could help CHC teams implement these suggested adjustments for patients with social risks.

RESULTS

Partners universally prioritized tools for social risk screening and documentation. Additional high-priority content included adjusting medication costs and changing follow-up plans based on reported social risks. Most content recommendations reflected partners' interests in encouraging provider-patient dialogue about care plan adaptations specific to patients' social needs. Partners recommended CDS tool functions such as alerts and shortcuts to facilitate and efficiently document social risk-informed care plan adjustments.

DISCUSSION AND CONCLUSION

CDS tools were designed to support CHC providers and staff to more consistently tailor care based on information about patients' social context and thereby enhance patients' ability to adhere to care plans. While such adjustments occur on an ad hoc basis in many care settings, these are among the first tools designed both to systematize and document these activities.

Adoption of shared decision-making and clinical decision support for reducing cardiovascular disease risk in community health centers

Objective

Electronic health record (EHR)-based shared decision-making (SDM) and clinical decision support (CDS) systems can improve cardiovascular disease (CVD) care quality and risk factor management. Use of the CV Wizard system showed a beneficial effect on high-risk community health center (CHC) patients' CVD risk within an effectiveness trial, but system adoption was low overall. We assessed which multi-level characteristics were associated with system use.

Materials and Methods

Analyses included 80 195 encounters with 17 931 patients with high CVD risk and/or uncontrolled risk factors at 42 clinics in September 2018-March 2020. Data came from the CV Wizard repository and EHR data, and a survey of 44 clinic providers. Adjusted, mixed-effects multivariate Poisson regression analyses assessed factors associated with system use. We included clinic- and provider-level clustering as random effects to account for nested data.

Results

Likelihood of system use was significantly higher in encounters with patients with higher CVD risk and at longer encounters, and lower when providers were >10 minutes behind schedule, among other factors. Survey participants reported generally high satisfaction with the system but were less likely to use it when there were time constraints or when rooming staff did not print the system output for the provider.

Discussion

CHC providers prioritize using this system for patients with the greatest CVD risk, when time permits, and when rooming staff make the information readily available. CHCs' financial constraints create substantial challenges to addressing barriers to improved system use, with health equity implications.

Conclusion

Research is needed on improving SDM and CDS adoption in CHCs.

Trial Registration

ClinicalTrials.gov, NCT03001713, https://clinicaltrials.gov/.

Clinician interest in clinical decision support for PSA-based prostate cancer screening

OBJECTIVE

To evaluate the interest of primary care clinicians in utilizing CDS for PSA screening. Evidence suggests that electronic clinical decision support (CDS) may decrease low-value prostate-specific antigen (PSA) testing. However, physician attitudes towards CDS for PSA screening are largely unknown.

METHODS

A survey was sent to 201 primary care clinicians, including both physicians and Advanced Practice Providers (APP), within a large academic health system. Eligible clinicians cared for male patients aged 40 to 80 years and ordered ≥5 PSA tests in the past year. Respondents were stratified into 3 groups, appropriate screeners, low-value screeners, or rare-screeners, based on responses to survey questions assessing PSA screening practices. The degree of interest in electronic CDS was determined via a composite Likert score comprising relevant survey items.

RESULTS

Survey response rate was 29% (59/201) consisting of 85% MD/DO and 15% APP respondents. All clinicians surveyed were interested in CDS (P < 0.001) without significant difference between screener groups. Clinicians agreed most uniformly that CDS be evidence-based. Clinicians disagreed on whether CDS would decrease professional discretion over patient decisions.

CONCLUSIONS

Primary care clinicians are interested in CDS for PSA screening regardless of their current screening practices. Prioritizing CDS features that clinicians value, such as ensuring CDS recommendations are evidence-based, may increase the likelihood of successful implementation, whereas perceived threat to autonomy may be a hinderance to utilization.

Nudging Health Care Providers' Adoption of Clinical Decision Support: Protocol for the User-Centered Development of a Behavioral Economics-Inspired Electronic Health Record Tool

BACKGROUND

The improvements in care resulting from clinical decision support (CDS) have been significantly limited by consistently low health care provider adoption. Health care provider attitudes toward CDS, specifically psychological and behavioral barriers, are not typically addressed during any stage of CDS development, although they represent an important barrier to adoption. Emerging evidence has shown the surprising power of using insights from the field of behavioral economics to address psychological and behavioral barriers. Nudges are formal applications of behavioral economics, defined as positive reinforcement and indirect suggestions that have a nonforced effect on decision-making.

OBJECTIVE

Our goal is to employ a user-centered design process to develop a CDS tool-the pulmonary embolism (PE) risk calculator-for PE risk stratification in the emergency department that incorporates a behavior theory-informed nudge to address identified behavioral barriers to use.

METHODS

All study activities took place at a large academic health system in the New York City metropolitan area. Our study used a user-centered and behavior theory-based approach to achieve the following two aims: (1) use mixed methods to identify health care provider barriers to the use of an active CDS tool for PE risk stratification and (2) develop a new CDS tool-the PE risk calculator-that addresses behavioral barriers to health care providers' adoption of CDS by incorporating nudges into the user interface. These aims were guided by the revised Observational Research Behavioral Information Technology model. A total of 50 clinicians who used the original version of the tool were surveyed with a quantitative instrument that we developed based on a behavior theory framework-the Capability-Opportunity-Motivation-Behavior framework. A semistructured interview guide was developed based on the survey responses. Inductive methods were used to analyze interview session notes and audio recordings from 12 interviews. Revised versions of the tool were developed that incorporated nudges.

RESULTS

Functional prototypes were developed by using Axure PRO (Axure Software Solutions) software and usability tested with end users in an iterative agile process (n=10). The tool was redesigned to address 4 identified major barriers to tool use; we included 2 nudges and a default. The 6-month pilot trial for the tool was launched on October 1, 2021.

CONCLUSIONS

Clinicians highlighted several important psychological and behavioral barriers to CDS use. Addressing these barriers, along with conducting traditional usability testing, facilitated the development of a tool with greater potential to transform clinical care. The tool will be tested in a prospective pilot trial.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/42653.

Barriers and Facilitators to the Use of Clinical Decision Support Systems in Primary Care: A Mixed-Methods Systematic Review

PURPOSE

To identify and quantify the barriers and facilitators to the use of clinical decision support systems (CDSSs) by primary care professionals (PCPs).

METHODS

A mixed-methods systematic review was conducted using a sequential synthesis design. PubMed/MEDLINE, PsycInfo, Embase, CINAHL, and the Cochrane library were searched in July 2021. Studies that evaluated CDSSs providing recommendations to PCPs and intended for use during a consultation were included. We excluded CDSSs used only by patients, described as concepts or prototypes, used with simulated cases, and decision supports not considered as CDSSs. A framework synthesis was performed according to the HOT-fit framework (Human, Organizational, Technology, Net Benefits), then a quantitative synthesis evaluated the impact of the HOT-fit categories on CDSS use.

RESULTS

A total of 48 studies evaluating 45 CDSSs were included, and 186 main barriers or facilitators were identified. Qualitatively, barriers and facilitators were classified as human (eg, perceived usefulness), organizational (eg, disruption of usual workflow), and technological (eg, CDSS user-friendliness), with explanatory elements. The greatest barrier to using CDSSs was an increased workload. Quantitatively, the human and organizational factors had negative impacts on CDSS use, whereas the technological factor had a neutral impact and the net benefits dimension a positive impact.

CONCLUSIONS

Our findings emphasize the need for CDSS developers to better address human and organizational issues, in addition to technological challenges. We inferred core CDSS features covering these 3 factors, expected to improve their usability in primary care.

An intervention to reduce stigma and improve management of depression, risk of suicide/self-harm and other significant emotional or medically unexplained complaints among adolescents living in urban slums: protocol for the ARTEMIS project

BACKGROUND

There are around 250 million adolescents in India. Adolescents are vulnerable to common mental disorders with depression and self-harm accounting for a major share of the burden of death and disability in this age group. Around 20% of children and adolescents are diagnosed with/ or live with a disabling mental illness. A national survey has found that suicide is the third leading cause of death among adolescents in India. The authors hypothesise that an intervention involving an anti-stigma campaign co-created by adolescents themselves, and a mobile technology-based electronic decision support system will help reduce stigma, depression, and suicide risk and improve mental health for high-risk adolescents living in urban slums in India.

METHODS

The intervention will be implemented as a cluster randomised control trial in 30 slum clusters in each of the cities of Vijayawada and New Delhi in India. Adolescents aged 10 to 19 years will be screened for depression and suicide ideation using the Patient Health Questionnaire (PHQ-9). Two evaluation cohorts will be derived-a high-risk cohort with an elevated PHQ-9 score ≥ 10 and/or a positive response (score ≥ 2) to the suicide risk question on the PHQ-9, and a non-high-risk cohort comprising an equal number of adolescents not at elevated risk based on these scores.

DISCUSSION

The key elements that ARTEMIS will focus on are increasing awareness among adolescents and the slum community on these mental health conditions as well as strengthening the skills of existing primary healthcare workers and promoting task sharing. The findings from this study will provide evidence to governments about strategies with potential for addressing the gaps in providing care for adolescents living in urban slums and experiencing depression, other significant emotional or medically unexplained complaints or increased suicide risk/self-harm and should have relevance not only for India but also for other low- and middle-income countries.

TRIAL STATUS

Protocol version - V7, 20 Dec 2021 Recruitment start date: tentatively after 15th July 2022 Recruitment end date: tentatively 14th July 2023 (1 year after the trial start date) TRIAL REGISTRATION: The trial has been registered in the Clinical Trial Registry India, which is included in the WHO list of Registries ( https://www.who.int/clinical-trials-registry-platform/network/primary-registries ) Reference No. CTRI/2022/02/040307 . Registered on 18 February 2022. The tentative start date of participant recruitment for the trial will begin after 15th July 2022.

Design, effectiveness, and economic outcomes of contemporary chronic disease clinical decision support systems: a systematic review and meta-analysis

Objectives

Electronic health record-based clinical decision support (CDS) has the potential to improve health outcomes. This systematic review investigates the design, effectiveness, and economic outcomes of CDS targeting several common chronic diseases.

Material and Methods

We conducted a search in PubMed (Medline), EBSCOHOST (CINAHL, APA PsychInfo, EconLit), and Web of Science. We limited the search to studies from 2011 to 2021. Studies were included if the CDS was electronic health record-based and targeted one or more of the following chronic diseases: cardiovascular disease, diabetes, chronic kidney disease, hypertension, and hypercholesterolemia. Studies with effectiveness or economic outcomes were considered for inclusion, and a meta-analysis was conducted.

Results

The review included 76 studies with effectiveness outcomes and 9 with economic outcomes. Of the effectiveness studies, 63% described a positive outcome that favored the CDS intervention group. However, meta-analysis demonstrated that effect sizes were heterogenous and small, with limited clinical and statistical significance. Of the economic studies, most full economic evaluations (n = 5) used a modeled analysis approach. Cost-effectiveness of CDS varied widely between studies, with an estimated incremental cost-effectiveness ratio ranging between USD$2192 to USD$151 955 per QALY.

Conclusion

We summarize contemporary chronic disease CDS designs and evaluation results. The effectiveness and cost-effectiveness results for CDS interventions are highly heterogeneous, likely due to differences in implementation context and evaluation methodology. Improved quality of reporting, particularly from modeled economic evaluations, would assist decision makers to better interpret and utilize results from these primary research studies.

Registration

PROSPERO (CRD42020203716)

Innovative mobile-health led participatory approach to comprehensive screening and treatment of diabetes (IMPACT diabetes): rationale, design, and baseline characteristics

Background

India has 66 million people with diabetes, of which a large proportion do not receive adequate care. The primary health centres can serve as platforms for early detection of diabetes and continuum of care.

Objectives

This project evaluates a community-level technology-enabled system-level intervention based around the community health workers and primary-care physicians. We hypothesize that incorporation of a mobile clinical decision support system, with other process-level changes will improve identification and management of individuals with diabetes in primary care settings.

Methods

A cluster-randomized trial in sixteen villages/peri-urban areas in Andhra Pradesh and Haryana will test the feasibility and preliminary effectiveness of this intervention. The effectiveness of the extended care intervention will be evaluated by the difference in HbA1c (glycosylated hemoglobin) measured at baseline and end-line between the two study arms. Qualitative interviews of physicians, ASHA, and community members will ascertain the intervention acceptability and feasibility.

Results

A total of 1785 adults (females: 53.2%; median age: 50 years) were screened. ASHAs achieved 100% completeness of data for anthropometric, blood-pressure, and blood-glucose measures. At baseline, 63% of the participants were overweight/obese, 27.8% had elevated blood pressure, 20.3% were at high-risk for cardiovascular disease (CVD), and 21.3% had elevated blood glucose. Half of the individuals with diabetes were newly diagnosed.

Conclusion

Technology enabled transfer of simple clinical procedures from physicians to nonphysician health workers can support the provision of healthcare in under-served communities. Community health workers can successfully screen and refer patients with diabetes and/or CVD to physicians in primary healthcare system.

Clinical decision support in cardiovascular medicine

Despite considerable progress in tackling cardiovascular disease over the past 50 years, many gaps in the quality of care for cardiovascular disease remain. Multiple missed opportunities have been identified at every step in the prevention and treatment of cardiovascular disease, such as failure to make risk factor modifications, failure to diagnose cardiovascular disease, and failure to use proper evidence based treatments. With the digital transformation of medicine and advances in health information technology, clinical decision support (CDS) tools offer promise to enhance the efficiency and effectiveness of delivery of cardiovascular care. However, to date, the promise of CDS delivering scalable and sustained value for patient care in clinical practice has not been realized. This article reviews the evidence on key emerging questions around the development, implementation, and regulation of CDS with a focus on cardiovascular disease. It first reviews evidence on the effectiveness of CDS on healthcare process and clinical outcomes related to cardiovascular disease and design features associated with CDS effectiveness. It then reviews the barriers encountered during implementation of CDS in cardiovascular care, with a focus on unintended consequences and strategies to promote successful implementation. Finally, it reviews the legal and regulatory environment of CDS with specific examples for cardiovascular disease.

The impact of clinical decision support systems on provider behavior in the inpatient setting: A systematic review and meta-analysis

BACKGROUND

Clinical decision support systems (CDSS) are used to improve processes of care. CDSS proliferation may have unintended consequences impacting effectiveness.

OBJECTIVE

To evaluate the effectiveness of CDSS in altering clinician behavior.

DESIGN

Electronic searches were performed in EMBASE, PubMed, and Cochrane Central Register of Control Trials for randomized controlled trials testing the impacted of CDSS on clinician behavior from 2000-2021. Extracted data included study design, CDSS attributed and outcomes, user characteristics, settings, and risk of bias. Eligible studies were analyzed qualitatively to describe CDSS types. Studies with sufficient outcome data were included in the meta-analysis.

SETTING AND PARTICIPANTS

Adult inpatients in the United States.

INTERVENTION

Clinical decision support system versus non-clinical decision support system.

MAIN OUTCOME AND MEASURE

A random-effects model measured the pooled risk difference (RD) and odds ratio of clinicians' adherence to CDSS; subgroup analyses tested differences in CDSS effectiveness over time and by CDSS type.

RESULTS

Qualitative synthesis included 22 studies. Eleven studies reported sufficient outcome data for inclusion in the meta-analysis. CDSS did not result in a statistically significant increase in clinician adoption of desired practicies (RD = 0.04 [95% confidence interval {CI} 0.00, 0.07]). CDSS from 2010-2015 (n = 5) did not increase clinician adoption of desired practice [RD -0.01, (95% CI -0.04, 0.02)].CDSS from 2016-2021 (n = 6) were associated with an increase in targeted practices [RD 0.07 (95% CI0.03, 0.12)], pInteraction = 0.004. EHR [RD 0.04 (95% CI 0.00, 0.08)] vs. non-EHR [RD 0.01 (95% CI -0.01, 0.04)] based CDSS interventions did not result in different adoption of desired practices (pInteraction = 0.27). The meta-analysis did not find an overall positive impact of CDSS on clinician behavior in the inpatient setting.

Implementation strategies for knowledge products in primary healthcare: a systematic review of systematic reviews (Preprint)

Background

The underuse or overuse of knowledge products leads to waste in health care, and primary care is no exception.

Objective

This study aimed to characterize which knowledge products are frequently implemented, the implementation strategies used in primary care, and the implementation outcomes that are measured.

Methods

We performed a systematic review (SR) of SRs using the Cochrane systematic approach to include eligible SRs. The inclusion criteria were any primary care contexts, health care professionals and patients, any Effective Practice and Organization of Care implementation strategies of specified knowledge products, any comparators, and any implementation outcomes based on the Proctor framework. We searched the MEDLINE, EMBASE, CINAHL, Ovid PsycINFO, Web of Science, and Cochrane Library databases from their inception to October 2019 without any restrictions. We searched the references of the included SRs. Pairs of reviewers independently performed selection, data extraction, and methodological quality assessment by using A Measurement Tool to Assess Systematic Reviews 2. Data extraction was informed by the Effective Practice and Organization of Care taxonomy for implementation strategies and the Proctor framework for implementation outcomes. We performed a descriptive analysis and summarized the results by using a narrative synthesis.

Results

Of the 11,101 records identified, 81 (0.73%) SRs were included. Of these 81, a total of 47 (58%) SRs involved health care professionals alone. Moreover, 15 SRs had a high or moderate methodological quality. Most of them addressed 1 type of knowledge product (56/81, 69%), common clinical practice guidelines (26/56, 46%) or management, and behavioral or pharmacological health interventions (24/56, 43%). Mixed strategies were used for implementation (67/81, 83%), predominantly education-based (meetings in 60/81, 74%; materials distribution in 59/81, 73%; and academic detailing in 45/81, 56%), reminder (53/81, 36%), and audit and feedback (40/81, 49%) strategies. Education meetings (P=.13) and academic detailing (P=.11) seemed to be used more when the population was composed of health care professionals alone. Improvements in the adoption of knowledge products were the most commonly measured outcome (72/81, 89%). The evidence level was reported in 12% (10/81) of SRs on 62 outcomes (including 48 improvements in adoption), of which 16 (26%) outcomes were of moderate or high level.

Conclusions

Clinical practice guidelines and management and behavioral or pharmacological health interventions are the most commonly implemented knowledge products and are implemented through the mixed use of educational, reminder, and audit and feedback strategies. There is a need for a strong methodology for the SR of randomized controlled trials to explore their effectiveness and the entire cascade of implementation outcomes.

Clinical Decision Support with or without Shared Decision Making to Improve Preventive Cancer Care: A Cluster-Randomized Trial

BACKGROUND

Innovative interventions are needed to address gaps in preventive cancer care, especially in rural areas. This study evaluated the impact of clinical decision support (CDS) with and without shared decision making (SDM) on cancer-screening completion.

METHODS

In this 3-arm, parallel-group, cluster-randomized trial conducted at a predominantly rural medical group, 34 primary care clinics were randomized to clinical decision support (CDS), CDS plus shared decision making (CDS+SDM), or usual care (UC). The CDS applied web-based clinical algorithms identifying patients overdue for United States Preventive Services Task Force-recommended preventive cancer care and presented evidence-based recommendations to patients and providers on printouts and on the electronic health record interface. Patients in the CDS+SDM clinic also received shared decision-making tools (SDMTs). The primary outcome was a composite indicator of the proportion of patients overdue for breast, cervical, or colorectal cancer screening at index who were up to date on these 1 y later.

RESULTS

From August 1, 2018, to March 15, 2019, 69,405 patients aged 21 to 74 y had visits at study clinics and 25,198 were overdue for 1 or more cancer screening tests at an index visit. At 12-mo follow-up, 9,543 of these (37.9%) were up to date on the composite endpoint. The adjusted, model-derived percentage of patients up to date was 36.5% (95% confidence interval [CI]: 34.0-39.1) in the UC group, 38.1% (95% CI: 35.5-40.9) in the CDS group, and 34.4% (95% CI: 31.8-37.2) in the CDS+SDM group. For all comparisons, the screening rates were higher than UC in the CDS group and lower than UC in the CDS+SDM group, although these differences did not reach statistical significance.

CONCLUSION

The CDS did not significantly increase cancer-screening rates. Exploratory analyses suggest a deeper understanding of how SDM and CDS interact to affect cancer prevention decisions is needed. Trial registration: ClinicalTrials.gov ID: NCT02986230, December 6, 2016.

Effect of Clinical Decision Support at Community Health Centers on the Risk of Cardiovascular Disease: A Cluster Randomized Clinical Trial

IMPORTANCE

Management of cardiovascular disease (CVD) risk in socioeconomically vulnerable patients is suboptimal; better risk factor control could improve CVD outcomes.

OBJECTIVE

To evaluate the impact of a clinical decision support system (CDSS) targeting CVD risk in community health centers (CHCs).

DESIGN, SETTING, AND PARTICIPANTS

This cluster randomized clinical trial included 70 CHC clinics randomized to an intervention group (42 clinics; 8 organizations) or a control group that received no intervention (28 clinics; 7 organizations) from September 20, 2018, to March 15, 2020. Randomization was by CHC organization accounting for organization size. Patients aged 40 to 75 years with (1) diabetes or atherosclerotic CVD and at least 1 uncontrolled major risk factor for CVD or (2) total reversible CVD risk of at least 10% were the population targeted by the CDSS intervention.

INTERVENTIONS

A point-of-care CDSS displaying real-time CVD risk factor control data and personalized, prioritized evidence-based care recommendations.

MAIN OUTCOMES AND MEASURES

One-year change in total CVD risk and reversible CVD risk (ie, the reduction in 10-year CVD risk that was considered achievable if 6 key risk factors reached evidence-based levels of control).

RESULTS

Among the 18 578 eligible patients (9490 [51.1%] women; mean [SD] age, 58.7 [8.8] years), patients seen in control clinics (n = 7419) had higher mean (SD) baseline CVD risk (16.6% [12.8%]) than patients seen in intervention clinics (n = 11 159) (15.6% [12.3%]; P < .001); baseline reversible CVD risk was similarly higher among patients seen in control clinics. The CDSS was used at 19.8% of 91 988 eligible intervention clinic encounters. No population-level reduction in CVD risk was seen in patients in control or intervention clinics; mean reversible risk improved significantly more among patients in control (-0.1% [95% CI, -0.3% to -0.02%]) than intervention clinics (0.4% [95% CI, 0.3% to 0.5%]; P < .001). However, when the CDSS was used, both risk measures decreased more among patients with high baseline risk in intervention than control clinics; notably, mean reversible risk decreased by an absolute 4.4% (95% CI, -5.2% to -3.7%) among patients in intervention clinics compared with 2.7% (95% CI, -3.4% to -1.9%) among patients in control clinics (P = .001).

CONCLUSIONS AND RELEVANCE

The CDSS had low use rates and failed to improve CVD risk in the overall population but appeared to have a benefit on CVD risk when it was consistently used for patients with high baseline risk treated in CHCs. Despite some limitations, these results provide preliminary evidence that this technology has the potential to improve clinical care in socioeconomically vulnerable patients with high CVD risk.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03001713.

Documentation of Behavioral Health Risk Factors in a Large Academic Primary Care Clinic

Objective:

To determine the prevalence of alcohol, smoking, and physical activity status documentation at a family health team in Toronto, Ontario, and to explore the patient characteristics that predict documentation of these lifestyle risk factor statuses.

Design:

Manual retrospective review of electronic medical records (EMRs).

Setting:

Large, urban, academic family health team in Toronto, Ontario.

Participants:

Patients over the age of 18 that had attended a routine clinical appointment in March, 2018.

Main Outcome Measures:

Prevalence and content of risk factor status in electronic medical records for alcohol, smoking, and physical activity.

Results:

The prevalence of alcohol, smoking, and physical activity documentation was 86.4%, 90.4%, and 66.1%, respectively. These lifestyle risk factor statuses were most often documented in the “risk factors” section of the EMR (83.7% for alcohol, 88.1% for smoking, and 47.9% for physical activity). Completion of a periodic health review within 1 year was most strongly associated with documentation (alcohol odds ratio [OR] 9.79, 95% Confidence Interval [CI] 2.12, 45.15; smoking OR 1.77 95% CI 0.51, 6.20; physical activity OR 3.52 95% CI 1.67, 7.40).

Conclusion:

Documentation of lifestyle risk factor statuses is strongly associated with having a recent periodic health review. If “annual physicals” continue to decline, primary care providers should final additional opportunities to address these key determinants of health.

Effect of computerised, knowledge-based, clinical decision support systems on patient-reported and clinical outcomes of patients with chronic disease managed in primary care settings: a systematic review

OBJECTIVES

Chronic diseases are the leading cause of disability globally. Most chronic disease management occurs in primary care with outcomes varying across primary care providers. Computerised clinical decision support systems (CDSS) have been shown to positively affect clinician behaviour by improving adherence to clinical guidelines. This study provides a summary of the available evidence on the effect of CDSS embedded in electronic health records on patient-reported and clinical outcomes of adult patients with chronic disease managed in primary care.

DESIGN AND ELIGIBILITY CRITERIA

Systematic review, including randomised controlled trials (RCTs), cluster RCTs, quasi-RCTs, interrupted time series and controlled before-and-after studies, assessing the effect of CDSS (vs usual care) on patient-reported or clinical outcomes of adult patients with selected common chronic diseases (asthma, chronic obstructive pulmonary disease, heart failure, myocardial ischaemia, hypertension, diabetes mellitus, hyperlipidaemia, arthritis and osteoporosis) managed in primary care.

DATA SOURCES

Medline, Embase, CENTRAL, Scopus, Health Management Information Consortium and trial register clinicaltrials.gov were searched from inception to 24 June 2020.

DATA EXTRACTION AND SYNTHESIS

Screening, data extraction and quality assessment were performed by two reviewers independently. The Cochrane risk of bias tool was used for quality appraisal.

RESULTS

From 5430 articles, 8 studies met the inclusion criteria. Studies were heterogeneous in population characteristics, intervention components and outcome measurements and focused on diabetes, asthma, hyperlipidaemia and hypertension. Most outcomes were clinical with one study reporting on patient-reported outcomes. Quality of the evidence was impacted by methodological biases of studies.

CONCLUSIONS

There is inconclusive evidence in support of CDSS. A firm inference on the intervention effect was not possible due to methodological biases and study heterogeneity. Further research is needed to provide evidence on the intervention effect and the interplay between healthcare setting features, CDSS characteristics and implementation processes.

PROSPERO REGISTRATION NUMBER

CRD42020218184.

Economic Aspects of Delivering Primary Care Services: An Evidence Synthesis to Inform Policy and Research Priorities

Policy Points The 2018 Declaration of Astana reemphasized the importance of primary health care and its role in achieving universal health coverage. While there is a large amount of literature on the economic aspects of delivering primary care services, there is a need for more comprehensive overviews of this evidence. In this article, we offer such an overview. Evidence suggests that there are several strategies involving coverage, financing, service delivery, and governance arrangements which can, if implemented, have positive economic impacts on the delivery of primary care services. These include arrangements such as worker task-shifting and telemedicine. The implementation of any such arrangements, based on positive economic evidence, should carefully account for potential impacts on overall health care access and quality. There are many opportunities for further research, with notable gaps in evidence on the impacts of increasing primary care funding or the overall supply of primary care services.

CONTEXT

The 2018 Declaration of Astana reemphasized the importance of primary health care and its role in achieving universal health coverage. To strengthen primary health care, policymakers need guidance on how to allocate resources in a manner that maximizes its economic benefits.

METHODS

We collated and synthesized published systematic reviews of evidence on the economic aspects of different models of delivering primary care services. Building on previous efforts, we adapted existing taxonomies of primary care components to classify our results according to four categories: coverage, financing, service delivery, and governance.

FINDINGS

We identified and classified 109 reviews that met our inclusion criteria according to our taxonomy of primary care components: coverage, financing, service delivery, and governance arrangements. A significant body of evidence suggests that several specific primary care arrangements, such as health workers' task shifting and telemedicine, can have positive economic impacts (such as lower overall health care costs). Notably absent were reviews on the impact of increasing primary care funding or the overall supply of primary care services.

CONCLUSIONS

There is a great opportunity for further research to systematically examine the broader economic impacts of investing in primary care services. Despite progress over the last decade, significant evidence gaps on the economic implications of different models of primary care services remain, which could help inform the basis of future research efforts.

Using Electronic Health Record-Based Clinical Decision Support to Provide Social Risk-Informed Care in Community Health Centers: Protocol for the Design and Assessment of a Clinical Decision Support Tool

Background

Consistent and compelling evidence demonstrates that social and economic adversity has an impact on health outcomes. In response, many health care professional organizations recommend screening patients for experiences of social and economic adversity or social risks—for example, food, housing, and transportation insecurity—in the context of care. Guidance on how health care providers can act on documented social risk data to improve health outcomes is nascent. A strategy recommended by the National Academy of Medicine involves using social risk data to adapt care plans in ways that accommodate patients’ social risks.

Objective

This study’s aims are to develop electronic health record (EHR)–based clinical decision support (CDS) tools that suggest social risk–informed care plan adaptations for patients with diabetes or hypertension, assess tool adoption and its impact on selected clinical quality measures in community health centers, and examine perceptions of tool usability and impact on care quality.

Methods

A systematic scoping review and several stakeholder activities will be conducted to inform development of the CDS tools. The tools will be pilot-tested to obtain user input, and their content and form will be revised based on this input. A randomized quasi-experimental design will then be used to assess the impact of the revised tools. Eligible clinics will be randomized to a control group or potential intervention group; clinics will be recruited from the potential intervention group in random order until 6 are enrolled in the study. Intervention clinics will have access to the CDS tools in their EHR, will receive minimal implementation support, and will be followed for 18 months to evaluate tool adoption and the impact of tool use on patient blood pressure and glucose control.

Results

This study was funded in January 2020 by the National Institute on Minority Health and Health Disparities of the National Institutes of Health. Formative activities will take place from April 2020 to July 2021, the CDS tools will be developed between May 2021 and November 2022, the pilot study will be conducted from August 2021 to July 2022, and the main trial will occur from December 2022 to May 2024. Study data will be analyzed, and the results will be disseminated in 2024.

Conclusions

Patients’ social risk information must be presented to care teams in a way that facilitates social risk–informed care. To our knowledge, this study is the first to develop and test EHR-embedded CDS tools designed to support the provision of social risk–informed care. The study results will add a needed understanding of how to use social risk data to improve health outcomes and reduce disparities.

International Registered Report Identifier (IRRID)

PRR1-10.2196/31733

Barriers to the Use of Clinical Decision Support for the Evaluation of Pulmonary Embolism: Qualitative Interview Study

BACKGROUND

Clinicians often disregard potentially beneficial clinical decision support (CDS).

OBJECTIVE

In this study, we sought to explore the psychological and behavioral barriers to the use of a CDS tool.

METHODS

We conducted a qualitative study involving emergency medicine physicians and physician assistants. A semistructured interview guide was created based on the Capability, Opportunity, and Motivation-Behavior model. Interviews focused on the barriers to the use of a CDS tool built based on Wells' criteria for pulmonary embolism to assist clinicians in establishing pretest probability of pulmonary embolism before imaging.

RESULTS

Interviews were conducted with 12 clinicians. Six barriers were identified, including (1) Bayesian reasoning, (2) fear of missing a pulmonary embolism, (3) time pressure or cognitive load, (4) gestalt includes Wells' criteria, (5) missed risk factors, and (6) social pressure.

CONCLUSIONS

Clinicians highlighted several important psychological and behavioral barriers to CDS use. Addressing these barriers will be paramount in developing CDS that can meet its potential to transform clinical care.

Evaluation of a Clinical Decision Support Strategy to Increase Seasonal Influenza Vaccination Among Hospitalized Children Before Inpatient Discharge

IMPORTANCE

Hospitalized children are at increased risk of influenza-related complications, yet influenza vaccine coverage remains low among this group. Evidence-based strategies about vaccination of vulnerable children during all health care visits are especially important during the COVID-19 pandemic.

OBJECTIVE

To design and evaluate a clinical decision support (CDS) strategy to increase the proportion of eligible hospitalized children who receive a seasonal influenza vaccine prior to inpatient discharge.

DESIGN, SETTING, AND PARTICIPANTS

This quality improvement study was conducted among children eligible for the seasonal influenza vaccine who were hospitalized in a tertiary pediatric health system providing care to more than half a million patients annually in 3 hospitals. The study used a sequential crossover design from control to intervention and compared hospitalizations in the intervention group (2019-2020 season with the use of an intervention order set) with concurrent controls (2019-2020 season without use of an intervention order set) and historical controls (2018-2019 season with use of an order set that underwent intervention during the 2019-2020 season).

INTERVENTIONS

A CDS intervention was developed through a user-centered design process, including (1) placing a default influenza vaccine order into admission order sets for eligible patients, (2) a script to offer the vaccine using a presumptive strategy, and (3) just-in-time education for clinicians addressing vaccine eligibility in the influenza order group with links to further reference material. The intervention was rolled out in a stepwise fashion during the 2019-2020 influenza season.

MAIN OUTCOMES AND MEASURES

Proportion of eligible hospitalizations in which 1 or more influenza vaccines were administered prior to discharge.

RESULTS

Among 17 740 hospitalizations (9295 boys [52%]), the mean (SD) age was 8.0 (6.0) years, and the patients were predominantly Black (n = 8943 [50%]) or White (n = 7559 [43%]) and mostly had public insurance (n = 11 274 [64%]). There were 10 997 hospitalizations eligible for the influenza vaccine in the 2019-2020 season. Of these, 5449 (50%) were in the intervention group, and 5548 (50%) were concurrent controls. There were 6743 eligible hospitalizations in 2018-2019 that served as historical controls. Vaccine administration rates were 31% (n = 1676) in the intervention group, 19% (n = 1051) in concurrent controls, and 14% (n = 912) in historical controls (P < .001). In adjusted analyses, the odds of receiving the influenza vaccine were 3.25 (95% CI, 2.94-3.59) times higher in the intervention group and 1.28 (95% CI, 1.15-1.42) times higher in concurrent controls than in historical controls.

CONCLUSIONS AND RELEVANCE

This quality improvement study suggests that user-centered CDS may be associated with significantly improved influenza vaccination rates among hospitalized children. Stepwise implementation of CDS interventions was a practical method that was used to increase quality improvement rigor through comparison with historical and concurrent controls.

Evaluation Framework for Successful Artificial Intelligence-Enabled Clinical Decision Support Systems: Mixed Methods Study

Background

Clinical decision support systems are designed to utilize medical data, knowledge, and analysis engines and to generate patient-specific assessments or recommendations to health professionals in order to assist decision making. Artificial intelligence–enabled clinical decision support systems aid the decision-making process through an intelligent component. Well-defined evaluation methods are essential to ensure the seamless integration and contribution of these systems to clinical practice.

Objective

The purpose of this study was to develop and validate a measurement instrument and test the interrelationships of evaluation variables for an artificial intelligence–enabled clinical decision support system evaluation framework.

Methods

An artificial intelligence–enabled clinical decision support system evaluation framework consisting of 6 variables was developed. A Delphi process was conducted to develop the measurement instrument items. Cognitive interviews and pretesting were performed to refine the questions. Web-based survey response data were analyzed to remove irrelevant questions from the measurement instrument, to test dimensional structure, and to assess reliability and validity. The interrelationships of relevant variables were tested and verified using path analysis, and a 28-item measurement instrument was developed. Measurement instrument survey responses were collected from 156 respondents.

Results

The Cronbach α of the measurement instrument was 0.963, and its content validity was 0.943. Values of average variance extracted ranged from 0.582 to 0.756, and values of the heterotrait-monotrait ratio ranged from 0.376 to 0.896. The final model had a good fit (χ₂₆²=36.984; P=.08; comparative fit index 0.991; goodness-of-fit index 0.957; root mean square error of approximation 0.052; standardized root mean square residual 0.028). Variables in the final model accounted for 89% of the variance in the user acceptance dimension.

Conclusions

User acceptance is the central dimension of artificial intelligence–enabled clinical decision support system success. Acceptance was directly influenced by perceived ease of use, information quality, service quality, and perceived benefit. Acceptance was also indirectly influenced by system quality and information quality through perceived ease of use. User acceptance and perceived benefit were interrelated.

The role of digital clinical decision support tool in improving quality of intrapartum and postpartum care: experiences from two states of India

BACKGROUND

Computerized clinical decision support (CDSS) -digital information systems designed to improve clinical decision making by providers - is a promising tool for improving quality of care. This study aims to understand the uptake of ASMAN application (defined as completeness of electronic case sheets), the role of CDSS in improving adherence to key clinical practices and delivery outcomes.

METHODS

We have conducted secondary analysis of program data (government data) collected from 81 public facilities across four districts each in two sates of Madhya Pradesh and Rajasthan. The data collected between August -October 2017 (baseline) and the data collected between December 2019 - March 2020 (latest) was analysed. The data sources included: digitized labour room registers, case sheets, referral and discharge summary forms, observation checklist and complication format. Descriptive, univariate and multivariate and interrupted time series regression analyses were conducted.

RESULTS

The completeness of electronic case sheets was low at postpartum period (40.5%), and in facilities with more than 300 deliveries a month (20.9%). In multivariate logistic regression analysis, the introduction of technology yielded significant improvement in adherence to key clinical practices. We have observed reduction in fresh still births rates and asphyxia, but these results were not statistically significant in interrupted time series analysis. However, our analysis showed that identification of maternal complications has increased over the period of program implementation and at the same time referral outs decreased.

CONCLUSIONS

Our study indicates CDSS has a potential to improve quality of intrapartum care and delivery outcome. Future studies with rigorous study design is required to understand the impact of technology in improving quality of maternity care.

The impact of the implementation of a clinical decision support system on the quality of healthcare services in a primary care setting

Background

In July 2015, King Faisal Hospital Family Medicine clinics (KFH-FMC) successfully implemented a paperless, fully integrated, electronic healthcare system. The aim of this study is to evaluate the impact of moving to a fully integrated electronic medical record system, with clinical decision support (CDS) systems, on the quality of healthcare services in a primary care setting. We aim to evaluate the impact of CDS on clinical outcomes such as screening and diagnosis of breast and colorectal cancers, as well as the management of chronic diseases such as diabetes and hypertension, and the uptake of immunizations.

Inclusion and Exclusion Criteria

Our study included all adult patients, over the age of 18, registered in the Family Medicine clinic linked to King Faisal Hospital, seen between January 2012 and December 2018.

Design

Retrospective cohort study.

Setting

Family Medicine clinics at King Faisal Hospital (KFH-FMC).

Materials and Methods

Data were collected retrospectively from the electronic health records of all adult patients above 18 years of age, who were seen in KFH-FMC between January 2012 and December 2018. We analyzed several processes of care and a number of clinical outcomes, comparing results for the three and a half years before CDS implementation with the three and a half years after implementation. Data collected included blood pressure measurements, lipid levels, HbA1c for diabetic patients, screening tests done, including PAP smear, mammogram, fecal occult blood tests, and bone densitometry. Other data included cancer diagnoses and immunizations received.

Results

Significant increases were found in adult vaccine uptake ranging from an 11-fold increase in influenza uptake, to a 22-fold increase in pneumococcal 23 uptake. The uptake of all the cancer screening tests increased (FOB 66%, mammogram 33%, PAP smear 16%). Diagnoses of breast and colorectal cancer showed significant increases. Breast cancer diagnoses increased from 2 to 14, and colorectal cancer from 3 to 11. No significant improvement was found in chronic disease outcomes.

Discussion

The electronic health record with CDS led to significantly improved uptake of immunizations and screening tests, with earlier diagnoses of breast and colon cancer. Evidence of improvement in chronic disease outcomes is still lacking.

An integrated community and primary healthcare worker intervention to reduce stigma and improve management of common mental disorders in rural India: protocol for the SMART Mental Health programme

BACKGROUND

Around 1 in 7 people in India are impacted by mental illness. The treatment gap for people with mental disorders is as high as 75-95%. Health care systems, especially in rural regions in India, face substantial challenges to address these gaps in care, and innovative strategies are needed.

METHODS

We hypothesise that an intervention involving an anti-stigma campaign and a mobile-technology-based electronic decision support system will result in reduced stigma and improved mental health for adults at high risk of common mental disorders. It will be implemented as a parallel-group cluster randomised, controlled trial in 44 primary health centre clusters servicing 133 villages in rural Andhra Pradesh and Haryana. Adults aged ≥ 18 years will be screened for depression, anxiety and suicide based on Patient Health Questionnaire (PHQ-9) and Generalised Anxiety Disorders (GAD-7) scores. Two evaluation cohorts will be derived-a high-risk cohort with elevated PHQ-9, GAD-7 or suicide risk and a non-high-risk cohort comprising an equal number of people not at elevated risk based on these scores. Outcome analyses will be conducted blinded to intervention allocation.

EXPECTED OUTCOMES

The primary study outcome is the difference in mean behaviour scores at 12 months in the combined 'high-risk' and 'non-high-risk' cohort and the mean difference in PHQ-9 scores at 12 months in the 'high-risk' cohort. Secondary outcomes include depression and anxiety remission rates in the high-risk cohort at 6 and 12 months, the proportion of high-risk individuals who have visited a doctor at least once in the previous 12 months, and change from baseline in mean stigma, mental health knowledge and attitude scores in the combined non-high-risk and high-risk cohort. Trial outcomes will be accompanied by detailed economic and process evaluations.

SIGNIFICANCE

The findings are likely to inform policy on a low-cost scalable solution to destigmatise common mental disorders and reduce the treatment gap for under-served populations in low-and middle-income country settings.

TRIAL REGISTRATION

Clinical Trial Registry India CTRI/2018/08/015355 . Registered on 16 August 2018.

Artificial intelligence-based clinical decision support in modern medical physics: Selection, acceptance, commissioning, and quality assurance

Background

Recent advances in machine and deep learning based on an increased availability of clinical data have fueled renewed interest in computerized clinical decision support systems (CDSSs). CDSSs have shown great potential to improve healthcare, increase patient safety and reduce costs. However, the use of CDSSs is not without pitfalls, as an inadequate or faulty CDSS can potentially deteriorate the quality of healthcare and put patients at risk. In addition, the adoption of a CDSS might fail because its intended users ignore the output of the CDSS due to lack of trust, relevancy or actionability.

Aim

In this article, we provide guidance based on literature for the different aspects involved in the adoption of a CDSS with a special focus on machine and deep learning based systems: selection, acceptance testing, commissioning, implementation and quality assurance.

Results

A rigorous selection process will help identify the CDSS that best fits the preferences and requirements of the local site. Acceptance testing will make sure that the selected CDSS fulfills the defined specifications and satisfies the safety requirements. The commissioning process will prepare the CDSS for safe clinical use at the local site. An effective implementation phase should result in an orderly roll out of the CDSS to the well‐trained end‐users whose expectations have been managed. And finally, quality assurance will make sure that the performance of the CDSS is maintained and that any issues are promptly identified and solved.

Conclusion

We conclude that a systematic approach to the adoption of a CDSS will help avoid pitfalls, improve patient safety and increase the chances of success.

Clinical decision support systems-based interventions to improve medication outcomes: A systematic literature review on features and effects

Background: Clinical decision support systems (CDSSs) interventions were used to improve the life quality and safety in patients and also to improve practitioner performance, especially in the field of medication. Therefore, the aim of the paper was to summarize the available evidence on the impact, outcomes and significant factors on the implementation of CDSS in the field of medicine. Methods: This study is a systematic literature review. PubMed, Cochrane Library, Web of Science, Scopus, EMBASE, and ProQuest were investigated by 15 February 2017. The inclusion requirements were met by 98 papers, from which 13 had described important factors in the implementation of CDSS, and 86 were medicated-related. We categorized the system in terms of its correlation with medication in which a system was implemented, and our intended results were examined. In this study, the process outcomes (such as; prescription, drug-drug interaction, drug adherence, etc.), patient outcomes, and significant factors affecting the implementation of CDSS were reviewed. Results: We found evidence that the use of medication-related CDSS improves clinical outcomes. Also, significant results were obtained regarding the reduction of prescription errors, and the improvement in quality and safety of medication prescribed. Conclusion: The results of this study show that, although computer systems such as CDSS may cause errors, in most cases, it has helped to improve prescribing, reduce side effects and drug interactions, and improve patient safety. Although these systems have improved the performance of practitioners and processes, there has not been much research on the impact of these systems on patient outcomes.

SMART Mental Health Project: process evaluation to understand the barriers and facilitators for implementation of multifaceted intervention in rural India

BACKGROUND

Globally, mental health problems are a growing public health concern. Resources and services for mental disorders are disproportionately low compared to disease burden. In order to bridge treatment gaps, The Systematic Medical Appraisal, Referral and Treatment (SMART) Mental Health Project was implemented across 12 villages in West Godavari district of the southern Indian state of Andhra Pradesh. This paper reports findings from a process evaluation of feasibility and acceptability of the intervention that focused on a mental health services delivery model to screen, diagnose and manage common mental disorders (CMDs).

METHODS

A mixed methods evaluation was undertaken using quantitative service usage analytics, and qualitative data from in-depth interviews and focus group discussions were conducted with stakeholders including primary care physicians, community health workers, field staff and community members. Barriers to and facilitators of intervention implementation were identified. Andersen's Behavioral Model for Health Services Use was the conceptual framework used to guide the process evaluation and interpretation of data.

RESULTS

In all, 41 Accredited Social Health Activists (ASHAs) and 6 primary health centre (PHC) doctors were trained in mental health symptoms and its management. ASHAs followed up 98.7% of screen positive cases, and 81.2% of these were clinically diagnosed and treated by the PHC doctors. The key facilitators of implementation were adequate training and supervision of field staff, ASHAs and doctors, use of electronic decision support, incorporation of a door-to-door campaign and use of culturally tailored dramas/videos to raise awareness about CMDs, and organising health camps at the village level facilitating delivery of intervention activities. Barriers to implementation included travel distance to receive care, limited knowledge about mental health, high level of stigma related to mental health issues, and poor mobile network signals and connectivity in the villages. Lack of familiarity with and access to mobile phones, especially among women, to accessing health related messages as part of the intervention.

CONCLUSIONS

The evaluation not only provides a context to the interventions delivered, but also allowed an understanding of possible factors that need to be addressed to make the programme scalable and of benefit to policy makers.

Design and rationale of an intervention to improve cancer prevention using clinical decision support and shared decision making: A clinic-randomized trial

BACKGROUND

Despite decades of research the gap in primary and secondary cancer prevention services in the U. S. remains unacceptably wide. Innovative interventions are needed to address this persistent challenge. Electronic health records linked with Web-based clinical decision support may close this gap, especially if delivered to both patients and their providers.

OBJECTIVES

The Cancer Prevention Wizard (CPW) study is an implementation, clinic-randomized trial designed to achieve these aims: 1) assess impact of the Cancer Prevention Wizard-Clinical Decision Support (CPW-CDS) alone and CPW-CDS plus Shared Decision Making Tools (CPW + SDMTs) compared to usual care (UC) on tobacco cessation counseling and drugs, HPV vaccinations, and screening tests for breast, cervical, colorectal, or lung cancer; 2) assess cost of the CPW-CDS intervention; and 3) describe critical facilitators and barriers for CPW-CDS implementation, use, and clinical impact using a mixed-methods approach supported by the CFIR and RE-AIM frameworks.

METHODS

34 predominantly rural, primary care clinics were randomized to CPW-CDS, CPW + SMDTs, or UC. Between August 2018 and October 2020, primary care providers and their patients who met inclusion criteria in intervention clinics were exposed to the CPW-CDS with or without SDMTs. Study outcomes at 12 months post index visit include patients up to date on screening tests and HPV vaccinations, overall healthcare costs, and diagnostic codes and billing levels for cancer prevention services.

CONCLUSIONS

We will test in rural primary care settings whether CPW-CDS with or without SDMTs can improve delivery of primary and secondary cancer prevention services. The trial and analyses are ongoing with results expected in 2021.

An Ontological Clinical Decision Support System Based on Clinical Guidelines for Diabetes Patients in Sri Lanka

Health professionals should follow the clinical guidelines to decrease healthcare costs to avoid unnecessary testing and to minimize the variations among healthcare providers. In addition, this will minimize the mistakes in diagnosis and treatment processes. To this end, it is possible to use Clinical Decision Support Systems that implement the clinical guidelines. Clinical guidelines published by international associations are not suitable for developing countries such as Sri Lanka, due to the economic background, lack of resources, and unavailability of some laboratory tests. Hence, a set of clinical guidelines has been formulated based on the various published international professional organizations from a Sri Lankan context. Furthermore, these guidelines are usually presented in non-computer-interpretable narrative text or non-executable flow chart formats. In order to fill this gap, this research study finds a suitable approach to represent/organize the clinical guidelines in a Sri Lankan context that is suitable to be used in a clinical decision support system. To this end, we introduced a novel approach which is an ontological model based on the clinical guidelines. As it is revealed that there are 4 million diabetes patients in Sri Lanka, which is approximately twenty percent of the total population, we used diabetes-related guidelines in this research. Firstly, conceptual models were designed to map the acquired diabetes-related clinical guidelines using Business Process Model and Notation 2.0. Two models were designed in mapping the diagnosis process of Type 1 and Type 2 Diabetes, and Gestational diabetes. Furthermore, several conceptual models were designed to map the treatment plans in guidelines by using flowcharting. These designs were validated by domain experts by using questionnaires. Grüninger and Fox’s method was used to design and evaluate the ontology based on the designed conceptual models. Domain experts’ feedback and several real-life diabetic scenarios were used to validate and evaluate the developed ontology. The evaluation results show that all suggested answers based on the proposed ontological model are accurate and well addressed with respect to the real-world scenarios. A clinical decision support system was implemented based on the ontological knowledge base using the Jena Framework, and this system can be used to access the diabetic information and knowledge in the Sri Lankan context. However, this contribution is not limited to diabetes or a local context, and can be applied to any disease or any context.

Challenges involved in establishing a web-based clinical decision support tool in community health centers

Implementation lessons: Establishing a shared 'hub-and-spoke,' web-based clinical decision support system (CDSS) in an EHR shared by >600 community health centers incurred a myriad of challenges, which are summarized here to guide others seeking to use similar CDSS. Legal and compliance challenges involved ensuring secure data exchanges, determining which entity maintains data records, and deciding which data are sent to the CDSS. Technical challenges involved using lab data from multiple sources and improving the CDSS' cache routine performance in its new setting. Clinical implementation challenges involved identifying optimal strategies for generating data on CDSS use rates, modifying the CDSS functionality for obtaining clinician/staff feedback, and customizing the risk thresholds that trigger the CDSS for the new setting.

A Conceptual Framework to Study the Implementation of Clinical Decision Support Systems (BEAR): Literature Review and Concept Mapping

Background

The implementation of clinical decision support systems (CDSSs) as an intervention to foster clinical practice change is affected by many factors. Key factors include those associated with behavioral change and those associated with technology acceptance. However, the literature regarding these subjects is fragmented and originates from two traditionally separate disciplines: implementation science and technology acceptance.

Objective

Our objective is to propose an integrated framework that bridges the gap between the behavioral change and technology acceptance aspects of the implementation of CDSSs.

Methods

We employed an iterative process to map constructs from four contributing frameworks—the Theoretical Domains Framework (TDF); the Consolidated Framework for Implementation Research (CFIR); the Human, Organization, and Technology-fit framework (HOT-fit); and the Unified Theory of Acceptance and Use of Technology (UTAUT)—and the findings of 10 literature reviews, identified through a systematic review of reviews approach.

Results

The resulting framework comprises 22 domains: agreement with the decision algorithm; attitudes; behavioral regulation; beliefs about capabilities; beliefs about consequences; contingencies; demographic characteristics; effort expectancy; emotions; environmental context and resources; goals; intentions; intervention characteristics; knowledge; memory, attention, and decision processes; patient–health professional relationship; patient’s preferences; performance expectancy; role and identity; skills, ability, and competence; social influences; and system quality. We demonstrate the use of the framework providing examples from two research projects.

Conclusions

We proposed BEAR (BEhavior and Acceptance fRamework), an integrated framework that bridges the gap between behavioral change and technology acceptance, thereby widening the view established by current models.

Designing, Conducting, and Reporting Clinical Decision Support Studies: Recommendations and Call to Action

By enabling more efficient and effective medical decision making, computer-based clinical decision support (CDS) could unlock widespread benefits from the significant investment in electronic health record (EHR) systems in the United States. Evidence from high-quality CDS studies is needed to enable and support this vision of CDS-facilitated care optimization, but limited guidance is available in the literature for designing and reporting CDS studies. To address this research gap, this article provides recommendations for designing, conducting, and reporting CDS studies to: 1) ensure that EHR data to inform the CDS are available; 2) choose decision rules that are consistent with local care processes; 3) target the right users and workflows; 4) make the CDS easy to access and use; 5) minimize the burden placed on users; 6) incorporate CDS success factors identified in the literature, in particular the automatic provision of CDS as a part of clinician workflow; 7) ensure that the CDS rules are adequately tested; 8) select meaningful evaluation measures; 9) use as rigorous a study design as is feasible; 10) think about how to deploy the CDS beyond the original host organization; 11) report the study in context; 12) help the audience understand why the intervention succeeded or failed; and 13) consider the financial implications. If adopted, these recommendations should help advance the vision of more efficient, effective care facilitated by useful and widely available CDS.

Impact of a Hybrid Lung Cancer Screening Model on Patient Outcomes and Provider Behavior

BACKGROUND

Lung cancer screening (LCS) implementation is complicated by the Centers for Medicare and Medicaid Services reimbursement requirements of shared decision-making and tobacco cessation counseling. LCS programs can utilize different structures to meet these requirements, but the impact of programmatic structure on provider behavior and screening outcomes is poorly described.

PATIENTS AND METHODS

In a retrospective chart review of 624 patients in a hybrid structure, academic LCS program, we compared characteristics and outcomes of primary care provider (PCP)- and specialist-screened patients. We also assessed the impact of the availability of an LCS specialty clinic and best practice advisory (BPA) on PCP ordering patterns using electronic medical record generated reports.

RESULTS

During the study period of July 1, 2014 through June 30, 2018, 48% of patients were specialist-screened and 52% were PCP-screened; there were no clinically relevant differences in patient characteristics or screening outcomes between these populations. PCPs demonstrate distinct practice patterns when offered the choice of specialist-driven or PCP-driven screening. Increased exposure to a LCS BPA is associated with increased PCP screening orders. The addition of a nurse navigator into the LCS program increased documentation of shared decision-making and tobacco cessation counseling to > 95% and virtually eliminated screening of ineligible patients.

CONCLUSIONS

Systematic interventions including a BPA and nurse navigator are associated with increased screening and improved program quality, as evidenced by reduced screening of ineligible patients, increased lung cancer risk of the screened population, and improved compliance with LCS guidelines. Individual PCPs demonstrate clear preferences regarding LCS that should be considered in program design.

The Effect of Higher Level Computerized Clinical Decision Support Systems on Oncology Care: A Systematic Review

BACKGROUND

To deal with complexity in cancer care, computerized clinical decision support systems (CDSSs) are developed to support quality of care and improve decision-making. We performed a systematic review to explore the value of CDSSs using automated clinical guidelines, Artificial Intelligence, datamining or statistical methods (higher level CDSSs) on the quality of care in oncology.

MATERIALS AND METHODS

The search strategy combined synonyms for 'CDSS' and 'cancer.' Pubmed, Embase, The Cochrane Library, Institute of Electrical and Electronics Engineers, Association of Computing Machinery digital library and Web of Science were systematically searched from January 2000 to December 2019. Included studies evaluated the impact of higher level CDSSs on process outcomes, guideline adherence and clinical outcomes.

RESULTS

11,397 studies were selected for screening, after which 61 full-text articles were assessed for eligibility. Finally, nine studies were included in the final analysis with a total population size of 7985 patients. Types of cancer included breast cancer (63.1%), lung cancer (27.8%), prostate cancer (4.1%), colorectal cancer (3.1%) and other cancer types (1.9%). The included studies demonstrated significant improvements of higher level CDSSs on process outcomes and guideline adherence across diverse settings in oncology. No significant differences were reported for clinical outcomes.

CONCLUSION

Higher level CDSSs seem to improve process outcomes and guidelines adherence but not clinical outcomes. It should be noticed that the included studies primarily focused on breast and lung cancer. To further explore the impact of higher level CDSSs on quality of care, high-quality research is required.

The Systematic Medical Appraisal Referral and Treatment Mental Health Project: Quasi-Experimental Study to Evaluate a Technology-Enabled Mental Health Services Delivery Model Implemented in Rural India

Background

Although around 10% of Indians experience depression, anxiety, or alcohol use disorders, very few receive adequate mental health care, especially in rural communities. Stigma and limited availability of mental health services contribute to this treatment gap. The Systematic Medical Appraisal Referral and Treatment Mental Health project aimed to address this gap.

Objective

This study aimed to evaluate the effectiveness of an intervention in increasing the use of mental health services and reducing depression and anxiety scores among individuals at high risk of common mental disorders.

Methods

A before-after study was conducted from 2014 to 2019 in 12 villages in Andhra Pradesh, India. The intervention comprised a community antistigma campaign, with the training of lay village health workers and primary care doctors to identify and manage individuals with stress, depression, and suicide risk using an electronic clinical decision support system.

Results

In total, 900 of 22,046 (4.08%) adults screened by health workers had increased stress, depression, or suicide risk and were referred to a primary care doctor. At follow-up, 731 out of 900 (81.2%) reported visiting the doctor for their mental health symptoms, compared with 3.3% (30/900) at baseline (odds ratio 133.3, 95% CI 89.0 to 199.7; P<.001). Mean depression and anxiety scores were significantly lower postintervention compared with baseline from 13.4 to 3.1 (P<.001) and from 12.9 to 1.9 (P<.001), respectively.

Conclusions

The intervention was associated with a marked increase in service uptake and clinically important reductions in depression and anxiety symptom scores. This will be further evaluated in a large-scale cluster randomized controlled trial.

Impact of the KidneyWise toolkit on chronic kidney disease referral practices in Ontario primary care: a prospective evaluation

OBJECTIVES

Chronic kidney disease (CKD) is common; therefore, coordination of care between primary care and nephrology is important. Ontario Renal Network's KidneyWise toolkit was developed to provide guidance on the detection and management of people with CKD in primary care (www.kidneywise.ca). The aim of this study was to evaluate the impact of the April 2015 KidneyWise toolkit release on the characteristics of primary care referrals to nephrology.

DESIGN AND SETTING

The study was a prospective pre-post design conducted at two nephrology sites (community site: Trillium Health Partners in Mississauga, Ontario, Canada, and academic site: St Joseph's Healthcare in Hamilton, Ontario, Canada). Referrals were compared during the 3-month time period immediately prior to, and during a 3-month period 1 year after, the toolkit release.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome was the change in proportion of referrals for CKD that met the KidneyWise criteria. Additional secondary referral and quality of care outcomes were also evaluated. Multivariable logistic regression was used to evaluate preselected variables for their independent association with referrals that met the KidneyWise criteria.

RESULTS

The proportion of referrals for CKD among people who met the KidneyWise referral criteria did not significantly change from pre-KidneyWise to post-KidneyWise implementation (44.7% vs 45.8%, respectively, adjusted OR 1.16, 95% CI 0.85 to 1.59, p=0.36). The proportion of referrals for CKD that provided a urine albumin-creatinine ratio significantly increased post-KidneyWise (25.8% vs 43.8%, adjusted OR 1.45, 95% CI 1.06 to 1.97, p=0.02). The significant independent predictors of meeting the KidneyWise referral criteria were academic site, increased age and use of the KidneyWise referral form.

CONCLUSIONS

We did not observe any change in the proportion of appropriate referrals for CKD at two large nephrology centres 1 year after implementation of the KidneyWise toolkit.

Accuracy and Effects of Clinical Decision Support Systems Integrated With BMJ Best Practice-Aided Diagnosis: Interrupted Time Series Study

BACKGROUND

Clinical decision support systems (CDSS) are an integral component of health information technologies and can assist disease interpretation, diagnosis, treatment, and prognosis. However, the utility of CDSS in the clinic remains controversial.

OBJECTIVE

The aim is to assess the effects of CDSS integrated with British Medical Journal (BMJ) Best Practice-aided diagnosis in real-world research.

METHODS

This was a retrospective, longitudinal observational study using routinely collected clinical diagnosis data from electronic medical records. A total of 34,113 hospitalized patient records were successively selected from December 2016 to February 2019 in six clinical departments. The diagnostic accuracy of the CDSS was verified before its implementation. A self-controlled comparison was then applied to detect the effects of CDSS implementation. Multivariable logistic regression and single-group interrupted time series analysis were used to explore the effects of CDSS. The sensitivity analysis was conducted using the subgroup data from January 2018 to February 2019.

RESULTS

The total accuracy rates of the recommended diagnosis from CDSS were 75.46% in the first-rank diagnosis, 83.94% in the top-2 diagnosis, and 87.53% in the top-3 diagnosis in the data before CDSS implementation. Higher consistency was observed between admission and discharge diagnoses, shorter confirmed diagnosis times, and shorter hospitalization days after the CDSS implementation (all P<.001). Multivariable logistic regression analysis showed that the consistency rates after CDSS implementation (OR 1.078, 95% CI 1.015-1.144) and the proportion of hospitalization time 7 days or less (OR 1.688, 95% CI 1.592-1.789) both increased. The interrupted time series analysis showed that the consistency rates significantly increased by 6.722% (95% CI 2.433%-11.012%, P=.002) after CDSS implementation. The proportion of hospitalization time 7 days or less significantly increased by 7.837% (95% CI 1.798%-13.876%, P=.01). Similar results were obtained in the subgroup analysis.

CONCLUSIONS

The CDSS integrated with BMJ Best Practice improved the accuracy of clinicians' diagnoses. Shorter confirmed diagnosis times and hospitalization days were also found to be associated with CDSS implementation in retrospective real-world studies. These findings highlight the utility of artificial intelligence-based CDSS to improve diagnosis efficiency, but these results require confirmation in future randomized controlled trials.

Computerized clinical decision support system for diabetes in primary care does not improve quality of care: a cluster-randomized controlled trial

Background

The EBMeDS system is the computerized clinical decision support (CCDS) system of EBPNet, a national computerized point-of-care information service in Belgium. There is no clear evidence of more complex CCDS systems to manage chronic diseases in primary care practices (PCPs). The objective of this study was to assess the effectiveness of EBMeDS use in improving diabetes care.

Methods

A cluster-randomized trial with before-and-after measurements was performed in Belgian PCPs over 1 year, from May 2017 to May 2018. We randomly assigned 51 practices to either the intervention group (IG), to receive the EBMeDS system, or to the control group (CG), to receive usual care. Primary and secondary outcomes were the 1-year pre- to post-implementation change in HbA1c, LDL cholesterol, and systolic and diastolic blood pressure. Composite patient and process scores were calculated. A process evaluation was added to the analysis. Results were analyzed at 6 and 12 months. Linear mixed models and logistic regression models based on generalized estimating equations were used where appropriate.

Results

Of the 51 PCPs that were enrolled and randomly assigned (26 PCPs in the CG and 25 in the IG), 29 practices (3815 patients) were analyzed in the study: 2464 patients in the CG and 1351 patients in the IG. No change differences existed between groups in primary or secondary outcomes. Change difference between CG and IG after 1-year follow-up was − 0.09 (95% CI − 0.18; 0.01, p-value = 0.06) for HbA1c; 1.76 (95% CI − 0.46; 3.98, p-value = 0.12) for LDL cholesterol; and 0.13 (95% CI − 0.91; 1.16, p-value = 0.81) and 0.12 (95% CI − 1.25;1.49, p-value = 0.86) for systolic and diastolic blood pressure respectively. The odds ratio of the IG versus the CG for the probability of no worsening and improvement was 1.09 (95% CI 0.73; 1.63, p-value = 0.67) for the process composite score and 0.74 (95% CI 0.49; 1.12, p-value = 0.16) for the composite patient score. All but one physician was satisfied with the EBMeDS system.

Conclusions

The CCDS system EBMeDS did not improve diabetes care in Belgian primary care. The lack of improvement was mainly caused by imperfections in the organizational context of Belgian primary care for chronic disease management and shortcomings in the system requirements for the correct use of the EBMeDS system (e.g., complete structured records). These shortcomings probably caused low-use rates of the system.

Trial registration

ClinicalTrials.gov, NCT01830569, Registered 12 April 2013.

Effectiveness of a Hospital-Based Computerized Decision Support System on Clinician Recommendations and Patient Outcomes: A Randomized Clinical Trial

IMPORTANCE

Sophisticated evidence-based information resources can filter medical evidence from the literature, integrate it into electronic health records, and generate recommendations tailored to individual patients.

OBJECTIVE

To assess the effectiveness of a computerized clinical decision support system (CDSS) that preappraises evidence and provides health professionals with actionable, patient-specific recommendations at the point of care.

DESIGN, SETTING, AND PARTICIPANTS

Open-label, parallel-group, randomized clinical trial among internal medicine wards of a large Italian general hospital. All analyses in this randomized clinical trial followed the intent-to-treat principle. Between November 1, 2015, and December 31, 2016, patients were randomly assigned to the intervention group, in which CDSS-generated reminders were displayed to physicians, or to the control group, in which reminders were generated but not shown. Data were analyzed between February 1 and July 31, 2018.

INTERVENTIONS

Evidence-Based Medicine Electronic Decision Support (EBMEDS), a commercial CDSS covering a wide array of health conditions across specialties, was integrated into the hospital electronic health records to generate patient-specific recommendations.

MAIN OUTCOMES AND MEASURES

The primary outcome was the resolution rate, the rate at which medical problems identified and alerted by the CDSS were addressed by a change in practice. Secondary outcomes included the length of hospital stay and in-hospital all-cause mortality.

RESULTS

In this randomized clinical trial, 20 563 patients were admitted to the hospital. Of these, 6480 (31.5%) were admitted to the internal medicine wards (study population) and randomized (3242 to CDSS and 3238 to control). The mean (SD) age of patients was 70.5 (17.3) years, and 54.5% were men. In total, 28 394 reminders were generated throughout the course of the trial (median, 3 reminders per patient per hospital stay; interquartile range [IQR], 1-6). These messages led to a change in practice in approximately 4 of 100 patients. The resolution rate was 38.0% (95% CI, 37.2%-38.8%) in the intervention group and 33.7% (95% CI, 32.9%-34.4%) in the control group, corresponding to an odds ratio of 1.21 (95% CI, 1.11-1.32; P < .001). The length of hospital stay did not differ between the groups, with a median time of 8 days (IQR, 5-13 days) for the intervention group and a median time of 8 days (IQR, 5-14 days) for the control group (P = .36). In-hospital all-cause mortality also did not differ between groups (odds ratio, 0.95; 95% CI, 0.77-1.17; P = .59). Alert fatigue did not differ between early and late study periods.

CONCLUSIONS AND RELEVANCE

An international commercial CDSS intervention marginally influenced routine practice in a general hospital, although the change did not statistically significantly affect patient outcomes.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT02577198.

Interface, information, interaction: a narrative review of design and functional requirements for clinical decision support

Objective

Provider acceptance and associated patient outcomes are widely discussed in the evaluation of clinical decision support systems (CDSSs), but critical design criteria for tools have generally been overlooked. The objective of this work is to inform electronic health record alert optimization and clinical practice workflow by identifying, compiling, and reporting design recommendations for CDSS to support the efficient, effective, and timely delivery of high-quality care.

Material and Methods

A narrative review was conducted from 2000 to 2016 in PubMed and The Journal of Human Factors and Ergonomics Society to identify papers that discussed/recommended design features of CDSSs that are associated with the success of these systems.

Results

Fourteen papers were included as meeting the criteria and were found to have a total of 42 unique recommendations; 11 were classified as interface features, 10 as information features, and 21 as interaction features.

Discussion

Features are defined and described, providing actionable guidance that can be applied to CDSS development and policy. To our knowledge, no reviews have been completed that discuss/recommend design features of CDSS at this scale, and thus we found that this was important for the body of literature. The recommendations identified in this narrative review will help to optimize design, organization, management, presentation, and utilization of information through presentation, content, and function. The designation of 3 categories (interface, information, and interaction) should be further evaluated to determine the critical importance of the categories. Future work will determine how to prioritize them with limited resources for designers and developers in order to maximize the clinical utility of CDSS.

Conclusion

This review will expand the field of knowledge and provide a novel organization structure to identify key recommendations for CDSS.

Development of Health Pathways to Standardize Cancer Care Pathways Informed by Patient-Reported Outcomes and Clinical Practice Guidelines

PURPOSE

High-quality symptom management and supportive care are essential components of comprehensive cancer care. We aimed to describe the development of an evidence-based automated decisional algorithm for patients with cancer that had specific, actionable, clinical, evidence-based recommendations to improve patient care, communication, and management.

METHODS

We reviewed existing literature and clinical practice guidelines to identify priority domains of patient care and potential clinical recommendations. Two multidisciplinary clinical advisory groups used a two-stage consensus decision-making approach to determine domains of care and patient-reported outcome (PRO) measures and subsequently developed automated algorithms with clear clinical recommendations amendable to intervention in clinical settings.

RESULTS

Algorithms were developed to inform management of patient symptoms, distress, and unmet needs. Three PRO measures were chosen: Distress Thermometer and problem checklist, Edmonton Symptom Assessment Scale, and the Supportive Care Needs Survey-Screening Tool 9. PRO items were mapped to five domains of patient well-being: physical, emotional, practical, social and family, and maintenance of well-being. A total of 15 actionable clinical recommendations tailored to specific issues of concern were established.

CONCLUSION

Using automated algorithms and clinical recommendations provides a platform for streamlining and systematizing the use of PROs to inform risk-stratified guideline-informed care. The series of algorithms, which set out systematized care pathways for the clinical care of patients with cancer, can be used to potentially inform patient-centered care.

Towards a Maturity Model for Clinical Decision Support Operations

Clinical decision support (CDS) systems delivered through the electronic health record are an important element of quality and safety initiatives within a health care system. However, managing a large CDS knowledge base can be an overwhelming task for informatics teams. Additionally, it can be difficult for these informatics teams to communicate their goals with external operational stakeholders and define concrete steps for improvement. We aimed to develop a maturity model that describes a roadmap toward organizational functions and processes that help health care systems use CDS more effectively to drive better outcomes. We developed a maturity model for CDS operations through discussions with health care leaders at 80 organizations, iterative model development by four clinical informaticists, and subsequent review with 19 health care organizations. We ceased iterations when feedback from three organizations did not result in any changes to the model. The proposed CDS maturity model includes three main "pillars": "Content Creation," "Analytics and Reporting," and "Governance and Management." Each pillar contains five levels-advancing along each pillar provides CDS teams a deeper understanding of the processes CDS systems are intended to improve. A "roof" represents the CDS functions that become attainable after advancing along each of the pillars. Organizations are not required to advance in order and can develop in one pillar separately from another. However, we hypothesize that optimal deployment of preceding levels and advancing in tandem along the pillars increase the value of organizational investment in higher levels of CDS maturity. In addition to describing the maturity model and its development, we also provide three case studies of health care organizations using the model for self-assessment and determine next steps in CDS development.

Using normalisation process theory to understand workflow implications of decision support implementation across diverse primary care settings

BACKGROUND

Effective implementation of technologies into clinical workflow is hampered by lack of integration into daily activities. Normalisation process theory (NPT) can be used to describe the kinds of 'work' necessary to implement and embed complex new practices. We determined the suitability of NPT to assess the facilitators, barriers and 'work' of implementation of two clinical decision support (CDS) tools across diverse care settings.

METHODS

We conducted baseline and 6-month follow-up quantitative surveys of clinic leadership at two academic institutions' primary care clinics randomised to the intervention arm of a larger study. The survey was adapted from the NPT toolkit, analysing four implementation domains: sense-making, participation, action, monitoring. Domains were summarised among completed responses (n=60) and examined by role, institution, and time.

RESULTS

The median score for each NPT domain was the same across roles and institutions at baseline, and decreased at 6 months. At 6 months, clinic managers' participation domain (p=0.003), and all domains for medical directors (p<0.003) declined. At 6 months, the action domain decreased among Utah respondents (p=0.03), and all domains decreased among Wisconsin respondents (p≤0.008).

CONCLUSIONS

This study employed NPT to longitudinally assess the implementation barriers of new CDS. The consistency of results across participant roles suggests similarities in the work each role took on during implementation. The decline in engagement over time suggests the need for more frequent contact to maintain momentum. Using NPT to evaluate this implementation provides insight into domains which can be addressed with participants to improve success of new electronic health record technologies.

TRIAL REGISTRATION NUMBER

NCT02534987.

Development of Decision Support System to Predict Neurofeedback Response in ADHD: an Artificial Neural Network Approach

INTRODUCTION

Clinical decision support system (CDSS) is an analytical tool that converts raw data into useful information to help clinicians make better decisions for patients.

AIM

The purpose of this study was to investigate the efficacy of neurofeedback (NF), in Attention Deficit Hyperactivity Disorder (ADHD) by the development of CDSS based on artificial neural network (ANN).

METHODS

This study analyzed 122 patients with ADHD who underwent NF in the Parand-Human Potential Empowerment Institute in Tehran. The patients were divided into two groups according to the effects of NF: effective and non-effective groups. The patients' record information was mined by data mining techniques to identify effective features. Based on unsaturated condition of data and imbalanced classes between the patient groups (patients with successful NF response and those without it), the SMOTE technique was applied on dataset. Using MATLAB 2014a, a modular program was designed to test both multiple architectures of neural networks and their performance. Selected architecture of the neural networks was then applied in the procedure.

RESULTS

Eleven features from 28 features of the initial dataset were selected as effective features. Using the SMOTE technique, number of the samples rose to around 300 samples. Based on the multiple neural networks architecture testing, a network by 11-20-16-2 neurons was selected (specify>00.91%, sensivity=100%) and applied in the software.

CONCLUSION

The ANN used in this study has led to good results in sensivity, specificity, and AUC. The ANN and other intelligent techniques can be used as supportive tools for decision making by healthcare providers.