How are ontologies implemented to represent clinical practice guidelines in clinical decision support systems: protocol for a systematic review

BACKGROUND

Clinical practice guidelines are statements which are based on the best available evidence, and their goal is to improve the quality of patient care. Integrating clinical practice guidelines into computer systems can help physicians reduce medical errors and help them to have the best possible practice. Guideline-based clinical decision support systems play a significant role in supporting physicians in their decisions. Meantime, system errors are the most critical concerns in designing decision support systems that can affect their performance and efficacy. A well-developed ontology can be helpful in this matter. The proposed systematic review will specify the methods, components, language of rules, and evaluation methods of current ontology-driven guideline-based clinical decision support systems.

METHODS

This review will identify literature through searching MEDLINE (via Ovid), PubMed, EMBASE, Cochrane Library, CINAHL, ScienceDirect, IEEEXplore, and ACM Digital Library. Gray literature, reference lists, and citing articles of the included studies will be searched. The quality of the included studies will be assessed by the mixed methods appraisal tool (MMAT-version 2018). At least two independent reviewers will perform the screening, quality assessment, and data extraction. A third reviewer will resolve any disagreements. Proper data analysis will be performed based on the type of system and ontology engineering evaluation data.

DISCUSSION

The study will provide evidence regarding applying ontologies in guideline-based clinical decision support systems. The findings of this systematic review will be a guide for decision support system designers and developers, technologists, system providers, policymakers, and stakeholders. Ontology builders can use the information in this review to build well-structured ontologies for personalized medicine.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018106501.

Automating risk of bias assessment in systematic reviews: a real-time mixed methods comparison of human researchers to a machine learning system

Background

Machine learning and automation are increasingly used to make the evidence synthesis process faster and more responsive to policymakers’ needs. In systematic reviews of randomized controlled trials (RCTs), risk of bias assessment is a resource-intensive task that typically requires two trained reviewers. One function of RobotReviewer, an off-the-shelf machine learning system, is an automated risk of bias assessment.

Methods

We assessed the feasibility of adopting RobotReviewer within a national public health institute using a randomized, real-time, user-centered study. The study included 26 RCTs and six reviewers from two projects examining health and social interventions. We randomized these studies to one of two RobotReviewer platforms. We operationalized feasibility as accuracy, time use, and reviewer acceptability. We measured accuracy by the number of corrections made by human reviewers (either to automated assessments or another human reviewer’s assessments). We explored acceptability through group discussions and individual email responses after presenting the quantitative results.

Results

Reviewers were equally likely to accept judgment by RobotReviewer as each other’s judgement during the consensus process when measured dichotomously; risk ratio 1.02 (95% CI 0.92 to 1.13; p = 0.33). We were not able to compare time use. The acceptability of the program by researchers was mixed. Less experienced reviewers were generally more positive, and they saw more benefits and were able to use the tool more flexibly. Reviewers positioned human input and human-to-human interaction as superior to even a semi-automation of this process.

Conclusion

Despite being presented with evidence of RobotReviewer’s equal performance to humans, participating reviewers were not interested in modifying standard procedures to include automation. If further studies confirm equal accuracy and reduced time compared to manual practices, we suggest that the benefits of RobotReviewer may support its future implementation as one of two assessors, despite reviewer ambivalence. Future research should study barriers to adopting automated tools and how highly educated and experienced researchers can adapt to a job market that is increasingly challenged by new technologies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01649-y.

Do providers use computerized clinical decision support systems? A systematic review and meta-regression of clinical decision support uptake

BACKGROUND

Computerized clinical decision support systems (CDSSs) are a promising knowledge translation tool, but often fail to meaningfully influence the outcomes they target. Low CDSS provider uptake is a potential contributor to this problem but has not been systematically studied. The objective of this systematic review and meta-regression was to determine reported CDSS uptake and identify which CDSS features may influence uptake.

METHODS

Medline, Embase, CINAHL, and the Cochrane Database of Controlled Trials were searched from January 2000 to August 2020. Randomized, non-randomized, and quasi-experimental trials reporting CDSS uptake in any patient population or setting were included. The main outcome extracted was CDSS uptake, reported as a raw proportion, and representing the number of times the CDSS was used or accessed over the total number of times it could have been interacted with. We also extracted context, content, system, and implementation features that might influence uptake, for each CDSS. Overall weighted uptake was calculated using random-effects meta-analysis and determinants of uptake were investigated using multivariable meta-regression.

RESULTS

Among 7995 citations screened, 55 studies involving 373,608 patients and 3607 providers met full inclusion criteria. Meta-analysis revealed that overall CDSS uptake was 34.2% (95% CI 23.2 to 47.1%). Uptake was only reported in 12.4% of studies that otherwise met inclusion criteria. Multivariable meta-regression revealed the following factors significantly associated with uptake: (1) formally evaluating the availability and quality of the patient data needed to inform CDSS advice; and (2) identifying and addressing other barriers to the behaviour change targeted by the CDSS.

CONCLUSIONS AND RELEVANCE

System uptake was seldom reported in CDSS trials. When reported, uptake was low. This represents a major and potentially modifiable barrier to overall CDSS effectiveness. We found that features relating to CDSS context and implementation strategy best predicted uptake. Future studies should measure the impact of addressing these features as part of the CDSS implementation strategy. Uptake reporting must also become standard in future studies reporting CDSS intervention effects.

REGISTRATION

Pre-registered on PROSPERO, CRD42018092337.

Emergency care with lay responders in underserved populations: a systematic review

Objective

To assess the individual and community health effects of task shifting for emergency care in low-resource settings and underserved populations worldwide.

Methods

We systematically searched 13 databases and additional grey literature for studies published between 1984 and 2019. Eligible studies involved emergency care training for laypeople in underserved or low-resource populations, and any quantitative assessment of effects on the health of individuals or communities. We conducted duplicate assessments of study eligibility, data abstraction and quality. We synthesized findings in narrative and tabular format.

Findings

Of 19 308 papers retrieved, 34 studies met the inclusion criteria from low- and middle-income countries (21 studies) and underserved populations in high-income countries (13 studies). Targeted emergency conditions included trauma, burns, cardiac arrest, opioid poisoning, malaria, paediatric communicable diseases and malnutrition. Trainees included the general public, non-health-care professionals, volunteers and close contacts of at-risk populations, all trained through in-class, peer and multimodal education and public awareness campaigns. Important clinical and policy outcomes included improvements in community capacity to manage emergencies (14 studies), patient outcomes (13 studies) and community health (seven studies). While substantial effects were observed for programmes to address paediatric malaria, trauma and opioid poisoning, most studies reported modest effect sizes and two reported null results. Most studies were of weak (24 studies) or moderate quality (nine studies).

Conclusion

First aid education and task shifting to laypeople for emergency care may reduce patient morbidity and mortality and build community capacity to manage health emergencies for a variety of emergency conditions in underserved and low-resource settings.

Instruments to measure fear of COVID-19: a diagnostic systematic review

BACKGROUND

The COVID-19 pandemic has become a source of fear across the world. Measuring the level or significance of fear in different populations may help identify populations and areas in need of public health and education campaigns. We were interested in diagnostic tests developed to assess or diagnose COVID-19-related fear or phobia.

METHODS

We performed a systematic review of studies that examined instruments diagnosing or assessing fear or phobia of COVID-19 (PROSPERO registration: CRD42020197100). We utilized the Norwegian Institute of Public Health's Live map of covid-19 evidence, a database of pre-screened and pre-categorized studies. The Live map of covid-19 evidence identified references published since 1 December 2019 in MEDLINE, Embase, and the Centers for Disease Control and Prevention. Following biweekly searches, two researchers independently categorized all studies according to topic (seven main topics, 52 subordinate topics), population (41 available groups), study design, and publication type. For this review, we assessed for eligibility all studies that had been categorized to the topic "Experiences and perceptions, consequences; social, political, economic aspects" as of 25 September 2020, in addition to hand-searching included studies' reference lists. We meta-analyzed correlation coefficients of fear scores to the most common reference tests (self-reports of anxiety, depression, and stress), and reported additional concurrent validity to other reference tests such as specific phobias. We assessed study quality using the QUADAS-2 for the minority of studies that presented diagnostic accuracy statistics.

RESULTS

We found 18 studies that validated fear instruments. Fifteen validated the Fear of COVID-19 scale (FCV-19S). We found no studies that proposed a diagnosis of fear of COVID-19 or a threshold of significant/clinical versus non-significant/subclinical fear. Study quality was low, with the most common potential biases related to sampling strategy and un-blinded data analysis. The FSV-19S total score correlated strongly with severe phobia (r = 0.703, 95%CI 0.634-0.761) in one study, and moderately with anxiety in a meta-analysis.

CONCLUSIONS

The accuracy of the FSV-19S needs to be measured further using fear-related reference instruments, and future studies need to provide cut-off scores and normative values. Further evaluation of the remaining three instruments is required.

The mental health impact of the covid-19 pandemic on healthcare workers, and interventions to help them: A rapid systematic review

The covid-19 pandemic has heavily burdened healthcare systems throughout the world. We performed a rapid systematic review to identify, assess and summarize research on the mental health impact of the covid-19 pandemic on HCWs (healthcare workers). We utilized the Norwegian Institute of Public Health's Live map of covid-19 evidence on 11 May and included 59 studies. Six reported on implementing interventions, but none reported on effects of the interventions. HCWs reported low interest in professional help, and greater reliance on social support and contact. Exposure to covid-19 was the most commonly reported correlate of mental health problems, followed by female gender, and worry about infection or about infecting others. Social support correlated with less mental health problems. HCWs reported anxiety, depression, sleep problems, and distress during the covid-19 pandemic. We assessed the certainty of the estimates of prevalence of these symptoms as very low using GRADE. Most studies did not report comparative data on mental health symptoms before the pandemic or in the general population. There seems to be a mismatch between risk factors for adverse mental health outcomes among HCWs in the current pandemic, their needs and preferences, and the individual psychopathology focus of current interventions.

Coding Systems for Clinical Decision Support: Theoretical and Real-World Comparative Analysis

Background

Effective clinical decision support systems require accurate translation of practice recommendations into machine-readable artifacts; developing code sets that represent clinical concepts are an important step in this process. Many clinical coding systems are currently used in electronic health records, and it is unclear whether all of these systems are capable of efficiently representing the clinical concepts required in executing clinical decision support systems.

Objective

The aim of this study was to evaluate which clinical coding systems are capable of efficiently representing clinical concepts that are necessary for translating artifacts into executable code for clinical decision support systems.

Methods

Two methods were used to evaluate a set of clinical coding systems. In a theoretical approach, we extracted all the clinical concepts from 3 preventive care recommendations and constructed a series of code sets containing codes from a single clinical coding system. In a practical approach using data from a real-world setting, we studied the content of 1890 code sets used in an internationally available clinical decision support system and compared the usage of various clinical coding systems.

Results

SNOMED CT and ICD-10 (International Classification of Diseases, Tenth Revision) proved to be the most accurate clinical coding systems for most concepts in our theoretical evaluation. In our practical evaluation, we found that International Classification of Diseases (Tenth Revision) was most often used to construct code sets. Some coding systems were very accurate in representing specific types of clinical concepts, for example, LOINC (Logical Observation Identifiers Names and Codes) for investigation results and ATC (Anatomical Therapeutic Chemical Classification) for drugs.

Conclusions

No single coding system seems to fulfill all the needs for representing clinical concepts for clinical decision support systems. Comprehensiveness of the coding systems seems to be offset by complexity and forms a barrier to usability for code set construction. Clinical vocabularies mapped to multiple clinical coding systems could facilitate clinical code set construction.

Computerized clinical decision support system for diabetes in primary care does not improve quality of care: a cluster-randomized controlled trial

Background

The EBMeDS system is the computerized clinical decision support (CCDS) system of EBPNet, a national computerized point-of-care information service in Belgium. There is no clear evidence of more complex CCDS systems to manage chronic diseases in primary care practices (PCPs). The objective of this study was to assess the effectiveness of EBMeDS use in improving diabetes care.

Methods

A cluster-randomized trial with before-and-after measurements was performed in Belgian PCPs over 1 year, from May 2017 to May 2018. We randomly assigned 51 practices to either the intervention group (IG), to receive the EBMeDS system, or to the control group (CG), to receive usual care. Primary and secondary outcomes were the 1-year pre- to post-implementation change in HbA1c, LDL cholesterol, and systolic and diastolic blood pressure. Composite patient and process scores were calculated. A process evaluation was added to the analysis. Results were analyzed at 6 and 12 months. Linear mixed models and logistic regression models based on generalized estimating equations were used where appropriate.

Results

Of the 51 PCPs that were enrolled and randomly assigned (26 PCPs in the CG and 25 in the IG), 29 practices (3815 patients) were analyzed in the study: 2464 patients in the CG and 1351 patients in the IG. No change differences existed between groups in primary or secondary outcomes. Change difference between CG and IG after 1-year follow-up was − 0.09 (95% CI − 0.18; 0.01, p-value = 0.06) for HbA1c; 1.76 (95% CI − 0.46; 3.98, p-value = 0.12) for LDL cholesterol; and 0.13 (95% CI − 0.91; 1.16, p-value = 0.81) and 0.12 (95% CI − 1.25;1.49, p-value = 0.86) for systolic and diastolic blood pressure respectively. The odds ratio of the IG versus the CG for the probability of no worsening and improvement was 1.09 (95% CI 0.73; 1.63, p-value = 0.67) for the process composite score and 0.74 (95% CI 0.49; 1.12, p-value = 0.16) for the composite patient score. All but one physician was satisfied with the EBMeDS system.

Conclusions

The CCDS system EBMeDS did not improve diabetes care in Belgian primary care. The lack of improvement was mainly caused by imperfections in the organizational context of Belgian primary care for chronic disease management and shortcomings in the system requirements for the correct use of the EBMeDS system (e.g., complete structured records). These shortcomings probably caused low-use rates of the system.

Trial registration

ClinicalTrials.gov, NCT01830569, Registered 12 April 2013.

A systematic review of trials evaluating success factors of interventions with computerised clinical decision support

BACKGROUND

Computerised clinical decision support (CDS) can potentially better inform decisions, and it can help with the management of information overload. It is perceived to be a key component of a learning health care system. Despite its increasing implementation worldwide, it remains uncertain why the effect of CDS varies and which factors make CDS more effective.

OBJECTIVE

To examine which factors make CDS strategies more effective on a number of outcomes, including adherence to recommended practice, patient outcome measures, economic measures, provider or patient satisfaction, and medical decision quality.

METHODS

We identified randomised controlled trials, non-randomised trials, and controlled before-and-after studies that directly compared CDS implementation with a given factor to CDS without that factor by searching CENTRAL, MEDLINE, EMBASE, and CINAHL and checking reference lists of relevant studies. We considered CDS with any objective for any condition in any healthcare setting. We included CDS interventions that were either displayed on screen or provided on paper and that were directed at healthcare professionals or targeted at both professionals and patients. The reviewers screened the potentially relevant studies in duplicate. They extracted data and assessed risk of bias in independent pairs or individually followed by a double check by another reviewer. We summarised results using medians and interquartile ranges and rated our certainty in the evidence using the GRADE system.

RESULTS

We identified 66 head-to-head trials that we synthesised across 14 comparisons of CDS intervention factors. Providing CDS automatically versus on demand led to large improvements in adherence. Displaying CDS on-screen versus on paper led to moderate improvements and making CDS more versus less patient-specific improved adherence modestly. When CDS interventions were combined with professional-oriented strategies, combined with patient-oriented strategies, or combined with staff-oriented strategies, then adherence improved slightly. Providing CDS to patients slightly increased adherence versus CDS aimed at the healthcare provider only. Making CDS advice more explicit and requiring users to respond to the advice made little or no difference. The CDS intervention factors made little or no difference to patient outcomes. The results for economic outcomes and satisfaction outcomes were sparse.

CONCLUSION

Multiple factors may affect the success of CDS interventions. CDS may be more effective when the advice is provided automatically and displayed on-screen and when the suggestions are more patient-specific. CDS interventions combined with other strategies probably also improves adherence. Providing CDS directly to patients may also positively affect adherence. The certainty of the evidence was low to moderate for all factors.

TRIAL REGISTRATION

PROSPERO, CRD42016033738.

The GUIDES checklist: development of a tool to improve the successful use of guideline-based computerised clinical decision support

Background

Computerised decision support (CDS) based on trustworthy clinical guidelines is a key component of a learning healthcare system. Research shows that the effectiveness of CDS is mixed.

Multifaceted context, system, recommendation and implementation factors may potentially affect the success of CDS interventions. This paper describes the development of a checklist that is intended to support professionals to implement CDS successfully.

Methods

We developed the checklist through an iterative process that involved a systematic review of evidence and frameworks, a synthesis of the success factors identified in the review, feedback from an international expert panel that evaluated the checklist in relation to a list of desirable framework attributes, consultations with patients and healthcare consumers and pilot testing of the checklist.

Results

We screened 5347 papers and selected 71 papers with relevant information on success factors for guideline-based CDS. From the selected papers, we developed a 16-factor checklist that is divided in four domains, i.e. the CDS context, content, system and implementation domains. The panel of experts evaluated the checklist positively as an instrument that could support people implementing guideline-based CDS across a wide range of settings globally. Patients and healthcare consumers identified guideline-based CDS as an important quality improvement intervention and perceived the GUIDES checklist as a suitable and useful strategy.

Conclusions

The GUIDES checklist can support professionals in considering the factors that affect the success of CDS interventions. It may facilitate a deeper and more accurate understanding of the factors shaping CDS effectiveness. Relying on a structured approach may prevent that important factors are missed.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0772-3) contains supplementary material, which is available to authorized users.

Involving general practice trainees in clinical practice guideline adaptation

BACKGROUND

It is unclear whether it is feasible to involve residents in guideline development or adaptation. We designed a multifaceted training program that combines training sessions, a handbook and a documentation tool to assist general practice (GP)-trainees in the adaptation of clinical practice guidelines (CPGs). The aim of this study is to adapt a database of CPGs by involving GP-trainees and to build evidence-based practice (EBP) learning capacity.

METHODS

We assessed each adaptation process and surveyed all GP-trainees who enrolled in our training program on their views on the program. They were asked to formulate an overall rating for the training and were asked to rate individual aspects of the training program (the training sessions, the handbook and the documentation tool).

RESULTS

To date, 122 GP-trainees followed the training and have adapted 60 different CPGs. Overall quality of their work was good. Based on an assessment of the content of the documentation tool, 24 (40%) adapted CPGs rated as good quality and 30 (50%) rated as moderate quality. Only 3 adapted CPGs (5%) were evaluated as being of poor quality. 51 (42%) GP-trainees completed the survey on user satisfaction. 98% (50) of the GP-trainees found the training to be of good overall quality. 86% of the GP-trainees were satisfied with the handbook but satisfaction was lowest for the documentation tool (47% satisfied).

CONCLUSION

It is possible to engage GP-trainees in CPG adaptation using a formal process when provided with training, feedback and documentation tools.

Development of a Tailored Intervention With Computerized Clinical Decision Support to Improve Quality of Care for Patients With Knee Osteoarthritis: Multi-Method Study

BACKGROUND

Clinical practice patterns greatly diverge from evidence-based recommendations to manage knee osteoarthritis conservatively before resorting to surgery.

OBJECTIVE

This study aimed to tailor a guideline-based computerized decision support (CDS) intervention that facilitates the conservative management of knee osteoarthritis.

METHODS

Experts with backgrounds in clinical medicine, research, implementation, or health informatics suggested the most important recommendations for implementation, how to develop an implementation strategy, and how to form the CDS algorithms. In 6 focus group sessions, 8 general practitioners and 22 patients from Norway, Belgium, and Finland discussed the suggested CDS intervention and identified factors that would be most critical for the success of the intervention. The focus group moderators used the GUideline Implementation with DEcision Support checklist, which we developed to support consideration of CDS success factors.

RESULTS

The experts prioritized 9 out of 22 recommendations for implementation. We formed the concept for 6 CDS algorithms to support implementation of these recommendations. The focus group suggested 59 unique factors that could affect the success of the presented CDS intervention. Five factors (out of the 59) were prioritized by focus group participants in every country, including the perceived potential to address the information needs of both patients and general practitioners; the credibility of CDS information; the timing of CDS for patients; and the need for personal dialogue about CDS between the general practitioner and the patient.

CONCLUSIONS

The focus group participants supported the CDS intervention as a tool to improve the quality of care for patients with knee osteoarthritis through shared, evidence-based decision making. We aim to develop and implement the CDS based on these study results. Future research should address optimal ways to (1) provide patient-directed CDS, (2) enable more patient-specific CDS within the context of patient complexity, and (3) maintain user engagement with CDS over time.

Electronic Laboratory Medicine ordering with evidence-based Order sets in primary care (ELMO study): protocol for a cluster randomised trial

BACKGROUND

Laboratory testing is an important clinical act with a valuable role in screening, diagnosis, management and monitoring of diseases or therapies. However, inappropriate laboratory test ordering is frequent, burdening health care spending and negatively influencing quality of care. Inappropriate tests may also result in false-positive results and potentially cause excessive downstream activities. Clinical decision support systems (CDSSs) have shown promising results to influence the test-ordering behaviour of physicians and to improve appropriateness. Order sets, a form of CDSS where a limited set of evidence-based tests are proposed for a series of indications, integrated in a computerised physician order entry (CPOE) have been shown to be effective in reducing the volume of ordered laboratory tests but convincing evidence that they influence appropriateness is lacking. The aim of this study is to evaluate the effect of order sets on the quality and quantity of laboratory test orders by physicians. We also aim to evaluate the effect of order sets on diagnostic error and explore the effect on downstream or cascade activities.

METHODS

We will conduct a cluster randomised controlled trial in Belgian primary care practices. The study is powered to measure two outcomes. We will primarily measure the influence of our CDSS on the appropriateness of laboratory test ordering. Additionally, we will also measure the influence on diagnostic error. We will also explore the effects of our intervention on cascade activities due to altered results of inappropriate tests.

DISCUSSION

We have designed a study that should be able to demonstrate whether the CDSS aimed at diagnostic testing is not only able to influence appropriateness but also safe with respect to diagnostic error. These findings will influence a lager, nationwide implementation of this CDSS.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT02950142 .

The Effects of Computerized Clinical Decision Support Systems on Laboratory Test Ordering: A Systematic Review

CONTEXT

- Inappropriate laboratory test ordering has been shown to be as high as 30%. This can have an important impact on quality of care and costs because of downstream consequences such as additional diagnostics, repeat testing, imaging, prescriptions, surgeries, or hospital stays.

OBJECTIVE

- To evaluate the effect of computerized clinical decision support systems on appropriateness of laboratory test ordering.

DATA SOURCES

- We used MEDLINE, Embase, CINAHL, MEDLINE In-Process and Other Non-Indexed Citations, Clinicaltrials.gov, Cochrane Library, and Inspec through December 2015. Investigators independently screened articles to identify randomized trials that assessed a computerized clinical decision support system aimed at improving laboratory test ordering by providing patient-specific information, delivered in the form of an on-screen management option, reminder, or suggestion through a computerized physician order entry using a rule-based or algorithm-based system relying on an evidence-based knowledge resource. Investigators extracted data from 30 papers about study design, various study characteristics, study setting, various intervention characteristics, involvement of the software developers in the evaluation of the computerized clinical decision support system, outcome types, and various outcome characteristics.

CONCLUSIONS

- Because of heterogeneity of systems and settings, pooled estimates of effect could not be made. Data showed that computerized clinical decision support systems had little or no effect on clinical outcomes but some effect on compliance. Computerized clinical decision support systems targeted at laboratory test ordering for multiple conditions appear to be more effective than those targeted at a single condition.

Adapting a large database of point of care summarized guidelines: a process description

RATIONALE, AIMS AND OBJECTIVES

Questions posed at the point of care (POC) can be answered using POC summarized guidelines. To implement a national POC information resource, we subscribed to a large database of POC summarized guidelines to complement locally available guidelines. Our challenge was in developing a sustainable strategy for adapting almost 1000 summarized guidelines. The aim of this paper was to describe our process for adapting a database of POC summarized guidelines.

METHODS

An adaptation process based on the ADAPTE framework was tailored to be used by a heterogeneous group of participants. Guidelines were assessed on content and on applicability to the Belgian context. To improve efficiency, we chose to first aim our efforts towards those guidelines most important to primary care doctors.

RESULTS

Over a period of 3 years, we screened about 80% of 1000 international summarized guidelines. For those guidelines identified as most important for primary care doctors, we noted that in about half of the cases, remarks were made concerning content. On the other hand, at least two-thirds of all screened guidelines required no changes when evaluating their local usability.

CONCLUSIONS

Adapting a large body of POC summarized guidelines using a formal adaptation process is possible, even when faced with limited resources. This can be done by creating an efficient and collaborative effort and ensuring user-friendly procedures. Our experiences show that even though in most cases guidelines can be adopted without adaptations, careful review of guidelines developed in a different context remains necessary. Streamlining international efforts in adapting international POC information resources and adopting similar adaptation processes may lessen duplication efforts and prove more cost-effective.

Lower Extremity Near-infrared Spectroscopy After Popliteal Block For Orthopaedic Foot Surgery

Background:

Noninvasive measurement of cutaneous tissue oxygenation using near-infrared spectroscopy (NIRS) has become common in peri-operative care. Following institution of peripheral nerve blocks, neurovascular alterations in the blocked region have been described.

Objective:

The primary aim of this study encompassed the assessment of the influence of a popliteal block on changes in regional oxygen saturation (SrO₂), and the location of most prominent changes.

Method:

We conducted a prospective randomised controlled trial. Hundred twenty patients who received a popliteal block for foot surgery were included. Popliteal block was performed under echographic guidance. The patients were randomized in 3 groups according to the location of the SrO₂ electrodes on the legs. Bilateral SrO₂ measurements were performed simultaneously. SrO₂ in the operated leg and in the control leg was measured at baseline and 1, 5, 10, 15, and 30 minutes after the perineural injection. We quantified the evolution in SrO₂ by calculating over time the differences in SrO₂ values between the operated and control leg (=ΔSrO₂).

Results:

At 30 minutes, ΔSrO₂ increased significantly (p<0.05) at the plantar side of the foot (11.3% ± 2.9%), above the ankle (4.9% ± 1.3%) and the popliteal fossa (3.6% ± 1.2%).

Conclusion:

At 30 minutes after institution of the popliteal block, ΔSrO₂ was most prominent at the plantar side of the foot as compared with measurement performed above the ankle or under the knee.

Health effects of training laypeople to deliver emergency care in underserviced populations: a systematic review protocol

INTRODUCTION

The Disease Control Priorities Project recommends emergency care training for laypersons in low-resource settings, but evidence for these interventions has not yet been systematically reviewed. This review will identify the individual and community health effects of educating laypeople to deliver prehospital emergency care interventions in low-resource settings.

METHODS AND ANALYSIS

This systematic review addresses the following question: in underserviced populations and low-resource settings (P), does first aid or emergency care training or education for laypeople (I) confer any individual or community health benefit for emergency health conditions (O), in comparison with no training or other forms of education (C)? We restrict this review to studies reporting quantitatively measurable outcomes, and search 12 electronic bibliographic databases and grey literature sources. A team of expert content and methodology reviewers will conduct title and abstract screening and full-text review, using a custom-built online platform. Two investigators will independently extract methodological variables and outcomes related to patient-level morbidity and mortality and community-level effects on resilience or emergency care capacity. Two investigators will independently assess external validity, selection bias, performance bias, measurement bias, attrition bias and confounding. We will summarise the findings using a narrative approach to highlight similarities and differences between the gathered studies.

ETHICS AND DISSEMINATION

Formal ethical approval is not required.

RESULTS

The results will be disseminated through a peer-reviewed publication and knowledge translation strategy.

REVIEW REGISTRATION NUMBER

CRD42014009685.

Tailoring implementation strategies for evidence-based recommendations using computerised clinical decision support systems: protocol for the development of the GUIDES tools

Background

A computerised clinical decision support system (CCDSS) is a technology that uses patient-specific data to provide relevant medical knowledge at the point of care. It is considered to be an important quality improvement intervention, and the implementation of CCDSS is growing substantially. However, the significant investments do not consistently result in value for money due to content, context, system and implementation issues. The Guideline Implementation with Decision Support (GUIDES) project aims to improve the impact of CCDSS through optimised implementation based on high-quality evidence-based recommendations. To achieve this, we will develop tools that address the factors that determine successful CCDSS implementation.

Methods/design

We will develop the GUIDES tools in four steps, using the methods and results of the Tailored Implementation for Chronic Diseases (TICD) project as a starting point: (1) a review of research evidence and frameworks on the determinants of implementing recommendations using CCDSS; (2) a synthesis of a comprehensive framework for the identified determinants; (3) the development of tools for use of the framework and (4) pilot testing the utility of the tools through the development of a tailored CCDSS intervention in Norway, Belgium and Finland. We selected the conservative management of knee osteoarthritis as a prototype condition for the pilot. During the process, the authors will collaborate with an international expert group to provide input and feedback on the tools.

Discussion

This project will provide guidance and tools on methods of identifying implementation determinants and selecting strategies to implement evidence-based recommendations through CCDSS. We will make the GUIDES tools available to CCDSS developers, implementers, researchers, funders, clinicians, managers, educators, and policymakers internationally. The tools and recommendations will be generic, which makes them scalable to a large spectrum of conditions. Ultimately, the better implementation of CCDSS may lead to better-informed decisions and improved care and patient outcomes for a wide range of conditions.

Protocol registration

PROSPERO, CRD42016033738

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0393-7) contains supplementary material, which is available to authorized users.

Technology for Large-Scale Translation of Clinical Practice Guidelines: A Pilot Study of the Performance of a Hybrid Human and Computer-Assisted Approach

BACKGROUND

The construction of EBMPracticeNet, a national electronic point-of-care information platform in Belgium, began in 2011 to optimize quality of care by promoting evidence-based decision making. The project involved, among other tasks, the translation of 940 EBM Guidelines of Duodecim Medical Publications from English into Dutch and French. Considering the scale of the translation process, it was decided to make use of computer-aided translation performed by certificated translators with limited expertise in medical translation. Our consortium used a hybrid approach, involving a human translator supported by a translation memory (using SDL Trados Studio), terminology recognition (using SDL MultiTerm terminology databases) from medical terminology databases, and support from online machine translation. This resulted in a validated translation memory, which is now in use for the translation of new and updated guidelines.

OBJECTIVE

The objective of this experiment was to evaluate the performance of the hybrid human and computer-assisted approach in comparison with translation unsupported by translation memory and terminology recognition. A comparison was also made with the translation efficiency of an expert medical translator.

METHODS

We conducted a pilot study in which two sets of 30 new and 30 updated guidelines were randomized to one of three groups. Comparable guidelines were translated (1) by certificated junior translators without medical specialization using the hybrid method, (2) by an experienced medical translator without this support, and (3) by the same junior translators without the support of the validated translation memory. A medical proofreader who was blinded for the translation procedure, evaluated the translated guidelines for acceptability and adequacy. Translation speed was measured by recording translation and post-editing time. The human translation edit rate was calculated as a metric to evaluate the quality of the translation. A further evaluation was made of translation acceptability and adequacy.

RESULTS

The average number of words per guideline was 1195 and the mean total translation time was 100.2 minutes/1000 words. No meaningful differences were found in the translation speed for new guidelines. The translation of updated guidelines was 59 minutes/1000 words faster (95% CI 2-115; P=.044) in the computer-aided group. Revisions due to terminology accounted for one third of the overall revisions by the medical proofreader.

CONCLUSIONS

Use of the hybrid human and computer-aided translation by a non-expert translator makes the translation of updates of clinical practice guidelines faster and cheaper because of the benefits of translation memory. For the translation of new guidelines, there was no apparent benefit in comparison with the efficiency of translation unsupported by translation memory (whether by an expert or non-expert translator).

Evidence-based prehospital management of severe traumatic brain injury: a comparative analysis of current clinical practice guidelines

OBJECTIVE

This study appraised the completeness and level of evidence behind prehospital recommendations in clinical practice guidelines (CPGs) for management of severe traumatic brain injury (TBI). Differences and similarities in key recommendations for prehospital emergency care were assessed between current CPGs.

METHODS

A systematic search identified current evidence-based CPGs for the management of severe TBI. The identified CPGs were screened for prehospital recommendations. Finally, an evaluation of the completeness and level of evidence for each of the identified recommendations was carried out. A review of the literature identified additional evidence. Designs of the retrieved publications were considered and classified according to the GRADE levels of evidence.

RESULTS

This study identified 12 current CPGs for the management of patients after traumatic brain injury. Of these, twenty-one prehospital recommendations were selected. Only a few CPGs made recommendations on temperature management and ventilation patterns. Statements on prehospital transport and advanced airway management were common to all of the guidelines. Statements on initial treatment demonstrated the greatest variability. The literature review identified several relevant publications not included in the CPGs even after we controlled for the indicated time-intervals of their literature search. In addition, evidence from more recent trials published outside the search-interval of the clinical practice guidelines was found.

CONCLUSIONS

The use of current guidelines on traumatic brain injury will not always facilitate decisions about best or most appropriate practice for prehospital practitioners. The amount of recommended prehospital interventions varied considerably, and there was large content variation in prehospital recommendations in these guidelines. Not all evidence was taken into account and not all CPGs were up-to-date.

Feasibility and impact of an evidence-based electronic decision support system for diabetes care in family medicine: protocol for a cluster randomized controlled trial

Background

In Belgium, the construction of the national electronic point-of-care information service, EBMPracticeNet, was initiated in 2011 to optimize quality of care by promoting evidence-based decision-making. The collaboration of the government, healthcare providers, Evidence-Based Medicine (EBM) partners, and vendors of Electronic Health Records (EHR) is unique to this project. All Belgian healthcare professionals get free access to an up-to-date database of validated Belgian and nearly 1,000 international guidelines, incorporated in a portal that also provides EBM information from sources other than guidelines, including computerized clinical decision support that is integrated in the EHRs.

The EBMeDS system is the electronic evidence-based decision support system of EBMPracticeNet. The EBMeDS system covers all clinical areas of diseases and could play a crucial role in response to the emerging challenge posed by chronic conditions. Diabetes was chosen as the analysis topic of interest. The objective of this study is to assess the effectiveness of EBMeDS use in improving diabetes care. This objective will be enhanced by a formal process evaluation to provide crucial information on the feasibility of using the system in daily Belgian family medicine.

Methods

The study is a cluster-randomized trial with before/after measurements conducted in Belgian family medicine. Physicians’ practices will be randomly assigned to the intervention or control group in a 1:1 ratio, to receive either the EBMeDS reminders or to follow the usual care process. Randomization will be performed by a statistical consultant with an electronic random list generator, anonymously for the researchers. The follow-up period of the study will be 12 months with interim analysis points at 3, 6 and 9 months. Primary outcome is the one-year pre- to post-implementation change in HbA1c. Patients will not be informed about the intervention. Data analysts will be kept blinded to the allocation.

Discussion

The knowledge obtained in this study will be useful for further integration in other Belgian software packages. Users’ perceptions and process evaluation will provide information for improving the feasibility of the system.

Trial registration

The trial is registered with the ClinicalTrials.gov registry: NCT01830569.

EBMPracticeNet: A Bilingual National Electronic Point-Of-Care Project for Retrieval of Evidence-Based Clinical Guideline Information and Decision Support

BACKGROUND

In Belgium, the construction of a national electronic point-of-care information service, EBMPracticeNet, was initiated in 2011 to optimize quality of care by promoting evidence-based decision-making. The collaboration of the government, health care providers, evidence-based medicine (EBM) partners, and vendors of electronic health records (EHR) is unique to this project. All Belgian health care professionals get free access to an up-to-date database of validated Belgian and nearly 1000 international guidelines, incorporated in a portal that also provides EBM information from other sources than guidelines, including computerized clinical decision support that is integrated in the EHRs.

OBJECTIVE

The objective of this paper was to describe the development strategy, the overall content, and the management of EBMPracticeNet which may be of relevance to other health organizations creating national or regional electronic point-of-care information services.

METHODS

Several candidate providers of comprehensive guideline solutions were evaluated and one database was selected. Translation of the guidelines to Dutch and French was done with translation software, post-editing by translators and medical proofreading. A strategy is determined to adapt the guideline content to the Belgian context. Acceptance of the computerized clinical decision support tool has been tested and a randomized controlled trial is planned to evaluate the effect on process and patient outcomes.

RESULTS

Currently, EBMPracticeNet is in "work in progress" state. Reference is made to the results of a pilot study and to further planned research including a randomized controlled trial.

CONCLUSIONS

The collaboration of government, health care providers, EBM partners, and vendors of EHRs is unique. The potential value of the project is great. The link between all the EHRs from different vendors and a national database held on a single platform that is controlled by all EBM organizations in Belgium are the strengths of EBMPracticeNet.

Human computation as a new method for evidence-based knowledge transfer in Web-based guideline development groups: proof of concept randomized controlled trial

BACKGROUND

Guideline developers use different consensus methods to develop evidence-based clinical practice guidelines. Previous research suggests that existing guideline development techniques are subject to methodological problems and are logistically demanding. Guideline developers welcome new methods that facilitate a methodologically sound decision-making process. Systems that aggregate knowledge while participants play a game are one class of human computation applications. Researchers have already proven that these games with a purpose are effective in building common sense knowledge databases.

OBJECTIVE

We aimed to evaluate the feasibility of a new consensus method based on human computation techniques compared to an informal face-to-face consensus method.

METHODS

We set up a randomized design to study 2 different methods for guideline development within a group of advanced students completing a master of nursing and obstetrics. Students who participated in the trial were enrolled in an evidence-based health care course. We compared the Web-based method of human-based computation (HC) with an informal face-to-face consensus method (IC). We used 4 clinical scenarios of lower back pain as the subject of the consensus process. These scenarios concerned the following topics: (1) medical imaging, (2) therapeutic options, (3) drugs use, and (4) sick leave. Outcomes were expressed as the amount of group (dis)agreement and the concordance of answers with clinical evidence. We estimated within-group and between-group effect sizes by calculating Cohen's d. We calculated within-group effect sizes as the absolute difference between the outcome value at round 3 and the baseline outcome value, divided by the pooled standard deviation. We calculated between-group effect sizes as the absolute difference between the mean change in outcome value across rounds in HC and the mean change in outcome value across rounds in IC, divided by the pooled standard deviation. We analyzed statistical significance of within-group changes between round 1 and round 3 using the Wilcoxon signed rank test. We assessed the differences between the HC and IC groups using Mann-Whitney U tests. We used a Bonferroni adjusted alpha level of .025 in all statistical tests. We performed a thematic analysis to explore participants' arguments during group discussion. Participants completed a satisfaction survey at the end of the consensus process.

RESULTS

Of the 135 students completing a master of nursing and obstetrics, 120 participated in the experiment. We formed 8 HC groups (n=64) and 7 IC groups (n=56). The between-group comparison demonstrated that the human computation groups obtained a greater improvement in evidence scores compared to the IC groups, although the difference was not statistically significant. The between-group effect size was 0.56 (P=.30) for the medical imaging scenario, 0.07 (P=.97) for the therapeutic options scenario, and 0.89 (P=.11) for the drug use scenario. We found no significant differences in improvement in the degree of agreement between HC and IC groups. Between-group comparisons revealed that the HC groups showed greater improvement in degree of agreement for the medical imaging scenario (d=0.46, P=.37) and the drug use scenario (d=0.31, P=.59). Very few evidence arguments (6%) were quoted during informal group discussions.

CONCLUSIONS

Overall, the use of the IC method was appropriate as long as the evidence supported participants' beliefs or usual practice, or when the availability of the evidence was sparse. However, when some controversy about the evidence existed, the HC method outperformed the IC method. The findings of our study illustrate the importance of the choice of the consensus method in guideline development. Human computation could be an acceptable methodology for guideline development specifically for scenarios in which the evidence shows no resonance with participants' beliefs. Future research is needed to confirm the results of this study and to establish practical significance in a controlled setting of multidisciplinary guideline panels during real-life guideline development.

Can training improve laypersons helping behaviour in first aid? A randomised controlled deception trial

BACKGROUND

There is limited evidence indicating that laypersons trained in first aid provide better help, but do not help more often than untrained laypersons. This study investigated the effect of conventional first aid training versus conventional training plus supplementary training aimed at decreasing barriers to helping.

METHODS

The authors conducted a randomised controlled trial. After 24 h of conventional first aid training, the participants either attended an experimental lesson to reduce barriers to helping or followed a control lesson. The authors used a deception test to measure the time between the start of the unannounced simulated emergency and seeking help behaviour and the number of particular helping actions.

RESULTS

The authors randomised 72 participants to both groups. 22 participants were included in the analysis for the experimental group and 36 in the control group. The authors found no statistically or clinically significant differences for any of the outcome measures. The time until seeking help (geometrical mean and 95% CI) was 55.5 s (42.9 to 72.0) in the experimental group and 56.5 s (43.0 to 74.3) in the control group. 57% of the participants asked a bystander to seek help, 40% left the victim to seek help themselves and 3% did not seek any help.

CONCLUSION

Supplementary training on dealing with barriers to helping did not alter the helping behaviour. The timing and appropriateness of the aid provided can be improved.

TRIAL REGISTRATION

The authors registered this trial at ClinicalTrials.gov as NCT00954161.

Evidence-based African first aid guidelines and training materials

Stijn Van de Velde and colleagues describe the African First Aid Materials project, which developed evidence-based guidelines on administering first aid in the African context as well as training materials to support the implementation of the guidelines.

Medicinal use of potato-derived products: conclusions of a rapid versus full systematic review

Vlachojannis et al reported a systematic review on the medicinal use of potato-derived products. The authors identified five trials for inclusion in the review, including one study on the treatment of burns. Based on this RCT the review authors concluded that potato peel is not recommended for burns. As the authors of a rapid review on the use of potato peels for burns, we read this systematic review with great interest. Although the concept of rapid review is rising, accelerating the review process might introduce bias and its conclusions may be subject to change once a systematic review is available. Since this rapid and systematic review were done at similar times, we explored if the results were consistent. We identified three trials on the use of potato peels. Two of these trials were not mentioned in the systematic review. The evidence indicates that sterile potato peel dressings are better than gauze alone during the healing phase.While there is no evidence of an antibacterial effect, we concluded that potato peels promote healing. Potato peel dressings might be the best available dressing in resource poor countries. Because systematic reviews have a major impact it is crucial that systematic reviews meet specified quality criteria. Therefore we draw attention to adherance to the PRISMA statement.

Effectiveness of electronic guideline-based implementation systems in ambulatory care settings - a systematic review

Background

Electronic guideline-based decision support systems have been suggested to successfully deliver the knowledge embedded in clinical practice guidelines. A number of studies have already shown positive findings for decision support systems such as drug-dosing systems and computer-generated reminder systems for preventive care services.

Methods

A systematic literature search (1990 to December 2008) of the English literature indexed in the Medline database, Embase, the Cochrane Central Register of Controlled Trials, and CRD (DARE, HTA and NHS EED databases) was conducted to identify evaluation studies of electronic multi-step guideline implementation systems in ambulatory care settings. Important inclusion criterions were the multidimensionality of the guideline (the guideline needed to consist of several aspects or steps) and real-time interaction with the system during consultation. Clinical decision support systems such as one-time reminders for preventive care for which positive findings were shown in earlier reviews were excluded. Two comparisons were considered: electronic multidimensional guidelines versus usual care (comparison one) and electronic multidimensional guidelines versus other guideline implementation methods (comparison two).

Results

Twenty-seven publications were selected for analysis in this systematic review. Most designs were cluster randomized controlled trials investigating process outcomes more than patient outcomes. With success defined as at least 50% of the outcome variables being significant, none of the studies were successful in improving patient outcomes. Only seven of seventeen studies that investigated process outcomes showed improvements in process of care variables compared with the usual care group (comparison one). No incremental effect of the electronic implementation over the distribution of paper versions of the guideline was found, neither for the patient outcomes nor for the process outcomes (comparison two).

Conclusions

There is little evidence at the moment for the effectiveness of an increasingly used and commercialised instrument such as electronic multidimensional guidelines. After more than a decade of development of numerous electronic systems, research on the most effective implementation strategy for this kind of guideline-based decision support systems is still lacking. This conclusion implies a considerable risk towards inappropriate investments in ineffective implementation interventions and in suboptimal care.

The attitude of Belgian social insurance physicians towards evidence-based practice and clinical practice guidelines

Background

Evidence-based medicine has broadened its scope and is starting to reach insurance medicine. Although still in its initial stages, physicians in the area of insurance medicine should keep up-to-date with the evidence on various diseases in order to correctly assess disability and to give appropriate advice about health care reimbursement. In order to explore future opportunities of evidence-based medicine to improve daily insurance medicine, there is a need for qualitative studies to better understand insurance physicians' perceptions of EBM. The present study was designed to identify the attitude of insurance physicians towards evidence-based medicine and clinical practice guidelines, and to determine their ability to access, retrieve and appraise the health evidence and the barriers for applying evidence to practice.

Methods

A cross-sectional survey study was carried out among all Dutch-speaking insurance physicians employed at one of the six Belgian social insurance sickness funds and at the National Institute of Disability and Health care Insurance (n = 224). Chi-square tests were used to compare nominal and ordinal variables. Student's t-tests, ANOVA, Mann-Whitney and Kruskal-Wallis were used to compare means of continuous variables for different groups.

Results

The response rate was 48.7%. The majority of respondents were positive towards evidence-based medicine and clinical practice guidelines. Their knowledge of EBM was rather poor. Perceived barriers for applying evidence to practice were mainly time and lack of EBM skills.

Conclusion

Although the majority of physicians were positive towards EBM and welcomed more guidelines, the use of evidence and clinical practice guidelines in insurance medicine is low at present. It is in the first place important to eradicate the perceived inertia which limits the use of EBM and to further investigate the EBM principles in the context of insurance medicine. Available high-quality evidence-based resources (at the moment mainly originating from other medical fields) need to be structured in a way that is useful for insurance physicians and global access to this information needs to be ensured.

The attitude of Flemish occupational health physicians toward evidence-based occupational health and clinical practice guidelines

PURPOSE

To identify the attitude of occupational health physicians toward evidence-based occupational health (EBOH) and clinical practice guidelines (CPGs); to determine their ability to access, retrieve and appraise the health evidence and the barriers to applying evidence to practice.

METHODS

A cross-sectional survey study was carried out among all Dutch-speaking occupational health physicians in Belgium (584 physicians could be reached).

RESULTS

A response rate of 25.5% was achieved. The majority of respondents were positive toward EBOH and CPGs. Most respondents were less confident in basic skills of EBM, except for their searching skills. Perceived barriers to applying evidence to practice were mainly time and lack of EBM skills.

CONCLUSIONS

Belgian occupational health physicians are interested in the implementation of EBOH in their daily occupational practice and have a general knowledge of EBM. However, there are barriers in the legislative framework, the education and the information infrastructure, which first have to be removed. The time has come for the responsible authorities to take educational initiatives and to take a huge leap forward in the integration of EBOH into occupational practice.

European first aid guidelines

AIM

Our objectives were to determine the most effective, safe, and feasible first aid (FA) techniques and procedures, and to formulate valid recommendations for training. We focussed on emergencies involving few casualties, where emergency medical services or healthcare professionals are not immediately present at the scene, but are available within a short space of time. Due to time and resource constraints, we limited ourselves to safety, emergency removal, psychosocial FA, traumatology, and poisoning. Cardiopulmonary resuscitation (CPR) was not included because guidelines are already available from the European Resuscitation Council (ERC). The FA guidelines are intended to provide guidance to authors of FA handbooks and those responsible for FA programmes. These guidelines, together with the ERC resuscitation guidelines, will be integrated into a European FA Reference Guide and a European FA Manual.

METHODS

To create these guidelines we used an evidence-based guideline development process, based on the methodology of the Scottish Intercollegiate Guidelines Network (SIGN).

RESULTS

The recommendations cover FA for bleeding, wounds, burns, spinal and head trauma, musculoskeletal trauma, and poisoning, as well as safety and psychosocial FA.

CONCLUSIONS

Where good evidence was available, we were able to turn science into practice. Where evidence was lacking, the recommendations were consensus-based. These guidelines provide systematically developed recommendations and justifications for the procedures and techniques that should be included in FA manuals and training programmes.