A Delphi survey to determine how educational interventions for evidence-based practice should be reported: stage 2 of the development of a reporting guideline

Adherence to the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) of studies on evidence-based healthcare e-learning: a cross-sectional study

OBJECTIVES

The objectives of this study are to assess reporting of evidence-based healthcare (EBHC) e-learning interventions using the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and explore factors associated with compliant reporting.

DESIGN

Methodological cross-sectional study.

METHODS

Based on the criteria used in an earlier systematic review, we included studies comparing EBHC e-learning and any other form of EBHC training or no EBHC training. We searched Medline, Embase, ERIC, CINAHL, CENTRAL, SCOPUS, Web of Knowledge, PsycInfo, ProQuest and Best Evidence Medical Education up to 4 January 2023. Screening of titles, abstracts, full-text articles and data extraction was done independently by two authors. For each study, we assessed adherence to each of the 17 GREET items and extracted information on possible predictors. Adequacy of reporting for each item of the GREET checklist was judged with yes (provided complete information), no (provided no information), unclear (when insufficient information was provided), or not applicable, when the item was clearly of no relevance to the intervention described (such as for item 8-details about the instructors-in the studies which used electronic, self-paced intervention, without any tutoring). Studies' adherence to the GREET checklist was presented as percentages and absolute numbers. We performed univariate analysis to assess the association of potential adherence predictors with the GREET checklist. We summarised results descriptively.

RESULTS

We included 40 studies, the majority of which assessed e-learning or blended learning and mostly involved medical and other healthcare students. None of the studies fully reported all the GREET items. Overall, the median number of GREET items met (received yes) per study was 8 and third quartile (Q3) of GREET items met per study was 9 (min. 4 max. 14). When we used Q3 of the number of items met as cut-off point, adherence to the GREET reporting checklist was poor with 7 out of 40 studies (17.5%) reporting items of the checklist on acceptable level (adhered to at least 10 items out of 17). None of the studies reported on all 17 GREET items. For 3 items, 80% of included studies well reported information (received yes for these items): item 1 (brief description of intervention), item 4 (evidence-based practice content) and item 6 (educational strategies). Items for which 50% of included studies reported complete information (received yes for these items) included: item 9 (modes of delivery), item 11 (schedule) and 12 (time spent on learning). The items for which 70% or more of included studies did not provide information (received no for these items) included: item 7 (incentives) and item 13 (adaptations; for both items 70% of studies received no for them), item 14 (modifications of educational interventions-95% of studies received no for this item), item 16 (any processes to determine whether the materials and the educational strategies used in the educational intervention were delivered as originally planned-93% of studies received no for this item) and 17 (intervention delivery according to schedule-100% of studies received no for this item). Studies published after September 2016 showed slight improvements in nine reporting items. In the logistic regression models, using the cut-off point of Q3 (10 points or above) the odds of acceptable adherence to GREET guidelines were 7.5 times higher if adherence to other guideline (Consolidated Standards of Reporting Trials, Strengthening the Reporting of Observational Studies in Epidemiology, etc) was reported for a given study type (p=0.039), also higher number of study authors increased the odds of adherence to GREET guidance by 18% (p=0.037).

CONCLUSIONS

Studies assessing educational interventions on EBHC e-learning still poorly adhere to the GREET checklist. Using other reporting guidelines increased the odds of better GREET reporting. Journals should call for the use of appropriate use of reporting guidelines of future studies on teaching EBHC to increase transparency of reporting, decrease unnecessary research duplication and facilitate uptake of research evidence or result.

STUDY REGISTRATION NUMBER

The Open Science Framework (https://doi.org/10.17605/OSF.IO/V86FR).

Implementation and use of the Safewards model in healthcare services: A scoping review

Safewards is a multi-intervention mental health nursing model of practice improvement aimed at preventing and reducing conflict and containment. The use of Safewards has now extended beyond mental health settings. Implementation of Safewards has been reported to be challenging and therefore requires an evidence-informed and structured approach. This review's objectives were to: (i) Comprehensively map approaches used to implement Safewards interventions; (ii) Characterise the outcomes measured in Safewards implementation studies; and (iii) Identify the facilitators and barriers to Safewards training and its implementation in practice. All quantitative, qualitative and mixed-methods publications of Safewards, the interventions, evaluations, barriers and facilitators from all healthcare services internationally were included. The Joanna Briggs Institute scoping review and Preferred Reporting Items for Systematic Reviews and Meta-Analysis extension for Scoping Reviews were used to guide methodology. Data were reported according to the 12 items of the TIDieR. Twenty-seven publications reported the implementation of Safewards. Descriptions were limited for reporting items such as intervention descriptions, materials, resources, specific procedures and processes, modifications made to interventions and delivery of interventions and training. No consistent theoretical implementation framework was reported. Collaboration, leadership, feedback and co-design were strong drivers for staff buy-in, engagement and success for implementation in mental health and acute settings. Transparency, replicability and generalisation require a detailed description of all elements of an intervention being implemented. Without adequate information, only assumptions can be drawn about the clinical governance and process of the implementation and training, and it is difficult to conclude when attempting to replicate the interventions.

Improving self-directed learning ability of medical students using the blended teaching method: a quasi-experimental study

BACKGROUND

Self-directed learning (SDL) is one of the most important abilities for medical students in terms of their future clinical medical practice. During the blended teaching process, teachers can design a variety of learning activities to cultivate students' SDL abilities. This study aimed to assess the differences between the SDL abilities of medical students using blended and traditional didactic teaching.

METHODS

This study included 239 medical students from eight administrative classes. The students were divided into two groups: (1) the experimental group (EG), which included 119 students from four administrative classes, and (2) the control group (CG), which included 120 students from the remaining four classes. From February to July 2022, blended teaching methods were applied to the EG group, and SDL abilities were assessed in comparison to the CG group receiving traditional didactic teaching methods.

RESULTS

At the end of the semester, significant differences (p < 0.05) were observed between EG and CG in all six SDL ability factors. Furthermore, when k-means cluster analysis was used to analyze the learning behavior of students in the EG after classifying them as comprehensive, interactive, and passive types, significant differences were observed in all six Self-directed learning factors of students with the comprehensive type, whereas significant differences were observed in four factors (setting learning goals and plans, self-monitoring and regulation, information processing, and communication and cooperation) of students with the interactive type. For students with passive type, only one factor of SDL (information processing) showed significant improvement. There were on differences between comprehensive, interactive, and passive types of CG.

CONCLUSION

The blended teaching approach is better than the conventional didactic teaching for cultivating clinical medical students' SDL abilities.

Towards optimal use of antithrombotic therapy of people with cancer at the end of life: A research protocol for the development and implementation of the SERENITY shared decision support tool

BACKGROUND

Even though antithrombotic therapy has probably little or even negative effects on the well-being of people with cancer during their last year of life, deprescribing antithrombotic therapy at the end of life is rare in practice. It is often continued until death, possibly resulting in excess bleeding, an increased disease burden and higher healthcare costs.

METHODS

The SERENITY consortium comprises researchers and clinicians from eight European countries with specialties in different clinical fields, epidemiology and psychology. SERENITY will use a comprehensive approach combining a realist review, flash mob research, epidemiological studies, and qualitative interviews. The results of these studies will be used in a Delphi process to reach a consensus on the optimal design of the shared decision support tool. Next, the shared decision support tool will be tested in a randomised controlled trial. A targeted implementation and dissemination plan will be developed to enable the use of the SERENITY tool across Europe, as well as its incorporation in clinical guidelines and policies. The entire project is funded by Horizon Europe.

RESULTS

SERENITY will develop an information-driven shared decision support tool that will facilitate treatment decisions regarding the appropriate use of antithrombotic therapy in people with cancer at the end of life.

CONCLUSIONS

We aim to develop an intervention that guides the appropriate use of antithrombotic therapy, prevents bleeding complications, and saves healthcare costs. Hopefully, usage of the tool leads to enhanced empowerment and improved quality of life and treatment satisfaction of people with advanced cancer and their care givers.

Delphi Consensus Recommendation for Optimization of Pulmonary Hypertension Therapy Focusing On Switching from a Phosphodiesterase 5 Inhibitor to Riociguat

Dual combination therapy with a phosphodiesterase‐5 inhibitor (PDE5i) and endothelin receptor antagonist is recommended for most patients with intermediate‐risk pulmonary arterial hypertension (PAH). The RESPITE and REPLACE studies suggest that switching from a PDE5i to a soluble guanylate cyclase (sGC) activator may provide clinical improvement in this situation. The optimal approach to escalation or transition of therapy in this or other scenarios is not well defined. We developed an expert consensus statement on the transition to sGC and other treatment escalations and transitions in PAH using a modified Delphi process. The Delphi process used a panel of 20 physicians with expertise in PAH. Panelists answered three questionnaires on the management of treatment escalations and transitions in PAH. The initial questionnaire included open‐ended questions. Later questionnaires consolidated the responses into statements that panelists rated on a Likert scale from −5 (strongly disagree) to +5 (strongly agree) to determine consensus. The Delphi process produced several consensus recommendations. Escalation should be considered for patients who are at high risk or not achieving treatment goals, by adding an agent from a new class, switching from oral to parenteral prostacyclins, or increasing the dose. Switching to a new class or within a class should be considered if tolerability or other considerations unrelated to efficacy are affecting adherence. Switching from a PDE5i to an SGC activator may benefit patients with intermediate risk who are not improving on their present therapy. These consensus‐based recommendations may be helpful to clinicians and beneficial for patients when evidence‐based guidance is unavailable.

Certificate Course in Occupational Safety and Health: A Capacity Building Program for Primary Care Physicians based on Adult Learning Model

Context

The need for physicians qualified in the field of occupational safety and health (OSH) is growing with the growth of manufacturing and service sectors where maximum number of work-related morbidity and mortality occur.

Aims

This article aims to assess the effectiveness and feasibility of the certificate course in occupational safety and health for training and capacity building of primary care physicians in OSH.

Methods and Material

Guideline for reporting evidence-based practice educational interventions and teaching (GREET) was used for describing educational interventions. The outcome was assessed by comparing the mean scores. t test with P < 0.005 was considered a level of significance.

Results

The result showed significant improvement in the cumulative pre- and post-test scores after each module. The certificate course is suitable for adult learning as there was no intergroup difference in knowledge after the program. Consistent with the findings, the improvement scores indicate that training has made a difference in the knowledge and learning of the trained physicians.

Conclusions

This customized training intervention has high potential for scaling up while optimally addressing the scarcity of trained physicians in the OSH in high population density settings like India where a high number of vulnerable workforces work in the informal economy.

Management of repository corticotropin injection therapy for non-infectious uveitis: a Delphi study

PURPOSE

Diagnosis and management of non-infectious uveitis (NIU), a major cause of blindness worldwide, are challenging. Corticosteroids, the cornerstone of therapy, are not appropriate for long-term use, and while non-biologic and biologic immunomodulators may be used for some patients, data on their efficacy and safety in this population are limited. Repository corticotropin injection (RCI), believed to affect uveitis by multiple mechanisms, has received regulatory approval for treatment of ophthalmic diseases including posterior uveitis, but is not widely used or discussed in guidelines for the management of uveitis and ocular inflammatory diseases.

METHODS

The index study employed a modified Delphi process with a panel of 14 US-based ophthalmologists. Consensus recommendations were developed through a series of three questionnaires. Panellists rated statements on a Likert scale from -5 (strongly disagree) to +5 (strongly agree).

RESULTS

The Delphi panel provided consensus recommendations on examinations and testing needed for diagnosis, treatment goals, and the use of corticosteroids, as well as the use of non-biologic and biologic immunomodulators. The panel reached consensus that RCI may be considered for posterior and pan-uveitis, and dosing should be individualized for each patient. Dose reduction/discontinuation should be considered for excessive RCI-related toxicity, hyperglycaemia and/or diabetic complications, excessive costs, or remission ≥ 2 years. Patients should be weaned from RCI if uveitis is stable and well controlled. Adverse events during RCI therapy can be managed by appropriate interventions, with dose reduction/discontinuation considered if events are severe or recurrent.

CONCLUSIONS

Expert consensus suggests RCI may be an appropriate treatment option for some patients with uveitis when other therapies are ineffective or intolerable.

Development and validation of patient-community pharmacist encounter toolkit regarding substance misuse: Delphi procedure

BACKGROUND

Pharmacists' roles and services for patients with substance use are not well defined and inconsistent from site to site. Several barriers have been identified that hinder pharmacists' care for people who use substances, such as a lack of training and resources. Clinical practice tools can aid in transferring evidence-based approaches to the practice sphere.

OBJECTIVES

The aim of the study was to develop a substance misuse management toolkit for community pharmacists to help them manage their encounters with people who use substances.

METHODS

A focused literature review was conducted and 2 needs assessment studies, one for community pharmacists and one for patients informed the development of the toolkit. The toolkit is an adaption of the screening, brief intervention, and referral to treatment (SBIRT) approach, which is one of the most well-defined and effective strategies for substance use management. However, SBIRT is a novel care model in community pharmacy settings. Therefore, a substance misuse management toolkit with 20 items was created for community pharmacists incorporating evidence-based strategies and clinical algorithms. Delphi procedure was used to validate the toolkit.

RESULTS

Two rounds of questions were sent to experts in the field of substance misuse, some of whom were pharmacists. In both rounds, these experts were asked to rate the appropriateness and clarity of items in the toolkit and provide comments and suggestions. Items with a median rating of 7 or more out of 10 were included in the toolkit. In the second round, the experts were asked to rerate the revised version and provide additional feedback. After the second round, agreement was reached for almost all items of the toolkit.

CONCLUSION

A Delphi procedure was successfully used to provide evidence of the validity of the new guiding toolkit for community pharmacists. The toolkit will be implemented and evaluated to provide additional evidence of validity in practice.

An expert panel delphi consensus statement on patient selection and management for transitioning between oral and inhaled treprostinil

Treprostinil, a prostacyclin analogue used in the treatment of pulmonary arterial hypertension (PAH), is available for administration by parenteral, oral, or inhaled routes. Transitioning between routes may be beneficial for appropriate patients; however, there is little published data on transitions between oral and inhaled treprostinil. We used a modified Delphi process to develop expert consensus recommendations on transitions between these formulations. Three questionnaires were used to develop statements about relevant aspects of transition management, which the panelists rated, using a Likert scale, from -5 (strongly disagree) to +5 (strongly agree). Eleven physicians with expertise in PAH treatment modalities, participated in the panel. Of the 492 statements evaluated, consensus was reached on 215 (43.7%). Key consensus recommendations included (1) accurately defining successful transition, as stable or improved PAH with good tolerability and adherence, and (2) patients with stable, low-risk PAH showing insufficient response or tolerability to their existing treprostinil therapy (and due to restrictions in up titration of dosing), as appropriate candidates for transitions between treprostinil formulations. Panelists did not reach consensus for an overall strategy for performing these transitions, mainly because of variability in their practice parameters. Consensus was also achieved on recommendations for adverse event management, including reassurance, administration of oral treprostinil 3 times daily with food, and dosing inhaled treprostinil at intervals ≥3 hours apart. The Delphi process aided in developing expert consensus recommendations that may provide clinically useful guidance for transitioning between treprostinil formulations. However, additional data from centers with high volumes of PAH patients undergoing treprostinil transitions would be optimal for defining more complete and robust strategies to facilitate successful transition.

A Modified Delphi Study to Prioritize Content for a Simulation-based Pediatric Curriculum for Emergency Medicine Residency Training Programs

OBJECTIVES

Pediatric training is an essential component of emergency medicine (EM) residency. The heterogeneity of pediatric experiences poses a significant challenge to training programs. A national simulation curriculum can assist in providing a standardized foundation of pediatric training experience to all EM trainees. Previously, a consensus-derived set of content for a pediatric curriculum for EM was published. This study aimed to prioritize that content to establish a pediatric simulation-based curriculum for all EM residency programs.

METHODS

Seventy-three participants were recruited to participate in a three-round modified Delphi project from 10 stakeholder organizations. In round 1, participants ranked 275 content items from a published set of pediatric curricular items for EM residents into one of four categories: definitely must, probably should, possibly could, or should not be taught using simulation in all residency programs. Additionally, in round 1 participants were asked to contribute additional items. These items were then added to the survey in round 2. In round 2, participants were provided the ratings of the entire panel and asked to rerank the items. Round 3 involved participants dichotomously rating the items.

RESULTS

A total of 73 participants participated and 98% completed all three rounds. Round 1 resulted in 61 items rated as definitely must, 72 as probably should, 56 as possibly could, 17 as should not, and 99 new items were suggested. Round 2 resulted in 52 items rated as definitely must, 91 as probably should, 120 as possibly could, and 42 as should not. Round 3 resulted in 56 items rated as definitely must be taught using simulation in all programs.

CONCLUSIONS

The completed modified Delphi process developed a consensus on 56 pediatric items that definitely must be taught using simulation in all EM residency programs (20 resuscitation, nine nonresuscitation, and 26 skills). These data will serve as a targeted needs assessment to inform the development of a standard pediatric simulation curriculum for all EM residency programs.

Expert consensus on the management of adverse events and prescribing practices associated with the treatment of patients taking pirfenidone for idiopathic pulmonary fibrosis: a Delphi consensus study

Background

In patients with idiopathic pulmonary fibrosis (IPF) treated with pirfenidone (Esbriet®, Genentech USA, Inc. South San Francisco, CA.), effectively managing treatment-related adverse events (AEs) may improve adherence. Due to a lack of clinical evidence and expertise, managing these AEs can be challenging for patients and physicians alike. In the absence of evidence, consensus recommendations from physicians experienced in using pirfenidone to treat IPF are beneficial.

Methods

Using a modified Delphi process, expert recommendations were developed by a panel of physicians experienced with using pirfenidone for IPF. Over three iterations, panelists developed and refined a series of statements on the use of pirfenidone in IPF. Their agreement on each statement was ranked using a Likert scale.

Results

A panel of 12 physicians participated and developed a total of 286 statements on dosing and administration, special populations, drug-drug interactions, laboratory analysis, warnings and precautions, and AE management. Expert recommendations were achieved with regard to slower initial titrations and slower titrations for AEs, dosing with meal(s) or substantial meals, and adding other prescribed pharmacological agents for AEs.

Conclusion

Until there is further clinical evidence, the resulting consensus recommendations are intended to provide direction on the practical management of IPF with pirfenidone, by encompassing a broad experience from the real world to complement data gleaned from clinical trials.

National Competencies for Registered Nurses in Primary Care: A Delphi Study

A Delphi (consensus) process was used to obtain national agreement on competencies for registered nurses (RNs) in primary care. A draft of competencies was developed by key informants. Following this, nurses with primary care experience/expertise completed a Delphi survey to rate the importance of competency statements on a six-point Likert scale. Statements not reaching consensus (agreement ≥80%) were modified and included in a second (final) round. The first survey was completed by 63% (n = 86/137) of participants and 84% (n = 72/86) of these participants completed the second survey. Most statements (n = 45) achieved agreement after the first survey; one statement was dropped and two were combined following the second round. The final list of competencies consists of 47 statements across six domains (professionalism; clinical practice; communication; collaboration and partnership; quality assurance, evaluation, and research; leadership). National competencies will help strengthen the RN workforce within primary care, improve team functioning, and support role integration/optimization.

Management of repository corticotrophin injection therapy for pulmonary sarcoidosis: a Delphi study

In patients treated with repository corticotrophin injection (RCI) for pulmonary sarcoidosis, effective management of adverse events may improve adherence. However, management of adverse events may be challenging due to limitations in real-world clinical experience with RCI and available published guidelines.

We surveyed 12 physicians with a modified Delphi process using three questionnaires. Questionnaire 1 consisted of open-ended questions. Panellists' answers were developed into a series of statements for Questionnaires 2 and 3. In these, physicians rated their agreement with the statements using a Likert scale.

Key consensus recommendations included a starting dose of 40 units twice a week for patients with less severe disease, continued at a maintenance dose for patients who responded, particularly those with chronic refractory sarcoidosis. Panellists reached consensus that concomitant steroids should be quickly tapered in patients receiving RCI, but that concomitant use of immunosuppressive medications should be continued. Panellists developed consensus recommendations for adverse event management, and reached consensus that RCI should be down-titrated or discontinued if other interventions for the adverse effects fail or if the adverse effect is severe.

In the absence of clinical evidence, our Delphi consensus opinions may provide practical guidance to physicians on the management of RCI to treat pulmonary sarcoidosis.

In this paper, a modified Delphi method was used to develop an expert consensus on the use of repository corticotrophin injection therapy for pulmonary sarcoidosis, including dosing, concomitant medications, contraindications and adverse event management.http://bit.ly/2TyauZp

Delphi consensus recommendations for a treatment algorithm in pulmonary sarcoidosis

Pulmonary sarcoidosis presents substantial management challenges, with limited evidence on effective therapies and phenotypes. In the absence of definitive evidence, expert consensus can supply clinically useful guidance in medicine. An international panel of 26 experts participated in a Delphi process to identify consensus on pharmacological management in sarcoidosis with the development of preliminary recommendations.

The modified Delphi process used three rounds. The first round focused on qualitative data collection with open-ended questions to ensure comprehensive inclusion of expert concepts. Rounds 2 and 3 applied quantitative assessments using an 11-point Likert scale to identify consensus.

Key consensus points included glucocorticoids as initial therapy for most patients, with non-biologics (immunomodulators), usually methotrexate, considered in severe or extrapulmonary disease requiring prolonged treatment, or as a steroid-sparing intervention in cases with high risk of steroid toxicity. Biologic therapies might be considered as additive therapy if non-biologics are insufficiently effective or are not tolerated with initial biologic therapy, usually with a tumour necrosis factor-α inhibitor, typically infliximab.

The Delphi methodology provided a platform to gain potentially valuable insight and interim guidance while awaiting evidenced-based contributions.

Expert consensus recommendations for a pulmonary sarcoidosis treatment algorithm from a modified Delphi process include corticosteroids as initial therapy, immunomodulators for steroid-sparing or severe disease, and biologics for very severe diseasehttp://bit.ly/2SmP3uG

Core Disaster Medicine Education (CDME) for Emergency Medicine Residents in the United States

OBJECTIVES

Disaster Medicine (DM) education for Emergency Medicine (EM) residents is highly variable due to time constraints, competing priorities, and program expertise. The investigators' aim was to define and prioritize DM core competencies for EM residency programs through consensus opinion of experts and EM professional organization representatives.

METHODS

Investigators utilized a modified Delphi methodology to generate a recommended, prioritized core curriculum of 40 DM educational topics for EM residencies.

RESULTS

The DM topics recommended and outlined for inclusion in EM residency training included: patient triage in disasters, surge capacity, introduction to disaster nomenclature, blast injuries, hospital disaster mitigation, preparedness, planning and response, hospital response to chemical mass-casualty incident (MCI), decontamination indications and issues, trauma MCI, disaster exercises and training, biological agents, personal protective equipment, and hospital response to radiation MCI.

CONCLUSIONS

This expert-consensus-driven, prioritized ranking of DM topics may serve as the core curriculum for US EM residency programs.

Getting messier with TIDieR: embracing context and complexity in intervention reporting

Background

The Template for Intervention Description and Replication (TIDieR) checklist and guide was developed by an international team of experts to promote full and accurate description of trial interventions. It is now widely used in health research. The aim of this paper is to describe the experience of using TIDieR outside of trials, in a range of applied health research contexts, and make recommendations on its usefulness in such settings.

Main body

We used the TIDieR template for intervention description in six applied health research projects. The six cases comprise a diverse sample in terms of clinical problems, population, settings, stage of intervention development and whether the intervention was led by researchers or the service deliverers. There was also variation in how the TIDieR description was produced in terms of contributors and time point in the project. Researchers involved in the six cases met in two workshops to identify issues and themes arising from their experience of using TIDieR.

We identified four themes which capture the difficulties or complexities of using TIDieR in applied health research: (i) fidelity and adaptation: all aspects of an intervention can change over time; (ii) voice: the importance of clarity on whose voice the TIDieR description represents; (iii) communication beyond the immediate context: the usefulness of TIDieR for wider dissemination and sharing; (iv) the use of TIDieR as a research tool.

Conclusion

We found TIDieR to be a useful tool for applied research outside the context of clinical trials and we suggest four revisions or additions to the original TIDieR which would enable it to better capture these complexities in applied health research:

An additional item, ‘voice’ conveys who was involved in preparing the TIDieR template, such as researchers, service users or service deliverers.

An additional item, ‘stage of implementation’ conveys what stage the intervention has reached, using a continuum of implementation research suggested by the World Health Organisation.

A new column, ‘modification’ reminds authors to describe modifications to any item in the checklist.

An extension of the ‘how well’ item encourages researchers to describe how contextual factors affected intervention delivery.

Electronic supplementary material

The online version of this article (10.1186/s12874-017-0461-y) contains supplementary material, which is available to authorized users.

Reaching consensus on communication of critical laboratory results using a collective intelligence method

Background:

There is no consensus in the literature about what analytes or values should be informed as critical results and how they should be communicated. The main aim of this project is to establish consensual standards of critical results for the laboratories participating in the study. Among the project’s secondary objectives, establishing consensual procedures for communication can be highlighted.

Methods:

Consensus was reached among all participating laboratories establishing the basis for the construction of the initial model put forward for consensus in conjunction with the clinicians. A real-time Delphi, methodology “health consensus” (HC), with motivating and participative questions was applied. The physician was expected to choose a numeric value within a scale designed for each analyte.

Results:

The medians of critical results obtained represent the consensus on critical results for outpatient and inpatient care. Both in primary care and in hospital care a high degree of consensus was observed for critical values proposed in the analysis of creatinine, digoxin, phosphorus, glucose, international normalized ratio (INR), leukocytes, magnesium, neutrophils, chloride, sodium, calcium and lithium. For the rest of critical results the degree of consensus obtained was “medium high”. The results obtained showed that in 72% of cases the consensual critical value coincided with the medians initially proposed by the laboratories.

Conclusions:

The real-time Delphi has allowed obtaining consensual standards for communication of critical results among the laboratories participating in the study, which can serve as a basis for other organizations.

Recommendations for the clinical management of patients receiving macitentan for pulmonary arterial hypertension (PAH): A Delphi consensus document

In patients treated with macitentan (Opsumit®, Actelion Pharmaceuticals Ltd., Basel, Switzerland) for pulmonary arterial hypertension (PAH), prevention and/or effective management of treatment-related adverse events may improve adherence. However, management of these adverse events can be challenging and the base of evidence and clinical experience for macitentan is limited. In the absence of evidence, consensus recommendations from physicians experienced in using macitentan to treat PAH may benefit patients and physicians who are using macitentan. Consensus recommendations were developed by a panel of physicians experienced with macitentan and PAH using a modified Delphi process. Over three iterations, panelists developed and refined a series of statements on the use of macitentan in PAH and rated their agreement with each statement on a Likert scale. The panel of 18 physicians participated and developed a total of 118 statements on special populations, add-on therapy, drug–drug interactions, warnings and precautions, hospitalization and functional class, and adverse event management. The resulting consensus recommendations are intended to provide practical guidance on real-world issues in using macitentan to treat patients with PAH.

A writer’s guide to education scholarship: Quantitative methodologies for medical education research (part 1)

Background

Quantitative research is one of the many research methods used to help educators advance their understanding of questions in medical education. However, little research has been done on how to succeed in publishing in this area.

Objective

We conducted a scoping review to identify key recommendations and reporting guidelines for quantitative educational research and scholarship.

Methods

Medline, ERIC, and Google Scholar were searched for English-language articles published between 2006 and January 2016 using the search terms, “research design,” “quantitative,” “quantitative methods,” and “medical education.” A hand search was completed for additional references during the full-text review. Titles/abstracts were reviewed by two authors (BT, PC) and included if they focused on quantitative research in medical education and outlined reporting guidelines, or provided recommendations on conducting quantitative research. One hundred articles were reviewed in parallel with the first 30 used for calibration and the subsequent 70 to calculate Cohen’s kappa coefficient. Two reviewers (BT, PC) conducted a full text review and extracted recommendations and reporting guidelines. A simple thematic analysis summarized the extracted recommendations.

Results

Sixty-one articles were reviewed in full, and 157 recommendations were extracted. The thematic analysis identified 86 items, 14 categories, and 3 themes. Fourteen quality evaluation tools and reporting guidelines were found.

Discussion

This paper provides guidance for junior researchers in the form of key quality markers and reporting guidelines. We hope that quantitative researchers in medical education will be informed by the results and that further work will be done to refine the list of recommendations.

A Modified Delphi Study for Development of a Pediatric Curriculum for Emergency Medicine Residents

OBJECTIVES

Emergency medicine (EM) trainees are expected to learn to provide acute care for patients of all ages. The American Council for Graduate Medical Education provides some guidance on topics related to caring for pediatric patients; however, education about pediatric topics varies across residency programs. The goal of this project was to develop a consensus curriculum for teaching pediatric emergency care.

METHODS

We recruited 13 physicians from six academic health centers to participate in a three-round electronic modified Delphi project. Participants were selected on the basis of expertise with both EM resident education and pediatric emergency care. The first modified Delphi survey asked participants to generate the core knowledge, skills, and experiences needed to prepare EM residents to effectively treat children in an acute care setting. The qualitative data from the first round was reformulated into a second-round questionnaire. During the second round, participants used rating scales to prioritize the curriculum content proposed during the first round. In round 3, participants were asked to make a determination about each curriculum topic using a three-point scale labeled required, optional, or not needed.

RESULTS

The first modified Delphi round yielded 400 knowledge topics, 206 clinical skills, and 44 specific types of experience residents need to prepare for acute pediatric patient care. These were narrowed to 153 topics, 84 skills, and 28 experiences through elimination of redundancy and two rounds of prioritization. The final lists contain topics classified by highly recommended, partially recommended, and not recommended. The partially recommended category is intended to help programs tailor their curriculum to the unique needs of their learners as well as account for variability between 3- and 4-year programs and the amount of time programs allocate to pediatric education.

CONCLUSION

The modified Delphi process yielded the broad outline of a consensus core pediatric emergency care curriculum.

Recommendations for the use of oral treprostinil in clinical practice: a Delphi consensus project pulmonary circulation

Oral treprostinil was recently labeled for treatment of pulmonary arterial hypertension. Similar to the period immediately after parenteral treprostinil was approved, there is a significant knowledge gap for practicing physicians who might prescribe oral treprostinil. Despite its oral route of delivery, use of the drug is challenging because of the requirement for careful titration and management of drug-related adverse effects. We aimed to create a consensus document combining available evidence with expert opinion to provide guidance for use of oral treprostinil. Following a methodology commonly used in business and social sciences (the 'Delphi Process'), two investigators from the oral treprostinil (Freedom) studies created a series of statements based on available evidence and the package insert. The set of 'best practice' statements was circulated to nine other Freedom trial investigators. Their comments were incorporated into the document as new line items for further vote and comment. The subsequent document was put to vote line by line (scale of -5 to +5) and a final statement was drafted. Consensus recommendations include initial therapy with 0.125 mg for treatment naÿ patients, three times daily dosing, aggressive use of antidiarrheal medication, and a strong preference for use of the drug in combination with other approved PAH therapies. This process was particularly valuable in providing guidance for the management of adverse events (where essentially no data is available). The Delphi process was useful to codify investigator experience and subsequently develop investigator consensus about practical issues for physicians who may wish to prescribe oral treprostinil.

Educating physicians in evidence based medicine: current practices and curricular strategies

INTRODUCTION

Evidence-based medicine (EBM) is an expectation of professional healthcare and a requisite component of medical school curricula. However, upon graduation medical students' EBM skills have been found lacking suggesting a need to examine EBM training.

METHODS

This PhD report presents two studies on EBM education. The first study is a literature review that describes and attempts to assess educational interventions for teaching medical students EBM. The second study presents a multi-institutional case study conducted in North America using interviews and curricular materials to identify EBM instructors' perceptions of challenges that may impede medical students' efforts to learn EBM.

RESULTS

The literature review analyzed 20 learning interventions from 12 countries that were presented in classrooms (75 %) and clinics (25 %). The steps of EBM were addressed to varying degrees. It was not possible to draw conclusions about the efficacy of the interventions due to lack of detailed reporting. The qualitative study identified four learning challenges: sub-optimal role models, student lack of willingness to admit uncertainty, lack of clinical context, and difficulty mastering EBM skills. To meet these challenges, participants described interventions such as integrating EBM skills with other content/courses, incorporating clinical content into EBM teaching, providing faculty development, using whole-task EBM activities, and longitudinal integration of EBM across the curriculum.

CONCLUSION

This PhD report takes steps to describe and assess EBM learning interventions, presents student learning challenges and looks at approaches institutions take to meet them. Educators can use these findings to examine their curriculum and learning environments and, if desired, adopt them for their training.

Development and validation of the guideline for reporting evidence-based practice educational interventions and teaching (GREET)

BACKGROUND

The majority of reporting guidelines assist researchers to report consistent information concerning study design, however, they contain limited information for describing study interventions. Using a three-stage development process, the Guideline for Reporting Evidence-based practice Educational interventions and Teaching (GREET) checklist and accompanying explanatory paper were developed to provide guidance for the reporting of educational interventions for evidence-based practice (EBP). The aim of this study was to complete the final development for the GREET checklist, incorporating psychometric testing to determine inter-rater reliability and criterion validity.

METHODS

The final development for the GREET checklist incorporated the results of a prior systematic review and Delphi survey. Thirty-nine items, including all items from the prior systematic review, were proposed for inclusion in the GREET checklist. These 39 items were considered over a series of consensus discussions to determine the inclusion of items in the GREET checklist. The GREET checklist and explanatory paper were then developed and underwent psychometric testing with tertiary health professional students who evaluated the completeness of the reporting in a published study using the GREET checklist. For each GREET checklist item, consistency (%) of agreement both between participants and the consensus criterion reference measure were calculated. Criterion validity and inter-rater reliability were analysed using intra-class correlation coefficients (ICC).

RESULTS

Three consensus discussions were undertaken, with 14 items identified for inclusion in the GREET checklist. Following further expert review by the Delphi panelists, three items were added and minor wording changes were completed, resulting in 17 checklist items. Psychometric testing for the updated GREET checklist was completed by 31 participants (n = 11 undergraduate, n = 20 postgraduate). The consistency of agreement between the participant ratings for completeness of reporting with the consensus criterion ratings ranged from 19 % for item 4 Steps of EBP, to 94 % for item 16 Planned delivery. The overall consistency of agreement, for criterion validity (ICC 0.73) and inter-rater reliability (ICC 0.96), was good to almost perfect.

CONCLUSION

The final GREET checklist comprises 17 items which are recommended for reporting EBP educational interventions. Further validation of the GREET checklist with experts in EBP research and education is recommended.

The effectiveness and feasibility of an online educational program for improving evidence-based practice literacy: an exploratory randomized study of US chiropractors

Background

Online education programs are becoming a popular means to disseminate knowledge about evidence-based practice (EBP) among healthcare practitioners. This mode of delivery also offers a viable and potentially sustainable solution for teaching consistent EBP content to learners over time and across multiple geographical locations. We conducted a study with 3 main aims: 1) develop an online distance-learning program about the principles of evidence-based practice (EBP) for chiropractic providers; 2) test the effectiveness of the online program on the attitudes, skills, and use of EBP in a sample of chiropractors; and 3) determine the feasibility of expanding the program for broader-scale implementation. This study was conducted from January 2013 to September 2014.

Methods

This was an exploratory randomized trial in which 293 chiropractors were allocated to either an online EBP education intervention or a waitlist control. The online EBP program consisted of 3 courses and 4 booster lessons, and was developed using educational resources created in previous EBP educational programs at 4 chiropractic institutions. Participants were surveyed using a validated EBP instrument (EBASE) with 3 rescaled (0 to 100) subscores: Attitudes, Skills, and Use of EBP. Multiple regression was used to compare groups, adjusting for personal and practice characteristics. Satisfaction and compliance with the program was evaluated to assess feasibility.

Results

The Training Group showed modest improvement compared to the Waitlist Group in attitudes (Δ =6.2, p < .001) and skills (Δ =10.0, p < .001) subscores, but not the use subscore (Δ = −2.3, p = .470). The majority of participants agreed that the educational program was ‘relevant to their profession’ (84 %) and ‘was worthwhile’ (82 %). Overall, engagement in the online program was less than optimal, with 48 % of the Training Group, and 42 % of the Waitlist Group completing all 3 of the program courses.

Conclusions

Online EBP training leads to modest improvements in chiropractors’ EBP attitudes and skill, but not their use of EBP. This online program can be delivered to a wide national audience, but requires modification to enable greater individualization and peer-to-peer interaction. Our results indicate that it is feasible to deliver an online EBP education on a broad scale, but that this mode of education alone is not sufficient for making large changes in chiropractors’ use of EBP.

Guiding CBT supervision: how well do manuals and guidelines fulfil their promise?

Written guidance is part of a coherent support system for CBT supervisors, consistent with general developments in evidence-based practice. In theory, training manuals can help by specifying educational procedures and providing support materials so as to foster fidelity, expertise and effectiveness among supervisors. Supervision guidelines can help by offering supervisors the fruits of research and expert consensus. Together with related resources, such written guidance promises to empower professionals and to benefit patients. But progress with written guidance is very limited within clinical supervision, particularly in controlled research. In marked contrast, locally developed guidance appears to have been implemented successfully within pragmatic studies, a marked achievement when also contrasted with the general healthcare literature. Using eight criteria, the available supervision research is reviewed and conclusions drawn on developing written guidance so as to better fulfil its promise.

How can video recordings best contribute to clinical supervisor training?

Significant progress has been made internationally in the training of clinical supervisors, yet much remains to be done, such as refining training through the improved use of video material. A review of currently available video recordings used in supervisor training indicates that they are generally used informally, are manifestly lacking in procedural detail, are narrowly applied, and lack an educational rationale or research support. After reviewing current options, a more precise, comprehensive and educationally informed framework is proposed, broadening the video options to 10 complementary applications. Systematic links are made between these applications, the core supervision competencies, and the related workshop content (especially training objectives and learning outcomes), culminating in an educational rationale for the use of video within a supervisor training curriculum. It is concluded that application of this framework can enhance the effectiveness and efficiency of supervisor training, helping to maintain progress.

How Comprehensively Is Evidence-Based Practice Represented in Australian Health Professional Accreditation Documents? A Systematic Audit

PHENONENON: In many developed countries, accreditation documents, which reflect the practice standards of health professions, form the basis for evaluation of education programs for meeting the requirements for registration. The 2005 Sicily statement proposed a 5-step model of training in evidence-based practice (ask, access, appraise, apply, and assess). A key recommendation was that evidence-based practice should be incorporated into entry-level health professional training and registration. No previous research has assessed the extent to which this has occurred.

APPROACH

We undertook a systematic audit of the accreditation documents for the registered health professions in Australia. The 11 health professional disciplines included in the audit were medicine, nursing and midwifery, pharmacy, physiotherapy, dentistry, psychology, occupational therapy, optometry, podiatry, osteopathy, and chiropractic. Two investigators independently identified the occurrence of the term evidence that related to "evidence-based practice" and the occurrences of terms related to the 5 steps in the accreditation documents.

FINDINGS

Occurrence of the term evidence as it relates specifically to "evidence-based practice" ranged from 0 (pharmacy, dentistry and occupational therapy) to 8 (physiotherapy) in the accreditation documents. Overall, there were 77 occasions when terms relating to any of the 5 steps of evidence-based practice were used across all 11 accreditation documents. All 5 steps were included in the physiotherapy and psychology documents; 4 steps in medicine and optometry; 3 steps in pharmacy; 2 steps each in documents for chiropractic, osteopathy, and podiatry; and 1 step for nursing. There was no inclusion of terms relating to any of the 5 steps in the dentistry and occupational therapy documents. Insights: Terminology relating explicitly to evidence-based practice and to the 5 steps of evidence-based practice appears to be lacking in the accreditation documents for health professions registered in Australia. This is not necessarily reflective of the curricular content or quality, or dedication to evidence-based practice teaching. However, recognition and demand by accreditation bodies for skills in evidence-based practice may act as a driver for education providers to give greater priority to embedding this training in entry-level programs. Consequently, accreditation bodies are powerfully positioned to shape future directions, focus, and boundaries within and across professions. Future international audits of accreditation documents could provide insight into the global breadth of this phenomenon and contribute to closer scrutiny of the representation of evidence-based practice in future iterations of accreditation documents.

A Systematic Review of Reporting Tools Applicable to Sexual and Reproductive Health Programmes: Step 1 in Developing Programme Reporting Standards

Background

Complete and accurate reporting of programme preparation, implementation and evaluation processes in the field of sexual and reproductive health (SRH) is essential to understand the impact of SRH programmes, as well as to guide their replication and scale-up.

Objectives

To provide an overview of existing reporting tools and identify core items used in programme reporting with a focus on programme preparation, implementation and evaluation processes.

Methods

A systematic review was completed for the period 2000–2014. Reporting guidelines, checklists and tools, irrespective of study design, applicable for reporting on programmes targeting SRH outcomes, were included. Two independent reviewers screened the title and abstract of all records. Full texts were assessed in duplicate, followed by data extraction on the focus, content area, year of publication, validation and description of reporting items. Data was synthesized using an iterative thematic approach, where items related to programme preparation, implementation and evaluation in each tool were extracted and aggregated into a consolidated list.

Results

Out of the 3,656 records screened for title and abstracts, full texts were retrieved for 182 articles, out of which 108 were excluded. Seventy-four full text articles corresponding to 45 reporting tools were retained for synthesis. The majority of tools were developed for reporting on intervention research (n = 15), randomized controlled trials (n = 8) and systematic reviews (n = 7). We identified a total of 50 reporting items, across three main domains and corresponding sub-domains: programme preparation (objective/focus, design, piloting); programme implementation (content, timing/duration/location, providers/staff, participants, delivery, implementation outcomes), and programme evaluation (process evaluation, implementation barriers/facilitators, outcome/impact evaluation).

Conclusions

Over the past decade a wide range of tools have been developed to improve the reporting of health research. Development of Programme Reporting Standards (PRS) for SRH can fill a significant gap in existing reporting tools. This systematic review is the first step in the development of such standards. In the next steps, we will draft a preliminary version of the PRS based on the aggregate list of identified items, and finalize the tool using a consensus process among experts and user-testing.

Aligning Theory and Design: The Development of an Online Learning Intervention to Teach Evidence-based Practice for Maximal Reach

Background:

Online educational interventions to teach evidence-based practice (EBP) are a promising mechanism for overcoming some of the barriers to incorporating research into practice. However, attention must be paid to aligning strategies with adult learning theories to achieve optimal outcomes.

Methods:

We describe the development of a series of short self-study modules, each covering a small set of learning objectives. Our approach, informed by design-based research (DBR), involved 6 phases: analysis, design, design evaluation, redesign, development/implementation, and evaluation. Participants were faculty and students in 3 health programs at a complementary and integrative educational institution.

Results:

We chose a reusable learning object approach that allowed us to apply 4 main learning theories: events of instruction, cognitive load, dual processing, and ARCS (attention, relevance, confidence, satisfaction). A formative design evaluation suggested that the identified theories and instructional approaches were likely to facilitate learning and motivation. Summative evaluation was based on a student survey (N=116) that addressed how these theories supported learning. Results suggest that, overall, the selected theories helped students learn.

Conclusion:

The DBR approach allowed us to evaluate the specific intervention and theories for general applicability. This process also helped us define and document the intervention at a level of detail that covers almost all the proposed Guideline for Reporting Evidence-based practice Educational intervention and Teaching (GREET) items. This thorough description will facilitate the interpretation of future research and implementation of the intervention. Our approach can also serve as a model for others considering online EBP intervention development.