Impact of guideline implementation on patient care: a cluster RCT

Web-based intervention to improve the evidence-practice gap in minimal intervention dentistry: Findings from a dental practice-based research network

OBJECTIVES

To determine whether: the evidence-practice gap (EPG) in minimal intervention dentistry (MID) can be improved by a tailored web-based intervention, and specific clinical situations might impede implementing MID.

METHODS

We conducted a before-after intervention study and a qualitative study. Two web-based questionnaire surveys were conducted among 197 Japanese dentists. In the first questionnaire, a baseline EPG was measured using six questionnaire items. Subsequently, feedback material about the EPG was electronically prepared, including results of the first questionnaire, international comparisons with a previous study from the US, and a summary of recent evidence on MID. In the second questionnaire, the EPG was re-measured after participants read the material. The primary outcome was mean overall concordance between published evidence and the dentist's clinical practice for all six questions. During the second questionnaire, we performed qualitative content analysis using free-text responses to a question about difficult situations encountered when conducting MID.

RESULTS

Regarding before and after comparisons of concordance between the first and second questionnaires, mean overall concordance improved significantly, from 66% to 89% (p<0.001). Qualitative content analysis identified five difficult situations: "cases where decision making for treatment and prognosis is difficult", "inadequate practice resources", "limitations on patient visit and treatment period", "discrepancy between MID and the patient's values", and "limitations on health insurance and social understanding".

CONCLUSIONS

These results suggest that it is possible to reduce the EPG in MID using a web-based educational intervention among Japanese dentists. Qualitative content analysis revealed five difficult situations that might hinder implementation of MID.

CLINICAL SIGNIFICANCE

Although this intervention demonstrated educational effects, perfect concordance was not achieved by all participants. This is possibly associated with the five situations that participants reported facing when conducting MID. Creating an environment to improve these situations may facilitate a reduction in the EPG.

Continuing education meetings and workshops: effects on professional practice and healthcare outcomes

BACKGROUND

Educational meetings are used widely by health personnel to provide continuing medical education and to promote implementation of innovations or translate new knowledge to change practice within healthcare systems. Previous reviews have concluded that educational meetings can result in small changes in behaviour, but that effects vary considerably. Investigations into which characteristics of educational meetings might lead to greater impact have yielded varying results, and factors that might explain heterogeneity in effects remain unclear. This is the second update of this Cochrane Review.

OBJECTIVES

• To assess the effects of educational meetings on professional practice and healthcare outcomes • To investigate factors that might explain the heterogeneity of these effects SEARCH METHODS: We searched CENTRAL, MEDLINE, Embase, ERIC, Science Citation Index Expanded (ISI Web of Knowledge), and Social Sciences Citation Index (last search in November 2016).

SELECTION CRITERIA

We sought randomised trials examining the effects of educational meetings on professional practice and patient outcomes.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data and assessed risk of bias. One review author assessed the certainty of evidence (GRADE) and discussed with a second review author. We included studies in the primary analysis that reported baseline data and that we judged to be at low or unclear risk of bias. For each comparison of dichotomous outcomes, we measured treatment effect as risk difference adjusted for baseline compliance. We expressed adjusted risk difference values as percentages, and we noted that values greater than zero favour educational meetings. For continuous outcomes, we measured treatment effect as per cent change relative to the control group mean post test, adjusted for baseline performance; we expressed values as percentages and noted that values greater than zero favour educational meetings. We report means and 95% confidence intervals (CIs) and, when appropriate, medians and interquartile ranges to facilitate comparisons to previous versions of this review. We analysed professional and patient outcomes separately and analysed 22 variables that were hypothesised a priori to explain heterogeneity. We explored heterogeneity by using univariate meta-regression and by inspecting violin plots.

MAIN RESULTS

We included 215 studies involving more than 28,167 health professionals, including 142 new studies for this update. Educational meetings as the single intervention or the main component of a multi-faceted intervention compared with no intervention • Probably slightly improve compliance with desired practice when compared with no intervention (65 comparisons, 7868 health professionals for dichotomous outcomes (adjusted risk difference 6.79%, 95% CI 6.62% to 6.97%; median 4.00%; interquartile range 0.29% to 13.00%); 28 comparisons, 2577 health professionals for continuous outcomes (adjusted relative percentage change 44.36%, 95% CI 41.98% to 46.75%; median 20.00%; interquartile range 6.00% to 65.00%)) • Probably slightly improve patient outcomes compared with no intervention (15 comparisons, 2530 health professionals for dichotomous outcomes (adjusted risk difference 3.30%, 95% CI 3.10% to 3.51%; median 0.10%; interquartile range 0.00% to 4.00%); 28 comparisons, 2294 health professionals for continuous outcomes (adjusted relative percentage change 8.35%, 95% CI 7.46% to 9.24%; median 2.00%; interquartile range -1.00% to 21.00%)) The certainty of evidence for this comparison is moderate. Educational meetings alone compared with other interventions • May improve compliance with desired practice when compared with other interventions (6 studies, 1402 health professionals for dichotomous outcomes (adjusted risk difference 9.99%, 95% CI 9.47% to 10.52%; median 16.5%; interquartile range 0.80% to 16.50%); 2 studies, 72 health professionals for continuous outcomes (adjusted relative percentage change 12.00%, 95% CI 9.16% to 14.84%; median 12.00%; interquartile range 0.00% to 24.00%)) No studies met the inclusion criteria for patient outcome measurements. The certainty of evidence for this comparison is low. Interactive educational meetings compared with didactic (lecture-based) educational meetings • We are uncertain of effects on compliance with desired practice (3 studies, 370 health professionals for dichotomous outcomes; 1 study, 192 health professionals for continuous outcomes) or on patient outcomes (1 study, 54 health professionals for continuous outcomes), as the certainty of evidence is very low Any other comparison of different formats and durations of educational meetings • We are uncertain of effects on compliance with desired practice (1 study, 19 health professionals for dichotomous outcomes; 1 study, 20 health professionals for continuous outcomes) or on patient outcomes (1 study, 113 health professionals for continuous outcomes), as the certainty of evidence is very low. Factors that might explain heterogeneity of effects Meta-regression suggests that larger estimates of effect are associated with studies judged to be at high risk of bias, with studies that had unit of analysis errors, and with studies in which the unit of analysis was the provider rather than the patient. Improved compliance with desired practice may be associated with: shorter meetings; poor baseline compliance; better attendance; shorter follow-up; professionals provided with additional take-home material; explicit building of educational meetings on theory; targeting of low- versus high-complexity behaviours; targeting of outcomes with high versus low importance; goal of increasing rather than decreasing behaviour; teaching by opinion leaders; and use of didactic versus interactive teaching methods. Pre-specified exploratory analyses of behaviour change techniques suggest that improved compliance with desired practice may be associated with use of a greater number of behaviour change techniques; goal-setting; provision of feedback; provision for social comparison; and provision for social support. Compliance may be decreased by the use of follow-up prompts, skills training, and barrier identification techniques.

AUTHORS' CONCLUSIONS

Compared with no intervention, educational meetings as the main component of an intervention probably slightly improve professional practice and, to a lesser extent, patient outcomes. Educational meetings may improve compliance with desired practice to a greater extent than other kinds of behaviour change interventions, such as text messages, fees, or office systems. Our findings suggest that multi-strategy approaches might positively influence the effects of educational meetings. Additional trials of educational meetings compared with no intervention are unlikely to change the review findings; therefore we will not further update this review comparison in the future. However, we note that randomised trials comparing different types of education are needed.

Printed educational materials: effects on professional practice and healthcare outcomes

BACKGROUND

Printed educational materials are widely used dissemination strategies to improve the quality of healthcare professionals' practice and patient health outcomes. Traditionally they are presented in paper formats such as monographs, publication in peer-reviewed journals and clinical guidelines. This is the fourth update of the review.

OBJECTIVES

To assess the effect of printed educational materials (PEMs) on the practice of healthcare professionals and patient health outcomes. To explore the influence of some of the characteristics of the printed educational materials (e.g. source, content, format) on their effect on healthcare professionals' practice and patient health outcomes.

SEARCH METHODS

We searched MEDLINE, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL), HealthStar, CINAHL, ERIC, CAB Abstracts, Global Health, and EPOC Register from their inception to 6 February 2019. We checked the reference lists of all included studies and relevant systematic reviews.

SELECTION CRITERIA

We included randomised trials (RTs), controlled before-after studies (CBAs) and interrupted time series studies (ITSs) that evaluated the impact of PEMs on healthcare professionals' practice or patient health outcomes. We included three types of comparisons: (1) PEM versus no intervention, (2) PEM versus single intervention, (3) multifaceted intervention where PEM is included versus multifaceted intervention without PEM. Any objective measure of professional practice (e.g. prescriptions for a particular drug), or patient health outcomes (e.g. blood pressure) were included.

DATA COLLECTION AND ANALYSIS

Two reviewers undertook data extraction independently. Disagreements were resolved by discussion. For analyses, we grouped the included studies according to study design, type of outcome and type of comparison. For controlled trials, we reported the median effect size for each outcome within each study, the median effect size across outcomes for each study and the median of these effect sizes across studies. Where data were available, we re-analysed the ITS studies by converting all data to a monthly basis and estimating the effect size from the change in the slope of the regression line between before and after implementation of the PEM. We reported median changes in slope for each outcome, for each study, and then across studies. We standardised all changes in slopes by their standard error, allowing comparisons and combination of different outcomes. We categorised each PEM according to potential effects modifiers related to the source of the PEMs, the channel used for their delivery, their content, and their format. We assessed the risks of bias of all the included studies.

MAIN RESULTS

We included 84 studies: 32 RTs, two CBAs and 50 ITS studies. Of the 32 RTs, 19 were cluster RTs that used various units of randomisation, such as practices, health centres, towns, or areas. The majority of the included studies (82/84) compared the effectiveness of PEMs to no intervention. Based on the RTs that provided moderate-certainty evidence, we found that PEMs distributed to healthcare professionals probably improve their practice, as measured with dichotomous variables, compared to no intervention (median absolute risk difference (ARD): 0.04; interquartile range (IQR): 0.01 to 0.09; 3,963 healthcare professionals randomised within 3073 units). We could not confirm this finding using the evidence gathered from continuous variables (standardised mean difference (SMD): 0.11; IQR: -0.16 to 0.52; 1631 healthcare professionals randomised within 1373 units ), from the ITS studies (standardised median change in slope = 0.69; 35 studies), or from the CBA study because the certainty of this evidence was very low. We also found, based on RTs that provided moderate-certainty evidence, that PEMs distributed to healthcare professionals probably make little or no difference to patient health as measured using dichotomous variables, compared to no intervention (ARD: 0.02; IQR: -0.005 to 0.09; 935,015 patients randomised within 959 units). The evidence gathered from continuous variables (SMD: 0.05; IQR: -0.12 to 0.09; 6,737 patients randomised within 594 units) or from ITS study results (standardised median change in slope = 1.12; 8 studies) do not strengthen these findings because the certainty of this evidence was very low. Two studies (a randomised trial and a CBA) compared a paper-based version to a computerised version of the same PEM. From the RT that provided evidence of low certainty, we found that PEM in computerised versions may make little or no difference to professionals' practice compared to PEM in printed versions (ARD: -0.02; IQR: -0.03 to 0.00; 139 healthcare professionals randomised individually). This finding was not strengthened by the CBA study that provided very low certainty evidence (SMD: 0.44; 32 healthcare professionals). The data gathered did not allow us to conclude which PEM characteristics influenced their effectiveness. The methodological quality of the included studies was variable. Half of the included RTs were at risk of selection bias. Most of the ITS studies were conducted retrospectively, without prespecifying the expected effect of the intervention, or acknowledging the presence of a secular trend.

AUTHORS' CONCLUSIONS

The results of this review suggest that, when used alone and compared to no intervention, PEMs may slightly improve healthcare professionals' practice outcomes and patient health outcomes. The effectiveness of PEMs compared to other interventions, or of PEMs as part of a multifaceted intervention, is uncertain.

Dental Decision Simulation (DDSim): Development of a virtual training environment

PURPOSE

Case-based simulations are powerful training tools that can enhance learning and drive behavior change. This is an overview of the design/development of Dental Decision Simulation (DDSim), a web-based simulation of an electronic dental record (EDR). The purpose was to use DDSim to train dentists to make evidence-based treatment planning decisions consistent with current evidence. This simulated EDR provides case-based information in support of a set of defined evidence-based learning objectives.

METHODS

The development of this complex simulation model required coordinated efforts to create several components: identify behavior changes, case authoring mechanism, create virtual patient visits, require users to make treatment plan decisions related to learning objectives, and a feedback mechanism to help users recognize departures from those learning objectives. This simulation was evaluated in a 2-arm, clinic-randomized, controlled pilot study examining the extent to which DDSim changed dentists' planned treatment to conform to evidence-based treatment guidelines relative to change in dentists not exposed to DDSim. Outcomes were measured by comparing preintervention and postintervention patient EDR treatment data.

RESULTS

Changes in behavior over time did not favor intervention or control clinics.

CONCLUSION

DDSim provides a standardized learning platform that cannot be achieved through the use of live patients. Both live patients and case-based simulations can be used to transfer knowledge and skill development. DDSim offers the advantage of providing a platform for developing treatment planning skills in a low-risk environment. However, further research examining behavior change is needed.

Evaluating the impact of caries prevention and management by caries risk assessment guidelines on clinical practice in a dental teaching hospital

BACKGROUND

Clinical practice guidelines on 'Dental caries prevention and management by caries risk assessment for pre-school children in Hong Kong' were developed using ADAPTE process and Delphi consensus technique. This study aimed to evaluate the feasibility of disseminating and implementing the guidelines, and to evaluate their effectiveness in changing clinical practice.

METHODS

The study was conducted in two phases, examining clinical records of pre-school aged patients being treated by non-academic clinical staff in the Paediatric Dentistry Clinic of a dental teaching hospital in Hong Kong. The clinical guidelines were introduced to the staff in a departmental seminar at the end of pre-intervention phase. Post-intervention phase began one month after the introduction of guidelines. Clinical records for three consecutive months were reviewed against standards and recommendations derived from the newly developed clinical guidelines in both phases. The results were assessed by Chi-square test, ANOVA and regression analyses.

RESULTS

A total of 237 and 147 clinical records were reviewed in pre-intervention and post-intervention phases, respectively. Guideline adherence percentage increased significantly on almost all aspects of the guidelines in the post-intervention phase (P < 0.05). There were a significant difference in the mean overall guideline adherence score (pre-intervention phase: [Formula: see text] = 14.86 ± 6.11; post-intervention phase: [Formula: see text] = 28.88 ± 8.75) and sub-domain adherence scores between the two phases (P < 0.001). The training grade of the clinicians was the factor associated with changes in evidence-based practice (P < 0.001).

CONCLUSIONS

The developed guidelines were effective in translating evidence into best practice. The findings have implication for widespread implementation.

Practice change toward better adherence to evidence-based treatment of early dental decay in the National Dental PBRN

Background

Significant national investments have aided the development of practice-based research networks (PBRNs) in both medicine and dentistry. Little evidence has examined the translational impact of these efforts and whether PBRN involvement corresponds to better adoption of best available evidence. This study addresses that gap in knowledge and examines changes in early dental decay among PBRN participants and non-participants with access to the same evidence-based guideline. This study examines the following questions regarding PBRN participation: are practice patterns of providers with PBRN engagement in greater concordance with current evidence? Does provider participation in a PBRNs increase concordance with current evidence? Do providers who participate in PBRN activities disseminate knowledge to their colleagues?

Methods

Logistic regression models adjusting for clustering at the clinic and provider levels compared restoration (dental fillings) rates from 2005–2011 among 35 providers in a large staff model practice. All new codes for early-stage caries (dental decay) and co-occurring caries were identified. Treatment was determined by codes occurring up to 6 months following the date of diagnosis. Provider PBRN engagement was determined by study involvement and meeting attendance.

Results

In 2005, restoration rates were high (79.5%), decreased to 47.6% by 2011 (p < .01), and differed by level of PBRN engagement. In 2005, engaged providers were less likely to use restorations compared to the unengaged (73.1% versus 88.2%; p < .01). Providers with high PBRN involvement decreased use of restorations by 15.4% from 2005 to 2008 (2005: 73%, 2008: 63%; p < .01). Providers with no PBRN involvement decreased use by only 7.5% (2005: 88%, 2008: 82%; p = .041). During the latter half of 2008 following the May PBRN meeting, attendees reduced restorations by 7.5%, compared to a 2.4% among non-attendees (OR = .64, p < .01).

Conclusions

Based on actual clinical data, PBRN engagement was associated with practice change consistent with current evidence on treatment of early dental decay. The impact of PBRN engagement was most significant for the most-engaged providers and consistent with a spillover effect onto same-clinic providers who were not PBRN-engaged. PBRNs can generate relevant evidence and expedite translation into practice.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-014-0177-x) contains supplementary material, which is available to authorized users.

Neonatal parenteral nutrition: Review of the pharmacist role as a prescriber

Introduction

In the context of the continuous quest to improve the care of the neonates especially the critically ill premature infants, the extended role of pharmacists in the process of parenteral nutrition order writing and effective participation in decision-making especially in the neonatal population is increasingly important.

This review aims to present results from the literature review of available evidence on the pharmacist role in neonatal parenteral nutrition therapy.

Material and methods

Key medical, clinical, and review databases were searched; relevant articles were retrieved and evaluated.

Results and discussion

A total of 19 papers out of 7127 searched papers met the criteria for inclusion, discussing the review topic.

The main focus of the selected papers was on parenteral nutrition practice as related to the pharmacy practice.

The overall quality of studies was mixed.

Conclusion

Overall, the review presents the up-to-date status of the most recent analysis being undertaken on the topic of pharmacist involvement in the parenteral nutrition order writing practices and more specific in the neonatal population over the period from 1979 to 2013.

The overall impression is that the practice of pharmacist writing neonatal parenteral nutrition orders already exists, but still limited if compared with the practice of pharmacist writing adult parenteral nutrition orders which is much more established in many countries.

There was no single clinical study evaluating this practice, as we were able to retrieve only two surveys, which make it difficult to evaluate the pharmacist role in this area.

Nevertheless, despite the wide variation in literature types, characteristics and quality, there are consistent patterns across all the reviewed literatures that competencies of the pharmacist in this field are well represented, which make it very important to carry out good quality clinical studies in this field.

Finally, we are currently conducting a prospective clinical study to evaluate the impact of clinical pharmacist as a neonatal PN prescriber, this impact will be judged through the study outcomes as reducing the metabolic and electrolyte complications and increasing the mean daily weight gain during PN therapy and reducing the average number of days of PN till enteral feeding is achieved.

Effectiveness of a quality management program in dental care practices

Background

Structured quality management is an important aspect for improving patient dental care outcomes, but reliable evidence to validate effects is lacking. We aimed to examine the effectiveness of a quality management program in primary dental care settings in Germany.

Methods

This was an exploratory study with a before-after-design. 45 dental care practices that had completed the European Practice Assessment (EPA) accreditation scheme twice (intervention group) were selected for the study. The mean interval between the before and after assessment was 36 months. The comparison group comprised of 56 dental practices that had undergone their first assessment simultaneously with follow-up assessment in the intervention group. Aggregated scores for five EPA domains: ‘infrastructure’, ‘information’, ‘finance’, ‘quality and safety’ and ‘people’ were calculated.

Results

In the intervention group, small non-significant improvements were found in the EPA domains. At follow-up, the intervention group had higher scores on EPA domains as compared with the comparison group (range of differences was 4.2 to 10.8 across domains). These differences were all significant in regression analyses, which controlled for relevant dental practice characteristics.

Conclusions

Dental care practices that implemented a quality management program had better organizational quality in contrast to a comparison group. This may reflect both improvements in the intervention group and a selection effect of dental practices volunteering for the first round of EPA practice assessment.

Adherence to NICE guidelines on recall intervals and the FGDP(UK) Selection Criteria for Dental Radiography

AIM

The study investigated adherence of GDPs to National Institute for Health and Clinical Excellence (NICE) guidelines on recall intervals and the FGDP (UK)'s Selection Criteria for Dental Radiography. It also explored any factors that might influence GDPs' compliance with the guidelines.

METHOD

A previously piloted questionnaire was circulated to all GDPs within the district of the Cardiff and Vale University Health Board (UHB). The questionnaire sought demographic data as well as answers to questions relating to compliance with guidelines.

RESULTS

Of 215 questionnaires, 133 (61.9%) were returned. One hundred a nd thirty (97.7%) respondents were familiar with NICE recall guidelines and 112 (84.2%) were familiar with the FGDP(UK) publication Selection Criteria for Dental Radiography. Thirty six (27.7%) 'always' followed the NICE recall guidelines and, overall, 108 (81.8%) 'always or mostly' followed the guidance. Fifty one (38.6%) respondents 'always' carried out a caries risk assessment for adult patients and 57 (43.5%) 'always' carried out a caries risk assessment for child patients. Seventy nine (59.8%) reported that they 'always or mostly' recorded the patient's disease risk category in the notes. Fifty two (39.7%) respondents 'always' took bitewing radiographs that corresponded to disease risk. Overall, however, 119 GDPs (90.8%) 'always or mostly' took bitewing radiographs at appropriate intervals according to disease risk. Bitewing radiographs for new adult patients were prescribed more often for new child patients. The dentist's length of experience, NHS commitment, country of graduation, access to digital radiography or panoramic machines, receipt of any postgraduate qualifications or involvement in dental foundation training were proven not to have any statistically significant association with adherence to NICE or FGDP(UK) guidelines.

CONCLUSIONS

Most dentists are familiar with NICE guidelines on recall intervals and the FGDP(UK)'s Selection Criteria for Dental Radiography. The number of dentists who always comply with these sets of guidance is low. None of the variables investigated were shown to have any statistically significant association with adherence to these guidelines.

Reasons for replacement of restorations: dentists' perceptions

OBJECTIVE

This study aimed at evaluating dentists' perceived reasons for replacement of restorations and ascertaining the differences arising from dentists' gender, time since graduation and working sector (salaried vs private).

MATERIALS AND METHODS

A postal questionnaire was sent to a total of 592 working-age general dental practitioners in Finland, 57% (n = 339) responded. The dentists were asked to rank in order of priority the six most common reasons for replacement of composite in the incisors and posterior teeth and amalgam in the posterior tooth from a list of 12 reasons. Ranking order 1 was worth six points and order 6 one point; the non-ranked reasons were equal to zero. Differences in the means of the summed scores of caries-related (RC), fracture- and failure-related (RF) and miscellaneous (RO) groups were evaluated by ANOVA. The level of significance was set at p = 0.05.

RESULTS

For each of three restorations, the RF group comprised 48-56% of the sum-scores. Of the single reasons, secondary caries predominated (20-24%). For composite restorations in the incisors, the mean sum-score of the RO group was greater for private-sector dentists (p = 0.04). For composite restorations in the posterior teeth, the mean sum-score of RF group was higher for male than female dentists (p = 0.009). For amalgam, mean score for RF was 10.2, followed by RC (8.5) and RO (1.1).

CONCLUSION

Secondary caries and various fractures and failures predominate as dentists' perceived reasons for replacement of restorations. Private dentists included miscellaneous reasons as one of their six reasons more often than did the salaried dentists. The complex process of treatment planning and decision-making is influenced by many as of yet unknown factors, calling for emphasis on investigating of perceptions.

Printed educational materials: effects on professional practice and healthcare outcomes

BACKGROUND

Printed educational materials are widely used passive dissemination strategies to improve the quality of clinical practice and patient outcomes. Traditionally they are presented in paper formats such as monographs, publication in peer-reviewed journals and clinical guidelines.

OBJECTIVES

To assess the effect of printed educational materials on the practice of healthcare professionals and patient health outcomes.To explore the influence of some of the characteristics of the printed educational materials (e.g. source, content, format) on their effect on professional practice and patient outcomes.

SEARCH METHODS

For this update, search strategies were rewritten and substantially changed from those published in the original review in order to refocus the search from published material to printed material and to expand terminology describing printed materials. Given the significant changes, all databases were searched from start date to June 2011. We searched: MEDLINE, EMBASE, the Cochrane Central Register of Controlled Trials (CENTRAL), HealthStar, CINAHL, ERIC, CAB Abstracts, Global Health, and the EPOC Register.

SELECTION CRITERIA

We included randomised controlled trials (RCTs), quasi-randomised trials, controlled before and after studies (CBAs) and interrupted time series (ITS) analyses that evaluated the impact of printed educational materials (PEMs) on healthcare professionals' practice or patient outcomes, or both. We included three types of comparisons: (1) PEM versus no intervention, (2) PEM versus single intervention, (3) multifaceted intervention where PEM is included versus multifaceted intervention without PEM. There was no language restriction. Any objective measure of professional practice (e.g. number of tests ordered, prescriptions for a particular drug), or patient health outcomes (e.g. blood pressure) were included.

DATA COLLECTION AND ANALYSIS

Two review authors undertook data extraction independently, and any disagreement was resolved by discussion among the review authors. For analyses, the included studies were grouped according to study design, type of outcome (professional practice or patient outcome, continuous or dichotomous) and type of comparison. For controlled trials, we reported the median effect size for each outcome within each study, the median effect size across outcomes for each study and the median of these effect sizes across studies. Where the data were available, we re-analysed the ITS studies and reported median differences in slope and in level for each outcome, across outcomes for each study, and then across studies. We categorised each PEM according to potential effects modifiers related to the source of the PEMs, the channel used for their delivery, their content, and their format.

MAIN RESULTS

The review includes 45 studies: 14 RCTs and 31 ITS studies. Almost all the included studies (44/45) compared the effectiveness of PEM to no intervention. One single study compared paper-based PEM to the same document delivered on CD-ROM. Based on seven RCTs and 54 outcomes, the median absolute risk difference in categorical practice outcomes was 0.02 when PEMs were compared to no intervention (range from 0 to +0.11). Based on three RCTs and eight outcomes, the median improvement in standardised mean difference for continuous profession practice outcomes was 0.13 when PEMs were compared to no intervention (range from -0.16 to +0.36). Only two RCTs and two ITS studies reported patient outcomes. In addition, we re-analysed 54 outcomes from 25 ITS studies, using time series regression and observed statistically significant improvement in level or in slope in 27 outcomes. From the ITS studies, we calculated improvements in professional practice outcomes across studies after PEM dissemination (standardised median change in level = 1.69). From the data gathered, we could not comment on which PEM characteristic influenced their effectiveness.

AUTHORS' CONCLUSIONS

The results of this review suggest that when used alone and compared to no intervention, PEMs may have a small beneficial effect on professional practice outcomes. There is insufficient information to reliably estimate the effect of PEMs on patient outcomes, and clinical significance of the observed effect sizes is not known. The effectiveness of PEMs compared to other interventions, or of PEMs as part of a multifaceted intervention, is uncertain.

Quality of cluster randomized controlled trials in oral health: a systematic review of reports published between 2005 and 2009

OBJECTIVES

To assess the quality of methods and reporting of recently published cluster randomized trials (CRTs) in oral health.

METHODS

We searched PubMed for CRTs that included at least one oral health-related outcome and were published from 2005 to 2009 inclusive. We developed a list of criteria for assessing trial quality and reporting. This was influenced largely by the extended CONSORT statement for CRTs but also included criteria suggested by other authors. We examined the extent to which trials were consistent with these criteria.

RESULTS

Twenty-three trials were included in the review. In 15 (65%) trials, clustering had been accounted for in sample size calculations, and in 18 (78%) authors had accounted for clustering in analysis. Intraclass correlation coefficients (ICCs) were reported for eight (35%) trials; the outcome assessor was reported as having been blinded to allocation in 12 (52%) trials; 17 (74%) described eligibility criteria at individual level, but only nine (39%) described such criteria at cluster level. Sixteen of 20 trials (80%), in which individuals were recruited, reported that individual informed consent was obtained.

CONCLUSIONS

These results suggest that the quality of recent CRTs in oral health is relatively high and appears to compare favourably with other fields. However, there remains room for improvement. Authors of future trials should endeavour to ensure sample size calculations and analyses properly account for clustering (and are reported as such), consider the potential for recruitment/identification bias at the design stage, describe the steps taken to avoid this in the final report and report observed ICCs and cluster-level eligibility criteria.