Evidence-based management: using serial firm trials to improve diabetes care quality

Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review

OBJECTIVE

There is a widely held assumption that engagement by clinicians and healthcare organisations in research improves healthcare performance at various levels, but little direct empirical evidence has previously been collated. The objective of this study was to address the question: Does research engagement (by clinicians and organisations) improve healthcare performance?

METHODS

An hourglass-shaped review was developed, consisting of three stages: (1) a planning and mapping stage; (2) a focused review concentrating on the core question of whether or not research engagement improves healthcare performance; and (3) a wider (but less systematic) review of papers identified during the two earlier stages, focusing on mechanisms.

RESULTS

Of the 33 papers included in the focused review, 28 identified improvements in health services performance. Seven out of these papers reported some improvement in health outcomes, with others reporting improved processes of care. The wider review demonstrated that mechanisms such as collaborative and action research can encourage some progress along the pathway from research engagement towards improved healthcare performance. Organisations that have deliberately integrated the research function into organisational structures demonstrate how research engagement can, among other factors, contribute to improved healthcare performance.

CONCLUSIONS

Current evidence suggests that there is an association between the engagement of individuals and healthcare organisations in research and improvements in healthcare performance. The mechanisms through which research engagement might improve healthcare performance overlap and rarely act in isolation, and their effectiveness often depends on the context in which they operate.

Engagement in research: an innovative three-stage review of the benefits for health-care performance

Background

There is a widely held assumption that research engagement improves health-care performance at various levels, but little direct empirical evidence.

Objectives

To conduct a theoretically and empirically grounded synthesis to map and explore plausible mechanisms through which research engagement might improve health services performance. A review of the effects on patients of their health-care practitioner's or institution's participation in clinical trials was published after submission of the proposal for this review. It identified only 13 relevant papers and, overall, suggested that the evidence that research engagement improves health-care performance was less strong than some thought. We aimed to meet the need for a wider review.

Methods

An hourglass review was developed, consisting of three stages: (1) a planning and mapping stage; (2) a focused review concentrating on the core question of whether or not research engagement improves health care; and (3) a wider (but less systematic) review of papers identified during the two earlier stages. Studies were included inthe focused review if the concept of ‘engagementinresearch’ was an input and some measure of ‘performance’ an output. The search strategy covered the period 1990 to March 2012. MEDLINE, EMBASE, PsycINFO, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Web of Science and other relevant databases were searched. A total of 10,239 papers were identified through the database searches, and 159 from other sources. A further relevance and quality check on 473 papers was undertaken, and identified 33 papers for inclusion in the review. A standard meta-analysis was not possible on the heterogeneous mix of papers in the focused review. Therefore an explanatory matrix was developed to help characterise the circumstances in which research engagement might improve health-care performance and the mechanisms that might be at work, identifying two main dimensions along which to categorise the studies:the degree of intentionalityandthe scope of the impact.

Results

Of the 33 papers in the focused review, 28 were positive (of which six were positive/mixed) in relation to the question of whether or not research engagement improves health-care performance. Five papers were negative (of which two were negative/mixed). Seven out of 28 positive papers reported some improvement in health outcomes. For the rest, the improved care took the form of improved processes of care. Nine positive papers were at a clinician level and 19 at an institutional level. The wider review demonstrated, for example, how collaborative and action research can encourage some progress along the pathway from research engagement towards improved health-care performance. There is also evidence that organisations in which the research function is fully integrated into the organisational structure out-perform other organisations that pay less formal heed to research and its outputs. The focused and wider reviews identified the diversity in the mechanisms through which research engagement might improve health care: there are many circumstances and mechanisms at work, more than one mechanism is often operative, and the evidence available for each one is limited.

Limitations

To address the complexities of this evidence synthesis of research we needed to spend significant time mapping the literature, and narrowed the research question to make it feasible. We excluded many potentially relevant papers (though we partially addressed this by conducting a wider additional synthesis). Studies assessing the impact made on clinician behaviour by small, locally conducted pieces of research could be difficult to interpret without full knowledge of the context.

Conclusions

Drawing on the focused and wider reviews, it is suggested that when clinicians and health-care organisations engage in research there is the likelihood of a positive impact on health-care performance. Organisations that have deliberately integrated the research function into organisational structures demonstrate how research engagement can, among other factors, contribute to improved health-care performance. Further explorations are required of research networks and schemes to promote the engagement of clinicians and managers in research. Detailed observational research focusing on research engagement within organisations would build up an understanding of mechanisms.

Study registration

PROSPERO: CRD42012001990.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Computer-generated reminders delivered on paper to healthcare professionals; effects on professional practice and health care outcomes

BACKGROUND

Clinical practice does not always reflect best practice and evidence, partly because of unconscious acts of omission, information overload, or inaccessible information. Reminders may help clinicians overcome these problems by prompting the doctor to recall information that they already know or would be expected to know and by providing information or guidance in a more accessible and relevant format, at a particularly appropriate time.

OBJECTIVES

To evaluate the effects of reminders automatically generated through a computerized system and delivered on paper to healthcare professionals on processes of care (related to healthcare professionals' practice) and outcomes of care (related to patients' health condition).

SEARCH METHODS

For this update the EPOC Trials Search Co-ordinator searched the following databases between June 11-19, 2012: The Cochrane Central Register of Controlled Trials (CENTRAL) and Cochrane Library (Economics, Methods, and Health Technology Assessment sections), Issue 6, 2012; MEDLINE, OVID (1946- ), Daily Update, and In-process; EMBASE, Ovid (1947- ); CINAHL, EbscoHost (1980- ); EPOC Specialised Register, Reference Manager, and INSPEC, Engineering Village. The authors reviewed reference lists of related reviews and studies. 

SELECTION CRITERIA

We included individual or cluster-randomized controlled trials (RCTs) and non-randomized controlled trials (NRCTs) that evaluated the impact of computer-generated reminders delivered on paper to healthcare professionals on processes and/or outcomes of care.

DATA COLLECTION AND ANALYSIS

Review authors working in pairs independently screened studies for eligibility and abstracted data. We contacted authors to obtain important missing information for studies that were published within the last 10 years. For each study, we extracted the primary outcome when it was defined or calculated the median effect size across all reported outcomes. We then calculated the median absolute improvement and interquartile range (IQR) in process adherence across included studies using the primary outcome or median outcome as representative outcome.

MAIN RESULTS

In the 32 included studies, computer-generated reminders delivered on paper to healthcare professionals achieved moderate improvement in professional practices, with a median improvement of processes of care of 7.0% (IQR: 3.9% to 16.4%). Implementing reminders alone improved care by 11.2% (IQR 6.5% to 19.6%) compared with usual care, while implementing reminders in addition to another intervention improved care by 4.0% only (IQR 3.0% to 6.0%) compared with the other intervention. The quality of evidence for these comparisons was rated as moderate according to the GRADE approach. Two reminder features were associated with larger effect sizes: providing space on the reminder for provider to enter a response (median 13.7% versus 4.3% for no response, P value = 0.01) and providing an explanation of the content or advice on the reminder (median 12.0% versus 4.2% for no explanation, P value = 0.02). Median improvement in processes of care also differed according to the behaviour the reminder targeted: for instance, reminders to vaccinate improved processes of care by 13.1% (IQR 12.2% to 20.7%) compared with other targeted behaviours. In the only study that had sufficient power to detect a clinically significant effect on outcomes of care, reminders were not associated with significant improvements.

AUTHORS' CONCLUSIONS

There is moderate quality evidence that computer-generated reminders delivered on paper to healthcare professionals achieve moderate improvement in process of care. Two characteristics emerged as significant predictors of improvement: providing space on the reminder for a response from the clinician and providing an explanation of the reminder's content or advice. The heterogeneity of the reminder interventions included in this review also suggests that reminders can improve care in various settings under various conditions.

A group randomized trial of a complexity-based organizational intervention to improve risk factors for diabetes complications in primary care settings: study protocol

Background

Most patients with type 2 diabetes have suboptimal control of their glucose, blood pressure (BP), and lipids – three risk factors for diabetes complications. Although the chronic care model (CCM) provides a roadmap for improving these outcomes, developing theoretically sound implementation strategies that will work across diverse primary care settings has been challenging. One explanation for this difficulty may be that most strategies do not account for the complex adaptive system (CAS) characteristics of the primary care setting. A CAS is comprised of individuals who can learn, interconnect, self-organize, and interact with their environment in a way that demonstrates non-linear dynamic behavior. One implementation strategy that may be used to leverage these properties is practice facilitation (PF). PF creates time for learning and reflection by members of the team in each clinic, improves their communication, and promotes an individualized approach to implement a strategy to improve patient outcomes.

Specific objectives

The specific objectives of this protocol are to: evaluate the effectiveness and sustainability of PF to improve risk factor control in patients with type 2 diabetes across a variety of primary care settings; assess the implementation of the CCM in response to the intervention; examine the relationship between communication within the practice team and the implementation of the CCM; and determine the cost of the intervention both from the perspective of the organization conducting the PF intervention and from the perspective of the primary care practice.

Intervention

The study will be a group randomized trial conducted in 40 primary care clinics. Data will be collected on all clinics, with 60 patients in each clinic, using a multi-method assessment process at baseline, 12, and 24 months. The intervention, PF, will consist of a series of practice improvement team meetings led by trained facilitators over 12 months. Primary hypotheses will be tested with 12-month outcome data. Sustainability of the intervention will be tested using 24 month data. Insights gained will be included in a delayed intervention conducted in control practices and evaluated in a pre-post design.

Primary and secondary outcomes

To test hypotheses, the unit of randomization will be the clinic. The unit of analysis will be the repeated measure of each risk factor for each patient, nested within the clinic. The repeated measure of glycosylated hemoglobin A1c will be the primary outcome, with BP and Low Density Lipoprotein (LDL) cholesterol as secondary outcomes. To study change in risk factor level, a hierarchical or random effect model will be used to account for the nesting of repeated measurement of risk factor within patients and patients within clinics.

This protocol follows the CONSORT guidelines and is registered per ICMJE guidelines:

Clinical Trial Registration Number

NCT00482768

Measuring prescribing improvements in pragmatic trials of educational tools for general practitioners

Randomized pragmatic trials of drugs, physician education and drug policies are needed to improve pharmacosurveillance and cost-effectiveness of prescribing. Since 1994, we have developed and tested methods for low-cost education and policy trials to improve prescribing in primary care in Canada. We review methodology for using drug claims and other health services data to evaluate prescribing improvement programs and policies. We apply the lessons to a proposed trial of physician education tools (PET) for quality improvement of prescribing. Design issues for the trial include defining the potential programme in causal terms using counterfactuals, narrowing the denominator to the population affected, excluding noise from the numerator, calculating the prescribing preference, adjusting for baseline differences, controlling for modifiers and confounders, accounting for uncertainty when measuring impacts, and grouping practices for feedback and recognition. Data from a randomized trial of academic detailing illustrate measurement challenges. A decade of progress on methods for evaluating prescribing improvement programs with drug claims data has enabled planning of routine randomized pragmatic trials of education and policies in primary care in Canada.

Economic effectiveness of disease management programs: a meta-analysis

The economic effectiveness of disease management programs, which are designed to improve the clinical and economic outcomes for chronically ill individuals, has been evaluated extensively. A literature search was performed with MEDLINE and other published sources for the period covering January 1995 to September 2003. The search was limited to empirical articles that measured the direct economic outcomes for asthma, diabetes, and heart disease management programs. Of the 360 articles and presentations evaluated, only 67 met the selection criteria for meta-analysis, which included 32,041 subjects. Although some studies contained multiple measurements of direct economic outcomes, only one average effect size per study was included in the meta-analysis. Based on the studies included in the research, a meta-analysis provided a statistically significant answer to the question of whether disease management programs are economically effective. The magnitude of the observed average effect size for equally weighted studies was 0.311 (95% CI = 0.272-0.350). Statistically significant differences of effect sizes by study design, disease type and intensity of disease management program interventions were not found after a moderating variable, disease severity, was taken into consideration. The results suggest that disease management programs are more effective economically with severely ill enrollees and that chronic disease program interventions are most effective when coordinated with the overall level of disease severity. The findings can be generalized, which may assist health care policy makers and practitioners in addressing the issue of providing economically effective care for the growing number of individuals with chronic illness.

Effectiveness Of A Disease Management Program For Patients With Diabetes

Diabetes disease management programs (DDMPs) are proliferating, but their effectiveness in improving quality and mitigating health care spending has been difficult to measure. Using two quasi-experimental methods, this study analyzed the first-year results of a multistate DDMP for people with diabetes sponsored by a national managed care organization. In both analyses, overall cost of care were significantly lower in DDMP sites, and the payer saved more than it spent. Pharmacy costs showed mixed results. Quality scores in the DDMP sites were significantly better than in sites without the program.

The effect of point-of-care personal digital assistant use on resident documentation discrepancies

BACKGROUND

We recently found documentation discrepancies in 60% of resident daily-progress notes with respect to patient weight, medications, or vascular lines. To what extent information systems can decrease such discrepancies is unknown.

OBJECTIVE

To determine whether a point-of-care personal digital assistant (PDA)-based patient record and charting system could reduce the number of resident progress-note documentation discrepancies in a neonatal intensive care unit (NICU).

DESIGN/METHODS

We conducted a before-and-after trial in an academic NICU. Our intervention was a PDA-based patient record and charting system used by all NICU resident physicians over the study period. We analyzed all resident daily-progress notes from 40 randomly selected days over 4 months in both the baseline and intervention periods. Using predefined reference standards, we determined the accuracy of recorded information for patient weights, medications, and vascular lines. Logistic and Poisson regression were used in analyses to control for potential confounding factors.

RESULTS

A total of 339 progress notes in the baseline period and 432 progress notes in the intervention period were reviewed. When controlling for covariates in the regression, there were significantly fewer documentation discrepancies of patient weights in notes written by using the PDA system (14.4%-4.4% of notes; odds ratio [OR]: 0.29; 95% confidence interval [CI]: 0.15-0.56). When using the PDA system, there were no significant changes in the numbers of notes with documentation discrepancies of medications (27.7%-17.1% of notes; OR: 0.63; 95% CI: 0.35-1.13) or vascular lines (33.6%-36.1% of notes; OR: 1.11; 95% CI: 0.66-1.87).

CONCLUSIONS

The use of our PDA-based point-of-care patient record and charting system showed a modest benefit in reducing the number of documentation discrepancies in resident daily-progress notes. Further study of PDAs in information systems is warranted before they are widely adopted.