Applying a 'stages of change' model to enhance a traditional evaluation of a research transfer course

A scoping review of theories, models and frameworks used or proposed to evaluate knowledge mobilization strategies

BACKGROUND

Evaluating knowledge mobilization strategies (KMb) presents challenges for organizations seeking to understand their impact to improve KMb effectiveness. Moreover, the large number of theories, models, and frameworks (TMFs) available can be confusing for users. Therefore, the purpose of this scoping review was to identify and describe the characteristics of TMFs that have been used or proposed in the literature to evaluate KMb strategies.

METHODS

A scoping review methodology was used. Articles were identified through searches in electronic databases, previous reviews and reference lists of included articles. Titles, abstracts and full texts were screened in duplicate. Data were charted using a piloted data charting form. Data extracted included study characteristics, KMb characteristics, and TMFs used or proposed for KMb evaluation. An adapted version of Nilsen (Implement Sci 10:53, 2015) taxonomy and the Expert Recommendations for Implementing Change (ERIC) taxonomy (Powell et al. in Implement Sci 10:21, 2015) guided data synthesis.

RESULTS

Of the 4763 search results, 505 were retrieved, and 88 articles were eligible for review. These consisted of 40 theoretical articles (45.5%), 44 empirical studies (50.0%) and four protocols (4.5%). The majority were published after 2010 (n = 70, 79.5%) and were health related (n = 71, 80.7%). Half of the studied KMb strategies were implemented in only four countries: Canada, Australia, the United States and the United Kingdom (n = 42, 47.7%). One-third used existing TMFs (n = 28, 31.8%). According to the adapted Nilsen taxonomy, process models (n = 34, 38.6%) and evaluation frameworks (n = 28, 31.8%) were the two most frequent types of TMFs used or proposed to evaluate KMb. According to the ERIC taxonomy, activities to "train and educate stakeholders" (n = 46, 52.3%) were the most common, followed by activities to "develop stakeholder interrelationships" (n = 23, 26.1%). Analysis of the TMFs identified revealed relevant factors of interest for the evaluation of KMb strategies, classified into four dimensions: context, process, effects and impacts.

CONCLUSIONS

This scoping review provides an overview of the many KMb TMFs used or proposed. The results provide insight into potential dimensions and components to be considered when assessing KMb strategies.

"I Need to Be the First One with a Different Approach and to Make a Difference to the People": A Mixed Methods Pilot Study on Non-Physician Clinicians Training in Malawi

PURPOSE

To improve child health care depends on the availability of sufficient numbers of skilled healthcare workers. To achieve this, the German Society of Tropical Paediatrics & International Child Health supported the existing three-year Bachelor of Science in Paediatrics and Child Health training for Clinical Officers, a non-physician clinician cadre, from 09/2017 to 08/2019. This study aims to evaluate the project to inform forthcoming training.

METHODS

All 17 students who were in training took part in this study. Quantitative data collection took place between 01/2018 to 06/2019 using the post-self-assessment bloc course survey, Research Self-Efficacy Scale (RSES), and Stages of Change (SOC) model. Students and key informants participated in three focus group discussions and five in-depth interviews during April 1-10, 2019.

RESULTS

Students mostly perceived bloc course contents "At their level" (92%) and "Very important/relevant" (61%) with "Good quality" teaching (70.5%). The mean (SD) score for RSES (10-point scale) was 9.10 (0.91). The SOC (4-point scale) scores were higher for "Attitude" and "Intention" statements than "Action". Students found the program well-paced, felt that their clinical knowledge and skills had improved, and valued the acquired holistic disease management approach. They reported increased confidence and being more prepared for leadership roles in their future work. The involvement of international teachers and supervisors enriched their global perspectives.

CONCLUSION

Students improved their clinical and non-clinical skills, developed self-efficacy and attitudes toward research, and were confident to build and utilize their networks. These transformative experiences could facilitate the development of change agents among current and future trainees.

Introducing primary care research teaching in Ukraine: description and evaluation of the 'ABC' research methods course

Ukraine has a developing and expanding system of general practice, but only a rudimentary academic primary care system and no research skills training for general practitioners (GPs). We designed and evaluated a transnational primary care research skills course for Ukrainian GPs.The ABC course is series of three 2-day workshops, designed to teach the basics of primary care research to early-career Ukrainian GPs. It was delivered by Ukrainian and British experts, using innovative, interactive teaching methods. Evaluation measures included participants' assessment of their research abilities, and changes in their attitudes, intentions and actions regarding their research practice.Seventeen Ukrainian GPs took part. There was a 1.32-point increase in research ability self-assessment 5-point Likert scores, with particular increases in literature review and budgeting abilities. Scores for research attitudes, intentions and actions increased by 4.0%, though limited by a ceiling effect. Many participants subsequently developed their own research projects, and some set up primary care research skills courses in their own Ukrainian academic organisations.The course resulted in increased levels of self-confidence and ability to plan primary care research, with improvements in participants' stages of change. It sets out a model for providing and evaluating innovative educational interventions in post-soviet countries, and gives them a basis for high-quality primary care research.

Readiness to change for interprofessional collaboration in healthcare: Development and validation of a theory-based instrument

This paper presents a study that aimed to develop and validate a theory-based instrument for the assessment of readiness to change for interprofessional collaboration in healthcare (IPC-TTM). The instrument was developed in the Persian language and tested in the Iranian context. Healthcare professionals from medical and nursing professions participated in the assessment of validity and reliability of the instrument. We conducted this psychometric study in two phases: First, the questionnaire was developed based on the transtheoretical model (TTM) through literature review and expert panel. Then, in the validation phase, we held three modified Delphi rounds to assess the content and face validity of the questionnaire. We used confirmatory factor analysis (CFA) to evaluate the fit of the questionnaire as applied to modified TTM. Reliability of the final instrument was tested by assessing the test-retest reliability of instrument items with Kappa coefficient. We also calculated the intraclass correlation coefficient (ICC) and Cronbach's alpha to assess the test-retest reliability and internal consistency of the instrument sub-scales. The initial item pool consisted of 30 items and three sub-scales (Attitude, Intention, and Action). The content validity of the questionnaire was confirmed with 17 items. Based on the CFA results two additional items were deleted to increase the fit of the model. The final instrument was confirmed with 15 items and three sub-scales. Reliability assessment on the 15-item instrument showed an acceptable test-retest reliability of the instrument items. ICC values for the Attitude, Intention, and Action sub-scales of the instrument were calculated as 0.82, 0.73, and 0.71, respectively. Moreover, Cronbach's alpha for the Attitude, Intention, and Action sub-scales were 0.85, 0.73, and 0.77, respectively. This study offers a new theory-based instrument to measure readiness to change for interprofessional collaboration in healthcare in the Iranian context. The questionnaire can be used for 'needs assessment' in developing tailored educational interventions and self-assessments in interprofessional education studies.

Effectiveness of an interprofessional education model based on the transtheoretical model of behaviour change to improve interprofessional collaboration

This study aimed to assess the effectiveness of an interprofessional education model (IPE) based on the transtheoretical model to improve the participants' interprofessional collaborative practice. The study was conducted in Iran using a controlled before-and-after study design. The participants (n = 91) were the residents of emergency medicine and nurses of the emergency units from two teaching hospitals affiliated to Iran University of Medical Sciences. The participants in the intervention group (n = 40) were 22 residents and 18 nurses. The control group (n = 51) consisted of 20 residents and 31 nurses. The participants were classified based on their stage of readiness to change. The interventions were two-day workshops for each stage (i.e., attitude and intention). We used the Interprofessional Collaborator Assessment Rubric (ICAR) to assess the effectiveness of the developed model. The interprofessional collaboration of the participants in the intervention and control groups was assessed at four time points before and after the intervention in the real emergency unit environment. Student's t-test and repeated measures analysis of variance (RM-ANOVA) were used to analyse the data. We used partial eta-squared (η²) for effect size calculations. The mean values of ICAR scores in the intervention and control groups were 95.63 ± 19.14 and 89.19 ± 16.11 before the intervention. The mean values of ICAR scores at 3 months after the intervention were 99.82 ± 22.32 and 88.29 ± 16.87 in the intervention and control groups, respectively. After 6 months, the mean values of ICAR scores of the intervention and control groups were 98.6 ± 23.40 and 87.98 ± 16.01, respectively. The results showed that the intervention had a medium educational effect size (partial η² = 0.06) on performance of the participants. Our results showed that an IPE model that is tailored to the learners' stage of readiness to change improves interprofessional collaboration in the participants. The developed model could be applied for improving interprofessional collaborative performance in other IPE programmes.

Theoretical and Practical Considerations for Combating Mental Illness Stigma in Health Care

Reducing the stigma and discrimination associated with mental illness is becoming an increasingly important focus for research, policy, programming and intervention work. While it has been well established that the healthcare system is one of the key environments in which persons with mental illnesses experience stigma and discrimination there is little published literature on how to build and deliver successful anti-stigma programs in healthcare settings, towards healthcare providers in general, or towards specific types of practitioners. Our paper intends to address this gap by providing a set of theoretical considerations for guiding the design and implementation of anti-stigma interventions in healthcare.

Linking medical faculty stress/burnout to willingness to implement medical school curriculum change: a preliminary investigation

RATIONALE, AIMS AND OBJECTIVES

Balancing administrative demands from the medical school while providing patient support and seeking academic advancement can cause personal hardship that ranges from high stress to clinically recognizable conditions such as burnout. Regarding the importance of clinical faculties' burnout and its effects on different aspects of their professional career, this study was conducted and aimed to evaluate the relationship between willingness to change teaching approaches as characterized by a modified stage-of-change model and measures of stress and burnout.

METHODS

This descriptive analytic study was conducted on 143 clinical faculty members of Tehran University of Medical Sciences in Iran. Participants were asked to complete three questionnaires: a modified stages of change questionnaire the Maslach Burnout Inventory and the General Health Questionnaire. Data were analysed by SPSS: 16 using non-parametric statistical tests such as multiple regression and ICC (intra-class coefficient) and Spearman correlation coefficient test.

RESULT

A significant relationship was found between faculty members' readiness to change teaching approaches and the subscales of occupational burnout. Specifically, participants with low occupational burnout were more likely to be in the action stage, while those with high burnout were in the attitude or intention stage, which could be understood as not being ready to implement change. There was no significant correlation between general health scores and stage of change.

CONCLUSIONS

We found it feasible to measure stages of change as well as stress/burnout in academic doctors. Occupational burnout directly reduces the readiness to change. To have successful academic reform in medical schools, it therefore would be beneficial to assess and manage occupational burnout among clinical faculty members.

Proposal of a linear rather than hierarchical evaluation of educational initiatives: the 7Is framework

Extensive resources are expended attempting to change clinical practice; however, determining the effects of these interventions can be challenging. Traditionally, frameworks to examine the impact of educational interventions have been hierarchical in their approach. In this article, existing frameworks to examine medical education initiatives are reviewed and a novel '7Is framework' discussed. This framework contains seven linearly sequenced domains: interaction, interface, instruction, ideation, integration, implementation, and improvement. The 7Is framework enables the conceptualization of the various effects of an intervention, promoting the development of a set of valid and specific outcome measures, ultimately leading to more robust evaluation.

Evaluation of health research capacity strengthening trainings on individual level: validation of a questionnaire

RATIONALE, AIMS AND OBJECTIVES

In the field of global health, research capacity strengthening is becoming a common concept for defining and improving research competencies on individual, organizational, national and supranational level. However, HRCS activities often lack evaluation procedures to measure their impact and to ensure their quality. The aim of this study was to develop and validate a short questionnaire to evaluate trainings in the field of health research capacity strengthening (HRCS).

METHOD

The questionnaire was developed by an interdisciplinary research team and tested in four different training settings at the Mbeya Medical Research Center and Mbeya Referral Hospital, Tanzania. Construct validity of the questionnaire was tested based on 97 responses of the participants of four trainings.

RESULTS

Iterative checking of Cronbach's alpha of the subscales and exploratory factor analysis revealed a four-factor solution that differed from the original structure and subscales of the questionnaire. The instrument was adapted accordingly and consists now of four subscales with 19 items, three global impression items, and open questions for participants' comments and recommendations.

CONCLUSIONS

The result of the study is a short, validated questionnaire for the evaluation of HRCS trainings on the individual level. The tool can be applied both to measure the short-term effects of international health research capacity trainings and to ensure their quality. In the future, after collecting larger sample sizes, a confirmatory factor analysis should be done to further support the four factors.

Educating family physicians to recognize and manage depression: where are we now?

OBJECTIVES

To consider what the barriers are to effective depression education; to understand what attitudes, knowledge, and skills doctors need to acquire, and finally to examine what we currently know about effective ways of training family physicians (FPs) about depression.

METHODS

A narrative review of the published literature compiled from searching reviews and original articles was conducted using the following key words: education, training, attitudes, depression, and primary care. Further relevant articles were identified from reference lists.

RESULTS

The identified barriers are FPs' attitudes and confidence toward recognizing and managing depression, the way in which they conceptualize depression, and the difficulties they face in implementing change in the systems in which they work. We, as educators, can identify what FPs need to know, and this should include novel ways of organizing care. However, of key importance is the need to address how more effective interventions may be provided, recognizing that FPs may be starting from many different points on 3 differing continua of attitude, skills, and knowledge in relation to depression.

CONCLUSIONS

We have to not only ensure that the content of what we teach is perceived as relevant to primary care but also review exactly how we go about providing it, using methods that will engage and stimulate doctors at differing stages of readiness to acquire new attitudes, skills, and knowledge about depression. However, we still need to find better ways of helping FPs to recognize and acknowledge their educational needs. Further research is also required to thoroughly evaluate these novel approaches to tailoring educational interventions.

A tailored educational intervention improves doctor's performance in managing depression: a randomized controlled trial

RATIONAL AND OBJECTIVES

To assess the effects of a tailored and activating educational intervention, based on a three-stage modified Prochaska model of readiness-to-change, on the performance of general physicians in primary care (GPs) regarding management of depressive disorders.

METHODS

Parallel group, randomized control trial. Primary hypothesis was that performance would improve by 20 percentage units in the intervention arm. The setting was primary care in southern Tehran. The participants were 192 GPs stratified on stage of readiness-to-change, sex, age and work experience. The intervention was a 2-day interactive workshop for a small group of GPs' at a higher stage of readiness-to-change ('intention') and a 2-day interactive large group meeting for those with lower propensity to change ('attitude') at the pre-assessment. GPs in the control arm participated in a standard educational programme on the same topic. The main outcome measures were validated tools to assess GPs' performance by unannounced standardized patients, regarding diagnosis and treatment of depressive disorders. The assessments were made 2 months before and 2 months after the intervention.

RESULTS

GPs in the intervention arm significantly improved their overall mean scores for performance regarding both diagnosis, with an intervention effect of 14 percentage units (P = 0.007), and treatment and referral, with an intervention effect of 20 percentage units (P < 0.0001). The largest improvement after the intervention appeared in the small group: 30 percentage units for diagnosis (P = 0.027) and 29 percentage units for treatment and referral (P < 0.0001).

CONCLUSIONS

Activating learning methods, tailored according to the participants' readiness to change, improved clinical performance of GPs in continuing medical education and can be recommended for continuing professional development.

Moving empirically supported practices to addiction treatment programs: recruiting supervisors to help in technology transfer

Federal and state funding agencies are encouraging or mandating the use of empirically supported treatments in addiction programs, yet many programs have not moved in this direction (Forman, Bovasso, and Woody, 2001 ; Roman and Johnson, 2002 ; Willenbring et al., 2004 ). To improve the skills of counselors in community addiction programs, the authors developed an innovative Web-based course on Cognitive Behavioral Therapy (CBT), a widely accepted empirically-supported practice (ESP) for addiction. Federal funding supports this Web course and a randomized controlled trial to evaluate its effectiveness. Since supervisors often play a pivotal role in helping clinicians transfer learned skills from training courses to the workplace, the authors recruited supervisor-counselor teams, engaging 54 supervisors and 120 counselors. Lessons learned focus on supervisor recruitment and involvement, supervisors' perceptions of CBT, their own CBT skills and their roles in the study, and implications for technology transfer for the addiction field as a whole. Recruiting supervisors proved difficult because programs lacked clinical supervisors. Recruiting counselors was also difficult because programs were concerned about loss of third-party reimbursement. Across the addiction field, technology transfer will be severely hampered unless such infrastructure problems can be solved. Areas for further investigation are identified.

Effects on knowledge and attitudes of using stages of change to train general practitioners on management of depression: a randomized controlled study

OBJECTIVE

To assess the impact on knowledge and attitudes of a tailored educational intervention on depression using a modified version of the Prochaska stages of change model, compared with standard continuing medical education, for general practitioners (GPs) in primary care in Iran.

METHOD

Using a randomized controlled trial, a total of 192 GPs were evenly randomized to intervention or control arm. The topic for the educational intervention was depressive disorders. The participants were divided in to small and large groups, depending on their initial stage of change. The GPs' knowledge and skills regarding management of depressive disorders were assessed through a questionnaire with 7 multiple choice questions, 11 Likert statements, 3 case vignettes, and 1 essay question. Attitudes toward management of depressive disorders were also assessed. Both questionnaires were validated.

RESULTS

There was a significant improvement in knowledge mean scores regarding multiple choice and Likert questions (intervention effect 6%; P = 0.002), as well as for the case vignettes and essay question (intervention effect 12%; P = 0.011) in the intervention arm, in comparison with the control arm. There were significant changes in mean attitude scores in both study arms, but no difference between them.

CONCLUSIONS

A theoretical model of medical learning and behavioural change can be used to devise educational formats that suit different stages of learning. Such tailored educational formats can improve GPs' knowledge and skills regarding management of depressive disorders.

Designing a knowledge transfer and exchange strategy for the Alberta Depression Initiative: contributions of qualitative research with key stakeholders

BACKGROUND

Depressive disorders are highly prevalent and of significant societal burden. In fall 2004, the 'Alberta Depression Initiative' (ADI) research program was formed with a mission to enhance the mental health of the Alberta population. A key expectation of the ADI is that research findings will be effectively translated to appropriate research users. To help ensure this, one of the initiatives funded through the ADI focused specifically on knowledge transfer and exchange (KTE). The objectives of this project were first to examine the state of the KTE literature, and then based on this review and a set of key informant interviews, design a KTE strategy for the ADI.

METHODS

Face to face interviews were conducted with 15 key informants familiar with KTE and/or mental health policy and programs in Alberta. Interviews were transcribed and analyzed using the constant comparison method.

RESULTS

This paper reports on findings from the qualitative interviews. Respondents were familiar with the barriers to and facilitators of KTE as identified in the existing literature. Four key themes related to the nature of effective KTE were identified in the data analysis: personal relationships, cultivating champions, supporting communities of practice, and building receptor capacity. These recommendations informed the design of a contextually appropriate KTE strategy for the ADI. The three-phased strategy involves preliminary research, public workshops, on-going networking and linkage activities and rigorous evaluation against pre-defined and mutually agreed outcome measures.

CONCLUSION

Interest in KTE on the part of ADI has led to the development of a strategy for engaging decision makers, researchers, and other mental health stakeholders in an on-going network related to depression programs and policy. A similarly engaged process might benefit other policy areas.

The impact of genetic outreach education and support to primary care on practitioner's confidence and competence in dealing with familial cancers

AIMS

To determine the level of competence and confidence in general practice in relation to the management of familial cancers and to determine the impact of providing genetic educational outreach on confidence and competence.

METHODS

Confidence and competence in dealing with familial cancers was measured using a postal questionnaire sent to all general practitioners and practice nurses in 4 geographical areas in central England. In 2 areas, genetic educational outreach was provided to 10 randomly selected practices and a matched analysis of questionnaire responses before and after intervention was done to determine the impact of the intervention.

RESULTS

Respondents were more confident in dealing with patient queries around familial breast cancer risk than those around bowel cancer. This was inconsistent with the ability to correctly assign familial risk, with 48% incorrectly assigning a high-risk categorisation to a low-risk breast cancer scenario. Respondents who had taken part in the intervention reported more confidence in dealing with issues related to the management of patient queries around bowel cancer. Following intervention, participants were more likely to report feeling confident in knowing the relevant family history to collect (72.4% of respondents from participating practices compared to 56.1% from non-participating practices; OR 2.39, p = 0.02, 95% CI 1.14-5.00) and in making a basic assessment of risk (72.4% compared to 38.9%; OR 3.65, p = 0.01, 95% CI 1.38-9.61).

CONCLUSIONS

Providing genetic educational outreach has a positive impact upon how confident primary care staff feel in dealing with patient queries over familial cancers, particularly in relation to bowel cancer. Further research is needed to explore the impact of providing this service on other relevant outcomes such as appropriateness of referrals to genetic services.

Evaluation of a learner-designed course for teaching health research skills in Ghana

BACKGROUND

In developing countries the ability to conduct locally-relevant health research and high quality education are key tools in the fight against poverty. The objective of our study was to evaluate the effectiveness of a novel UK accredited, learner-designed research skills course delivered in a teaching hospital in Ghana.

METHODS

Study participants were 15 mixed speciality health professionals from Komfo Anokye Teaching Hospital, Kumasi, Ghana. Effectiveness measures included process, content and outcome indicators to evaluate changes in learners' confidence and competence in research, and assessment of the impact of the course on changing research-related thinking and behaviour. Results were verified using two independent methods.

RESULTS

14/15 learners gained research competence assessed against UK Quality Assurance Agency criteria. After the course there was a 36% increase in the groups' positive responses to statements concerning confidence in research-related attitudes, intentions and actions. The greatest improvement (45% increase) was in learners' actions, which focused on strengthening institutional research capacity. 79% of paired before/after responses indicated positive changes in individual learners' research-related attitudes (n = 53), 81% in intention (n = 52) and 85% in action (n = 52). The course had increased learners' confidence to start and manage research, and enhanced life-long skills such as reflective practice and self-confidence. Doing their own research within the work environment, reflecting on personal research experiences and utilising peer support and pooled knowledge were critical elements that promoted learning.

CONCLUSION

Learners in Ghana were able to design and undertake a novel course that developed individual and institutional research capacity and met international standards. Learning by doing and a supportive peer community at work were critical elements in promoting learning in this environment where tutors were scarce. Our study provides a model for delivering and evaluating innovative educational interventions in developing countries to assess whether they meet external quality criteria and achieve their objectives.

Applying a modified Prochaska's model of readiness to change for general practitioners on depressive disorders in CME programmes: validation of tool

OBJECTIVE

To assess the validity and reliability of an 11-item questionnaire for stages of readiness to change according to a modified Prochaska model (including attitude, intention and action stage) in the context of continuing medical education (CME) on depressive disorders for general practitioners (GPs) in Tehran, Iran.

METHODS

Three hundred and fifty GPs were recruited for filling in a questionnaire in order to assess content validity and modifying the questionnaire. Fifty-nine GPs were involved for testing reliability and 39 GPs for testing concurrent validity. Content validity of the questionnaire was assessed by expert consensus. Concurrent validity was assessed by correlating the results of a semi-structured interview with those of the self-assessment questionnaire. For testing reliability there was a test-retest approach with an interval of 3-7 days.

RESULTS

A panel of experts was held at four times and the final version of modified Prochaska questionnaire (MPQ) was compiled by the panel. Total kappa coefficient for concurrent validity of the whole questionnaire was 0.80. Only two of the questions had a kappa coefficient lower than 0.70. In the test-retest, 96% of participants reassigned to the same stage and the total kappa coefficient of reliability was 0.89 for the whole questionnaire.

CONCLUSION

The validity and reliability of the MPQ for assessing GPs' readiness to change in the field of depressive disorders were found to be high in the Iranian context. These findings support its application in tailoring and evaluating CME programmes for GPs in Iran.

A systematic review of evaluation in formal continuing medical education

INTRODUCTION

Physicians spend a considerable amount of time in Continuing Medical Education (CME) to maintain their medical licenses. CME evaluation studies vary greatly in evaluation methods, levels of evaluation, and length of follow-up. Standards for CME evaluation are needed to enable comparison among different studies and to detect factors influencing CME evaluation.

METHODS

A review of the CME evaluation literature was conducted on primary research studies published from January 2000 to January 2006. Studies assessing only satisfaction with CME were excluded, as were studies where fewer than 50% of the participants were practicing physicians. Thirty-two studies were included in the analyses. Determinations were made about evaluation methods, outcome measures, and follow-up assessment.

RESULTS

Only 2 of 32 reviewed studies addressed all evaluation levels: physician changes in knowledge and attitudes (level 2), practices (level 3), and improved patient health status (level 4). None of the studies using self-developed instruments (n = 10) provided reliability and validity information. Only 6 studies used validated scales. Twenty studies had a follow-up period of 6 months or less, and 11 had a follow-up period between 1 and 2 years.

DISCUSSION

A gold standard for evaluating the effectiveness of CME would include assessment of all 4 levels of evaluation. A valid, reliable, and adaptable CME evaluation questionnaire addressing variables in the second level is needed to allow comparison of effectiveness across CME interventions. A minimum 1-year postintervention follow-up period may also be indicated to investigate the sustainability of intervention outcomes.

Defining the effect and mediators of two knowledge translation strategies designed to alter knowledge, intent and clinical utilization of rehabilitation outcome measures: a study protocol [NCT00298727]

BACKGROUND

A substantial number of valid outcome measures have been developed to measure health in adult musculoskeletal and childhood disability. Regrettably, national initiatives have merely resulted in changes in attitude, while utilization remains unacceptably low. This study will compare the effectiveness and mediators of two different knowledge transfer (KT) interventions in terms of their impact on changing knowledge and behavior (utilization and clinical reasoning) related to health outcome measures.

METHOD/DESIGN

Physical and occupational therapists (n = 144) will be recruited in partnership with the national professional associations to evaluate two different KT interventions with the same curriculum: 1) Stakeholder-Hosted Interactive Problem-Based Seminar (SHIPS), and 2) Online Problem-Based course (e-PBL). SHIPS will consist of face-to-face problem-based learning (PBL) for 2 1/2 days with outcome measure developers as facilitators, using six problems generated in consultation with participants. The e-PBL will consist of a 6-week web-based course with six generic problems developed by content experts. SHIPS will be conducted in three urban centers in Canada. Participants will be block-allocated by a minimization procedure to either of the two interventions to minimize any prognostic differences. Trained evaluators at each site will conduct chart audits and chart-stimulated recall. Trained interviewers will conduct semi-structured interviews focused on identifying critical elements in KT and implementing practice changes. Interviews will be transcribed verbatim. Baseline predictors including demographics, knowledge, attitudes/barriers regarding outcome measures, and Readiness to Change will be assessed by self-report. Immediately post-intervention and 6 months later, these will be re-administered. Primary qualitative and quantitative evaluations will be conducted 6-months post-intervention to assess the relative effectiveness of KT interventions and to identify elements that contribute to changing clinical behavior. Chart audits will determine the utilization of outcome measures (counts). Incorporation of outcome measures into clinical reasoning will be assessed using an innovative technique: chart-stimulated recall.

DISCUSSION

A strategy for optimal transfer of health outcome measures into practice will be developed and shared with multiple disciplines involved in primary and specialty management of musculoskeletal and childhood disability.

The Alberta Ambassador Program: delivering Health Technology Assessment results to rural practitioners

BACKGROUND

The purpose of Health Technology Assessment (HTA) is to make the best possible summary of the evidence regarding specific health interventions in order to influence health care and policy decisions. The need for decision makers to find relevant HTA data when it is needed is a barrier to its usefulness. These barriers are highest in rural areas and amongst isolated practitioners.

METHODS

A multidisciplinary team developed an interactive case-based instructional strategy on the topic of chronic non-cancer pain (CNCP) management using clinical evidence derived by HTA. The evidence for each of 18 CNCP interventions was distilled into single-sheet summaries. Clinicians and HTA specialists ('Ambassadors') conducted 11 two-hour interactive sessions on CNCP in eight of Alberta's nine health regions. Pre- and post-session evaluations were conducted.

RESULTS

The sessions were attended by 130 individuals representing 14 health and administrative disciplines. The ambassador model was well received. The use of content experts as ambassadors was highly rated. The educational strategy was judged to be effective. Awareness of the best evidence in CNCP management was increased. Although some participants reported practice changes as a result of the workshops, the program was not designed to measure changes in patient outcome.

CONCLUSION

The ambassador program was successful in increasing awareness of the best evidence in CNCP management, and positively influenced treatment decisions. Its teaching methods were felt to be unique and innovative by participants. Its methods could be applied to other clinical content areas in order to increase the uptake of the results of HTA.