Utility of a brief self-efficacy scale in clinical training program evaluation

Impact of the Canadian Diabetes Association guideline dissemination strategy on clinician knowledge and behaviour change outcomes

AIM

Implementation of clinical practice guideline (CPG) into clinical practice remains limited. Using the Knowledge-To-Action framework, a guideline dissemination and implementation strategy for the Canadian Diabetes Association's 2013 CPG was developed and launched to clinicians and people with diabetes.

METHODS

The RE-AIM framework guided evaluation of this strategy clinician; we report here one aspect of the effectiveness dimension using mixed methods. We measured impact of the strategy on clinican knowledge and behaviour change constructs using evaluation forms, national online survey and individual interviews.

RESULTS

After attending a lecture, clinician confidence (n = 915) increased (3.7(SD 0.7) to 4.5 (SD 0.6) on a 5-point scale (p < 0.001)), with 55% (n = 505) intending to make a practice change (e.g. clinical management regarding glycemic control). Ninety-four percent of survey respondents (n = 907) were aware of the guidelines, attributed to communications from professional associations, continuing professional development events, and colleagues. Forty to 98% of respondents (total n 462-485) were correct in their interpretation of CPG messages, and 33-65%(total n 351-651) reported that they had made changes to their practice. Interviews with 28 clinicians revealed that organizational credibility, online access to tools, clarity of tool content, and education sessions facilitated uptake; lack of time, team-based consensus, and seamless integration into care and patient complexity were barriers.

CONCLUSION

The complexity of diabetes care requires systemic adoption of organization of care interventions, including interprofessional collaboration and consensus. Augmenting our strategy to include scalable models for professional development, integration of guidelines into electronic medical records, and expansion of our target audience to include health care teams and patients, may optimize guideline uptake.

A Methodological Review of Confidence Measurement Scales for Dietitians Working with Individuals

Confidence is frequently used to assess practitioner effectiveness, and its evaluation requires valid and reliable domain-specific tools. However, the quality of existing measures is unknown. This review identifies studies measuring dietitians' confidence in working with individuals; assesses psychometric (measurement) qualities of relevant tools; and identifies areas for future research. Seven electronic databases, the internet, and reference lists were searched to identify the development or use of relevant confidence measurement scales. A quality assessment of psychometric properties was conducted using guidelines developed by Terwee and colleagues. Of the 15 measures reviewed, 4 were subject to factor analysis. Overall, content validity was strong. However, many measures rated poorly due to lack of factor analysis, inadequate sampling, or poor reporting. Of the dietetics-specific instruments, the Dietetic Confidence Scale and Nutrition Counselling Self-Efficacy Scale received the best ratings. The General Self-Efficacy Subscale also rated highly due to validation with the general population. This article highlights the need for dietitians to incorporate evidence-based methods into practice evaluation and instrument development. Dietitians need an awareness of the terminology and key criteria used to evaluate instrument quality to effectively collaborate with statisticians and scale development experts, and critically evaluate the quality of existing measures. Future scale development and reporting must incorporate psychometric evaluation, such as factor analysis, which should be used to explore and/or confirm scale dimensionality. There is broad scope for future methodological research with existing and new measures for nutrition and dietetics practitioners.

Building system capacity for the integration of mental health at the level of primary care in Tunisia: a study protocol in global mental health

BACKGROUND

In low- and middle-income countries (LMICs), addressing the high prevalence of mental disorders is a challenge given the limited number and unequal distribution of specialists, as well as scarce resources allocated to mental health. The Mental Health Gap Action Programme (mhGAP) and its accompanying Intervention Guide (IG), developed by the World Health Organization (WHO), aim to address this challenge by training non-specialists such as general practitioners (GPs) in mental health care. This trial aims to implement and evaluate an adapted version of the mhGAP-IG (version 1.0) offered to GPs in 2 governorates of Tunisia (i.e., Tunis and Sousse), in order to uncover important information regarding implementation process and study design before country-wide implementation and evaluation.

METHODS/DESIGN

First, a systematic review will be conducted to explore types and effectiveness of mental health training programs offered to GPs around the world, with a specific focus on programs implemented and evaluated in LMICs. Second, a cluster randomized controlled trial (RCT) will be conducted to evaluate the effectiveness of the implemented training based on the mhGAP-IG (version 1.0). Third, multiple case study design will be used to explore how contextual factors impact the successful implementation of the training and desired outcomes.

DISCUSSION

In Tunisia, an important need exists to further develop proximity health services and to address the growing mental health treatment gap. One solution is to train GPs in the detection, treatment, and management of mental health problems, given their strategic role in the healthcare system. This trial thus aims to implement and evaluate an adapted version of a training based on the mhGAP-IG (version 1.0) in Tunis and Sousse before country-wide implementation and evaluation. Several contributions are envisioned: adding to the growing evidence on the mhGAP and its accompanying guide, especially in French-speaking nations; building research capacity in Tunisia and more generally in LMICs by employing rigorous designs; evaluating an adapted version of the mhGAP-IG (version 1.0) on a sample of GPs; generating important information regarding implementation process and study design before country-wide implementation; and complimenting the trial results with implementation analysis, a priority in global mental health.

An Integrated Model of Training Evaluation and Effectiveness

A decade of training evaluation and training effectiveness research was reviewed to construct an integrated model of training evaluation and effectiveness. This model integrates four prior evaluation models and results of 10 years of training effectiveness research. It is the first to be constructed using a set of strict criteria and to investigate the evaluation and effectiveness relationships with an evaluation measure proposed several years ago, post training attitudes. Evaluation measures found to be related to posttraining attitudes were cognitive learning, training performance, and transfer performance. Training effectiveness variables found to be related to posttraining attitudes were pretraining self-efficacy, experience, posttraining mastery orientation, learning principles, and post-training interventions. Overall, 10 training effectiveness variables were found to consistently influence training outcomes. Results also reveal that reaction measures and training motivation are two areas needing further development and research. These findings as well as other areas requiring research attention are discussed.

Improving health professionals' self-efficacy to support cardiac patients' emotional recovery: the 'Cardiac Blues Project'

BACKGROUND

Many patients experience the 'cardiac blues' at the time of an acute cardiac event, and one in five go on to develop severe depression. These emotional responses often go undetected and unacknowledged. We initiated the 'Cardiac Blues Project' in order to help support patients' emotional recovery. As part of the project, we developed online training in order to support health professionals in the identification and management of the cardiac blues and depression. The aim of this study was to assess the acceptability of the training and its impacts on health professionals' self-efficacy.

METHOD

In July 2014, a 'cardiac blues' pack of patient resources, including access to health professional online training, was mailed to 606 centres across Australia. In the first 3 months after distribution, 140 health professionals registered to undertake the online training and participated in the present study. Participants provided information via a six-item pre- and post-training self-efficacy scale and on 10 post-training acceptability items.

RESULTS

Health professionals' self-efficacy improved significantly after undertaking the online training across the six domains assessed and for the total score. Acceptability of the training was high across all 10 items assessed. Ratings of usefulness of the training in clinical practice were particularly favourable amongst those who worked directly with cardiac patients.

CONCLUSIONS

The health professional training significantly improves health professionals' confidence in identifying and managing the 'cardiac blues' and depression. Monitoring of uptake is ongoing and future studies will investigate patient outcomes.

Exploring the Connection Between Self-Efficacy and Effective Diabetes Self-f Management

PURPOSE

the purpose of this study was to review the existing empirical evidence about factors that contribute to effective diabetes self-management as indicated by healthy outcomes in persons with the disease, with a specific focus on self-efficacy, to determine the link between learned self-efficacy and effective diabetes self-management in adults.

METHODS

A systematic review was conducted of the extant literature from 1985-2001 that described factors related to effective self-management of diabetes. The review included theoretical and empirical articles. The search engines included CINAHL, MEDLINE, PUBMED, and COCHRANE.

RESULTS

Empirical evidence supports the following factors to improve the education outcomes for adults with diabetes: involve people with diabetes in their own care, guide them in actively learning about the disease, explore their feelings about having the disease, and teach them the skills necessary to adjust their behavior to control their own health outcomes. Thus, the goal for educating people with diabetes is to improve their individual self-efficacy and, accordingly, their self-management ability.

CONCLUSIONS

Education sessions need to involve fewer lectures and more practical, interactive exercises that focus on developing specific skills. Follow-up contact is a valuable method for helping people make a healthy adjustment to living with diabetes.

Effects of a Longitudinal Interprofessional Educational Outreach Program on Collaboration

INTRODUCTION

Interprofessional education (IPE) interventions lack clarity regarding development and implementation, impeding a clear understanding of their role and effectiveness. The aim of this study was to identify whether and how an outreach program targeting interprofessional health care teams can improve self-efficacy and interprofessional collaboration (IPC).

METHODS

A cohort study was conducted to explore the effect of the program on individual self-efficacy and perceived IPC and investigate factors affecting interprofessional learning and collaboration. The program was a two-year IPE program consisting of workshops, educational materials, and interworkshop support. Participants were physicians, nurses, dietitians, pharmacists, and social workers at two primary care teams in Toronto. Self-efficacy and team function were measured five times throughout the program. We used analysis of variance and t-tests to compare between teams and used Pearson correlations to estimate the relationship between self-efficacy and team function. One-on-one interviews investigated factors affecting IPC and the program's effect on IPC.

RESULTS

Team function improved as the program progressed (P = .02); although it did not affect self-efficacy, there was an increasing correlation between self-efficacy and team function as the program progressed (P < .01 for workshop 5). Interviews revealed that trust, liability concerns, and geographic proximity were mediators of IPC. The workshops were perceived to enable trust building by increasing knowledge and allowing nonphysician team members to showcase their expertise.

DISCUSSION

Our findings demonstrate that an IPE workshop, through role clarification, cultivation of trust, and a community of practice, can promote these elements. Trust in team members and geographic proximity are potential facilitators to IPC developed during an interprofessional program.

Adequacy Criteria of Rapid On-Site Evaluation for Endobronchial Ultrasound-Guided Transbronchial Needle Aspiration: A Simple Algorithm to Assess the Adequacy of ROSE

BACKGROUND

Rapid on-site evaluation (ROSE) for endobronchial ultrasound-guided transbronchial needle aspiration (EBUS-TBNA) has been advocated to qualitatively diagnose biopsy samples. However, adequate ROSE criteria during EBUS-TBNA are unclear. The aim of this study was to determine adequacy criteria of ROSE in EBUS-TBNA samples and suggest an appropriate algorithm.

METHODS

Patients who underwent EBUS-TBNA for nodal evaluation between March and July 2013 at Seoul National University Hospital were included prospectively. The ROSE slides were reviewed independently by two pathologists, and the results were compared to the final pathologic results. Diagnostic yields, sensitivity, specificity, and accuracy were calculated in order to make nodal evaluations.

RESULTS

EBUS-TBNA was performed on 300 lymph nodes in 133 patients. Samples were nondiagnostic in 7.7%, 6.3%, and 1.7% of the cytologic, histologic, and overall pathologic results, respectively. On the ROSE slides, a large tissue core size (≥2 cm), microscopic anthracotic pigment (MAP), and increased lymphocyte density (LD; ≥40 cells/field [40×, mean of 10 fields]) were significantly associated with adequate final cytologic or histologic results. Malignant cells were not statistically associated with adequacy but were considered a parameter indicating an adequate diagnosis. Using four sequential criteria, tissue core size, the presence of malignant cell, MAP, and LD ≥40 cells/field, the sensitivity and accuracy rates of ROSE increased from 64.4% to 98.6% and from 64.7% to 97.3%, respectively.

CONCLUSIONS

A high adequacy rate of ROSE in EBUS-TBNA can be achieved by sequentially applying four criteria: tissue core size, malignant cells, MAP, and increased LD.

Simulation Strategies to Teach Patient Transfers: Self-Efficacy by Strategy

OBJECTIVE. We evaluated the effects of transfer training—after training in the classroom and in the high-technology simulation laboratory (WISER Center)—on students’ perceptions of their self-efficacy for knowledge, skill, and safety in executing dependent transfers.

METHOD. After classroom training, occupational therapy students were randomized to three teaching groups on the basis of the amount of participation and observation opportunities provided at the WISER Center—observation dominant, participation dominant, and participation only.

RESULTS. The participation-dominant group reported an increase in knowledge self-efficacy over time compared with the observation-dominant and participation-only groups. Over time, self-efficacy ratings increased for all students, regardless of group.

CONCLUSION. Simulation scenarios implemented at the WISER Center provided a useful adjunct to classroom training in transfer skills. Both participatory and observational experiences contributed to the development of students’ perceptions of their ability to manage acutely ill and medically complex patients.

Confidence in the face of risk: the Risk Assessment and Management Self-Efficacy Study (RAMSES)

Aims and method To evaluate a comprehensive risk management programme. A Risk Assessment and Management Self-Efficacy Scale (RAMSES) was used to evaluate the impact of a clinical guideline and training course. Fifty-three psychological therapists were randomly allocated to training v. waiting list in a controlled, delayed-intervention design. Differences in mean self-efficacy scores between groups were examined using analysis of covariance (ANCOVA).

Results The RAMSES measure had adequate factor structure, internal consistency and construct validity. When adjusting for baseline scores and cluster design, the group exposed to training had a higher mean self-efficacy score than controls. Mean differences between groups were not significant after the control group received training, nor at 6 months’ follow-up.

Clinical implications Exposure to training and clinical guidelines can improve self-efficacy in risk assessment and management. An important advance put forward by this study is the specification of areas of competence in risk assessment and management, which can be measured using a psychometrically sound tool.

Supporting chronic disease self-management: translating policies and principles into clinical practice

To support self-management, health professionals need to adopt a client-centred approach and learn to deliver evidence-based behaviour change interventions. This paper reports on the evaluation of 1- and 2-day training programs developed to improve health professionals' capacity to support chronic disease self-management (CDSM). The 321 participants attended one of eighteen supporting CDSM courses held in urban and rural settings. Participants included nurses, allied health professionals, Aboriginal health workers and general practitioners. Data were collected at three time points: before participation; immediately after the training; and, for a sub-sample of 37 participants, 2 months after the training. Results revealed a significant and sustained increase in CDSM self-efficacy following training regardless of participants' gender, age or qualifications. A thematic analysis of the responses concerning intended practice revealed four main areas of intended practice change, namely: use behavioural strategies; improve communication with clients; adopt a client-centred approach; and improve goal setting. The number of practice changes at 2 months reported by a sub-sample of participants ranged from 1 to 20 with a mean of 14 (s.d.=4). The three most common areas of practice change point to the adoption by health professionals of a collaborative approach with chronic disease patients. Lack of staff trained in CDSM was seen as a major barrier to practice change, with lack of support and finance also named as barriers to practice change. Participants identified that increased training, support and awareness of the principles of supporting CDSM would help to overcome barriers to practice change. These results indicate a readiness among health professionals to adopt a more collaborative approach given the skills and the tools to put this approach into practice.

Targeting educational interventions to clinician's stage of change

This before-after mixed-method study assessed the effect of a diabetes education and self-efficacy training workshop on clinician knowledge, intention and self-efficacy. This workshop demonstrated and narrowed a knowledge gap but did not change intention or self-efficacy. Neither the intervention nor the measured outcomes were targeted to clinicians' stage of change.

Assessment of dietitians' nutrition counselling self-efficacy and its positive relationship with reported skill usage

BACKGROUND

Previous studies on nutrition counselling self-efficacy have assessed small groups of dietitians in focused practice areas or evaluated the effectiveness of skills training on only a few skills. This descriptive study developed a comprehensive scale to examine self-efficacy in a large, cross-sectional sample of practising dietitians when performing various counselling skills that promote dietary behaviour changes.

METHODS

A valid and reliable instrument was developed and administered through the Internet to survey dietitians in the USA from various areas of dietetics and with varying years of experience. Items included counselling self-efficacy, skill usage and counselling-related job characteristics. Of the 612 respondents, one group (n = 486) conducted counselling for more than 50% of their work week, and the other group (n = 126) for less than 50%. Factor analysis was used for scale development. Independent samples t-tests and chi-square tests were performed for group comparisons. Correlations and multiple regression analyses further assessed the relationships among variables.

RESULTS

The resultant unidimensional scale contained 25 items. Dietitians reported high self-efficacy scores and frequent skill usage. Those who counsel for more than 50% of their work week were more likely to work in outpatient settings and private practice, reported higher self-efficacy scores, and held longer and repeated sessions. Self-efficacy scores were positively correlated with counselling-related job characteristics. Years of counselling experience and skill usage significantly predicted self-efficacy scores.

CONCLUSIONS

Dietitians perceive themselves to be highly self-efficacious in using counselling skills which may contribute positively to their professional practice. However, the relationship between counselling self-efficacy and actual performance warrants further investigation.

HIV care providers' implementation of routine alcohol reduction support for their patients

Screening and brief intervention (SBI) for alcohol reduction is an important health promoting strategy for patients with HIV, and HIV care providers are optimally situated to support their patients' reduction efforts. We report results from analyses that use data collected from providers (n = 115) in 7 hospital-based HIV care centers in the New York City metropolitan area in 2007 concerning their routine use of 11 alcohol SBI components with their patients. Providers routinely implemented 5 or more of these alcohol SBI components if they (1) had a specific caseload (and were therefore responsible for a smaller number of patients), (2) had greater exposure to information about alcohol's effect on HIV, (3) had been in their present positions for at least 1 year, and (4) had greater self efficacy to support patients' alcohol reduction efforts. Findings suggest the importance of educating all HIV care providers about both the negative impact of excessive alcohol use on patients with HIV and the importance and value of alcohol SBIs. Findings also suggest the value of promoting increased self efficacy for at least some providers in implementing alcohol SBI components, especially through targeted alcohol SBI training.

The effectiveness of self-assessment on the identification of learner needs, learner activity, and impact on clinical practice: BEME Guide no. 10

BACKGROUND

Health professionals are increasingly expected to identify their own learning needs through a process of ongoing self-assessment. Self-assessment is integral to many appraisal systems and has been espoused as an important aspect of personal professional behaviour by several regulatory bodies and those developing learning outcomes for clinical students. In this review we considered the evidence base on self-assessment since Gordon's comprehensive review in 1991. The overall aim of the present review was to determine whether specific methods of self-assessment lead to change in learning behaviour or clinical practice. Specific objectives sought evidence for effectiveness of self-assessment interventions to: a. improve perception of learning needs; b. promote change in learning activity; c. improve clinical practice; d. improve patient outcomes.

METHODS

The methods for this review were developed and refined in a series of workshops with input from an expert BEME systematic reviewer, and followed BEME guidance. Databases searched included Medline, CINAHL, BNI, Embase, EBM Collection, Psychlit, HMIC, ERIC, BEI, TIMElit and RDRB. Papers addressing self-assessment in all professions in clinical practice were included, covering under- and post-graduate education, with outcomes classified using an extended version of Kirkpatrick's hierarchy. In addition we included outcome measures of accuracy of self-assessment and factors influencing it. 5,798 papers were retrieved, 194 abstracts were identified as potentially relevant and 103 papers coded independently by pairs using an electronic coding sheet adapted from the standard BEME form. This total included 12 papers identified by hand-searches, grey literature, cited references and updating. The identification of a further 12 papers during the writing-up process resulted in a total of 77 papers for final analysis.

RESULTS

Although a large number of papers resulted from our original search only a small proportion of these were of sufficient academic rigour to be included in our review. The majority of these focused on judging the accuracy of self-assessment against some external standard, which raises questions about assumed reliability and validity of this 'gold standard'. No papers were found which satisfied Kirkpatrick's hierarchy above level 2, or which looked at the association between self-assessment and resulting changes in either clinical practice or patient outcomes. Thus our review was largely unable to answer the specific research questions and provide a solid evidence base for effective self-assessment. Despite this, there was some evidence that the accuracy of self-assessment can be enhanced by feedback, particularly video and verbal, and by providing explicit assessment criteria and benchmarking guidance. There was also some evidence that the least competent are also the least able to self-assess accurately. Our review recommends that these areas merit future systematic research to further our understanding of self-assessment.

CONCLUSION

As in other BEME reviews, the methodological issues emerging from this review indicate a need for more rigorous study designs. In addition, it highlights the need to consider the potential for combining qualitative and quantitative data to further our understanding of how self-assessment can improve learning and professional clinical practice.

Determinants of dental referral practices among WIC nutritionists in North Carolina

OBJECTIVES

To examine the effects of knowledge and confidence on dental referral practices among WIC nutritionists in North Carolina.

METHODS

A questionnaire consisting of 118 Likert scale-type questions was administered to 92% of all WIC nutritionists in North Carolina (n = 324). The relationship of knowledge and confidence with frequent dental referrals was tested using logistic regression.

RESULTS

Regression results found that confidence in performing oral health risk assessments (OR = 2.12; 95% CI = 1.13, 3.96), confidence in making dental referrals (OR = 3.02; 95% CI = 1.45-6.29), and confidence in expected outcomes that parents would seek dental care when advised to do so (OR = 3.11; 95% CI = 1.62, 5.97) were associated with more frequent dental referrals.

CONCLUSIONS

The more confident WIC nutritionists feel about oral health, the more likely they are to make dental referrals. Screening and referral by WIC workers may benefit children by improving access to dental care, as the WIC clinic is frequently the first point of contact with a health professional.

Applying the theory of planned behavior to promotion of whole-grain foods by dietitians

The objective of this preliminary study was to apply the theory of planned behavior to explain dietitians' intentions to promote whole-grain foods. Surveys were mailed to a random national sample of registered dietitians to assess knowledge and attitudinal, normative, and control beliefs regarding intention to promote whole-grain foods, with a 39% return rate (n=776, with 628 usable surveys from those working in direct patient care). About half of the respondents had a master's degree, and 58% had substantial experience in the dietetics field. The theory of planned behavior explained intention to promote whole grains to a moderate extent (df=3, F=74.5, R(2)=0.278, P<.001). Most were positive about the health benefits, and few perceived barriers to promotion. However, many had low levels of knowledge and self-efficacy regarding ability to help clients consume more whole-grain foods. Continuing education for dietitians should use strategies that enhance self-efficacy regarding ability to promote whole-grain foods.

Patient-centered diabetes self-management education

Diabetes self-management education (DSME) has been shown to improve health outcomes. Yet, relatively little is known about how DSME has its effects. Literature reviewed from the past 3 years indicates that if DSME is to become more effective interventions need to be theory-based, to increase patient involvement in their care, and to encompass a broader array of evidenced-based outcomes. Outcomes reviewed go beyond knowledge and glycemic control to include prevention of diabetes, quality of life, and reduction of cardiovascular risk. The ability of practitioners and health care systems to implement, adopt, and maintain patient-centered interventions over time is discussed. By linking theory to behavior, and broadening the outcomes examined, advances can continue to be made in closing the gap between the scientific base for the treatment of diabetes, and the care and outcomes patients experience. Further research on patient-centered approaches that promote self-management is seen as critical in closing this gap.