Assessing the quality of clinical teaching: a preliminary study

Constructing an evaluation index system for clinical nursing practice teaching quality using a Delphi method and analytic hierarchy process-based approach

BACKGROUND

The key step in evaluating the quality of clinical nursing practice education lies in establishing a scientific, objective, and feasible index system. Current assessments of clinical teaching typically measure hospital learning environments, classroom teaching, teaching competency, or the internship quality of nursing students. As a result, clinical evaluations are often insufficient to provide focused feedback, guide faculty development, or identify specific areas for clinical teachers to implement change and improvement. Therefore, the purpose of our study was to to construct a scientific, systematic, and clinically applicable evaluation index system of clinical nursing practice teaching quality and determine each indicator's weight to provide references for the scientific and objective evaluation of clinical nursing practice teaching quality.

METHODS

Based on the "Structure-Process-Outcome" theoretical model, a literature review and Delphi surveys were conducted to establish the evaluation index system of clinical nursing practice teaching quality. Analytic Hierarchy Process (AHP) was employed to determine the weight of each indicator.

RESULTS

The effective response rate for the two rounds of expert surveys was 100%. The expert authority coefficients were 0.961 and 0.975, respectively. The coefficient of variation for the indicators at each level ranged from 0 to 0.25 and 0 to 0.21, and the Kendall harmony coefficients were 0.209 and 0.135, respectively, with statistically significant differences (P < 0.001). The final established index system included 3 first-level, 10 second-level, and 29 third-level indicators. According to the weights computed by the AHP, first-level indicators were ranked as "Process quality" (39.81%), "Structure quality" (36.67%), and "Outcome quality" (23.52%). Among the secondary indicators, experts paid the most attention to "Teaching staff" (23.68%), "Implementation of teaching rules and regulations (14.14%), and "Teaching plans" (13.20%). The top three third-level indicators were "Level of teaching staff" (12.62%), "Structure of teaching staff" (11.06%), and "Implementation of the management system for teaching objects" (7.54%).

CONCLUSION

The constructed evaluation index system of clinical nursing practice teaching quality is scientific and reliable, with reasonable weight. The managers' focus has shifted from outcome-oriented to process-oriented approaches, and more focus on teaching team construction, teaching regulations implementation, and teaching design is needed to improve clinical teaching quality.

Clinical educator self-efficacy, self-evaluation and its relationship with student evaluations of clinical teaching

BACKGROUND

In a whole-of-system approach to evaluation of teaching across any degree, multiple sources of information can help develop an educators' understanding of their teaching quality. In the health professions, student evaluations of clinical teaching are commonplace. However, self-evaluation of teaching is less common, and exploration of clinical educators' self-efficacy even less so. The aim of the study was to evaluate how a clinical educator's self-evaluation of teaching intersects with their self-efficacy, to ascertain if that matches student evaluation of their teaching. This information may assist in facilitating targeted professional development to improve teaching quality.

METHODS

Clinical educators in the osteopathy program at Victoria University (VU) were invited to complete: a) self-evaluation version of the Osteopathy Clinical Teaching Questionnaire (OCTQ); and b) the Self-Efficacy in Clinical Teaching (SECT) questionnaire. Students in the VU program completed the OCTQ for each of the clinical educators they worked with during semester 2, 2017.

RESULTS

Completed OCTQ and SECT were received from 37 clinical educators. These were matched with 308 student evaluations (mean of 6 student ratings per educator). Three possible educator cohorts were identified: a) high clinical eductor self-OCTQ with low student evaluation; b) low clinical educator self-evaluation and high student evaluations; and, c) no difference between self- and student evaulations. Clinical educators in the first cohort demonstrated significantly higher SECT subscale scores (effect size > 0.42) than their colleagues. Age, gender, teaching qualification, and years practicing or years as a clinical educator were not associated with clinical educator OCTQ scores or the SECT subscales.

CONCLUSIONS

Targeted professional development directed towards fostering self-efficacy may provide an avenue for engaging those clinical educators whose self-efficacy is low and/or those who did not receive high student evaluations. Given there is no gold standard measure of clinical teaching quality, educators should engage with multiple sources of feedback to benchmark their current performance level, and identify opportunities to improve. Student and self-evaluations using the OCTQ and evaluation of self-efficacy using the SECT, are useful tools for inclusion in a whole-of-system approach to evaluation of the clinical learning environment.

Measuring and assessing the competencies of preceptors in health professions: a systematic scoping review

BACKGROUND

In healthcare, preceptors act as a role model and supervisor, thereby facilitating the socialisation and development of the preceptee into a professional fit to practice. To ensure a consistent approach to every preceptorship experience, preceptor competencies should be measured or assessed to ensure that the desired outcomes are achieved. Defining these would ensure quality management and could inform development of an preceptor competency framework. This review aimed to evaluate the evidence for preceptor competencies and assessment in health professions.

METHODS

This study followed the PRISMA ScR scoping review guidelines. A database search was conducted in Embase, Medline, CINAHL and IPA in 2019. Articles were included if they defined criteria for competency, measured or assessed competency, or described performance indicators of preceptors. A modified GRADE CERQual approach and CASP quality assessment were used to appraise identified competencies, performance indicators and confidence in evidence.

RESULTS

Forty one studies identified 17 evidence-based competencies, of which 11 had an associated performance indicator. The competency of preceptors was most commonly measured using a preceptee completed survey (moderate to high confidence as per CERQual), followed by preceptor self-assessment, and peer-assessment. Preceptee outcomes as a measure of preceptor performance had good but limited evidence.

CONCLUSIONS

Competencies with defined performance indicators allow for effective measurement and may be modifiable with training. To measure preceptor competency, the preceptor perspective, as well as peer and preceptee assessment is recommended. These findings can provide the basis for a common preceptor competency framework in health professions.

Students' Perspectives on Their Experience in Clinical Placements: Using a Modified Delphi Methodology to Engage Physiotherapy Stakeholders in Revising the National Form

Purpose: We developed an evidence-informed Student Evaluation of the Clinical Placement form. This form gives students the opportunity to share their feedback and perceptions of their clinical placement experiences and provides meaningful data to all stakeholders. Method: We used a modified Delphi process to engage a sample of national stakeholders: physiotherapy clinical education leads of academic departments, centre coordinators of clinical education, clinical instructors, and students. An expert consultant panel, in addition to the investigators, reviewed the responses from each round and helped develop the questionnaire for the subsequent round and finalize the evaluation form. Results: The response rate was 65.3% (47 of 72) for Round 1, 76.6% (36 of 47) for Round 2, and 100% (36 of 36) for Round 3. After three rounds of questionnaires, 89% of participants thought that the evaluation form met their needs. Conclusions: We developed a revised Student Evaluation of the Clinical Placement form that is informed by the literature and meaningful to all stakeholders. This form is being implemented in physiotherapy university programmes across Canada to enable students to share their experiences at clinical sites.

Twelve tips for providing feedback to peers about their teaching

When healthcare professionals provide feedback to peers after a teaching observation, there are benefits for both parties. In this article, we outline strategies to use before, during, and after teaching observations to engage in mutually-beneficial conversations that highlight best practices, identify solutions for teaching dilemmas, and initiate teaching relationships. We discuss the importance of choosing words wisely; giving feedback about teaching skills, not the teacher as a person; recognizing how colleagues view their teaching identities; and ensuring peers are emotionally ready for a post-observation conversation. We also explain how to use pronouns, questions, and active listening during feedback conversations. Finally, we explore the impact of biases on observations, how to establish peer observer credibility, and how to make the teaching observation process and feedback discussion valuable experiences for both parties so that it leads to long-lasting partnerships in the quest to improve educational quality.

Dental students' perceptions on the contribution and impact role of a clinical teacher

OBJECTIVES

This study was conducted in University of Malaya to evaluate student perceptions on the contribution and role of an effective clinical teacher based on the cognitive apprenticeship model in clinical practice.

METHODS

Self-administered questionnaires were distributed to 233 undergraduate dental students involved with clinical teaching. This modified and validated questionnaire focusing on students' learning environment was used in order to gain relevant information related to dental clinical teaching. Six domains with different criteria applicable to clinical teaching in dentistry were selected consisting of modelling (four criteria), coaching (four criteria), scaffolding (four criteria), articulation (four criteria), reflection (two criteria) and general learning environment (six criteria). Data analyses were performed using IBM SPSS Statistics 20.

RESULTS

Majority of the students expressed positive perceptions on their clinical learning experience towards the clinical teachers in the Faculty of Dentistry, University of Malaya, in all criteria of the domains. Few negative feedbacks concerning the general learning environment were reported.

CONCLUSION

Further improvement in the delivery of clinical teaching preferably by using wide variety of teaching-learning activities can be taken into account through students' feedback on their learning experience.

Perceived effectiveness of one-minute preceptor in microskills by residents in dental residency training at National Dental Centre Singapore

Background: The National Dental Centre Singapore (NDCS) has collaborated with the National University of Singapore’s Division of Graduate Dental Studies to provide supervised clinical training in the five dental specialties for residents since the early 1990s. Faculty were first introduced to the one-minute preceptor (OMP) in microskills through in-house workshops held from May to August 2014. Purpose: The aim of this study was to assess if a OMP faculty workshop impacts residents’ perceptions of clinical teaching in NDCS. Methods: First- and second-year residents were asked to rate their perceptions of the quantity and quality of clinical teaching, and the adequacy of the five OMP microskills performed by faculty before and one month after the OMP workshops. Data were analysed using descriptive statistics and non-parametric statistical tests. Results: A total of 37 NDCS clinical faculty participated in the OMP workshops and 34 residents participated in the pre- and post-workshop survey. The short-term impact of the OMP faculty workshop revealed minimal effect on residents’ perceptions of the clinical teaching in both quantity and quality. Conclusion: A one-time OMP workshop for faculty members does not substantially improve residents’ perceptions of the quality or quantity of clinical teaching. The results of this study indicate that the effort to improve clinical teaching by faculty must be a sustained effort. Future studies should investigate the barriers to faculty members’ performance of the OMP microskills.

A Pilot Study of the Creation and Implementation of a Teaching Development Assessment Tool

BACKGROUND

The importance of effective clinical teaching skills is well established in the literature. However, reliable tools with validity evidence that are able to measure the development of these skills and can effectively be used by nonphysician raters do not exist.

OBJECTIVE

Our initiative had 2 aims: (1) to develop a teaching development assessment tool (TDAT) that allows skill assessment along a continuum, and (2) to determine if trained nonphysicians can assess clinical teachers with this tool.

METHODS

We describe the development of the TDAT, including identification of 6 global teaching domains and observable teaching behaviors along a 3-level continuum (novice/beginner, competent/proficient, expert) and an iterative revision process involving local and national content experts. The TDAT was studied with attending physicians during inpatient rounds with trained physician and nonphysician observers over 6 months.

RESULTS

The TDAT showed emerging evidence of content, construct, and viable validity (the degree to which an assessment tool is practical, affordable, suitable, evaluable, and helpful in the real world) for the evaluation of attending physicians on inpatient rounds. Moderate to near perfect interrater reliability was seen between physician and nonphysician raters for the domains of promotion of clinical reasoning, control of the learning environment, ability to teach to multiple levels of learners, and provision of feedback.

CONCLUSIONS

The TDAT holds potential as a valid and reliable assessment tool for clinical teachers to track the development of each individual's teaching skills along the continuum from early development to mastery.

Developing a clinical teaching quality questionnaire for use in a university osteopathic pre-registration teaching program

BACKGROUND

Clinical education is an important component of many health professional training programs. There is a range of questionnaires to assess the quality of the clinical educator however none are in student-led clinic environments. The present study developed a questionnaire to assess the quality of the clinical educators in the osteopathy program at Victoria University.

METHODS

A systematic search of the literature was used to identify questionnaires that evaluated the quality of clinical teaching. Eighty-three items were extracted and reviewed for their appropriateness to include in a questionnaire by students, clinical educators and academics. A fifty-six item questionnaire was then trialled with osteopathy students. A variety of statistics were used to determine the number of factors to extract. Exploratory factor analysis (EFA) was used to investigate the factor structure.

RESULTS

The number of factors to extract was calculated to be between 3 and 6. Review of the factor structures suggested the most appropriate fit was four and five factors. The EFA of the four-factor solution collapsed into three factors. The five-factor solution demonstrated the most stable structure. Internal consistency of the five-factor solution was greater than 0.70.

CONCLUSIONS

The five factors were labelled Learning Environment (Factor 1), Reflective Practice (Factor 2), Feedback (Factor 3) and Patient Management (Factor 4) and Modelling (Factor 5). Further research is now required to continue investigating the construct validity and reliability of the questionnaire.

First impressions count: does FAIRness affect adaptation of clinical clerks in their first clinical placement?

BACKGROUND

FAIRness (Feedback, Activity, Individualisation, Relevance) teaching is a structured program, comprising series of classes in which student work is anonymised and reviewed by the whole class, as well as students receiving private feedback on their written work. The class work emphasises logic, structure and order in history and examination, with a diagnostic and management focus.

AIM

The effect of FAIRness teaching methods on the adaptation of medical students entering their first clinical rotations was studied.

METHODS

18 students in FAIRness placements and 72 students in conventional placements, all in medical/surgical units in the same University teaching hospital were studied. They completed questionnaires relating to effectiveness and quality of clinical teaching. Some students additionally attended focus groups, at the start of placement to discuss their expectations, and after 3 weeks, to discuss their adaptation to the clinical learning environment.

RESULTS

All students entering clinical placements had low expectations of their future teaching. Students in standard placements still expressed negative attitudes after 3 weeks, while students on FAIRness placements felt positive. Students in FAIRness placements scored significantly higher on questions related to feedback and review of student work.

CONCLUSION

FAIRness teaching practices help students to adapt to their first clinical placements.