A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8

Faculty development on item writing substantially improves item quality

The quality of items written for in-house examinations in medical schools remains a cause of concern. Several faculty development programs are aimed at improving faculty's item writing skills. The purpose of this study was to evaluate the effectiveness of a faculty development program in item development. An objective method was developed and used to assess improvement in faculty's competence to develop high quality test items. This was a quasi experimental study with a pretest-midtest-posttest design. A convenience sample of 51 faculty members participated. Structured checklists were used to assess the quality of test items at each phase of the study. Group scores were analyzed using repeated measures analysis of variance. The results showed a significant increase in participants' mean scores on Multiple Choice Questions, Short Answer Questions and Objective Structured Clinical Examination checklists from pretest to posttest (p < .0005). The effect sizes were 1.38, 3.84 and 2.20 for Multiple Choice Questions, Short Answer Questions and Objective Structured Clinical Examination, respectively. This study emphasizes that items written by faculty without faculty development are generally lacking in quality. It also provides evidence of the value of faculty development in improving the quality of items generated by faculty.

Student perceptions of evaluation in undergraduate medical education: A qualitative study from one medical school

BACKGROUND

Evaluation is an integral part of medical education. Despite a wide use of various evaluation tools, little is known about student perceptions regarding the purpose and desired consequences of evaluation. Such knowledge is important to facilitate interpretation of evaluation results. The aims of this study were to elicit student views on the purpose of evaluation, indicators of teaching quality, evaluation tools and possible consequences drawn from evaluation data.

METHODS

This qualitative study involved 17 undergraduate medical students in Years 3 and 4 participating in 3 focus group interviews. Content analysis was conducted by two different researchers.

RESULTS

Evaluation was viewed as a means to facilitate improvements within medical education. Teaching quality was believed to be dependent on content, process, teacher and student characteristics as well as learning outcome, with an emphasis on the latter. Students preferred online evaluations over paper-and-pencil forms and suggested circulating results among all faculty and students. Students strongly favoured the allocation of rewards and incentives for good teaching to individual teachers.

CONCLUSIONS

In addition to assessing structural aspects of teaching, evaluation tools need to adequately address learning outcome. The use of reliable and valid evaluation methods is a prerequisite for resource allocation to individual teachers based on evaluation results.

Hospice and palliative medicine: curriculum evaluation and learner assessment in medical education

Major efforts have been pursued to improve palliative care education for physicians at all levels of their training. Such changes include the incorporation of palliative care curriculum and guidelines, an established process for competency-based evaluation and certification, faculty development, innovative educational experiences, the improvement of textbooks, and the establishment of accredited palliative medicine fellowships. Hospice and palliative medicine (HPM) has been clearly defined as a subspecialty and a crucial area of medical education. As innovative curricular approaches have become available to educate medical and other interprofessional trainees, this article aims to describe different models and methods applied in curriculum evaluation, tailoring such approaches to the field of palliative medicine. A stepwise process of curriculum development and evaluation is described, focusing on available curriculum evaluation competency-based tools for each level of learners. As HPM evolves and its educational programs grow, curriculum evaluation will provides invaluable feedback to institutions and programs in many ways.

Ethnic disparities in undergraduate pre-clinical and clinical performance

CONTEXT

Research from numerous medical schools has shown that students from ethnic minorities underperform compared with those from the ethnic majority. However, little is known about why this underperformance occurs and whether there are performance differences among ethnic minority groups.

OBJECTIVES

This study aimed to investigate underperformance across ethnic minority groups in undergraduate pre-clinical and clinical training.

METHODS

A longitudinal prospective cohort study of progress on a 6-year undergraduate medical course was conducted in a Dutch medical school. Participants included 1661 Dutch and 696 non-Dutch students who entered the course over a consecutive 6-year period (2002-2007). Main outcome measures were performance in Year 1 and in the pre-clinical and clinical courses. Odds ratios (ORs) with 95% confidence intervals (CIs) were estimated by logistic regression analysis for ethnic subgroups (Surinamese/Antillean, Turkish/Moroccan/African, Asian, Western) compared with Dutch students, adjusted for age, gender, pre-university grade point average (pu-GPA), additional socio-demographic variables (first-generation immigrant, urban background, first-generation university student, first language, medical doctor as parent) and previous performance at medical school.

RESULTS

Compared with Dutch students, Surinamese and Antillean students specifically underperformed in the Year 1 course (pass rate: 37% versus 64%; adjusted OR 0.40, 95% CI 0.27-0.60) and the pre-clinical course (pass rate: 19% versus 41%; adjusted OR 0.57, 95% CI 0.35-0.93). On the clinical course all non-Dutch subgroups were less likely than Dutch students to receive a grade of ≥ 8.0 (at least three of five grades: 54-77% versus 88%; adjusted ORs: 0.17-0.45).

CONCLUSIONS

Strong ethnic disparities exist in medical school performance even after adjusting for age, gender, pu-GPA and socio-demographic variables. More subjective grading cannot be ruled out as a cause of lower grades in clinical training, but other possible explanations should be studied further to mitigate the disparities.

Evidence based curriculum design--use of a surgical database to identify non-perceived needs in educating otolaryngologists on pediatric conditions

OBJECTIVE

To identify the common non-otolaryngological diagnoses (N-OD) encountered by a pediatric otolaryngologist in surgical patients. This information may help in educating otolaryngologists on relevant pediatric conditions among their patients.

DESIGN/SETTING

Cross-sectional, retrospective database review of entries from 2006 to 2008 at a tertiary pediatric centre involving the patient case load of a busy pediatric otolaryngologist.

MAIN OUTCOME MEASURES

The main outcome of this study was to identify the common (N-OD) encountered by a pediatric otolaryngologist. Conditions excluded from the analysis were surgical complications and diagnoses managed primarily by otolaryngologists (sleep disordered breathing and allergic rhinitis).

RESULTS

1357 pediatric surgical entries were identified. Of these, 524 (38.6%) entries contained N-OD and underwent surgery. A total of 580 N-OD were identified for these 524 patients. The N-OD were identified and categorized into the following subheadings: syndromes/associations, respiratory and cardiac, gastrointestinal, developmental, metabolic, and miscellaneous conditions. The most common N-OD are gastro-esophageal reflux disease, obesity, history of prematurity, congenital heart disease, asthma developmental delay and Down syndrome. The commonest categories encountered were developmental (138), cardio-respiratory (114) and gastro-intestinal (114).

CONCLUSIONS

This paper identifies the common N-OD encountered by a pediatric otolaryngologist in children managed at a tertiary level. The highlighted N-OD may help as a guide on curriculum content for training residents and fellows and continued medical education.

The Donald W. Reynolds Consortium for Faculty Development to Advance Geriatrics Education (FD~AGE): a model for dissemination of subspecialty educational expertise

PURPOSE

Most U.S. medical schools and training programs lack sufficient faculty expertise in geriatrics to train future physicians to care for the growing population of older adults. Thus, to reach clinician-educators at institutions and programs that have limited resources for enhancing geriatrics curricula, the Donald W. Reynolds Foundation launched the Faculty Development to Advance Geriatrics Education (FD~AGE) program. This consortium of four medical schools disseminates expertise in geriatrics education through support and training of clinician-educators. The authors conducted this study to measure the effects of FD~AGE.

METHOD

Program leaders developed a three-pronged strategy to meet program goals: FD~AGE offers (1) advanced fellowships in clinical education for geriatricians who have completed clinical training, (2) mini-fellowships and intensive courses for faculty in geriatrics, teaching skills, and curriculum development, and (3) on-site consultations to assist institutions with reviewing and redesigning geriatrics education programs. FD~AGE evaluators tracked the number and type of participants and conducted interviews and follow-up surveys to gauge effects on learners and institutions.

RESULTS

Over six years (2004-2010), FD~AGE trained 82 fellows as clinician-educators, hosted 899 faculty scholars in mini-fellowships and intensive courses, and conducted 65 site visits. Participants taught thousands of students, developed innovative curricula, and assumed leadership roles. Participants cited as especially important to program success expanded knowledge, improved teaching skills, mentoring, and advocacy.

CONCLUSIONS

The FD~AGE program represents a unique model for extending concentrated expertise in geriatrics education to a broad group of faculty and institutions to accelerate progress in training future physicians.

Do GP trainers use feedback in drawing up their Personal Development Plans (PDPs)? Results from a quantitative study

BACKGROUND

General practice (GP) trainers play a key role GP trainees' education. To stimulate development of trainer competencies a Personal Development Plan (PDP) can be helpful, especially when feedback is incorporated.

AIMS

To investigate to what extent GP trainers use feedback in PDPs.

METHODS

GP trainers were provided with three feedback sources: trainees' rating scores, trainees' narrative comments, and self-assessment scores. Trainers were instructed to use these while drawing up PDP goals. With quantitative analyses the extent to feedback sources were used was determined.

RESULTS

Of the trainers 93% submitted a PDP. More than 75% of goals were based on provided feedback. Multiple sources addressing the same issue increased feedback use. If two sources pointed in the same direction, feedback was used more often if one of them concerned "narrative comments". Ratings were lowest for GP-related Expertise and Teaching Skills. Most goals defined concerned these domains. Fewer goals regarded Personal Functioning. Proportion of feedback used concerning Personal functioning was lowest.

CONCLUSIONS

GP trainers use most feedback and address issues most commented upon. Narrative comments deserve a profound place when eliciting feedback. Research into the quality with which feedback is used in PDP goals should complement these results.

Career development needs of vice chairs for education in departments of surgery

AIM

To identify the career development needs Vice Chair for Education in Surgery Departments (VCESDs).

METHODS

In all, 33 VCESDs were invited to complete an online survey to identify the scope of duties, scholarly activity, job satisfaction, and career development needs.

RESULTS

A total of 29/33 (88%) VCESDs responded. Time constraints were the most frequent impediment for MDs vs. PhDs (p < 0.05). Dominant faculty development needs were conducting educational research (2.0 ± 0.78 for MDs, 1.33 ± 0.76 for PhDs), developing resident selection systems (1.68 ± 0.73), and mentorship programs (1.95 ± 0.77) for MDs, and developing teach the teacher programs (1 ± 0), and program performance evaluation systems (1.33 ± 0.76) for PhDs. The skills deemed to be of greatest importance were ability to communicate effectively (1.27 ± 0.55), resolve personnel conflicts (1.32 ± 0.57), and introduce change (1.41 ± 0.59). PhDs revealed a greater need to learn strategies for dealing with disruptive faculty (1.0 ± 0 vs 2.15 ± 0.87).

CONCLUSIONS

This information will inform the future career development of VCESDs and will assist Department Chairs who wish to recruit and retain VCESDs.

Perspectives on faculty development: aiming for 6/6 by 2020

Faculty development has a key role to play in individual and organizational development. This perspective on faculty development, which builds on the 2020 Vision of Faculty Development Across the Medical Education Continuum Conference and the First International Conference on Faculty Development in the Health Professions, describes six recommendations that we should consider as the field of faculty development moves forward: grounding faculty development in a theoretical framework; broadening the focus of faculty development to address the various roles that clinicians and basic scientists play; recognizing the role that faculty development can play in promoting curricular and organizational change; expanding our notion of how faculty members develop and moving beyond formal, structured activities to incorporate notions of work-based learning and communities of practice; making faculty development an expectation for all faculty members; and promoting scholarship in faculty development to ensure that research informs practice. Looking ahead, we should also consider strategies for leading change, collaborate across institutions and international borders, and work together to share lessons learned in research and practice.

Developing expert-derived rating standards for the peer assessment of lectures

PURPOSE

For peer review of teaching to be credible and reliable, peer raters must be trained to identify and measure teaching behaviors accurately. Peer rater training, therefore, must be based on expert-derived rating standards of teaching performance. The authors sought to establish precise lecture rating standards for use in peer rater training at their school.

METHOD

From 2008 to 2010, a panel of experts, who had previously helped to develop an instrument for the peer assessment of lecturing, met to observe, discuss, and rate 40 lectures, using a consensus-building model to determine key behaviors and levels of proficiency for each of the instrument's 11 criteria. During this process, the panelists supplemented the original instrument with precise behavioral descriptors of lecturing. The reliability of the derived rating standards was assessed by having the panelists score six sample lectures independently.

RESULTS

Intraclass correlation coefficients of the panelists' ratings of the lectures ranged from 0.75 to 0.96. There was moderate to high positive association between 10 of the 11 instrument's criteria and the overall performance score (r = 0.752-0.886). There were no statistically significant differences among raters in terms of leniency or stringency of scores.

CONCLUSIONS

Two relational themes, content and style, were identified within the instrument's variables. Recommendations for developing expert-derived ratings standards include using an interdisciplinary group for observation, discussion, and verbal identification of behaviors; asking members to consider views that contrast with their own; and noting key teaching behaviors for use in future peer rater training.

Impact of a Resident-as-Teacher Workshop on Teaching Behavior of Interns and Learning Outcomes of Medical Students

BACKGROUND

Residents and interns are recognized as important clinical teachers and mentors. Resident-as-teacher training programs are known to improve resident attitudes and perceptions toward teaching, as well as their theoretical knowledge, skills, and teaching behavior. The effect of resident-as-teacher programs on learning outcomes of medical students, however, remains unknown. An intervention cohort study was conducted to prospectively investigate the effects of a teacher-training workshop on teaching behavior of participating interns and on the clerkship learning outcomes of instructed fourth-year medical students.

METHODS

The House Officer-as-Teacher Training Workshop was implemented in November 2009 over 1.5 days and attended by all 34 interns from one teaching hospital. Subsequently, between February and August 2010, 124 fourth-year medical students rated the observable teaching behavior of interns during 6-week general surgery clerkships at this intervention hospital as well as at 2 comparable hospitals serving as control sites. Ratings were collected using an anonymous 15-item Intern Clinical Teaching Effectiveness Instrument. Student achievement of clerkship learning outcomes during this period was evaluated using a validated and centralized objective structured clinical examination.

RESULTS

Medical students completed 101 intern clinical teaching effectiveness instruments. Intern teaching behavior at the intervention hospital was found to be significantly more positive, compared with observed behavior at the control hospitals. Objective structured clinical examination results, however, did not demonstrate any significant intersite differences in student achievement of general surgery clerkship learning outcomes.

CONCLUSIONS

The House Officer-as-Teacher Training Workshop noticeably improved teaching behavior of surgical interns during general surgery clerkships. This improvement did not, however, translate into improved achievement of clerkship learning outcomes by medical students during the study period.

Three ways of understanding development as a teacher

The demands on faculty in terms of teaching are increasing, but until recently there has been little discussion of how faculty perceive that development as a teacher can be achieved or what approaches they use or suggest themselves. The aim of this study is to explore how teachers in dentistry and medicine understand development as teachers. For this study, 20 teachers were interviewed. The interviews were analysed using a phenomenographic approach. Three different ways of understanding development were identified: 1) Development as a dental or medical clinician/expert as the teacher role is seen as a tacit part of the role of the clinician. 2) Experience and professional and personal maturation, related to personal and professional development and confidence in ones clinical role. 3) Knowledge in education and systematic teacher training as in this category, being a teacher is seen as a separate role from that of being a clinician. The differences in these three ways of understanding development as a teacher are shown in their different aims of development, what kind of knowledge that may be used and what methods they suggested. The way teachers understand what it means to develop as a teacher will affect their motivation for engaging in development activities, which activities they choose and their own aims of development. This means that awareness of teachers' understanding of development is central when developing support or faculty development activities for teachers.

Changing assessment practice through in situ faculty development

The aim of this article is to describe the process of an in situ staff development process with the objective to influence change in assessment practice. An in situ training course focusing on writing questions for written examinations, but also including some contextual aspects of assessment practice, was therefore developed and implemented. The anticipated change was measured against Kirkpatrick's four levels for evaluating training programmes. As a whole, the reaction from the participants was positive (Kirkpatrick Level 1), and in a number of instances, learning, which includes changes in attitude, knowledge and skills (Kirkpatrick Level 2) and change in behaviour (Kirkpatrick Level 3), was observed. To conclude, the staff development initiative in the form of in situ assessment training facilitated change resulting in an improvement in assessment practice in the School in a relatively short period of time.

Faculty development initiatives designed to promote leadership in medical education. A BEME systematic review: BEME Guide No. 19

BACKGROUND

Due to the increasing complexity of medical education and practice, the preparation of healthcare professionals for leadership roles and responsibilities has become increasingly important. To date, the literature on faculty development designed to promote leadership in medical education has not been reviewed in a systematic fashion.

AIM

The objective of this review is to synthesize the existing evidence that addresses the following question: 'What are the effects of faculty development interventions designed to improve leadership abilities on the knowledge, attitudes, and skills of faculty members in medicine and on the institutions in which they work?'

SEARCH STRATEGY

The search, which covered the period 1980-2009, included six databases (Medline, EMBASE, CINAHL, Web of Science, ERIC, and ABI/Inform) and used the following keywords: faculty development; in-service training; doctor; medic; physician; faculty; leadership; management; administration; executive; and change agent. Hand searches were also conducted, and expert recommendations were solicited.

INCLUSION AND EXCLUSION CRITERIA

Articles with a focus on faculty development to improve leadership, targeting basic science and clinical faculty members, were reviewed. All study designs that included outcome data beyond participant satisfaction were examined. From an initial 687 unique records, 48 articles met the review criteria in three broad categories: (1) reports in which leadership was the primary focus of the intervention; (2) reports in which leadership was a component of a broader focus on educational development; and (3) reports in which leadership was a component of a broader focus on academic career development.

DATA EXTRACTION

Data were extracted by three coders using the standardized Best Evidence Medical Education coding sheet adapted for our use. One reviewer coded all of the articles, and two reviewers each coded half of the dataset. Coding differences were resolved through discussion.

DATA SYNTHESIS

Data were synthesized using Kirkpatrick's four levels of educational outcomes. Findings were grouped by intervention type and level of outcome.

RESULTS

Forty-eight articles described 41 studies of 35 different interventions. The majority of the interventions targeted clinical faculty members and included workshops, short courses, fellowships, and other longitudinal programs. The majority of studies were quantitative in nature, though five studies used a qualitative design, and 12 studies used mixed methods. All quantitative studies were quasi-experimental and most employed a single group design; only two studies had a comparison group. Qualitative study designs were typically not specified. The majority of evaluation data, primarily collected post-intervention, consisted of participants' responses to questionnaires and interviews. KEY POINTS AND SUMMARY OF OUTCOMES: Despite methodological limitations, the faculty development literature tends to support the following outcomes: ▪ High satisfaction with faculty development programs. Participants consistently found programs to be useful and of both personal and professional benefit. They also valued the practical relevance and applicability of the instructional methods used. ▪ A change in attitudes toward organizational contexts and leadership roles. Participants reported positive changes in attitudes toward their own organizations as well as their leadership capabilities. Some reported an increased awareness of--and commitment to--their institution's vision and challenges, whereas others reported greater self-awareness of personal strengths and limitations, increased motivation, and confidence in their leadership roles. A greater sense of community and appreciation of the benefits of networking were also identified. ▪ Gains in knowledge and skills. Participants reported increased knowledge of leadership concepts, principles, and strategies (e.g., leadership styles and strategic planning), gains in specific leadership skills (e.g., personal effectiveness and conflict resolution), and increased awareness of leadership roles in academic settings. ▪ Changes in leadership behavior. Self-perceived changes in leadership behavior were consistently reported and included a change in leadership styles, the application of new skills to the workplace (e.g., departmental reorganization and team building), the adoption of new leadership roles and responsibilities, and the creation of new collaborations and networks. Observed changes primarily suggested new leadership positions. ▪ Limited changes in organizational practice. Although not frequently examined, changes in organizational practice included the implementation of specific educational innovations, an increased emphasis on educational scholarship, and the establishment of collegial networks. ▪ Key features of faculty development. Features contributing to positive outcomes included the use of: multiple instructional methods within single interventions; experiential learning and reflective practice; individual and group projects; peer support and the development of communities of practice; mentorship; and institutional support. ▪ Avenues for future development: Moving forward, faculty development programs should: ground their work in a theoretical framework; articulate their definition of leadership; consider the role of context; explore the value of extended programs and follow-up sessions; and promote the use of alternative practices including narrative approaches, peer coaching, and team development. METHODOLOGICAL ISSUES: More rigorous and diverse research designs are needed to capture the complexity of interventions in this area. Varied methods of assessment, utilizing multiple data sources to tap changes at the interpersonal and organizational level should be explored, as should the maintenance of change over time. Process-oriented studies, comparing different faculty development strategies and clarifying the process of change through faculty development, should also become a priority.

CONCLUSION

Participants value leadership development activities and report changes in attitudes, knowledge, skills and behavior. Moreover, despite methodological limitations, certain program characteristics seem to be associated with positive outcomes. Further research is required to explore these associations and document changes at both the individual and organizational level.

A randomised controlled study of role play in a faculty development programme

BACKGROUND

The Stanford Faculty Development Center at Stanford University has developed a teaching improvement course for medical teachers that has been widely disseminated using a train-the-trainer model.

AIMS

This study was designed to investigate the relative impact of role playing as an instructional technique within that course for facilitating change in teaching behaviours.

METHOD

From January 2009 to April 2010, six faculty development courses were delivered at Uppsala University Hospital to 48 physicians from different departments. The standard course presentation includes a range of instructional methods including short lectures, small group discussion, review of video re-enactments, role-play exercises and personal goal setting. For this study, participants were randomised to participate in (1) a 'standard' course with role play or (2) an 'alternative' course with no role play. The effects of the course on teaching performance were assessed with retrospective pre- and post-course self-ratings of 29 specific teaching behaviours.

RESULTS

Self-assessment ratings indicated significantly greater positive changes in teaching behaviour among faculty who attended the standard course (with role play) as compared to those in the alternative course (p = 0.015).

CONCLUSIONS

This study validates the commonly held view that role play is a useful instructional method for improving teaching.

Quality evaluation reports: Can a faculty development program make a difference?

BACKGROUND

The quality of medical student and resident clinical evaluation reports submitted by rotation supervisors is a concern. The effectiveness of faculty development (FD) interventions in changing report quality is uncertain.

AIMS

This study assessed whether faculty could be trained to complete higher quality reports.

METHOD

A 3-h interactive program designed to improve evaluation report quality, previously developed and tested locally, was offered at three different Canadian medical schools. To assess for a change in report quality, three reports completed by each supervisor prior to the workshop and all reports completed for 6 months following the workshop were evaluated by three blinded, independent raters using the Completed Clinical Evaluation Report Rating (CCERR): a validated scale that assesses report quality.

RESULTS

A total of 22 supervisors from multiple specialties participated. The mean CCERR score for reports completed after the workshop was significantly higher (21.74 ± 4.91 versus 18.90 ± 5.00, p = 0.02).

CONCLUSIONS

This study demonstrates that this FD workshop had a positive impact upon the quality of the participants' evaluation reports suggesting that faculty have the potential to be trained with regards to trainee assessment. This adds to the literature which suggests that FD is an important component in improving assessment quality.

Changing conceptions of teaching in medical faculty

BACKGROUND

The Association of American Medical Colleges and Howard Hughes Medical Institute have urged medical faculty to develop in themselves, and in their students, habits of critical professional inquiry, while stressing the importance of integrating formal learning with hands-on, practical experience.

AIMS

This study describes the impact of a year-long faculty development program on conceptions of teaching held by 16 faculty in a medical school affiliated with a large private Midwestern research university.

METHOD

Using pre and post semi-structured interviews, researchers identified three conceptions of teaching - transmission, acquisition, and engagement - found in participants, comparing how participants' conceptions changed by the end of the program.

RESULTS

Researchers found that 13 out of 16 (81%) faculty developed or maintained a more complex understanding of teaching by the end of the program.

CONCLUSIONS

This study recommends that faculty developers (1) encourage their faculty to reflect substantially on what it means to teach - with the goal of assisting faculty to move beyond the idea of teaching as transmission or even as acquisition, to conceiving it as helping their students ask deep questions and make conceptual connections for themselves; (2) help faculty understand that developing a more complex understanding of teaching is as crucial to their professional development as developing their research and clinical skills; and (3) provide substantial opportunities for participants to explore teaching and learning as an evidence-based practice with colleagues and students.

Development of an instrument (the COLT) to measure conceptions on learning and teaching of teachers, in student-centred medical education

BACKGROUND

Conceptions of medical teachers regarding learning and teaching affect their teaching practice. Therefore conceptions should be addressed in faculty development.

AIM

To facilitate this, we constructed the Conceptions Of Learning and Teaching (COLT) instrument.

METHOD

The COLT was adapted based on experts' comments during a meeting and interviews, followed by a Delphi procedure (Part I). It was administered to teachers from two Dutch medical schools with different traditions in student-centred education (Part II; N=646). The data were analyzed using confirmatory factor analysis and reliability analysis.

RESULTS

324 Teachers (50.2%) completed the questionnaire. Confirmatory factor analysis did not confirm the underlying theoretical model, but an alternative model demonstrated a good fit. This led to an instrument with eighteen items reflecting three underlying factors: 'teacher centredness', 'appreciation of active learning', and 'orientation to professional practice'. We found significant differences in COLT scores between the faculty of the two medical schools.

CONCLUSIONS

The COLT appears to be a construct valid tool resulting in reliable scores of teachers' conceptions of learning and teaching, in student-centred medical education. Two of the three factors are new and may be specific for student-centred medical education. The COLT may be a promising tool to improve faculty development.

An innovative process for faculty development in residency training

BACKGROUND

For programs to accomplish the goals of the Accreditation Council of Graduate Medical Education (ACGME) Outcome Project, faculty must be trained to deliver and assess education that is level-specific, competency-based, standardized, integrated, and easily accessible.

DESCRIPTION

An innovative faculty development model that accomplishes these goals is described. This model trained faculty to analyze curricular needs and then to design, disseminate, and evaluate their curricula. This model utilized guided experiential learning that promoted the creation of residency program curricula and faculty buy-in.

EVALUATION

Key outcomes included high levels of resident satisfaction and use of the curricula, improved tracking of rotation progress, improved faculty satisfaction with their role as educators, perceived improvement in resident evaluations, and increased involvement of faculty in creating and teaching to the curriculum.

CONCLUSION

This process may be adapted by other programs based on their available resources to address faculty development needs. The process serves as a model for meeting ACGME requirements.

Kirkpatrick's levels and education 'evidence'

OBJECTIVES

This study aims to review, critically, the suitability of Kirkpatrick's levels for appraising interventions in medical education, to review empirical evidence of their application in this context, and to explore alternative ways of appraising research evidence.

METHODS

The mixed methods used in this research included a narrative literature review, a critical review of theory and qualitative empirical analysis, conducted within a process of cooperative inquiry.

RESULTS

Kirkpatrick's levels, introduced to evaluate training in industry, involve so many implicit assumptions that they are suitable for use only in relatively simple instructional designs, short-term endpoints and beneficiaries other than learners. Such conditions are met by perhaps one-fifth of medical education evidence reviews. Under other conditions, the hierarchical application of the levels as a critical appraisal tool adds little value and leaves reviewers to make global judgements of the trustworthiness of the data.

CONCLUSIONS

Far from defining a reference standard critical appraisal tool, this research shows that 'quality' is defined as much by the purpose to which evidence is to be put as by any invariant and objectively measurable quality. Pending further research, we offer a simple way of deciding how to appraise the quality of medical education research.

The impact of an objective structured teaching evaluation on faculty teaching skills

BACKGROUND

Objective structured teaching evaluations (OSTEs) have been utilized to evaluate educational curricula and for resident and faculty development.

PURPOSE

This study examines the impact of an OSTE on faculty teaching effectiveness and faculty satisfaction.

METHODS

From 2004 to 2007, 46 faculty members participated in the OSTE. Faculty assessed their teaching abilities with a retrospective pre-post-test analysis. Faculty teaching evaluations for the 6 months before and after the OSTE were compared. Faculty participants completed satisfaction questionnaires regarding their OSTE experience and made teaching plans for the future.

RESULTS

After the OSTE, faculty reported statistically significant improvements in all self-assessed teaching skills. There was, however, no improvement in their teaching evaluations. Faculty satisfaction with the OSTE experience was high. They indicated teaching plans incorporating lessons from the OSTE.

CONCLUSIONS

Faculty felt the OSTE was a rewarding experience and reported improvement in their teaching abilities; however, faculty teaching evaluations did not improve.

Twelve tips for using the objective structured teaching exercise for faculty development

BACKGROUND

The importance of faculty development to improve clinicians' teaching skills has been well articulated in the literature. There are few objective measures of the impact of faculty development on teaching skills. The objective structured teaching exercise (OSTE) is a faculty development tool that may meet this challenge. It also has great potential to be used in the development and enhancement of teaching skills. The OSTE consists of a simulated teaching scenario involving a standardized learner with objective and immediate feedback given to the teacher, and includes a pre-determined behaviourally based scale or checklist to assess teaching performance.

AIM

There is little information in the literature on the practical aspects of how to develop and deliver an OSTE in a faculty development context. Based on our experience, we created a framework to guide the use of the OSTE for faculty development.

METHODS

Twelve tips for using the OSTE for faculty development are outlined in this article. These include: clarifying the goal and target audience, identifying what teaching skills to focus on, developing the scenario and the assessment tool, choosing and training the standardized learner, holding a dry run, protecting the teacher, integrating the OSTE into one's own context and promoting buy-in, and evaluating the activity.

CONCLUSIONS

The OSTE is a novel tool to enhance faculty development. We describe 12 key elements that are important for its successful development and delivery.

Evidence shaped health professional education: can we talk about a new paradigm?

This commentary accompanies the publication of several Best Evidence Medical Education systematic reviews together with invited discussion papers on issues of key importance to evidence shaped health professional education. The reviews and papers signify positive developments for the still very young movement where practitioners and policy makers use evidence to help their decision making in health professional education. They provide this commentary with exemplar material to highlight some key points about the process of education systematic review and some of the conceptual and methodological issues presently being discussed by colleagues interested in taking the evidence shaped education movement in a forward direction. These include the need for more funded reviews, greater commitment to question-driven research (primary and secondary) and further debate about power, evidence and change. It is hoped that this will lead to a broader debate, a widening of interest and a deepening of scholarship related to evidence shared health professional education.

Faculty development program: a guide for medical schools in Arabian Gulf (GCC) countries

The effective faculty development (FD) program is the most essential component for creating successful educational innovation. The purpose of having any FD program is to enable faculty to accomplish their goals, and through accomplishing these goals, the missions of their departments, colleges, and universities would be accomplished. FD programs will need to provide an extensive set of developmental-educational skills and strategies in order to serve faculty members who come from various disciplines and at different stages of their careers, have different faculty responsibilities, backgrounds, appointments, and bring unique personal circumstances to their work. Although, there are quite a large number of medical schools in Arabian Gulf (GCC) countries, still the FD programs are extremely lacking. To analyze the situation in Gulf medical schools with regards to FD programs, we would emphasize the following needs: (1) FD committees or departments; (2) formal programs in FD for new or junior and for also senior and established faculty members; and (3) funding support has to be devoted to improve the skills of faculty members in academia. This aimed at presenting a proposed FD program to be considered as a tool for qualitative improvement in medical schools of GCC countries.

Spaced education faculty development may not improve faculty teaching performance ratings in a surgery department

OBJECTIVES

To determine the effectiveness of spaced education as a faculty development tool designed to improve teaching skills in a surgery department.

DESIGN

Faculty members were randomized to receive either weekly spaced education e-mails with content designed to improve teaching skills (group A) or no e-mails (group B). Using qualitative and quantitative surveys, we assessed both medical students' perception of faculty members' teaching effectiveness and faculty members' perception of the usefulness of the spaced education e-mails.

SETTING

Academic medical center.

PARTICIPANTS

Twenty-nine surgery faculty members with teaching responsibility for medical students in their Core Surgery Clerkship.

RESULTS

All 41 medical students who rotated through the Core Surgery Clerkship rated the quality of teaching for each faculty members; 172 online rating surveys were completed. Overall, faculty members received high ratings on the teaching skills included on the surveys. Additionally, no significant differences were found between the perceived skill level of the faculty members who received the weekly e-mails and those who did not. Specifically, 53.8% and 54% (p = 0.47) of the faculty were felt to deliver feedback more than three times per week; 87.1% and 89.9% (p = 0.15) of faculty were felt to deliver useful feedback; 89.2% and 90.8% (p = 0.71) of faculty were perceived to encourage student autonomy; and 78.1% and 81.9% (p = 0.89) of faculty were felt to set clear learning expectations for students. Postprogram comments from faculty revealed they did not find the e-mails useful as a faculty development tool.

CONCLUSIONS

Students perceived high levels of teaching skills among the clinical faculty. Faculty members who received e-mail-based spaced education-based faculty development were not rated to be more effective teachers on the student surveys. Electronically based faculty development does not satisfy faculty expectations.

Foundation observation of teaching project--a developmental model of peer observation of teaching

INTRODUCTION

Peer observation of teaching is important in the development of educators. The foundation curriculum specifies teaching competencies that must be attained. We created a developmental model of peer observation of teaching to help our foundation doctors achieve these competencies and develop as educators.

METHODS

A process for peer observation was created based on key features of faculty development. The project consisted of a pre-observation meeting, the observation, a post-observation debrief, writing of reflective reports and group feedback sessions. The project was evaluated by completion of questionnaires and focus groups held with both foundation doctors and the students they taught to achieve triangulation.

RESULTS

Twenty-one foundation doctors took part. All completed reflective reports on their teaching. Participants described the process as useful in their development as educators, citing specific examples of changes to their teaching practice. Medical students rated the sessions as better or much better quality as their usual teaching.

DISCUSSION

The study highlights the benefits of the project to individual foundation doctors, undergraduate medical students and faculty. It acknowledges potential anxieties involved in having teaching observed.

CONCLUSION

A structured programme of observation of teaching can deliver specific teaching competencies required by foundation doctors and provides additional benefits.

Neophyte facilitator experiences of interprofessional education: implications for faculty development

The facilitation of learners from different professional groups requires a range of interprofessional knowledge and skills (e.g. an understanding of possible sources of tension between professions) in addition to those that are more generic, such as how to manage a small group of learners. The development and delivery of interprofessional education (IPE) programs tends to rely on a small cohort of facilitators who have typically gained expertise through 'hands-on' involvement in facilitating IPE and through mentorship from more experienced colleagues. To avoid burn-out and to meet a growing demand for IPE, a larger number of facilitators are needed. However, empirical evidence regarding effective approaches to prepare for this type of work is limited. This article draws on data from a multiple case study of four IPE programs based in an urban setting in North America with a sample of neophyte facilitators and provides insight into their perceptions and experiences in preparing for and delivering IPE. Forty-one semi-structured interviews were conducted before (n = 20) and after (n = 21) program delivery with 21 facilitators. Findings indicated that despite participating in a three-fold faculty development strategy designed to support them in their IPE facilitation work, many felt unprepared and continued to have a poor conceptual understanding of core IPE and interprofessional collaboration principles, resulting in problematic implications (e.g. 'missed teachable moments') within their IPE programs. Findings from this study are discussed in relation to the IPE, faculty development and wider educational literature before implications are offered for the future delivery of interprofessional faculty development activities.

Value placed on formal training in education by pediatric department chairs and residency program directors

BACKGROUND

While much is known about how educational leaders at the medical school level (eg, deans) view the importance of formal training in education for medical school teachers, little is known about how leaders at the clinical level (eg, department chairs) view such training. We sought to determine how pediatric department chairs and residency program directors view the value of formal training in education, such as that at a Master of Education (MEd) level, and to estimate the number of clinical pediatric faculty with or pursuing such training.

METHODS

A survey designed to assess the value placed on formal training in education and to estimate the number of clinical faculty with or pursuing such training was mailed to pediatric department chair persons and residency program directors at all 131 allopathic medical schools in the United States and Puerto Rico.

RESULTS

Eighty department chairs (61%) responded, and most indicated that when hiring new faculty, they view an applicant with an MEd as having an advantage. Both chairs and residency directors considered an MEd to be advantageous for a residency director by a ranking of 4.5 and 4.2, respectively, on a scale of 1 to 5 (P  =  .008). Of the 80 chairs who responded, 58.8% of respondents reported one or more faculty in their department had or was pursuing an MEd. Of the 72 responding residency directors (55%), 11 respondents (15.3%) indicated that they had an MEd.

CONCLUSION

More than half the medical school pediatric chairs responding to the survey had one or more clinical faculty with or pursuing an MEd in their departments. Survey results indicated that such training is valued by both department chairs and residency directors. Given the time and expense involved in obtaining an MEd, awareness of these data 5 be helpful to those considering pursuing, offering, or requiring such training.

Feedback in the emergency medicine clerkship

OBJECTIVE

Feedback is a technique used in medical education to help develop and improve clinical skills. A comprehensive review article specifically intended for the emergency medicine (EM) educator is lacking, and it is the intent of this article to provide the reader with an in-depth, up-to-date, and evidence-based review of feedback in the context of the EM clerkship.

METHODS

The review article is organized in a progressive manner, beginning with the definition of feedback, the importance of feedback in medical education, the obstacles limiting the effective delivery of feedback, and the techniques to overcome these obstacles then follows. The article concludes with practical recommendations to implement feedback in the EM clerkship. To advance the literature on feedback, the concept of receiving feedback is introduced.

RESULTS

The published literature regarding feedback is limited but generally supportive of its importance and effectiveness. Obstacles in the way of feedback include time constraints, lack of direct observation, and fear of negative emotional responses from students. Feedback should be timely, expected, focused, based on first-hand data, and limited to behaviors that are remediable. Faculty development and course structure can improve feedback in the EM clerkship. Teaching students to receive feedback is a novel educational technique that can improve the feedback process.

CONCLUSION

Feedback is an important educational technique necessary to improve clinical skills. Feedback can be delivered effectively in the EM clerkship.

The effect of patient feedback on physicians' consultation skills: a systematic review

PURPOSE

The effect of patient feedback interventions as a method of improving physicians' consultation (i.e., communication, interpersonal) skills is equivocal; research is scarce, and methods and rigor vary. The authors conducted this systematic review to analyze the educational effect of feedback from real patients on physicians' consultation skills at the four Kirkpatrick levels.

METHOD

The authors searched five databases (PubMed, EMBASE, Cochrane, PsycInfo, ERIC; April 2010). They included empirical studies of all designs (randomized controlled, quasi-experimental, cross-sectional, and qualitative) if the studies concerned physicians in general health care who received formal feedback regarding their consultation skills from real patients. The authors have briefly described aspects of the included studies, analyzed their quality, and examined their results by Kirkpatrick educational effect level.

RESULTS

The authors identified 15 studies (10 studies in primary care; 5 in other specialties) in which physicians received feedback in various ways (e.g., aggregated patient reports or educator-mediated coaching sessions), conducted in the United States, the Netherlands, the United Kingdom, Australia, and Canada. All studies that assessed level 1 (valuation), level 2 (learning), and level 3 (intended behavior) demonstrated positive results; however, only four of the seven studies that assessed level 4 (change in actual performance or results) found a beneficial effect.

CONCLUSIONS

Some evidence for the effectiveness of using feedback from real patients to improve knowledge and behavior exists; however, before implementing patient feedback into training programs, educators and policy makers should realize that the evidence for effecting actual improvement in physicians' consulting skills is rather limited.

Academic advancement of clinician educators: why is it so difficult?

The role of clinician educators (CEs) in institutions and medical centres continues to be vital without any doubt. Although there has been more than a century since Sir William Osler established the role of the CE and the tradition by encouraging bed-side teaching, there is still a lack of consensus on the attributes that define a 'clinician-educator'. The concept of a superior clinician who is also a dedicated teacher seems to fit the description of a CE but most often seems insufficient to support the CE's academic advancement.

New tools for systematic evaluation of teaching qualities of medical faculty: results of an ongoing multi-center survey

Background

Tools for the evaluation, improvement and promotion of the teaching excellence of faculty remain elusive in residency settings. This study investigates (i) the reliability and validity of the data yielded by using two new instruments for evaluating the teaching qualities of medical faculty, (ii) the instruments' potential for differentiating between faculty, and (iii) the number of residents' evaluations needed per faculty to reliably use the instruments.

Methods and Materials

Multicenter cross-sectional survey among 546 residents and 629 medical faculty representing 29 medical (non-surgical) specialty training programs in the Netherlands. Two instruments—one completed by residents and one by faculty—for measuring teaching qualities of faculty were developed. Statistical analyses included factor analysis, reliability and validity exploration using standard psychometric methods, calculation of the numbers of residents' evaluations needed per faculty to achieve reliable assessments and variance components and threshold analyses.

Results

A total of 403 (73.8%) residents completed 3575 evaluations of 570 medical faculty while 494 (78.5%) faculty self-evaluated. In both instruments five composite-scales of faculty teaching qualities were detected with high internal consistency and reliability: learning climate (Cronbach's alpha of 0.85 for residents' instrument, 0.71 for self-evaluation instrument, professional attitude and behavior (0.84/0.75), communication of goals (0.90/0.84), evaluation of residents (0.91/0.81), and feedback (0.91/0.85). Faculty tended to evaluate themselves higher than did the residents. Up to a third of the total variance in various teaching qualities can be attributed to between-faculty differences. Some seven residents' evaluations per faculty are needed for assessments to attain a reliability level of 0.90.

Conclusions

The instruments for evaluating teaching qualities of medical faculty appear to yield reliable and valid data. They are feasible for use in medical residencies, can detect between-faculty differences and supply potentially useful information for improving graduate medical education.

Evaluation of the University of Virginia Leadership in Academic Medicine Program

BACKGROUND

Developing effective transformational leaders is essential for a diverse and complex academic medical system. The purpose of this study is to elicit faculty perspectives on their leadership skills before and after participation in a Leadership in Academic Medicine (LAM) Program.

SUMMARY

The two-part study consisted of a prospective Pre- and Immediate Post-LAM Survey distributed to 32 participants in the 2008 Program and a Long-Term Post-LAM Survey distributed to 110 prior participants (2004-2007). Both surveys were designed to assess participant leadership perspectives and career outcomes. Institutional Review Board approval was obtained. Data were analyzed quantitatively and qualitatively. All participants reported improved leadership skills, but the percentages were lower for Long-Term Post-LAM participants than for the Immediate Post-LAM participants. In addition, although 58% of Immediate Post-LAM women, compared with 19% of Immediate Post-LAM men (p < .05), were actively seeking leadership roles, this was reversed in the long-term group (26% of women vs. 66% of men; p < .05).

CONCLUSIONS

Although the benefits of LAM participation are sustained for some years, there is a need for reinforcement to prevent skill attrition. That lower percentages of Long-Term Post-LAM women had leadership aspirations requires further study.

Instruction in developmental psychiatry: preliminary findings at the undergraduate level in Brazil

BACKGROUND: Although knowledge on developmental psychiatry is fundamental for the early recognition, treatment, and prevention of mental disorders, this subject has not been incorporated into the medical curriculum or psychiatric practice in Brazil. OBJECTIVE: To evaluate the effect of a short course on developmental psychiatry for undergraduate students and to expand education policies concerning developmental psychiatry in Brazil. METHODS: Before and after attending an extracurricular 12-hour, 4-day course on the fundamentals of developmental psychiatry, undergraduate health sciences students were tested regarding their knowledge of the subject. The pre-test/post-test included 12 randomly selected multiple-choice questions designed to evaluate knowledge related to developmental psychiatry and was administered together with a questionnaire designed to evaluate students' attitudes. To compare performances between groups, nonparametric analyses of ordinal categorical data were employed. RESULTS: The final sample comprised 43 students. The mean post-test score was significantly higher than the mean pre-test score (65.0% vs 39.9%; p < 0.0001). We found that strongly positive attitudes correlated with better performance. The 3rd and 4th year medical students performed better than the 1st and 2nd year medical students and the non-medical students. Sex differences favoring males were also observed. CONCLUSION: Our findings encourage additional educational policies related to developmental psychiatry which may result in direct clinical implications.

To the point: medical education reviews-ongoing call for faculty development

This article in the To the Point series will focus on best practices regarding faculty development in medical education in the field of obstetrics and gynecology. Faculty development is an essential component in achieving teacher and learner satisfaction as well as improving learner outcomes. The Liaison Committee on Medical Education requires medical school faculty to have the capability and longitudinal commitment to be effective teachers. Although many programs have been created to address faculty development, there remains a paucity of literature documenting the impact of these programs on learner outcomes. We reviewed the qualities of an excellent medical educator, expectations regarding medical school teaching faculty, elements of comprehensive faculty development programs, and outcome measures for evaluating the effectiveness of these programs.

Developing future faculty: a program targeting internal medicine fellows' teaching skills

INTRODUCTION

The increased demand for clinician-educators in academic medicine necessitates additional training in educational skills to prepare potential candidates for these positions. Although many teaching skills training programs for residents exist, there is a lack of reports in the literature evaluating similar programs during fellowship training.

AIM

To describe the implementation and evaluation of a unique program aimed at enhancing educational knowledge and teaching skills for subspecialty medicine fellows and chief residents.

SETTING

Fellows as Clinician-Educators (FACE) program is a 1-year program open to fellows (and chief residents) in the Department of Internal Medicine at the University of Iowa.

PROGRAM DESCRIPTION

The course involves interactive monthly meetings held throughout the academic year and has provided training to 48 participants across 11 different subspecialty fellowships between 2004 and 2009.

PROGRAM EVALUATION

FACE participants completed a 3-station Objective Structured Teaching Examination using standardized learners, which assessed participants' skills in giving feedback, outpatient precepting, and giving a mini-lecture. Based on reviews of station performance by 2 independent raters, fellows demonstrated statistically significant improvement on overall scores for 2 of the 3 cases. Participants self-assessed their knowledge and teaching skills prior to starting and after completing the program. Analyses of participants' retrospective preassessments and postassessments showed improved perceptions of competence after training.

CONCLUSION

The FACE program is a well-received intervention that objectively demonstrates improvement in participants' teaching skills. It offers a model approach to meeting important training skills needs of subspecialty medicine fellows and chief residents in a resource-effective manner.

A train the trainer model for integrating evidence-based medicine into a complementary and alternative medicine training program

BACKGROUND

Public interest in complementary and alternative medicine (CAM) has grown over the past decade, accompanied by increased demand for evidence-based approaches to CAM practice. In order to define the role evidence-based decision making has in CAM practice, CAM professionals must have a full understanding of evidence-based medicine (EBM) concepts.

OBJECTIVE

This paper describes the design, implementation, and evaluation of a week-long intensive EBM short course for CAM faculty at a naturopathic and classical Chinese medicine institution.

INTERVENTION

This 20-hour course, entitled Principles of EBM for CAM Professionals, teaches participants how to access and appraise biomedical literature, apply it to their work, and teach these concepts to their students.

RESULTS

Results from precourse and postcourse evaluations suggest that, in a small group of participants, there were significant changes in EBM practice attitudes, self-appraised skills, and objectively assessed skills as a result of this course. Participants indicated they were committed to increasing their use of EBM in practice, enhancing EBM skills, using EBM in teaching, and working to change the culture at their institution to support use of EBM. At six months, 80% of participants had fully or partially followed through on their commitment to change plans.

A critical appraisal of and recommendations for faculty development

The 2009-2010 American Association of Colleges of Pharmacy (AACP) Council of Faculties Faculty Affairs Committee reviewed published literature assessing the scope and outcomes of faculty development for tenure and promotion. Relevant articles were identified via a PubMed search, review of pharmacy education journals, and identification of position papers from major healthcare professions academic organizations. While programs intended to enhance faculty development were described by some healthcare professions, relatively little specific to pharmacy has been published and none of the healthcare professions have adequately evaluated the impact of various faculty-development programs on associated outcomes.The paucity of published information strongly suggests a lack of outcomes-oriented faculty-development programs in colleges and schools of pharmacy. Substantial steps are required toward the development and scholarly evaluation of faculty-development programs. As these programs are developed and assessed, evaluations must encompass all faculty subgroups, including tenure- and nontenure track faculty members, volunteer faculty members, women, and underrepresented minorities. This paper proposes AACP, college and school, and department-level recommendations intended to ensure faculty success in achieving tenure and promotion.

The Questionnaire "SFDP26-German": a reliable tool for evaluation of clinical teaching?

Aims: Evaluation of the effectiveness of clinical teaching is an important contribution for the quality control of medical teaching. This should be evaluated using a reliable instrument in order to be able to both gauge the status quo and the effects of instruction. In the Stanford Faculty Development Program (SFDP), seven categories have proven to be appropriate:

Establishing the Learning Climate,

Controlling a Teaching Session,

Communication of Goals,

Encouraging Understanding and Retention,

Evaluation,

Feedback and

Self-directed Learning.

Since 1998, the SFDP26 questionnaire has established itself as an evaluation tool in English speaking countries. To date there is no equivalent German-language questionnaire available which evaluates the overall effectiveness of teaching.

Question:

Development and theoretical testing of a German-language version of SFDP26 (SFDP26-German),

Check the correlation of subscale of SFDPGerman against overall effectiveness of teaching.

Methods: 19 anaesthetists (7 female, 12 male) from the University of Lübeck were evaluated at the end of a teaching seminar on emergency medical care using SFDP-German. The sample consisted of 173 medical students (119 female (68.8%) and 54 male (31.2%), mostly from the fifth semester (6.6%) and sixth semester (80.3%). The mean age of the students was 23±3 years.

Results: The discriminatory power of all items ranged between good and excellent (r_it=0.48-0.75). All subscales displayed good internal consistency (α=0.69-0.92) and significant positive inter-scale correlations (r=0.40-0.70). The subscales and “overall effectiveness of teaching” showed significant correlation, with the highest correlation for the subscale “communication of goals (p< 0.001; r = 0.61).

Conclusion: The analysis of SFDP26-German confirms high internal consistency. Future research should investigate the effectiveness of the individual categories on the overall effectiveness of teaching and validate according to external criteria.