Psychometric properties of an instrument to measure the clinical learning environment

Psychometric properties of the arabic version of PHEEM applied on a sample of medical residents in Syria

BACKGROUND

The clinical learning environment (CLE) plays a crucial role in shaping the learning experiences and professional development of medical professionals. Understanding and optimising this environment is essential for improving doctors' knowledge acquisition, clinical skills, and overall well-being. The development of the Postgraduate Hospital Educational Environment Measure (PHEEM) and its translation to numerous languages has been a milestone in clinical education. Even though PHEEM was recently translated into Arabic, its psychometric properties in this form remain unevaluated. Therefore, this study aims to conduct a comprehensive psychometric analysis of the Arabic version of the PHEEM questionnaire.

METHODS

This is a cross-sectional questionnaire survey validation study. The defined population were medical residents in Damascus, Syria. A paper-based survey as well as an online-based one were conducted using several non-probability sampling methods namely, convenience, river and, snowball sampling between June 15, 2023, and June 21, 2023. Both exploratory (EFA) and confirmatory (CFA) factor analyses were conducted. Several psychometric criteria were applied including scree plot, eigenvalue > 1.5 and the 'proportion of variance accounted for' criterion.

RESULTS

A total of 543 participants completed the questionnaire (56.9% female). Kaiser-Meyer-Olkin measure for sample adequacy was high (0.937) and the P-value for Bartlett's test was < 0.001. EFA revealed five meaningful factors which were labelled: perception of teachers, learner's engagement and social participation, external regulation, work culture, and living conditions. These factors had the following eigenvalues: 12.6, 2.18, 2.03, 1.86, and 1.41 respectively, with a total explained variance of 43.45%. Cronbach's Alpha was 0.938. CFA confirmed the model structure of EFA (SRMR = 0.067 and RMSEA = 0.066). The Average Variance Explained (AVE) value of any given factor was > 0.7.

DISCUSSION

The Arabic PHEEM inventory demonstrated satisfactory psychometric properties. The extracted domains are of theoretical relevance to the psychosocial-material conceptual framework for learning environment. Nonetheless, this validation was performed in the Syrian context; therefore, future studies in other Arabic countries are recommended to support the applicability of Arabic PHEEM in the wide Arab World.

Violence and Clinical Learning Environments in Medical Residencies

INTRODUCTION

The objective of this study was to describe and analyze residents' perceptions of characteristics on the expansive/restrictive continuum of their clinical learning environment.

METHODS

We conducted a quantitative, descriptive and cross-sectional study. A self-administered questionnaire was designed, programmed and applied to residents at the Faculty of Medicine of the National Autonomous University of Mexico. The instrument was structured in eight sections, and for this article, Section 3, which referred to clinical environments and violence was considered. The questionnaire had an 85% response rate, with 12,612 residents from 113 medical units and 78 specialties participating. The reliability and internal consistency measured with alpha omega obtained a value of ω 0.835 (CI; 0.828-0.843).

RESULTS

Unpleasant, competitive, tense and conflictive contexts were related to restrictive environments. Sexual orientation influenced the perception of intolerance in the clinical setting with respect to discriminatory comments, such that for gender minorities, the environment was experienced as exclusionary. First-year residents perceived environments as more aggressive, a perception that tended to decrease in later years of residency.

DISCUSSION

Abuses in power relations, rigid hierarchical positions and offensive clinical interactions may foster restrictive environments. In such settings, the reproduction of socio-culturally learned violence is feasible; however, asymmetrical relationships may be deconstructed and transformed.

Evaluation of learning environment of different family medicine programs, Jeddah-KSA, 2021

Background

A supportive educational climate in which learners are educated and monitored is a significant determinant of the medical trainees' performance and progress and eventually contributes to better patient care. Therefore, residents' training programs need to be assessed and incorporated according to the views and expectations of the candidates. The objectives of this research are to compare and evaluate the learning environment of Family Medicine (FM) programs in Jeddah and make practical recommendations for improvement of the learning environment.

Methods

A descriptive causal-comparative study was designed at FM programs in Jeddah, Saudi Arabia during January, and February 2021, including all residents from R1 to R4. A Postgraduate Hospital Educational Environment Measure (PHEEM) questionnaire was used in this study. The overall score and the three domains were calculated.

Results

Two hundred and eighty-three questionnaires were distributed, and 262 were completed and collected, making a response rate of 92.6%. Females represented were 150 (57.3%), and males were 112 (42.7%). The study shows that 78 (29.8%) of the residents reported that the learning environment was excellent, and 154 (58.8%) described it as a more positive than the negative environment, whereas 29 (11.1%) described it as having a lot of problems. Out of 160 maximum scores, the highest total score (127) was observed among residents at the National Guard program, whereas the lowest score (108) was in the FM joint program, P < 0.001.

Conclusion

Evaluation of the training programs is an essential component of quality assurance procedures. The satisfaction of the FM residents at different programs in Jeddah, Saudi Arabia, is better than reported previously, but however, further improvement is warranted. Juniors and residents of the National Guard program were more satisfied than their peers. We recommend low score items should have special consideration from high authorities and ongoing evaluation of the FM training program for further improvement of the learning environment.

When Feedback Backfires: Influences of Negative Discrepancies Between Physicians' Self and Assessors' Scores on Their Subsequent Multisource Feedback Ratings

INTRODUCTION

With multisource feedback (MSF) physicians might overrate their own performance compared with scores received from assessors. However, there is limited insight into how perceived divergent feedback affects physicians' subsequent performance scores.

METHODS

During 2012 to 2018, 103 physicians were evaluated twice by 684 peers, 242 residents, 999 coworkers, and themselves in three MSF performance domains. Mixed-effect models quantified associations between the outcome variable "score changes" between first and second MSF evaluations, and the explanatory variable "negative discrepancy score" (number of items that physicians rated themselves higher compared with their assessors' scores) at the first MSF evaluation. Whether associations differed across assessor groups and across a physician's years of experience as a doctor was analyzed too.

RESULTS

Forty-nine percent of physicians improved their total MSF score at the second evaluation, as assessed by others. Number of negative discrepancies was negatively associated with score changes in domains "organization and (self)management" (b = -0.02; 95% confidence interval [CI], -0.03 to -0.02; SE = 0.004) and "patient-centeredness" (b = -0.03; 95% CI, -0.03 to -0.02; SE = 0.004). For "professional attitude," only negative associations between score changes and negative discrepancies existed for physicians with more than 6-year experience (b6-10yearsofexperience = -0.03; 95% CI, -0.05 to -0.003; SE = 0.01; b16-20yearsofexperience = -0.03; 95% CI, -0.06 to -0.004; SE = 0.01).

DISCUSSION

The extent of performance improvement was less for physicians confronted with negative discrepancies. Performance scores actually declined when physicians overrated themselves on more than half of the feedback items. PA score changes of more experienced physicians confronted with negative discrepancies and were affected more adversely. These physicians might have discounted feedback due to having more confidence in own performance. Future work should investigate how MSF could improve physicians' performance taking into account physicians' confidence.

Mainstreaming genomics: training experience of hospital medical officers at the Royal Melbourne Hospital

The rapid evolution and wide applicability of genomic testing means that medical practitioners outside the field are not appropriately skilled to understand the utility of genomics for their patients. Rotating junior doctors through genomic medicine provides them with the hands-on experience necessary to understand the complexities in this field. In this study, we analysed the training experience of 12 hospital medical officers who rotated through genomic medicine at the Royal Melbourne Hospital. Here, we demonstrate that immersion in clinical genomics aids in mainstreaming genomics knowledge.

Validation of the French translation of the Dutch residency educational climate test

BACKGROUND

The learning environment is one of the most influential factors in training of medical residents. The Dutch Residency Educational Climate Test (D-RECT) is one of the strongest instruments for measuring the learning environment. However, it has not been translated in French. The objective of this study is the psychometric validation of the DRECT French version.

MATERIAL AND METHODS

After translation of the D-RECT questionnaire into French, residents of five Moroccan hospitals were invited to complete the questionnaire between July and September 2018. Confirmatory factor analysis was used to evaluate the validity of the construct using the standardized root mean square residual (SRMR), the root mean square error approximation (RMSEA), the Comparative Fit Index (CFI) and the Tucker- Lewis Index (TLI). Reliability analysis was analysed using Internal consistency and Test-retest.

RESULTS

During the study period, 211 residents completed the questionnaire. Confirmatory factor analysis showed an adequate model fit with the following indicators: SRMR = 0.058 / RMSEA = 0.07 / CFI = 0.88 / TLI = 0.87. The French translation had a good internal consistency (Cronbach alpha score >  0.7 for all subscales) and a good temporal stability (correlation score between two measurements = 0.89).

CONCLUSION

This French version has an acceptable validity of the construct, a good internal consistency and good temporal reliability, and may be used to evaluate the learning climate. Additional research is necessary in other French-speaking contexts, in order to confirm these results.

Standard errors of two-level scalability coefficients

For the construction of tests and questionnaires that require multiple raters (e.g., a child behaviour checklist completed by both parents) a novel ordinal scaling technique is currently being further developed, called two-level Mokken scale analysis. The technique uses within-rater and between-rater coefficients to assess the scalability of the test. These coefficients are generalizations of Mokken's scalability coefficients. In this paper we derived standard errors for the two-level coefficients and for their ratios. The coefficients, the estimates, the estimated standard errors and the software implementation are discussed and illustrated using a real-data example, and a small-scale simulation study demonstrates the accuracy of the estimates.

Validity of the scan of postgraduate educational environment domains (SPEED) questionnaire in a rural general practice training setting

BACKGROUND

The educational environment is critical to learning and is determined by social interactions. Trainee satisfaction translates to career commitment, retention and a positive professional attitude as well as being an important factor in assessing the impact of the training program. This study aimed to validate the Scan of Postgraduate Educational Environment Domain (SPEED) tool and assess its appropriateness in evaluating the quality of General Practice (GP) rural postgraduate educational environment.

METHODS

A questionnaire containing the 15-item SPEED tool was administered to GP registrars to examine their perceptions of the educational environment. Principal component analysis (PCA) and exploratory factor analysis (EFA) were used to gather evidences of the validity of the instrument based on its internal structure. Additional validity evidence and reliability estimates were obtained using many-facet Rasch model analysis (MFRM).

RESULTS

The survey was completed by 351 registrars with a response rate of 60%. Parallel analysis performed using principal component analysis and exploratory factor analysis suggests that the SPEED tool is unidimensional. The MFRM analysis demonstrated an excellent degree of infit and outfit for items and training sites, but not for persons. The MFRM analysis also estimated high reliability levels for items (0.98), training sites (0.95) and persons within training sites (ranging from 0.87 to 0.93 in each training sites). Overall, the registrars agreed that the educational environment had high quality, with most (13 out of 15) of the items rated above 4 out of 5.

CONCLUSIONS

This study demonstrated a high degree of validity and reliability of the SPEED tool for the measurement of the quality of the educational environment in a rural postgraduate GP training context. However, when applied in a new setting, the tool may not function as a multidimensional tool consistent with its theoretical grounding.

Health system dynamics analysis of eyecare services in Trinidad and Tobago and progress towards Vision 2020 Goals

Avoidable blindness is an important global public health concern. This study aimed to assess Trinidad and Tobago's progress towards achieving the Pan American Health Organization, 'Strategic Framework for Vision 2020: The Right to Sight-Caribbean Region,' indicators through comprehensive review of the eyecare system, in order to facilitate health system priority setting. We administered structured surveys to six stakeholder groups, including eyecare providers, patients and older adult participants in the National Eye Survey of Trinidad and Tobago. We reviewed reports, registers and policy documents, and used a health system dynamics framework to synthesize data. In 2014, the population of 1.3 million were served by a pluralistic eyecare system, which had achieved 14 out of 27 Strategic Framework indicators. The Government provided free primary, secondary and emergency eyecare services, through 108 health centres and 5 hospitals (0.26 ophthalmologists and 1.32 ophthalmologists-in-training per 50 000 population). Private sector optometrists (4.37 per 50 000 population), and ophthalmologists (0.93 per 50 000 population) provided 80% of all eyecare. Only 19.3% of the adult population had private health insurance, revealing significant out-of-pocket expenditure. We identified potential weaknesses in the eyecare system where investment might reduce avoidable blindness. These included a need for more ophthalmic equipment and maintenance in the public sector, national screening programmes for diabetic retinopathy, retinopathy of prematurity and neonatal eye defects, and pathways to ensure timely and equitable access to subspecialized surgery. Eyecare for older adults was responsible for an estimated 9.5% (US$22.6 million) of annual health expenditure. This study used the health system dynamics framework and new data to identify priorities for eyecare system strengthening. We recommend this approach for exploring potential health system barriers to addressing avoidable blindness, and other important public health problems.

The learning environment of paediatric interns in South Africa

BACKGROUND

South African (SA) paediatric interns (recently qualified medical graduates) work in a high disease burdened and resource deficient environment for two years, prior to independent practice. Perceptions of this learning environment (LE) influences their approaches to training as well as the outcomes of this period of development. Obstacles to creating a supportive LE and supervisor interaction affects the quality of this training. Measuring perceptions of the LE with validated instruments can help inform improvements in learning during this crucial period of medical education.

METHODS

The aims of this study was to determine the psychometric qualities of the Postgraduate Hospital Educational Environment Measure (PHEEM) amongst paediatric interns across four hospital complexes in South Africa and to measure the LE as perceived by both interns and their supervisors. Construct validity was tested using factor analysis and internal consistency was measured with Cronbach's alpha.

RESULTS

A total of 209 interns and 60 supervisors (69% intern response rate) responded to the questionnaire. The PHEEM was found to be very reliable with an overall Cronbach's alpha of 0.943 and 0.874 for intern and supervisors respectively. Factor analysis using a 3-factor solution accounted for 42% of the variance with the teaching subscale having the best fit compared with the other sub-scales of the original tool. Most interns perceived the learning environment as being more positive than negative however, their perceptions differed significantly from that of their supervisors. Poor infrastructural support from institutions, excessive workloads and inadequate supervision were factors preventing optimal training of paediatric interns.

CONCLUSIONS

The SA version of the PHEEM tool used was found to be a reliable and valid instrument for use in interns amongst high disease burdened contexts. Various obstacles to creating an ideal learning environment for paediatric interns were identified to be in need of urgent review. Key differences in perceptions of an ideal learning environment between interns and their supervisors need to be fully explored as these may result in sub-optimal supervision and mentoring.

Novice students navigating the clinical environment in an early medical clerkship

CONTEXT

The black box that is student learning in clinical environments is an ongoing research project. Our previous research showed that despite the time that students are given to learn with, about and from patients, some lack confidence for those encounters and see few patients. The study reported here investigated individual and environmental factors affecting medical students' self-directed learning time in hospital.

METHODS

We studied second year students in the four-year postgraduate Melbourne Medical School programme as they undertook the first of their four 9-week hospital placements in medical wards. Each week approximately 10 hours of structured teaching is offered; the remaining time is spent in self-directed learning. Over six weeks, we observed 31 medical students and interviewed 17 of them. The interviews were subjected to content analysis procedures and the observation notes added contextual information to what was said in interviews. We considered the findings through the Experience-based Learning framework.

RESULTS

We found four main themes in the data: finding and contacting patients challenges all students and overwhelms some; the educational design of the placement is a flawed navigational device providing inadequate clarity and security; the physical and social terrain of a large tertiary hospital is replete with obstacles making it easy for some students to stumble and retreat; finally, any positive connection with peers, staff and patients is empowering.

CONCLUSIONS

This study throws light on to the uncertain path of the novice clinical learner illuminating both the intractable aspects of hospital environments and opportunities for pedagogical and affective supports that can compensate. The processes and conditions for self-directed learning time need attention in order to provide for a safe, efficient or successful clerkship experience for all students. Particular effort is needed to sensitively identify those individuals who struggle and suffer on the journey, and provide appropriate support.

Validation of the INCEPT: A Multisource Feedback Tool for Capturing Different Perspectives on Physicians' Professional Performance

INTRODUCTION

Multisource feedback (MSF) instruments are used to and must feasibly provide reliable and valid data on physicians' performance from multiple perspectives. The "INviting Co-workers to Evaluate Physicians Tool" (INCEPT) is a multisource feedback instrument used to evaluate physicians' professional performance as perceived by peers, residents, and coworkers. In this study, we report on the validity, reliability, and feasibility of the INCEPT.

METHODS

The performance of 218 physicians was assessed by 597 peers, 344 residents, and 822 coworkers. Using explorative and confirmatory factor analyses, multilevel regression analyses between narrative and numerical feedback, item-total correlations, interscale correlations, Cronbach's α and generalizability analyses, the psychometric qualities, and feasibility of the INCEPT were investigated.

RESULTS

For all respondent groups, three factors were identified, although constructed slightly different: "professional attitude," "patient-centeredness," and "organization and (self)-management." Internal consistency was high for all constructs (Cronbach's α ≥ 0.84 and item-total correlations ≥ 0.52). Confirmatory factor analyses indicated acceptable to good fit. Further validity evidence was given by the associations between narrative and numerical feedback. For reliable total INCEPT scores, three peer, two resident and three coworker evaluations were needed; for subscale scores, evaluations of three peers, three residents and three to four coworkers were sufficient.

DISCUSSION

The INCEPT instrument provides physicians performance feedback in a valid and reliable way. The number of evaluations to establish reliable scores is achievable in a regular clinical department. When interpreting feedback, physicians should consider that respondent groups' perceptions differ as indicated by the different item clustering per performance factor.

Adoption and correlates of Postgraduate Hospital Educational Environment Measure (PHEEM) in the evaluation of learning environments - A systematic review. MEDICAL TEACHER 2016;38:1248-1255. [PMID: 27564848 DOI: 10.1080/0142159x.2016.1210108]

BACKGROUND

The Postgraduate Hospital Educational Environment Measure (PHEEM) is a highly reliable and valid instrument to measure the educational environment during post graduate medical training. This review extends earlier reports by evaluating the extant adoption of PHEEM in various international clinical training sites, and its significant correlations in order to expand our understanding on the use of PHEEM and facilitate future applications and research.

METHOD

A systematic literature review was conducted on all articles between 2005 and October 2015 that adopted and reported data using the PHEEM.

RESULTS

Overall 30 studies were included, encompassing data from 14 countries internationally. Notable differences in the PHEEM scores were found between different levels of training, disciplines, and clinical training sites. Common strengths and weaknesses in learning environments were observed and there were significant correlations between PHEEM scores and In-Training Exam (ITE) performance (positive correlation) and level of burnout (negative correlation), respectively.

CONCLUSIONS

PHEEM is widely adopted in different learning settings, and is a useful tool to identify the strengths and weaknesses of an educational environment. Future research can examine other correlates of PHEEM and longitudinal changes in interventional studies.

A multicenter study: how do medical students perceive clinical learning climate?

BACKGROUND

The relationship between students and instructors is of crucial importance for the development of a positive learning climate. Learning climate is a multifaceted concept, and its measurement is a complicated process. The aim of this cross-sectional study was to determine medical students' perceptions about the clinical learning climate and to investigate differences in their perceptions in terms of various variables.

METHODS

Medical students studying at six medical schools in Turkey were recruited for the study. All students who completed clinical rotations, which lasted for 3 or more weeks, were included in the study (n=3,097). Data were collected using the Clinical Learning Climate Scale (CLCS). The CLCS (36 items) includes three subscales: clinical environment, emotion, and motivation. Each item is scored using a 5-point Likert scale (1: strongly disagree to 5: strongly agree).

RESULTS

The response rate for the trainees was 69.67% (n=1,519), and for the interns it was 51.47% (n=917). The mean total CLCS score was 117.20±17.19. The rotation during which the clinical learning climate was perceived most favorably was the Physical Therapy and Rehabilitation rotation (mean score: 137.77). The most negatively perceived rotation was the General Internal Medicine rotation (mean score: 104.31). There were significant differences between mean total scores in terms of trainee/intern characteristics, internal medicine/surgical medicine rotations, and perception of success.

CONCLUSION

The results of this study drew attention to certain aspects of the clinical learning climate in medical schools. Clinical teacher/instructor/supervisor, clinical training programs, students' interactions in clinical settings, self-realization, mood, students' intrinsic motivation, and institutional commitment are important components of the clinical learning climate. For this reason, the aforementioned components should be taken into consideration in studies aiming to improve clinical learning climate.

How educational innovations and attention to competencies in postgraduate medical education relate to preparedness for practice: the key role of the learning environment

INTRODUCTION

Many training programmes in postgraduate medical education (PGME) have introduced competency frameworks, but the effects of this change on preparedness for practice are unknown. Therefore, we explored how elements of competency-based programmes in PGME (educational innovations, attention to competencies and learning environment) were related to perceived preparedness for practice among new consultants.

METHODS

A questionnaire was distributed among 330 new consultants. Respondents rated how well their PGME training programme prepared them for practice, the extent to which educational innovations (portfolio, Mini-CEX) were implemented, and how much attention was paid to CanMEDS competencies during feedback and coaching, and they answered questions on the learning environment and general self-efficacy. Multiple regression and mediation analyses were used to analyze data.

RESULTS

The response rate was 43 % (143/330). Controlling for self-efficacy and gender, the learning environment was the strongest predictor of preparedness for practice (B = 0.42, p < 0.001), followed by attention to competencies (B = 0.29, p < 0.01). Educational innovations were not directly related to preparedness for practice. The overall model explained 52 % of the variance in preparedness for practice. Attention to competencies mediated the relationship between educational innovations and preparedness for practice. This mediation became stronger at higher learning environment values.

CONCLUSIONS

The learning environment plays a key role in determining the degree to which competency-based PGME prepares trainees for independent practice.

Validation of the Postgraduate Hospital Educational Environment Measure (PHEEM) in a sample of 731 Greek residents

Background

The Greek version of the Postgraduate Hospital Educational Environment Measure (PHEEM) was evaluated to determine its psychometric properties, i.e., validity, internal consistency, sensitivity and responsiveness to be used for measuring the learning environment in Greek hospitals.

Methods

The PHEEM was administered to Greek hospital residents. Internal consistency was measured using Cronbach’s alpha. Root Mean Square Error of Approximation (RMSEA) was used to evaluate the fit of Structural Equation Models. Content validity was addressed by the original study. Construct validity was tested using confirmatory (to test the set of underlying dimensions suggested by the original study) and exploratory (to explore the dimensions needed to explain the variability of the given answers) factor analysis using Varimax rotation. Convergent validity was calculated by Pearson’s correlation coefficient regarding the participant’s PHEEM score and participant’s overall satisfaction score of the added item “Overall, I am very satisfied with my specialization in this post”. Sensitivity was checked by comparing good versus poor aspects of the educational environment and by satisfied versus unsatisfied participants.

Results

A total of 731 residents from 83 hospitals and 41 prefectures responded to the PHEEM. The original three-factor model didn’t fit better compared to one factor model that is accounting for 32 % of the variance. Cronbach’s α was 0.933 when assuming one-factor model. Using a three-factor model (autonomy, teaching, social support), Cronbach’s α were 0.815 (expected 0.830), 0.908 (0.839), 0.734 (0.793), respectively. The three-factor model gave an RMSEA value of 0.074 (90 % confidence interval 0.071, 0.076), suggesting a fair fit. Pearson’s correlation coefficient between total PHEEM and global satisfaction was 0.765. Mean question scores ranged from 19.0 (very poor) to 73.7 (very good), and mean participant scores from 5.5 (very unsatisfied) to 96.5 (very satisfied).

Conclusions

The Greek version of PHEEM is a valid, reliable, and sensitive instrument measuring the educational environment among junior doctors in Greek hospitals and it can be used for evidence-based SWOT analysis and policy.

What motivates young physicians? - a qualitative analysis of the learning climate in specialist medical training

BACKGROUND

Not least the much-invoked shortage of physicians in the current and the next generation has resulted in a wide range of efforts to improve postgraduate medical training. This is also in the focus of the current healthcare policy debate. Furthermore, quality and scope of available postgraduate training are important locational advantages in the competition for medical doctors. This study investigates the preferences and concerns that German house officers (HOs) have about their current postgraduate training. It also highlights how HOs evaluate the quality of their current postgraduate training and the learning environment.

METHODS

HOs were asked to answer the question: "Which things are of capital importance to you personally in your medical training?", using a free text format. The survey was conducted web based (Lime survey) and all data was anonymized. Summarizing qualitative analyses were performed using the software tool MaxQDA.

RESULTS

A total of 255 HOs participated in this study (female: n = 129/50.6 %; male: n = 126/49.4 %; age: 32 + 6 years) associated with 17 different German hospitals and from four medical specialties. Ten categories were generated from a total of 366 free text answers: 1. methodology of learning (n = 66), 2. supervision (n = 66), 3. learning structure (n = 61), 4. teaching competence (n = 37), 5. dedication (n = 34), 6. work climate (n = 29), 7. feedback/communication (n = 22), 8. challenge/patient safety (n = 21), 9. time/resources (n = 17), 10. personal security/safety (n = 13).

CONCLUSIONS

HOs want a reliable and curriculum-guided learning structure. Different studying techniques should be used with sufficient (time) resources available in a trusting and communicative learning environment. Competent and dedicated instructors are expected to give individual and specific feedback to the HOs on individual strengths and deficits. Instructors should develop educational concepts in cooperation with the HOs and at the same time avoid excessive demands on HOs or hazards to patients.

The threat of the dyscompetent resident: A plea to make the implicit more explicit!

Although several examples of frameworks dealing with students' unprofessional behaviour are available, guidance on how to deal locally or regionally with dysfunctional residents is limited (Hickson et al. in Acad Med 82(11):1040-1048, 2007b; Leape and Fromson in Ann Intern Med 144(2):107-115, 2006). Any 'rules' are mostly unwritten, and often emerge by trial and error within the specialty training programme (Stern and Papadakis in N Engl J Med 355(17):1794-1799, 2006). It is nevertheless of utmost importance that objectives, rules and guidelines comparable to those existing in undergraduate training (Project Team Consilium Abeundi van Luijk in Professional behaviour: teaching, assessing and coaching students. Final report and appendices. Mosae Libris, 2005; van Mook et al. in Neth J Crit Care 16(4):162-173, 2010a) are developed for postgraduate training. And that implicit rules are made explicit. This article outlines a framework based on the lessons learned from contemporary postgraduate medical training programmes.

Evaluation of hospital-learning environment for pediatric residency in eastern region of Saudi Arabia

PURPOSE

No study had been conducted to assess the hospitals' environment for learning purposes in multicenter sites in Saudi Arabia. It aims to evaluate the environment of hospitals for learning purposes of pediatric residents.

METHODS

We applied Postgraduate Hospital Educational Environment Measure (PHEEM) to measure the learning environment at six teaching hospitals in the Eastern Region of Saudi Arabia from September to December 2013.

RESULTS

The number of respondents was 104 (86.7%) out of 120 residents and 37 females and 67 male residents have responded. The residents' response scored 100 out of 160 maximum score in rating of PHEEM that showed overall learning environment is favorable for training. There were some items in the social support domain suggesting improvements. There was no significant difference between male and female residents. There was a difference among the participant teaching hospitals (p<0.05).

CONCLUSION

The result pointed an overall positive rating. Individual item scores suggested that their social life during residency could be uninspiring. They have the low satisfactory level and they feel racism, and sexual discrimination. Therefore, there is still a room for improvement.

Evaluation of the educational environment of the Saudi family medicine residency training program

Objectives:

The study was conducted to evaluate the educational environment (EE) in Family Medicine Training Programs.

Materials and Methods:

A cross-sectional survey, The Postgraduate Hospital Educational Environment Measure (PHEEM), was distributed to all residents at the four training centers in the central region. Cronbach's alpha was used to test the reliability. The mean and standard deviation (SD) for each item, the overall score and the three domains were calculated. A multiple linear regression model was developed with PHEEM scores as an outcome. The Mann–Whitney–Wilcoxon test was used to compare each item based on the selected factors.

Results:

The overall score was 67.1/160 (SD: 20.1). The PHEEM's domains scores: 24.2/56 (SD: 7.13) for perception of role autonomy; 25.3/60 (SD: 8.88), for perception of teaching; and 17/44 (SD: 5.6), for perception of social support. Training center and Level of training were the significant outcome predictors. Centre 1 (Joint Program) significantly had better scores than Centre 2. The instrument showed great reliability with a Cronbach's alpha of 0.92.

Conclusions:

There are many problems in the training program. Urgent actions are needed to improve the residents' learning experience particularly during rotations. Also, the curriculum should be restructured, and effective training methods introduced using the Best Evidence in Medical Education to meet the expectations and learning needs of family physicians.

The Work Place Educational Climate in Gynecological Oncology Fellowships Across Europe: The Impact of Accreditation

Background

A good educational climate/environment in the workplace is essential for developing high-quality medical (sub)specialists. These data are lacking for gynecological oncology training.

Objective

This study aims to evaluate the educational climate in gynecological oncology training throughout Europe and the factors affecting it.

Methods

A Web-based anonymous survey sent to ENYGO (European Network of Young Gynecological Oncologists) members/trainees to assess gynecological oncology training. This included sociodemographic information, details regarding training posts, and a 50-item validated Dutch Residency Educational Climate Test (D-RECT) questionnaire with 11 subscales (1–5 Likert scale) to assess the educational climate. The χ2 test was used for evaluating categorical variables, and the Mann-Whitney U (nonparametric) test was used for continuous variables between 2 independent groups. Cronbach α assessed the questionnaire reliability. Multivariable linear regression assessed the effect of variables on D-RECT outcome subscales.

Results

One hundred nineteen gynecological oncological fellows responded. The D-RECT questionnaire was extremely reliable for assessing the educational environment in gynecological oncology (subscales’ Cronbach α, 0.82–0.96). Overall, trainees do not seem to receive adequate/effective constructive feedback during training. The overall educational climate (supervision, coaching/assessment, feedback, teamwork, interconsultant relationships, formal education, role of the tutor, patient handover, and overall consultant’s attitude) was significantly better (P = 0.001) in centers providing accredited training in comparison with centers without such accreditation. Multivariable regression indicated the main factors independently associated with a better educational climate were presence of an accredited training post and total years of training.

Conclusions

This study emphasizes the need for better feedback mechanisms and the importance of accreditation of centers for training in gynecological oncology to ensure training within higher quality clinical learning climates.

Clinical learning environment measurement for medical trainees at transitions: relations with socio-cultural factors and mental distress

BACKGROUND

Measuring clinical learning environment is crucial for the quality improvement of medical education, especially for medical trainees at transition stages. Medical education in Taiwan is shaped by multiple socio-cultural influences. The aims of this study were to construct an instrument for measuring clinical learning environment in alignment with the local socio-cultural factors and medical training settings, and further investigate the relationship between the quality of the clinical learning environment and the status of mental distress among medical trainees.

METHODS

Participants consisted of 189 medical trainees (62 interns, 52 postgraduate year one (PGY1) residents, and 75 senior residents). Instruments included a designed 40-item Clinical Learning Environment Questionnaire (CLENQ) and a five-item Brief Symptoms Rating Scale (BSRS-5) for evaluating mental distress. Constructs of CLENQ were examined using factor analysis. Correlations were calculated between BSRS-5 and CLENQ across the three levels of medical trainees.

RESULTS

Factor analysis of CLENQ yielded five factors- I: Teaching (13 items), II: Workload (7 items), III: Relationship pressure (9 items), IV: Organisational support (4 items) and V: Mutual trust (6 items). Intern trainees reported the lower total CLENQ scores in comparison to PGY1 and senior resident trainees. Mental distress using BSRS-5 was negatively correlated with total CLENQ scores and several key factors in all three groups.

CONCLUSIONS

Our study using CLENQ has identified five major factors of clinical learning environment that are closely linked with our local socio-cultural factors and medical training settings. Medical trainee's mental distress status was negatively correlated with the quality of CLENQ. These findings have socio-cultural relevance and medical contextual significance and might be applicable to other countries. It warrants further study to investigate the impact of clinical learning environment improvement on the medical trainee's mental distress and performance.

Development and psychometric evaluation of the Undergraduate Clinical Education Environment Measure (UCEEM)

BACKGROUND

There is a paucity of instruments designed to evaluate the multiple dimensions of the workplace as an educational environment for undergraduate medical students.

AIM

The aim was to develop and psychometrically evaluate an instrument to measure how undergraduate medical students perceive the clinical workplace environment, based on workplace learning theories and empirical findings.

METHOD

Development of the instrument relied on established standards including theoretical and empirical grounding, systematic item development and expert review at various stages to ensure content validity. Qualitative and quantitative methods were employed using a series of steps from conceptualization through psychometric analysis of scores in a Swedish medical student population.

RESULTS

The final result was a 25-item instrument with two overarching dimensions, experiential learning and social participation, and four subscales that coincided well with theory and empirical findings: Opportunities to learn in and through work & quality of supervision; Preparedness for student entry; Workplace interaction patterns & student inclusion; and Equal treatment. Evidence from various sources supported content validity, construct validity and reliability of the instrument.

CONCLUSION

The Undergraduate Clinical Education Environment Measure represents a valid, reliable and feasible multidimensional instrument for evaluation of the clinical workplace as a learning environment for undergraduate medical students. Further validation in different populations using various psychometric methods is needed.

The hospital educational environment and performance of residents in the General Medicine In-Training Examination: a multicenter study in Japan

Background

It is believed that the type of educational environment in teaching hospitals may affect the performance of medical knowledge base among residents, but this has not yet been proven.

Objective

We aimed to investigate the association between the hospital educational environment and the performance of the medical knowledge base among resident physicians in Japanese teaching hospitals.

Methods

To assess the knowledge base of medicine, we conducted the General Medicine InTraining Examination (GM-ITE) for second-year residents in the last month of their residency. The items of the exam were developed based on the outcomes designated by the Japanese Ministry of Health, Labor, and Welfare. The educational environment was evaluated using the Postgraduate Hospital Educational Environment Measure (PHEEM) score, which was assessed by a mailed survey 2 years prior to the exam. A mixed-effects linear regression model was employed for the analysis of variables associated with a higher score.

Results

Twenty-one teaching hospitals participated in the study and a total of 206 residents (67 women) participated and completed the exam. There were no residents who declined to participate in the exam. The mean GM-ITE score was 58 (standard deviation 8.4). The mixed-effects linear regression analysis showed that a higher PHEEM score was associated with a higher GM-ITE score (P = 0.02). Having a department of general medicine, and hospital location in a provincial community (versus an urban setting), were also shown to have a significant relationship with the higher score (P = 0.03, and P = 0.02, respectively).

Conclusion

We found that the performance of the medical knowledge base of resident physicians was significantly associated with the educational environment of their hospitals. Improvement of the educational environment in teaching hospitals might be crucial for enhancing the performance of resident physicians in Japan.

Attributes of residents as teachers and role models - a mixed methods study of stakeholders

BACKGROUND

Residents are at the forefront of student education in the hospital, yet valid tools to assess their performance as teachers are lacking.

AIMS

To develop a valid evaluation tool for assessing resident performance as educators for clerkship students.

METHOD

A mixed-methods design was used. Focus groups of residents and medical students explored desired behaviors in resident educators. Using grounded theory, a list of behaviors was generated inductively through iterative review and categorized into themes. After thematic saturation, behaviors were rated on a Likert scale by stakeholders based on "importance" and "accuracy of measurement." Items which were both important and accurate were used in the final tool.

RESULTS

Eighty-five desirable behaviors for resident educators were identified and consolidated into a 14-item tool. Twenty met both "importance" and "accuracy" criteria and fell under themes of respect, safe environment, balancing supervision with autonomy, relevant teaching and feedback. Nineteen "important" behaviors deemed not accurately measurable fell under themes of professionalism, communication, management skills and leadership.

CONCLUSIONS

Evaluation of residents as teachers and development of resident-as-teacher curricula should emphasize aforementioned areas. Professionalism and organizational skills may not be measurable reliably by learners. Complementary tools to assess these aspects of resident performance are necessary.

Factor analysis methods and validity evidence: a review of instrument development across the medical education continuum

PURPOSE

Instrument development consistent with best practices is necessary for effective assessment and evaluation of learners and programs across the medical education continuum. The author explored the extent to which current factor analytic methods and other techniques for establishing validity are consistent with best practices.

METHOD

The author conducted electronic and hand searches of the English-language medical education literature published January 2006 through December 2010. To describe and assess current practices, she systematically abstracted reliability and validity evidence as well as factor analysis methods, data analysis, and reported evidence from instrument development articles reporting the application of exploratory factor analysis and principal component analysis.

RESULTS

Sixty-two articles met eligibility criteria. They described 64 instruments and 95 factor analyses. Most studies provided at least one source of evidence based on test content. Almost all reported internal consistency, providing evidence based on internal structure. Evidence based on response process and relationships with other variables was reported less often, and evidence based on consequences of testing was not identified. Factor analysis findings suggest common method selection errors and critical omissions in reporting.

CONCLUSIONS

Given the limited reliability and validity evidence provided for the reviewed instruments, educators should carefully consider the available supporting evidence before adopting and applying published instruments. Researchers should design for, test, and report additional evidence to strengthen the argument for reliability and validity of these measures for research and practice.

Evaluation and feedback for effective clinical teaching in postgraduate medical education: validation of an assessment instrument incorporating the CanMEDS roles

BACKGROUND

Providing clinical teachers in postgraduate medical education with feedback about their teaching skills is a powerful tool to improve clinical teaching. A systematic review showed that available instruments do not comprehensively cover all domains of clinical teaching. We developed and empirically test a comprehensive instrument for assessing clinical teachers in the setting of workplace learning and linked to the CanMEDS roles.

METHODS

In a Delphi study, the content validity of a preliminary instrument with 88 items was studied, leading to the construction of the EFFECT (evaluation and feedback for effective clinical teaching) instrument. The response process was explored in a pilot test and focus group research with 18 residents of 6 different disciplines. A confirmatory factor analyses (CFA) and reliability analyses were performed on 407 evaluations of 117 supervisors, collected in 3 medical disciplines (paediatrics, pulmonary diseases and surgery) of 6 departments in 4 different hospitals.

RESULTS

CFA yielded an 11 factor model with a good to excellent fit and internal consistencies ranged from 0.740 to 0.940 per domain; 7 items could be deleted.

CONCLUSION

The model of workplace learning showed to be a useful framework for developing EFFECT, which incorporates the CanMEDS competencies and proved to be valid and reliable.

Workplace learning

This critical review found Dutch research to be strong at the undergraduate and residency levels and more or less absent in continuing medical education. It confirms the importance of coaching medical students, giving constructive feedback, and ensuring practice environments are conducive to learning though it has proved hard to improve them. Residents learn primarily from experiences encountered in the course of clinical work but the fine balance between delivering clinical services and learning can easily be upset by work pressure. More intervention studies are needed. Qualitative research designs need to be more methodologically sophisticated and use a wider range of data sources including direct observation, audio-diaries, and text analysis. Areas for improvement are clear but achieving results will require persistence and patience.

Building bridges between theory and practice in medical education using a design-based research approach: AMEE Guide No. 60

Medical education research has grown enormously over the past 20 years, but it does not sufficiently make use of theories, according to influential leaders and researchers in this field. In this AMEE Guide, it is argued that design-based research (DBR) studies should be conducted much more in medical education design research because these studies both advance the testing and refinement of theories and advance educational practice. In this Guide, the essential characteristics of DBR as well as how DBR differs from other approach such as formative evaluation are explained. It is also explained what the pitfalls and challenges of DBR are. The main challenges deal with how to insure that DBR studies reveal findings that are of a broader relevance than the local situation and how to insure that DBR contributes toward theory testing and refinement. An example of a series of DBR studies on the design of a teaching portfolio in higher education that is aimed at stimulating a teacher's professional development is described, to illustrate how DBR studies actually work in practice. Finally, it is argued that DBR-studies could play an important role in the advancement of theory and practice in the two broad domains of designing or redesigning work-based learning environments and assessment programs.

What factors affect students' overall ratings of a course?

PURPOSE

Medical students are typically asked to complete course evaluations, but little is known about how students decide to rate courses. This study sought to examine the student feedback process by exploring the dimensionality of a course evaluation tool and examining the relationship between resulting factors and the overall rating of a course.

METHOD

During the 2007-2008 academic year, all first- and second-year students were asked to provide feedback on various aspects of curricular content, delivery, and assessment for seven courses taught in the first two years of a clinical presentation curriculum. The authors examined the structure of the evaluation instrument using principal component factor analysis and used multiple linear regression to study the relationship between factors and overall course ratings.

RESULTS

Four stable and reliable factors were identified (assessment of students, small-group learning, basic science teaching, and teaching diagnostic approaches) that accounted for about 50% of the total variance in overall course ratings. Student assessment displayed the strongest association with overall course ratings, and for second-year students it was the only variable associated with overall course ratings.

CONCLUSIONS

Of the four factors, student assessment was by far the strongest predictor of overall course ratings, and this association strengthened over time. These results are consistent with the "peak-end rule" and "negativity dominance" for rating emotional experiences.

Development and analysis of D-RECT, an instrument measuring residents' learning climate

BACKGROUND

Measurement of learning climates can serve as an indicator of a department's educational functioning.

AIM

This article describes the development and psychometric qualities of an instrument to measure learning climates in postgraduate specialist training: the Dutch Residency Educational Climate Test (D-RECT).

METHOD

A preliminary questionnaire was evaluated in a modified Delphi procedure. Simultaneously, all residents in the Netherlands were invited to fill out the preliminary questionnaire. We used exploratory factor analysis to analyze the outcomes and construct the definitive D-RECT. Confirmatory factor analysis tested the questionnaire's goodness of fit. Generalizability studies tested the number of residents needed for a reliable outcome.

RESULTS

In two rounds, the Delphi panel reached consensus. In addition, 1278 residents representing 26 specialties completed the questionnaire. The Delphi panel's input in combination with the exploratory factor analysis of 600 completed surveys led to the definitive D-RECT, consisting of 50 items and 11 subscales (e.g., feedback, supervision, patient handover and professional relations between attendings). Confirmatory factor analyses of the remaining surveys confirmed the construct. The results showed that a feasible number of residents is needed for a reliable outcome.

CONCLUSION

D-RECT appears to be a valid, reliable and feasible tool to measure the quality of clinical learning climates.

Educational climate seems unrelated to leadership skills of clinical consultants responsible of postgraduate medical education in clinical departments

BACKGROUND

The educational climate is crucial in postgraduate medical education. Although leaders are in the position to influence the educational climate, the relationship between leadership skills and educational climate is unknown. This study investigates the relationship between the educational climate in clinical departments and the leadership skills of clinical consultants responsible for education.

METHODS

The study was a trans-sectional correlation study. The educational climate was investigated by a survey among all doctors (specialists and trainees) in the departments. Leadership skills of the consultants responsible for education were measured by multi-source feedback scores from heads of departments, peer consultants, and trainees.

RESULTS

Doctors from 42 clinical departments representing 21 specialties participated. The response rate of the educational climate investigation was moderate 52% (420/811), Response rate was high in the multisource-feedback process 84.3% (420/498). The educational climate was scored quite high mean 3.9 (SD 0.3) on a five-point Likert scale. Likewise the leadership skills of the clinical consultants responsible for education were considered good, mean 5.4 (SD 0.6) on a seven-point Likert scale. There was no significant correlation between the scores concerning the educational climate and the scores on leadership skills, r = 0.17 (p = 0.29).

CONCLUSIONS

This study found no relation between the educational climate and the leadership skills of the clinical consultants responsible for postgraduate medical education in clinical departments with the instruments used. Our results indicate that consultants responsible for education are in a weak position to influence the educational climate in the clinical department. Further studies are needed to explore, how heads of departments and other factors related to the clinical organisation could influence the educational climate.

Research in medical education: pratical impact on medical training and future challenges

Medical education research has changed over the years from merely descriptive studies towards justification or curriculum comparison studies and, nowadays, towards a slow introduction of more clarification studies. In clarification studies quantitative and qualitative methods are used to explain why or how educational interventions work or do not work. This shift is described in this paper. In addition, it is explained how research into workplace learning and assessment has impacted developments in educational practice. Finally, it is argued that the participation of teachers within the medical domain in conducting and disseminating research should be cherished, because they play a crucial role in ensuring that medical education research is applied in educational practice.

PHEEM 'downunder'

BACKGROUND

The use of the Postgraduate Hospital Education Environment Measure (PHEEM) has been previously described in this journal.

AIMS

We established a nine-hospital project to test the acceptability of the PHEEM in Australia.

METHOD

We adapted the language of some items in the PHEEM in order to localize the terminology (such as 'beeped'/'paged') and adjusted the demographics section to facilitate tracking of individual hospitals in the project.

RESULTS

Over two years, more than 400 PHEEMs were returned. Eight of the nine hospitals have an educational environment that is 'more positive than negative but with room for improvement'. One has an 'excellent' environment. None are in the two lowest scoring categories. The lowest scoring items in the collaborative project related to 'feedback', 'information and support', 'infrastructure' and 'interruptions'. The highest scoring items related to 'teachers', 'personal security' and 'working together'.

CONCLUSION

The PHEEM is valuable for systematically collecting information about the educational environment of hospitals. It has brought particular attention to problems associated with protected training time for first year trainees, inappropriate paging and lack of feedback.

Is PHEEM a multi-dimensional instrument? An international perspective

AIM

To look at the characteristics of Postgraduate Hospital Educational Environment Measure (PHEEM) using data from the UK, Brazil, Chile and the Netherlands, and to examine the reliability and characteristics of PHEEM, especially how the three PHEEM subscales fitted with factors derived statistically from the data sets.

METHODS

Statistical analysis of PHEEM scores from 1563 sets of data, using reliability analysis, exploratory factor analysis and correlations of factors derived with the three defined PHEEM subscales.

RESULTS

PHEEM was very reliable with an overall Cronbach's alpha of 0.928. Three factors were derived by exploratory factor analysis. Factor One correlated most strongly with the teaching subscale (R = 0.802), Factor Two correlated most strongly with the role autonomy subscale (R = 0.623) and Factor Three correlated most strongly with the social support subscale (R = 0.538).

CONCLUSIONS

PHEEM is a multi-dimensional instrument. Overall, it is very reliable. There is a good fit of the three defined subscales, derived by qualitative methods, with the three principal factors derived from the data by exploratory factor analysis.

The learning environment of paediatric trainees in New Zealand

AIMS

To assess the learning environment of New Zealand paediatric trainees, examine the factors influencing the trainee's perceptions of their learning environment and to assess the suitability of using the Postgraduate Hospital Education Environment Measure (PHEEM) in New Zealand.

METHODS

Paediatric trainees completed the PHEEM questionnaire and returned this anonymously by email.

RESULTS

Both basic and advanced trainees' perceptions of the environment were generally positive but significant concerns were identified. There were differences in perceptions of the learning environment between the basic and advanced trainees and between sites with more than six trainees compared with those with six or fewer trainees. The PHEEM is practical to use, showed good internal consistency and was acceptable to trainees.

CONCLUSIONS

Both basic and advanced trainees perceived the learning environment positively but significant concerns must be addressed. The advanced trainees perceived the overall learning environment and the teaching they receive more positively than basic trainees. Trainees at smaller hospitals perceived that their social supports were better than at larger hospitals. The PHEEM can be used to assess the paediatric training environment in New Zealand.

Time spent on clerkship activities by students in relation to their perceptions of learning environment quality

CONTEXT

Students' perceptions of their learning environment are of great importance to their learning process. In this study we assessed the time allocated by students to clerkship activities and the relationship between students' allocations of time and their perceptions of the quality of their clinical learning environment.

METHODS

Participants were 133 undergraduate students from eight hospitals taking part in four clerkship rotations. All students recorded the time they spent on eight clerkship activities over 2 weeks and completed the Postgraduate Hospital Educational Environment Measure (PHEEM). Partial correlation analyses were undertaken to examine the relationship between the amount of time students spent on each activity and their PHEEM scores.

RESULTS

Students spent nearly 8 hours a day on clerkship activities. Most time was spent on observing doctors (40%), followed by participating in consultations without direct supervision (12%). The time students spent on observing doctors (r = 0.206, P < 0.05) and in consultations without direct supervision (r = 0.211, P < 0.05) was significantly related to the students' PHEEM scores. There was a significant relationship at the P < 0.10 level between the time spent on directly supervised activities and students' PHEEM scores (r = 0.165, P < 0.10).

CONCLUSIONS

The results suggest that the time spent on activities involving direct patient contact is positively related to students' perceptions of the quality of their learning environment. None of the activities were significantly negatively related to the students' perceptions of their clinical learning environment. Future research should examine the optimal time allocations required to enhance the perceived quality of the clinical learning environment.

Psychometric analyses and internal consistency of the PHEEM questionnaire to measure the clinical learning environment in the clerkship of a Medical School in Chile

BACKGROUND AND AIMS

The Spanish version of the Postgraduate Hospital Educational Environment Measure (PHEEM) was evaluated in this study to determine its psychometric properties, validity and internal consistency to measure the clinical learning environment in the hospital setting of Pontificia Universidad Católica de Chile Medical School's Internship.

METHODS

The 40-item PHEEM questionnaire was translated from English to Spanish and retranslated to English. Content validity was tested by a focus group and minor differences in meaning were adjusted. The PHEEM was administered to clerks in years 6 and 7. Construct validity was carried out using exploratory factor analysis followed by a Varimax rotation. Internal consistency was measured using Cronbach's alpha.

RESULTS

A total of 125 out of 220 students responded to the PHEEM. The overall response rate was 56.8% and compliances with each item ranged from 99.2% to 100%. Analyses indicate that five factors instrument accounting for 58% of the variance and internal consistency of the 40-item questionnaire is 0.955 (Cronbach's alpha). The 40-item questionnaire had a mean score of 98.21 +/- 21.2 (maximum score of 160).

CONCLUSIONS

The Spanish version of PHEEM is a multidimensional, valid and highly reliable instrument measuring the educational environment among undergraduate medical students working in hospital-based clerkships.

Necessary steps in factor analysis: enhancing validation studies of educational instruments. The PHEEM applied to clerks as an example

BACKGROUND

The validation of educational instruments, in particular the employment of factor analysis, can be improved in many instances.

AIMS

To demonstrate the superiority of a sophisticated method of factor analysis, implying an integration of recommendations described in the factor analysis literature, over often employed limited applications of factor analysis. We demonstrate the essential steps, focusing on the Postgraduate Hospital Educational Environment Measure (PHEEM).

METHOD

The PHEEM was completed by 279 clerks. We performed Principal Component Analysis (PCA) with varimax rotation. A combination of three psychometric criteria was applied: scree plot, eigenvalues >1.5 and a minimum percentage of additionally explained variance of approximately 5%. Furthermore, four interpretability criteria were used. Confirmatory factor analysis was performed to verify the original scale structure.

RESULTS

Our method yielded three interpretable and practically useful dimensions: learning content and coaching, beneficial affective climate and external regulation. Additionally, combining several criteria reduced the risk of overfactoring and underfactoring. Furthermore, the resulting dimensions corresponded with three learning functions essential to high-quality learning, thus strengthening our findings. Confirmatory factor analysis disproved the original scale structure.

CONCLUSIONS

Our sophisticated approach yielded several advantages over methods applied in previous validation studies. Therefore, we recommend this method in validation studies to achieve best practice.

The power of one: looking beyond the teacher in clinical instruction

BACKGROUND

Research on clinical teaching in medicine tends to focus on preceptors and senior attending physicians as the primary source of learning for medical students. As a result, there is an artificial separation of 'teacher' from context in much of the research on clinical teaching in medicine.

AIMS

The central aim of this study was to challenge the taken-for-granted assumption that student learning can be attributed primarily to a preceptor or attending physician on a rotation.

METHODOLOGY

Twenty-two medical students and forty-one clinical faculty members generated explanations for a study that showed a positive effect on NMBE results for 3rd year clerkship students who had at least one highly effective clinical teacher during their clinical rotation in medicine.

RESULTS

Student and faculty explanations resulted in fourteen factors and six propositions describing the nature of highly effective clinical teaching. Students believed contextual factors influenced their own learning, but did not comment on that possibility in the study (Griffith CH, Georgesen JC, Wilson JF. 2000. Six-year documentation of the association between excellent clinical teaching and improved student examination performance. Acad Med 75(10): October Supplement). Most clinical faculty did question the assumption that one teacher could have that effect.

CONCLUSIONS

We recommend refocusing research on clinical instruction toward engagement within a community of professionals, rather than attributing 'power' to a single clinical teacher.