Psychometric properties of an instrument to measure the clinical learning environment

Assessment of Psychometric Properties of a Modified PHEEM Questionnaire

Background: An effective tool in analysing the learning environment, customised to the Sri Lankan setting, is vital for the assessment and delivery of quality healthcare training of preregistration house officers. Such a tool should be reliable and valid. We assessed psychometric properties such as internal reliability and construct validity of a modified version of the Postgraduate Hospital Educational Environment Measure (PHEEM). Materials and Methods: A modified PHEEM questionnaire customised to the Sri Lankan context was developed in accordance to the Sri Lanka Medical Council guidelines. The questionnaire was distributed to all interns at the National Hospital of Sri Lanka, Colombo North Teaching Hospital and Wathupitiwala Base Hospital during a calendar year (n = 100, response rate = 86%). Internal reliability and construct validity of the inventory were assessed by using Cronbach’s alpha and exploratory factor analysis respectively as statistical methods. Results: PHEEM consists of 3 subscales: perceptions of autonomy, social support and teaching, which are factors perceived to be influencing the educational environment. This administration demonstrated high internal reliability as reflected by a Cronbach’s alpha value of 0.84. Exploratory factor analysis identified 12 factors with eigenvalue >1. However, the first factor had an eigenvalue of 6.7 (accounting for 19.7% of variance), while the rest had eigenvalues < 2.5. These results suggest a single predictive factor and thus a one-dimensional scale as opposed to the three-dimensional scale which is used in the current questionnaire. Conclusions: The psychometric properties of this tool reflect a high degree of internal reliability in assessing the educational environment of intern doctors in Sri Lanka. It is possible that the clinical educational environment is collectively represented as a single dimension. This may be due to the complex interplay between individual items in the questionnaire. Therefore the psychometric properties do not justify the interpretation of the educational environment through specified subscales. Key words: Clinical training, Learning environment, Psychometric analysis

The postgraduate hospital educational environment measure (PHEEM) questionnaire identifies quality of instruction as a key factor predicting academic achievement

OBJECTIVE

This study analyzes the reliability of the PHEEM questionnaire translated into Portuguese. We present the results of PHEEM following distribution to doctors in three different medical residency programs at a university hospital in Brazil.

INTRODUCTION

Efforts to understand environmental factors that foster effective learning resulted in the development of a questionnaire to measure medical residents' perceptions of the level of autonomy, teaching quality and social support in their programs.

METHODS

The questionnaire was translated using the modified Brislin back-translation technique. Cronbach's alpha test was used to ensure good reliability and ANOVA was used to compare PHEEM results among residents from the Surgery, Anesthesiology and Internal Medicine departments. The Kappa coefficient was used as a measure of agreement, and factor analysis was employed to evaluate the construct strength of the three domains suggested by the original PHEEM questionnaire.

RESULTS

The PHEEM survey was completed by 306 medical residents and the resulting Cronbach's alpha was 0.899. The weighted Kappa was showed excellent reliability. Autonomy was rated most highly by Internal Medicine residents (63.7% +/- 13.6%). Teaching was rated highest in Anesthesiology (66.7% +/- 15.4%). Residents across the three areas had similar perceptions of social support (59.0% +/- 13.3% for Surgery; 60.5% +/- 13.6% for Internal Medicine; 61.4% +/- 14.4% for Anesthesiology). Factor analysis suggested that nine factors explained 58.9% of the variance.

CONCLUSIONS

This study indicates that PHEEM is a reliable instrument for measuring the quality of medical residency programs at a Brazilian teaching hospital. The results suggest that quality of teaching was the best indicator of overall response to the questionnaire.

How undergraduate clinical learning climates differ: a multi-method case study

CONTEXT

The clinical learning climate affects undergraduate medical students' behaviour, satisfaction and success. Most studies predominantly describe aspects of the clinical learning climate using quantitative methodologies, such as questionnaires. This study aimed to illuminate medical students' perceptions of the clinical learning climate, and which factors and their interactions explain differences in clinical learning climates.

METHODS

We carried out a multi-method case study. Twelve departments of obstetrics and gynaecology distributed the Postgraduate Hospital Educational Environment Measure (PHEEM), a reliable questionnaire measuring the clinical learning environment, among medical students. After analysis (using anova and post hoc tests), 14 medical students from the highest- and lowest-scoring departments participated in semi-structured interviews. We analysed the transcribed recordings using a content analysis approach. Researchers agreed on coding and an expert group reached consensus on the themes of the analysis.

RESULTS

We found a significant difference between departments in PHEEM scores. The interviews indicated that department and medical student characteristics determine the clinical learning climate. For departments, 'legitimacy', 'clerkship arrangements' and 'focus on personal development' were the main themes. For medical students, 'initial initiatives', 'continuing development' and 'clerkship fatigue' were the principal themes. The amount and nature of participation played a central role in all themes.

CONCLUSIONS

Differences between clinical learning climates appear to be related to differing approaches to participation among departments. Participation depends on characteristics of both departments and students, and the interactions among them. The outcomes give valuable clues to how a favourable clinical learning climate is shaped.

A controlled study of the short- and long-term effects of a Train the Trainers course

Objectives This study aimed to establish the longterm effects of a 3-day 'Training for Trainers' course (TTC) on doctors' knowledge, teaching behaviour and clinical learning climate. Methods The study was designed as an intervention study with pre-, post- and long-term measurements. The intervention group (I-group) included 118 doctors from the departments of internal medicine and orthopaedic surgery at one university hospital. The control group (C-group) consisted of 125 doctors from the corresponding departments at another university hospital. Gains in knowledge about teaching skills were assessed by a written test. Teaching behaviour and learning climate were evaluated by questionnaires. Results In the I-group, 98.4% of doctors, both specialists and trainees, participated in a TTC. Response rates on the written test varied from 90% at baseline to 70% at 6 months after the intervention. Knowledge about teaching skills increased in the I-group by 25% after the TTC and was sustained at 6 months. Questionnaire response rates varied from 98.4% at baseline to 84.8% at 6 months. Post-course, the teaching behaviour of the I-group significantly changed and its learning climate improved compared with the C-group. Scores for use of feedback and supervision in the I-group increased from 4-5 to 6-7 (maximum score = 9). This was significantly higher than in the C-group. Conclusions A 3-day residential TTC has a significant impact in terms of gains of knowledge concerning teaching skills, teaching behaviour and learning climate after 6 months. The positive effects demonstrated in this study were rooted in both the specialists and trainees who attended the course.

Factors determining medical students' and residents' satisfaction during VA-based training: findings from the VA Learners' Perceptions Survey

PURPOSE

To compare medical students' and physician residents' satisfaction with Veterans Affairs (VA) training to determine the factors that were most strongly associated with trainee satisfaction ratings.

METHOD

Each year from 2001 to 2006, all medical students and residents in VA teaching facilities were invited to complete the Learners' Perceptions Survey. Participants rated their overall training satisfaction on a 100-point scale and ranked specific satisfaction in four separate educational domains (learning environment, clinical faculty, working environment, and physical environment) on a five-point Likert scale. Each domain was composed of unique items.

RESULTS

A total of 6,527 medical students and 16,583 physician residents responded to the survey. The overall training satisfaction scores for medical students and physician residents were 84 and 79, respectively (P < .001), with significant differences in satisfaction reported across the training continuum. For both medical students and residents, the rating of each of the four educational domains was statistically significantly associated with the overall training satisfaction score (P < .001). The learning environment domain had the strongest association with overall training satisfaction score, followed by the clinical preceptor, working environment, and physical environment domains; no significant differences were found between medical students and physician residents in the rank order. Satisfaction with quality of care and faculty teaching contributed significantly to training satisfaction.

CONCLUSIONS

Factors that influence training satisfaction were similar for residents and medical students. The domain with the highest association was the learning environment; quality of care was a key item within this domain.

Japanese dental trainees' perceptions of educational environment in postgraduate training

BACKGROUND

Compulsory postgraduate dental training was introduced in April 2006 to meet social requirements in Japan.

AIM

The aim of the present study was to identify the difference in trainees' perception of the educational environment in two different training settings in the Hiroshima University Hospital postgraduate training program: (1) main hospitals, in Hiroshima's case, the University Hospital: (2) community dental hospitals or offices (cooperating hospitals).

METHODS

In order to determine how trainees perceive their educational environment, the Postgraduate Hospital Educational Environment Measure (PHEEM) Inventory was administered to 50 vocational dental trainees for two different training settings, both of which all trainees had experienced.

RESULT

The mean total PHEEM score in the main hospital was 102.4, and that in cooperating hospitals was 108.5. The scores for each of the three subscales were as follows: 'Perception of role autonomy': 33.9/56 (main hospital), 36.8/56 (cooperating hospitals); 'Perception of teaching': 38.1/60 (main hospital), 41,9/60 (cooperating hospitals), and 'Perception of social support': 30.4/44 (main hospital), 29.8/44 (cooperating hospitals) respectively.

CONCLUSIONS

PHEEM provided meaningful diagnostic information on the educational environment, and showed that cooperating hospitals met trainees' needs more than main hospitals.