Development of the clinical learning evaluation questionnaire for undergraduate clinical education: factor structure, validity, and reliability study

A scoping review of the questionnaires used for the assessment of the perception of undergraduate students of the learning environment in healthcare professions education programs

The learning environment (LE) includes social interactions, organizational culture, structures, and physical and virtual spaces that influence the learning experiences of students. Despite numerous studies exploring the perception of healthcare professional students (HCPS) of their LE, the validity evidence of the utilized questionnaires remains unclear. This scoping review aimed to identify questionnaires used to examine the perception of undergraduate HCPS of their LE and to assess their validity evidence. Five key concepts were used: (1) higher education; (2) questionnaire; (3) LE; (4) perception; and (5) health professions (HP). PubMed, ERIC, ProQuest, and Cochrane databases were searched for studies developing or adapting questionnaires to examine LE. This review employed the APERA standards of validity evidence and Beckman et al. (J Gen Intern Med 20:1159-1164, 2005) interpretation of these standards according to 5 categories: content, internal structure, response process, relation to other variables, and consequences. Out of 41 questionnaires included in this review, the analysis revealed a predominant emphasis on content and internal structure categories. However, less than 10% of the included questionnaires provided information in relation to other variables, consequences, and response process categories. Most of the identified questionnaires received extensive coverage in the fields of medicine and nursing, followed by dentistry. This review identified diverse questionnaires utilized for examining the perception of students of their LE across different HPs. Given the limited validity evidence for existing questionnaires, future research should prioritize the development and validation of psychometric measures. This will ultimately ensure sound and evidence-based quality improvement measures of the LE in HP education programs.

Assessment of clinical competence of graduating medical students and associated factors in Ethiopia

BACKGROUND

Ethiopia has scaled up medical education to improve access to healthcare which presented challenges to maintaining training quality. We conducted a study to assess the clinical competence of graduating medical students and the associated factors.

METHODS AND MATERIALS

A pretest assessment of a quasi-experimental study was conducted in 10 medical schools with a sample size of 240 students. We randomly selected 24 students per school. Clinical competence was assessed in a 12-station objective structured clinical examination. The clinical learning environment (CLE), simulation training, and practice exposure were self-rated. Mean scores for clinical competence, and satisfaction in the CLE and simulation training were calculated. Proportions of students with practice exposure, and who agreed on CLE and simulation items were done. Independent t-tests were used to look at competence differences among subgroups. Bivariate and multiple linear regression models were fitted for the outcome variable: competence score. A 95% statistical confidence interval and p-value < 0.05 were used for making statistical decisions. A 75% cut-off score was used to compare competence scores.

RESULTS

Graduating medical students had a mean competence score of 72%. Low scores were reported in performing manual vacuum aspiration (62%), lumbar puncture (64%), and managing childbirth (66%). Female students (73%) had a significantly higher competence score than males (70%). Higher cumulative grade point average (CGPA), positive appraisal of the CLE, and conducting more clinical procedures were associated with greater competence scores. Nearly half of the students were not satisfied with the clinical practice particularly due to the large student number and issues affecting the performance assessment. About two-thirds of the students were not satisfied with the sufficiency of models and equipment, and the quality of feedback during simulation training. Nearly one-third of the students never performed lumbar puncture, manual vacuum aspiration, and venipuncture.

CONCLUSIONS

Medical students had suboptimal clinical competence. A better clinical learning environment, higher cumulative GPA, and more practice exposure are associated with higher scores. There is a need to improve student clinical practice and simulation training. Strengthening school accreditation and graduates' licensing examinations is also a way forward.

Development and validation of the Workplace Learning Inventory in Health Sciences Education: a multimethod study

Health sciences students face many challenges in regard to clinical practical learning. A better understanding of student learning is required to address student needs in this crucial phase. The theory of self-regulated learning provides a comprehensive view of learning and could serve as a basis for further research. There are instruments to assess self-regulated learning in preclinical academic learning. However, there are no such instruments for workplace learning. The aim of the present study is to provide a comprehensive inventory from which researchers can select those scales that are relevant to their research questions in the investigation of workplace learning. Hence, the aim is to develop and validate a set of scales to assess undergraduates' workplace learning in health sciences education in four areas (cognition, motivation, emotion, and context) on two levels (the learning process level and the metalevel). Study 1 is a qualitative multimethod study to identify indicators and develop items. It integrates the perspectives of students, teachers, and researchers and includes six steps: literature review, interviews, synthesis, item development, expert review, and cognitive pretesting. This study yields a set of scales for each area on both levels. Study 2 is a quantitative study to assess the psychometric properties. The results show acceptable values in terms of unidimensionality, reliability and validity for each of the 31 scales. The newly developed Workplace Learning Inventory is comprehensive; the scales are relevant to workplace learning and short enough that their administration is feasible in the workplace setting. The rigorous process of questionnaire development contributes to the validity of scales. By providing the Workplace Learning Inventory, we hope to encourage research on workplace learning in health sciences education from an educational psychology perspective.

Clinical learning evaluation questionnaire: a reliable and valid tool for the evaluation of clinical education by educators and students

BACKGROUND

The clinical learning evaluation questionnaire can be used in the clinical trial period of medical students to measure the effectiveness of the clinical learning environment. The purpose of this study was to collect validity evidence of an adapted questionnaire to measure the transcultural adaptation of a Persian version of CLEQ.

METHODS

A total of 200 questionnaires were completed by students who were at the end of their clinical rotation. The study instrument was the latest version of the CLEQ consists of 18 Items in four dimensions. The CLEQ was translated into Persian language through a four-step process of forward and backward translation. Data analysis was performed on two softwares, SPSS, version 22 and Lisrel, version 8.8.

RESULTS

The results showed that the 18-question CLEQ could be applied to the Persian translation of the tool. The response process evidence of the Persian questionnaire was established through feedback from 15 students in the sample group. The content validity index (CVI) for the items were between 0.8 and 0.9, and the content validity ratio (CVR) for the entire questionnaire was 0.9. The 4-factor feature of CLEQ was good model fit. The internal consistency analysis indicated that the Cronbach's alpha values for all items of the 18-item questionnaire were equal to 0.87 and for the subscales were 0.68 to 0.79.

CONCLUSION

The Persian translation of the 4-factor CLEQ has sufficient validity evidence to measure the transcultural adaptability of clinical education activities by instructors and students. The validity evidence are content, response process and internal structure. We recommend that the English 6-factor and 6-factor versions of CLEQ be tested on medical students at multiple foreign academic institutions to assess their efficiency.

What Aspects of Supervised Patient Encounters Affect Students' Perception of Having an Excellent Learning Outcome? A Survey Among European Medical Students

Background

To investigate whether supervisor behavior, students' participation and approach, and psychological safety were associated with self-reported excellent learning outcome from supervised encounters with patients among European medical students.

Methods

A cross-sectional, online survey among European medical students asking about their latest clinical supervision experience. Associations were examined with logistic regression.

Results

Students (N=908) from >25 countries reported on experiences from supervised patient encounters in most types of hospital departments and general practice. One out of six (17%) students perceived the learning outcome as excellent. In the multivariable logistic regression, this was independently associated with supervisor role modelling (odds ratio (OR) 2.1, 95% confidence interval (CI) 1.5-3.0) and addressing learning goals (OR 1.4, 95% CI 1.1-1.7), students' approach to learning (OR 1.7, 95% CI 1.0-3.0) and psychological safety (OR 1.5, 95% CI 1.1-2.0). Supervisors being present during the patient encounter, coaching students or asking questions to have students express their thinking, and student participation in examination and/or history taking was not associated with perceived excellent learning outcome.

Conclusion

We encourage supervisors to recognize that students are beginners in most supervised clinical settings and often appreciate having learning goals addressed, behavior and thinking role modelled, and psychological safety established before they participate more fully.

Clinical Learning Evaluation Questionnaire: A Confirmatory Factor Analysis

INTRODUCTION

The Clinical Learning Evaluation Questionnaire (CLEQ) is a multidimensional, reliable instrument designed to measure the effectiveness of the clinical learning environment for undergraduate medical students. This study seeks to measure and examine the underlying construct along with the latent variables by conducting a confirmatory factor analysis, using structural equation modeling (SEM) so that the instrument can be utilized as an evaluation tool for the continuous improvement of educational environments and curricula.

METHODS

A cross-sectional study was carried out on 185 third- and fourth-year medical students. A confirmatory factor analysis was conducted, beginning with principal component analysis for standardized factor loadings, using varimax rotation in SPSS to explore the underlying construct of items. The constructs to which each item was tied were determined, and then the data were run through AMOS to assess construct validity through item reduction based on the modification indices, and estimates were made of the standardized residual covariance of each item in order to determine the best model fit.

RESULTS

A total of 185 students completed the CLEQ Inventory. The original six-factor structure of the CLEQ did not achieve model fit (X²=1587.475, RMSEA=0.092, RMR=0.146, GFI=0.651, AGFI=0.601, CFI=0.728, NFI=0.626). However, the suggested four-factor model of CLEQ displayed good model fit with the improvement of values (X²=86.184, RMSEA=0.052, RMR=0.062, GFI=0.903, AGFI=0.865, CFI=0.951, NFI=0.871). Internal consistency analysis showed that Cronbach's alpha values of the original six-factor model ranged from 0.68 to 0.88, while four-factor model ranged from 0.72 to 0.87.

CONCLUSION

This study did not support the proposed six-factor structure of the CLEQ tool. However, the four-factor CLEQ structure demonstrated an adequate degree of good fit and was found to be as reliable as the original structure. Further research on the predictive validity of CLEQ is required as well as a comparison of the psychometric properties across different institutions and countries.

Development of a brief learning environment measure for use in healthcare professions education: the Healthcare Education Micro Learning Environment Measure (HEMLEM)

BACKGROUND

The learning environment impacts many aspects of healthcare education, including student outcomes. Rather than being a single and fixed phenomenon, it is made up of multiple micro learning environments. The standard clinical learning environment measurement tools do not consider such diversity and may fail to adequately capture micro learning environments. Moreover, the existing tools are often long and may take a prohibitive amount of time to complete properly. This may have a negative impact on their usefulness in educational improvement strategies. In addition, there is no universal tool available which could be utilised across several healthcare student groups and placement settings.

AIM

To create an evidence-based measurement tool for assessing clinical micro learning environments across several healthcare profession student groups.

METHODS

The measurement tool was developed through a step-wise approach: 1) literature review with iterative analysis of existing tools; 2) generation of new items via thematic analysis of student experiences; 3) a Delphi process involving healthcare educators; 4) piloting of the prototype; and 5) item reduction.

RESULTS

The literature review and experiential data from healthcare students resulted in 115 and 43 items respectively. These items were refined, leaving 75 items for the Delphi process, which produced a prototype with 57 items. This prototype was then completed by 257 students across the range of healthcare professions, with item reduction resulting in a 12-item tool.

CONCLUSION

This paper describes a mixed methods approach to developing a brief micro learning environment measurement tool. The generated tool can be used for measuring student perceptions of clinical environments across several healthcare professions. Further cross-cultural and cross-professional validation studies are needed to support widespread use, possibly through mobile application.

Can a family medicine rotation improve medical students' knowledge, skills and attitude towards primary care in Vietnam? A pre-test-post-test comparison and qualitative survey

OBJECTIVES

Well-designed studies on the impact of a family medicine rotation on medical students are rare, and very few studies include a qualitative component. This study aimed to determine the improvement of medical students' knowledge, communication skills and attitude towards primary care and explore their perceptions after rotations, in comparison with a control group.

METHODS

We used a mixed-methods design, comprising a pre-test-post-test comparison between a sample of trained students who took family medicine rotations and a control group and a qualitative survey. The measurement of improvement included (i) multiple choice question testing, (ii) objective structured checklist examinations, (iii) self-reporting and (iv) interviews and focus group discussions. Data were collected from August 2017 to June 2018.

RESULTS

There were 696 students in the trained group and 617 controls. The two groups' baseline scores in knowledge, communication skills and attitude were not significantly different. Knowledge covering five domains of family medicine (Pearson's r from 0.6 to 0.9) improved significantly, as did attitudes towards primary care in the trained group. There were no differences in communication and counselling skills between the two groups for four situations, but for two-health check-ups and mental health care-skills were significantly improved (Pearson's r from 0.28 to 0.43). The qualitative survey showed highly positive feedback from trained students.

CONCLUSIONS

The family medicine rotation significantly improved students' knowledge and attitude towards primary care and some communication skills. Further studies should investigate students' interest in and career choice for this discipline.

Exploring undergraduate medical students' perception of learning procedural skills and its outcomes in clinical settings

INTRODUCTION

Learning procedural skills is one of the essential aspects of undergraduate medical education. However, learning procedural skills in clinical settings is less widely considered. This study aimed to explore the Iranian undergraduate medical students' perception of learning procedural skills and its outcomes in three universities of medical sciences in Iran.

METHODS

A descriptive exploratory qualitative methodology with an in-depth unstructured, face-to-face interview, and content analysis was used in this study. Sixteen students in clinical phases of general medical education programs from educational hospitals were selected using purposive sampling. According to the preferences of the participants, the interviews were conducted in medical schools or in hospitals.

RESULTS

The students participating in this study included 7 females and 9 males (totally 16 people) with a mean age of 23.7 years old with a range of 21-27 years. The three main themes of this study were "the gap of transferring formal teaching from skill lab to clinical placement", "learning self-leading procedural skills in clinical settings", and "students' dissatisfaction with patients' vulnerability" with 8 subthemes which were extracted and explained based on the students' perception.

CONCLUSION

Unsupervised and self-learning by medical students and weakness in controlling the learning process have undesirable results for patients and students.

Does performance on United States national board of medical examiners reflect student clinical experiences in United Arab Emirates?

This article was migrated. The article was marked as recommended. Background: A number of medical schools around the world use the United States National Board of Medical Examiners Subject Examinations as a clerkship assessment of student performance, yet these exams were blueprinted against the United States national core clerkship curriculum which might not be the same as the local curricula to which they are applied in other parts of the world. In this study, we investigated the correlations between the internal medicine clinical experiences at United Arab Emirates University with student performance on the National Board of Medical Examiners subject of internal medicine (NBME). Methods: One hundred and seven junior clerkship students out of 145 (74%) who finished their Internal Medicine clerkship during academic years 2014-2015 and 2015-2016 participated in this study. The students' clinical experiences were measured by the clinical learning evaluation questionnaire (CLEQ) and by the logged number of meaningful patient contacts during their internal medicine clerkship. Results: Linear regression analysis showed no significant association between performance on the subject test and student clinical experiences measured by the CLEQ or the number of logged patients. NBME scores were weakly correlated with OSCEs scores (ɸ 0.20). Conclusions: The study findings raised uncertainties about the suitability of using NBME in the clerkship assessment program in the United Arab Emirates.

Exploring variability of teaching & supervision at clinical clerkship teaching sites

Objective:

To explore undergraduate medical students’ perception of variation in teaching and supervision at different clinical teaching sites.

Methods:

This descriptive cross-sectional study was conducted at the College of Medicine & Health Sciences, United Arab Emirates University, UAE during 2017. Four clinical teaching sites affiliated with CMHS were evaluated namely Shaikh Khalifa Medical City (SKMC), Ambulatory Care Clinics (AC), Tawam Hospital (TH) and Al-Ain Hospital (AH). An online questionnaire was administered to year five and six students.

Results:

The response rate was 84.4%. Overall perception of the students about their clinical clerkship experience was positive. SKMC was rated as the best teaching site with mean rating of 3.79±0.97-4.79±0.43. The highest rated item was clinical teacher’s promotion of critical thinking in students while the lowest rated item was the opportunity to take responsibility for patient care. Ambulatory Care site had a mean rating of 2.33±1.23-4.13±1.19. The highest rated item at this site was the clinical teacher encouraging students to ask questions and participate actively. At Tawam Hospital, the mean ratings ranged between 2.65±1.64-4.31±0.86 with highest rated item being ability of the students to see cases with positive clinical findings. At the Al-Ain Hospital, the mean rating was in the range of 2.79±1.45-3.81±1.11. The item rated highest here was the ability of students to see cases with positive clinical findings. The lowest rated item at all three sites was the availability of on-call rooms and lockers. Significant variability was seen across training sites in the clinical teacher’s ability to act as professional role models, the opportunity for students to apply their previous knowledge to patient care and to independently assess patients before discussion with teachers.

Conclusion:

This study tool highlights variation in clinical teaching and supervision at four clinical teaching sites. It provides specific, actionable information which can be utilized to deliver equitable learning experiences across clinical clerkships and teaching sites. It places emphasis on the fact that lack of physical facilities hampers clinical teaching and supervision, hence, on call rooms, lockers and separate rooms for independent student interaction with patients should be provided at all clinical teaching sites.

Measuring assessment standards in undergraduate medical programs: Development and validation of AIM tool

Objective:

To develop a tool to evaluate faculty perceptions of assessment quality in an undergraduate medical program.

Methods:

The Assessment Implementation Measure (AIM) tool was developed by a mixed method approach. A preliminary questionnaire developed through literature review was submitted to a panel of 10 medical education experts for a three-round ‘Modified Delphi technique'. Panel agreement of > 75% was considered the criterion for inclusion of items in the questionnaire. Cognitive pre-testing of five faculty members was conducted. Pilot study was done with 30 randomly selected faculty members. Content validity index (CVI) was calculated for individual items (I-CVI) and composite scale (S-CVI). Cronbach's alpha was calculated to determine the internal consistency reliability of the tool.

Results:

The final AIM tool had 30 items after the Delphi process. S-CVI was 0.98 with the S-CVI/Avg method and 0.86 by S-CVI/UA method, suggesting good content validity. Cut-off value of < 0.9 I-CVI was taken as criterion for item deletion. Cognitive pre-testing revealed good item interpretation. Cronbach's alpha calculated for the AIM was 0.9, whereas Cronbach's alpha for the four domains ranged from 0.67 to 0.80.

Conclusions:

‘AIM' is a relevant and useful instrument with good content validity and reliability of results, and may be used to evaluate the teachers´ perceptions about assessment quality.

Preparing for practice: Nursing intern and faculty perceptions on clinical experiences

PURPOSE

Clinical experience and exposure to real patients are required elements of nursing education. Trainees in nursing are expected to be prepared adequately for the hard-working environment, increasing patient complexity, and higher-level competencies. This study investigates differences between nursing interns and clinical faculty on actual and perceived importance of educational preparation and development of clinical competencies, focusing on the nursing curriculum and transition to practice.

METHODS

A convenient sampling technique with a mixed-methods design was used to collect quantitative and qualitative data, by surveying and interviewing nursing interns and faculty members from King Abdul-Aziz University in Saudi Arabia; data collection occurred in December 2015. The survey (23 items) and focused interviews measured perceptions of clinical instruction and experience. Descriptive statistics and t-tests were used to analyze differences in mean ratings between actual and perceived importance. Themes collected from narrative interview data were summarized.

RESULTS

Significant differences were found between nursing interns (n = 46) and faculty (n = 29) perceptions of actual clinical teaching and experiences and its importance including the clinical teaching and the development of clinical competence, p < .01. Moreover, nursing interns rated actual experiences of knowledge base and skills significantly lower than faculty perceptions, p = .001. Narrative data provided in-depth information on factors contributing and hindering the learning and teaching environment.

CONCLUSIONS

Findings from this study call for clinical instruction and experiences to take a step further to meet current practice standards and to improve patient safety in the health professions education of nurses.

Nursing Instructor and Students' Perspectives on Clinical Education Apprenticeship Problems

INTRODUCTION

Different problems might lead to reduction in the efficiency of nursing students' apprenticeship education and solving those problems.

AIM

This study was conducted to determine nursing internship problems from the perspective of trainers and students.

MATERIALS AND METHODS

In this descriptive cross-sectional study, 20 trainers and 43 senior students of nursing were selected using census method. A researcher-made questionnaire was used to collect data. Total internal consistency (α) of the questionnaire was 0.88 and had proper convergent validity. SPSS was used to analyse data with applying descriptive and inferential statistics.

RESULTS

More than 50% considered their course relatively weak or weak in helping intern nurses to acquire comprehensive view of the nursing profession that takes a social perspective into account, skills required for the processes of nursing and instructing patients. Problems such as insufficient access to welfare and educational facilities, lack of co-operation among clinical team and scattering of internship sessions during a course were mentioned by nurses and their trainers. They believed that changes must take place in the way internship courses are carried out.

CONCLUSION

Although the internship courses seem to achieve their aim of improving students' skills and independence in providing different nursing services to students and their trainers generally have a positive attitude towards such courses, there are still problems in effective exercise of the training.

Clinical learning environment measurement for medical trainees at transitions: relations with socio-cultural factors and mental distress

BACKGROUND

Measuring clinical learning environment is crucial for the quality improvement of medical education, especially for medical trainees at transition stages. Medical education in Taiwan is shaped by multiple socio-cultural influences. The aims of this study were to construct an instrument for measuring clinical learning environment in alignment with the local socio-cultural factors and medical training settings, and further investigate the relationship between the quality of the clinical learning environment and the status of mental distress among medical trainees.

METHODS

Participants consisted of 189 medical trainees (62 interns, 52 postgraduate year one (PGY1) residents, and 75 senior residents). Instruments included a designed 40-item Clinical Learning Environment Questionnaire (CLENQ) and a five-item Brief Symptoms Rating Scale (BSRS-5) for evaluating mental distress. Constructs of CLENQ were examined using factor analysis. Correlations were calculated between BSRS-5 and CLENQ across the three levels of medical trainees.

RESULTS

Factor analysis of CLENQ yielded five factors- I: Teaching (13 items), II: Workload (7 items), III: Relationship pressure (9 items), IV: Organisational support (4 items) and V: Mutual trust (6 items). Intern trainees reported the lower total CLENQ scores in comparison to PGY1 and senior resident trainees. Mental distress using BSRS-5 was negatively correlated with total CLENQ scores and several key factors in all three groups.

CONCLUSIONS

Our study using CLENQ has identified five major factors of clinical learning environment that are closely linked with our local socio-cultural factors and medical training settings. Medical trainee's mental distress status was negatively correlated with the quality of CLENQ. These findings have socio-cultural relevance and medical contextual significance and might be applicable to other countries. It warrants further study to investigate the impact of clinical learning environment improvement on the medical trainee's mental distress and performance.