A psychometric evaluation of the University of Auckland General Practice Report of Educational Environment: UAGREE

Development and validation of the student ratings in clinical teaching scale in Australia: a methodological study

PURPOSE

This study aimed to devise a valid measurement for assessing clinical students' perceptions of teaching practices.

METHODS

A new tool was developed based on a meta-analysis encompassing effective clinical teaching-learning factors. Seventy-nine items were generated using a frequency (never to always) scale. The tool was applied to the University of New South Wales year 2, 3, and 6 medical students. Exploratory and confirmatory factor analysis (exploratory factor analysis [EFA] and confirmatory factor analysis [CFA], respectively) were conducted to establish the tool’s construct validity and goodness of fit, and Cronbach’s α was used for reliability.

RESULTS

In total, 352 students (44.2%) completed the questionnaire. The EFA identified student-centered learning, problem-solving learning, self-directed learning, and visual technology (reliability, 0.77 to 0.89). CFA showed acceptable goodness of fit (chi-square P<0.01, comparative fit index=0.930 and Tucker-Lewis index=0.917, root mean square error of approximation=0.069, standardized root mean square residual=0.06).

CONCLUSION

The established tool—Student Ratings in Clinical Teaching (STRICT)—is a valid and reliable tool that demonstrates how students perceive clinical teaching efficacy. STRICT measures the frequency of teaching practices to mitigate the biases of acquiescence and social desirability. Clinical teachers may use the tool to adapt their teaching practices with more active learning activities and to utilize visual technology to facilitate clinical learning efficacy. Clinical educators may apply STRICT to assess how these teaching practices are implemented in current clinical settings.

A meta-analysis for comparing effective teaching in clinical education

Aim: Many factors affect learning outcomes, however studies comparing the effectiveness of different clinical teaching methods are limited. We utilize the list of influences on educational achievement compiled by John Hattie to inform a meta-analysis of learning effect sizes (ESs) associated with teaching-learning factors (TLFs) in clinical education. Methods: A literature search was conducted in PubMed to identify articles examining clinically relevant TLFs. Selection criteria were applied to identify learner-focused studies, with subsequent categorization by study design (pretest-posttest or controlled group). The Cohen's ES (d) for each TLF was extracted and a pooled ES determined. Results: From 3454 studies, 132 suitable articles enabled analysis of 16 TLFs' ESs. In general, ESs derived from pretest-posttest data were larger than those from controlled group designs, probably due to learner maturation effect. The TLFs of mastery learning, small group learning and goal settings possessed the largest ESs (d ≥ 0.8), while worked examples, play programs, questioning, concept mapping, meta-cognitive strategies, visual-perception programs and teaching strategies demonstrated ESs between 0.4 and 0.8. Conclusions: This is the first study to provide a rigorous and comprehensive overview of the effectiveness of TLFs in clinical education. We discuss the practical traits shared by effective TLFs which may assist teaching design.