Instruments to measure the ability to self-reflect: A systematic review of evidence from workplace and educational settings including health care

Illustration of self-perceived knowledge, skills, and interests in undergraduate dental students using a visual metaphor- results of a monocentric cross-sectional study

BACKGROUND

Self-assessment and self-reflection of competencies are crucial skills for undergraduate students. This monocentric cross-sectional study aims to assess the self-perceived knowledge, skills and interests in conservative dentistry and periodontology of third-, fourth-, and fifth-year dental students by the Pictorial Representation of Illness and Self-measure (PRISM).

METHODS

Seventy-five undergraduate dental students (n = 25 of each year) who studied between 2021 and 2022 at the Department of Cariology, Endodontology and Periodontology at the University of Leipzig, Germany, were included. All of them underwent a PRISM-based interview regarding their perceived knowledge, practical skills, and interests in conservative dentistry as well as its sub-disciplines. The distances in the PRISM task (in millimeters) were measured and compared between the groups. Spearman's Rho was used to reveal correlations between knowledge, skills, and interests in the cohort.

RESULTS

Perceived theoretical knowledge and practical skills differed significantly between groups for the sub-disciplines periodontology, cariology, restorative dentistry and preventive dentistry (p < 0.05). However, students' interests did not significantly vary between groups (p > 0.05). In the field of conservative dentistry and its sub-disciplines, significant moderate to high positive correlations were found between knowledge and skills (p < 0.01), and weak to moderate positive correlations were found between interests and knowledge (p < 0.05). Regarding the relationship between perceived interests and skills, only restorative dentistry, endodontology and periodontology were significant and only moderate to weak correlations were found (p < 0.05).

CONCLUSION

PRISM revealed differences in perceived knowledge and skills between third-, fourth-, and fifth-year dental students. Correlations were found between perceived knowledge and skills, as well as between interests and knowledge. PRISM may be a promising tool to support students and teachers in dental education.

Comparison of PRISM and numeric scale for self-assessment of learning progress during a clinical course in undergraduate dental students

OBJECTIVES

To compare Pictorial Representation of Illness and Self-Measure (PRISM) and a numeric scale for self-reflection in dental students.

METHODS

Fourth year dental students were randomly assigned to each receive one interview based on PRISM or a numeric scale to self-assess their competencies at the beginning (t1), the middle (t2) and the end (t3) of integrated clinical course. Questionnaires were used to assess self-perceived benefit of the interviews at each time points.

RESULTS

Students in PRISM group perceived a higher benefit regarding the self-assessment of their practical skills at all time points (P < 0.05), for theoretical knowledge at t2 and t3 (P < 0.05) and reaching the course objectives at t3 (P = 0.04). At all time points, PRISM group rated their interview (P = 0.04), the applied instrument (PRISM, P = 0.01) and the benefit of the combination of both higher than numeric scale group (P < 0.05).

CONCLUSION

In this preliminary study, PRISM was superior against a numeric scale and can be recommended for dental education to facilitate self-assessment.

PRISM: A Novel Visual Instrument to Facilitate Self-Reflection and Learning Progress in Undergraduate Dental Education

Objectives

PRISM (Pictorial Representation of Illness and Self-Measure) is a simple visual tool that has been successfully used as a visual metaphor in medicine. In this pilot study, PRISM was used for the first time to test its potential to support self-reflection and expectations of learning in dental students.

Methods

Dental student volunteers (25 3^rd year, 10 4^th year, and 10 5^th year) participated. Using both quantitative and qualitative methods, PRISM interviews were compared with a numerical scale in assessing learning objectives concerning theoretical knowledge, practical skills, interest, and training need in the field of conservative dentistry.

Results

Overall, 71% of total student group stated that they would draw personal consequences for their studies due to participating in the PRISM interviews. Compared to the numeric scales, PRISM was rated as more helpful regarding appraisal of students' theoretical knowledge (p = 0.02), practical skills (p < 0.01), training needs (p < 0.01), importance of dental subspecialties (p < 0.01), and facilitating self-reflection (p = 0.02). In focus groups, students commented that PRISM fostered the development of a trusting relationship with their teacher. Strengths of PRISM mentioned by the students included being able to observe and manipulate a visual summary of their individual learning needs and seeing their different learning needs in relation to one another.

Conclusion

In this pilot study, dental students evaluated PRISM to be superior against numeric scales. Furthermore, it ameliorated the communication with teachers. The PRISM task is both simple and brief and warrants further exploration as a useful tool for self-reflection in dental education.

Pilot study to evaluate a novel measure of self-perceived competencies among dental students

BACKGROUND

PRISM is a novel approach to support self-reflection and learning appraisal in dental students, based on a visual metaphor. The aim of this study was to evaluate whether PRISM measurments would be reproducible and sensitive to detect learning progress in undergraduate dental students in their clinical years.

METHODS

Voluntarily participating dental students were included. To evaluate reproducibility, a mixed cohort of 10 students each in 3^rd, 4^th and 5^th year (total n = 30) was recruited and received three identically structured PRISM interviews within one week without any other teaching events. To assess perceived learning progress, 29 volunteer 3^rd year students participated in three interviews during their clinical simulation course (beginning, middle, end). Distances between Subject and Objects was measured in millimeter; objects were classified into close or branched clusters depending on their distance from each other on the PRISM board.

RESULTS

Values for perceived competencies within PRISM interviews during one week were comparable between the three time points in the mixed cohort (n = 30; p > 0.05). Comparing the three subgroups (3^rd, 4^th and 5^th year, each n = 10), PRISM indicated that 3^rd year students perceived their competencies are significantly lower than the 4^th and 5^th year students (p < 0.01). 3^rd year students had less often a branched cluster of objects than the other two groups (p < 0.05). PRISM showed that over time, 3^rd year students perceived a gain in their competencies in conservative dentistry and its sub-disciplines (p ≤ 0.01). The PRISM data indicated that by the end of the simulation course, the students appeared to show higher discrimination of their self-perceptions between sub-topics in conservative dentistry than at the start of the course (p = 0.01).

CONCLUSION

PRISM yields a reproducible measure of individual students' learning progress. It is a promising novel approach for appraisal in dental education. Further work is needed to confirm the generalisability of the findings.

Is reflection like soap? a critical narrative umbrella review of approaches to reflection in medical education research

Reflection is a complex concept in medical education research. No consensus exists on what reflection exactly entails; thus far, cross-comparing empirical findings has not resulted in definite evidence on how to foster reflection. The concept is as slippery as soap. This leaves the research field with the question, 'how can research approach the conceptual indeterminacy of reflection to produce knowledge?'. The authors conducted a critical narrative umbrella review of research on reflection in medical education. Forty-seven review studies on reflection research from 2000 onwards were reviewed. The authors used the foundational literature on reflection from Dewey and Schön as an analytical lens to identify and critically juxtapose common approaches in reflection research that tackle the conceptual complexity. Research on reflection must deal with the paradox that every conceptualization of reflection is either too sharp or too broad because it is entrenched in practice. The key to conceptualizing reflection lies in its use and purpose, which can be provided by in situ research of reflective practices.

Reliability of a grading rubric designed to evaluate the reflective ability of predoctoral dental students

INTRODUCTION

Reflection and self-assessment are critical skills for healthcare providers. Identification of gaps in knowledge, skills and attitudes, along with the ability to critically think and problem solve to fill gaps, is the ultimate outcome for lifelong learning. The aims of this study were to (a) refine an instrument used for measuring reflective ability, and conduct comprehensive reliability testing, and (b) describe a process for rater calibration.

MATERIALS AND METHODS

Students develop e-portfolios over a four-year span with assignments that require reflection and self-assessment. The final piece of the portfolio includes a global reflection written the last semester of the programme. Three faculty raters independently evaluated 106 dental students' global reflections using the revised grading rubric. An intraclass correlation coefficient measured the level of agreement between the three raters.

RESULTS

Analysis of the 318 faculty ratings (106/rater) resulted in an intraclass correlation of .708. Based on a 5-point grading scale (0 = does not respond to the assignment to 5 = reconstructing), the ratings of the 106 global reflections ranged from 1.3 to 5.0 (M = 3.1, SD =0.66).

DISCUSSION

This study provides confidence in the reliability of a grading rubric designed to assess reflective ability, along with suggestions for calibration. An overall mean of 3.1 (Level 3 = relating-includes evidence of lessons learned) illustrates the complexity of teaching reflection and self-assessment.

CONCLUSION

Use of a reliable grading rubric for assessing reflective writing could assist schools interested in incorporating reflection and self-assessment into the curriculum, ultimately supporting lifelong and enhanced health care.