Understanding the influence of teacher-learner relationships on learners' assessment perception

Blended teaching mode based on small private online course and case-based learning in analgesia and sedation education in China: a comparison with an offline mode

BACKGROUND

Standardized training for pain, agitation-sedation, and delirium (PAD) management is urgently needed for Chinese intensivists' continuing education. Since 2020, because of the COVID-19 pandemic, the Chinese Analgesia and Sedation Education and Research (CASER) group has used an online blended teaching mode based on a small private online course (SPOC) and case-based learning (CBL). This study evaluated whether an online blended teaching mode has similar effects on PAD management training when an offline mode cannot be used.

MATERIALS AND METHODS

Since 2020, the CASER group has provided offline training and online SPOC&CBL training three times each, targeting intensivists and ICU nurses in China. All participants were divided into an offline group and SPOC&CBL group. A final examination was offered in each training session to assess the students' mastery of professional knowledge. Teachers' and students' perceptions regarding the online SPOC&CBL mode were evaluated through questionnaires.

RESULTS

Of all participants (n = 117), 106 completed all examinations and questionnaires. Most participants were aged 31-40 years (53, 50.0%), had an academic degree (60, 56.6%), and worked in a tertiary hospital (100, 94.34%). We assessed the learning effect on participants from two aspects: theory and clinical practice. There was no significant difference between the SPOC&CBL and offline groups in terms of theoretical, case analysis, and total scores (p > 0.05). In terms of the participants' perceptions regarding the SPOC&CBL mode, 91.5% considered the online mode to be a useful and accessible alternative to improve knowledge and skills. A total of 95.7% of the participants believed that they could interact well with group members, and 87.2% believed that they had a good degree of participation. Of these participants, 76.6% believed that they had received valuable learning resources. All instructors believed that the SPOC&CBL mode was more flexible than the offline mode in terms of teaching time and location, and they were all willing to carry out training with the SPOC&CBL mode.

CONCLUSION

Compared to the offline mode, the SPOC&CBL mode can also enhance participants' knowledge and skills and meets their expectations. Therefore, an online mode can be considered a potential method in PAD management education in China.

Factors affecting residents' internal motivation, grit, and well-being

BACKGROUND

Residents completing competency-based medical education for postgraduate training face many challenging situations that may compromise their well-being or result in exhaustion or burnout. Factors described in self-determination theory and grit are important for residents' achievement of learning outcomes and well-being. This study explored the relationships among internal motivation, grit, well-being, and related factors among non-Western Asian residents.

METHODS

We conducted an explanatory sequential mixed-methods survey-based study to explore correlations among satisfaction with basic psychological needs, grit, and well-being from September to November 2021 among residents at Ramathibodi Hospital, Mahidol University, Thailand. Data were collected with the Basic Psychological Needs Scale, Short Grit Scale, and World Health Organization-Five Well-Being Index. Next, participants with the highest and lowest scores for each scale were purposively invited to participate in semi-structured interviews. Interview data underwent thematic analysis and data collection continued until saturation was reached.

RESULTS

In total, 245 residents (51% major ward, 65% female) completed the survey. There were strong associations between internal motivation, grit, and well-being (r = 0.46-0.90). Female residents had higher autonomy and relatedness scores than males (p = 0.04 and p = 0.01, respectively), and residents with less family responsibility had higher relatedness scores than other residents (p = 0.01). Residents who got more sleep had higher autonomy, relatedness, and well-being scores than those that slept less (p < 0.05). Residents who exercised > 5 times/week had higher well-being scores than other residents (p < 0.01). Thirty residents completed interviews. The thematic analysis revealed internal motivation, grit, and well-being were promoted by a supportive learning environment, a well-designed curriculum, actions and personalities of faculty members, and good personal factors.

CONCLUSION

Internal motivation is significantly correlated with residents' grit, well-being, gender, family burdens, exercise, and sleep hours. Priority should be given to promoting internal motivation, grit, and well-being among residents by enhancing a positive learning environment, creating well-designed curricula, fostering good characteristics and actions among faculty members, and supporting residents' personal lives.

Prosocial modelling matters: the association between parent and faculty involvement in fighting COVID-19 with medical students' career commitment

BACKGROUND

Role models are essential in medical education, yet empirical research is relatively insufficient on the influence of prosocial modelling on medical students' career commitment. The prosocial behaviour of medical staff involved in the fight against the novel coronavirus disease 2019 (COVID-19) at the beginning of 2020 presents an opportunity to fill the research gap. We explored and compared the different associations of the two most important role models for medical students - parents and faculty- with medical students' career commitment.

METHODS

The cross-sectional study was conducted with 99,559 undergraduate students majoring in clinical medicine in mainland China. Questions were asked to collect information about participants in the battle against COVID-19, medical students' determination to practice medicine after graduation, as well as students' socio-demographic characteristics. Chi-square tests and hierarchical regressions were performed to examine the associations between parent and faculty involvement and students' career commitment.

RESULTS

The results showed statistically significant associations between prosocial modelling during the COVID-19 pandemic in China and students' intention to pursue medical careers. The association of faculty involvement (OR = 1.165, p < .001) with students' career commitment was greater than that of parents (OR = 0.970, p > .05). For faculty involvement, the association was stronger among male students (OR = 1.323, p < .001) and students who were already determined to be doctors (OR = 1.219, p < .001) before the pandemic.

CONCLUSIONS

Our study provides new evidence on the potential roles of parents and faculty in shaping medical students' career commitment. Encouraging faculty to act as positive role models could help medical students increase their intention to become doctors.KEY MESSAGESProsocial modelling could enhance students' intention to pursue medical careers.The association of prosocial behaviour of faculty is larger than that of parents on medical students.Those who have prior medical career commitment are much more likely to persist in the medical profession, and prosocial modelling of faculty is positively associated with their medical career commitment.

Student perspectives on programmatic assessment in a large medical programme: A critical realist analysis

BACKGROUND

Fundamental challenges exist in researching complex changes of assessment practice from traditional objective-focused 'assessments of learning' towards programmatic 'assessment for learning'. The latter emphasise both the subjective and social in collective judgements of student progress. Our context was a purposively designed programmatic assessment system implemented in the first year of a new graduate entry curriculum. We applied critical realist perspectives to unpack the underlying causes (mechanisms) that explained student experiences of programmatic assessment, to optimise assessment practice for future iterations.

METHODS

Data came from 14 in-depth focus groups (N = 112/261 students). We applied a critical realist lens drawn from Bhasker's three domains of reality (the actual, empirical and real) and Archer's concept of structure and agency to understand the student experience of programmatic assessment. Analysis involved induction (pattern identification), abduction (theoretical interpretation) and retroduction (causal explanation).

RESULTS

As a complex educational and social change, the assessment structures and culture systems within programmatic assessment provided conditions (constraints and enablements) and conditioning (acceptance or rejection of new 'non-traditional' assessment processes) for the actions of agents (students) to exercise their learning choices. The emergent underlying mechanism that most influenced students' experience of programmatic assessment was one of balancing the complex relationships between learner agency, assessment structures and the cultural system.

CONCLUSIONS

Our study adds to debates on programmatic assessment by emphasising how the achievement of balance between learner agency, structure and culture suggests strategies to underpin sustained changes (elaboration) in assessment practice. These include; faculty and student learning development to promote collective reflexivity and agency, optimising assessment structures by enhancing integration of theory with practice, and changing learning culture by both enhancing existing and developing new social structures between faculty and the student body to gain acceptance and trust related to the new norms, beliefs and behaviours in assessing for and of learning.

Promoting a sense of belonging, engagement, and collegiality to reduce burnout: a mixed methods study among undergraduate medical students in a non-Western, Asian context

BACKGROUND

Burnout is a psychological condition induced by work-related chronic interpersonal stressors. Interventions creating a sense of belonging and collegiality have been proposed as approaches for alleviating burnout. The current study aimed to: (1) explore the relationships between burnout, sense of belonging (relatedness with others), and work engagement; and (2) identify the key elements perceived by undergraduate medical students as positively contributing to collegiality, engagement, and a sense of belonging, in an undergraduate medical training setting.

METHODS

An exploratory sequential mixed-methods design using questionnaires and semi-structured individual interviews collected quantitative and qualitative data among undergraduate medical students at Mahidol University, Thailand. The Maslach Burnout Inventory-Student Survey questionnaire was used to measure burnout. The Basic Psychological Need Satisfaction at Work Scale (BPNSS-21) and the Utrecht Work Engagement Scale-Student Version (UWESS-9) measured students' basic psychological needs satisfaction at work and work engagement, respectively. Descriptive statistical analysis and confirmatory factor analysis were performed on BPNSS-21 and UWESS-9 data. Spearman's correlation coefficient was used to identify the correlation between burnout and other factors. Twenty undergraduate medical students participated in the qualitative study. Qualitative analysis was conducted iteratively using constant comparison and the standard principles of primary, secondary, and tertiary coding for thematic analysis.

RESULTS

Thai versions of the BPNSS-21 and UWESS-9 showed an acceptable fit for the Thai cultural context. Burnout had significant weak inverse associations with engagement (r = - 0.39, p < 0.005) and basic psychological needs satisfaction (r = - 0.37, p < 0.005). Sense of belonging had a significant weak inverse relationship with burnout (r = - 0.25, p < 0.005). The main themes emerging from qualitative analysis were relevant tasks and learning activities, safety in the learning environment, peer interaction, program design factors, dynamics of collegiality while progressing through medical school, and personal stance and social skills.

CONCLUSIONS

Sense of belonging, engagement, and collegiality were related to burnout. The key features for promoting collegiality, the sense of belonging, and engagement were relevant tasks and learning activities, safety in the learning environment, peer interaction, program design factors, dynamics of collegiality while progressing through medical school, and personal stance and social skills.

Design and First Impressions of a Small Private Online Course in Clinical Workplace Learning: Questionnaire and Interview Study

BACKGROUND

Clinical workplace learning takes place in a dynamic and complex learning environment that is designated as a site for patient care and education. Challenges in clinical training can be overcome by implementing blended learning, as it offers flexible learning programs suitable for student-centered learning, web-based collaboration, and peer learning.

OBJECTIVE

The aim of this study is to evaluate the Small Private Online Course (SPOC) by interns' first impressions and satisfaction measures (N=20) on using the SPOC. This study describes the design process of a SPOC from a theoretical and practical perspective and how it has been integrated into a clinical internship in internal medicine.

METHODS

The design of the SPOC was based on general theoretical principles that learning should be constructive, contextual, collaborative, and self-regulated, and the self-determination theory to stimulate intrinsic motivation. Interns' impressions and level of satisfaction were evaluated with a web-based questionnaire and group interview.

RESULTS

Interns thought the web-based learning environment to be a useful and accessible alternative to improve knowledge and skills. Peer learning and web-based collaboration through peer interaction was perceived as less effective, as student feedback was felt inferior to teacher feedback. The interns would prefer more flexibility within the course, which could improve self-regulated learning and autonomy.

CONCLUSIONS

The evaluation shows that the SPOC is a useful and accessible addition to the clinical learning environment, providing an alternative opportunity to improve knowledge and skills. Further research is needed to improve web-based collaboration and interaction in our course.

Exploring complexities in the reform of assessment practice: a critical realist perspective

Although the principles behind assessment for and as learning are well-established, there can be a struggle when reforming traditional assessment of learning to a program which encompasses assessment for and as learning. When introducing and reporting reforms, tensions in faculty may arise because of differing beliefs about the relationship between assessment and learning and the rules for the validity of assessments. Traditional systems of assessment of learning privilege objective, structured quantification of learners' performances, and are done to the students. Newer systems of assessment promote assessment for learning, emphasise subjectivity, collate data from multiple sources, emphasise narrative-rich feedback to promote learner agency, and are done with the students. This contrast has implications for implementation and evaluative research. Research of assessment which is done to students typically asks, "what works", whereas assessment that is done with the students focuses on more complex questions such as "what works, for whom, in which context, and why?" We applied such a critical realist perspective drawing on the interplay between structure and agency, and a systems approach to explore what theory says about introducing programmatic assessment in the context of pre-existing traditional approaches. Using a reflective technique, the internal conversation, we developed four factors that can assist educators considering major change to assessment practice in their own contexts. These include enabling positive learner agency and engagement; establishing argument-based validity frameworks; designing purposeful and eclectic evidence-based assessment tasks; and developing a shared narrative that promotes reflexivity in appreciating the complex relationships between assessment and learning.

Ottawa 2020 consensus statements for programmatic assessment - 2. Implementation and practice

INTRODUCTION

Programmatic assessment is a longitudinal, developmental approach that fosters and harnesses the learning function of assessment. Yet the implementation, a critical step to translate theory into practice, can be challenging. As part of the Ottawa 2020 consensus statement on programmatic assessment, we sought to provide descriptions of the implementation of the 12 principles of programmatic assessment and to gain insight into enablers and barriers across different institutions and contexts.

METHODS

After the 2020 Ottawa conference, we surveyed 15 Health Profession Education programmes from six different countries about the implementation of the 12 principles of programmatic assessment. Survey responses were analysed using a deductive thematic analysis.

RESULTS AND DISCUSSION

A wide range of implementations were reported although the principles remained, for the most part, faithful to the original enunciation and rationale. Enablers included strong leadership support, ongoing faculty development, providing students with clear expectations about assessment, simultaneous curriculum renewal and organisational commitment to change. Most barriers were related to the need for a paradigm shift in the culture of assessment. Descriptions of implementations in relation to the theoretical principles, across multiple educational contexts, coupled with explanations of enablers and barriers, provided new insights and a clearer understanding of the strategic and operational considerations in the implementation of programmatic assessment. Future research is needed to further explore how contextual and cultural factors affect implementation.

How Do Clerkship Students Use EPA Data? Illuminating Students' Perspectives as Partners in Programs of Assessment

INTRODUCTION

The implementation of programs of assessment based on Entrustable Professional Activities (EPAs) offers an opportunity for students to obtain unique data to guide their ongoing learning and development. Although authors have explored factors that contribute to trust-based decisions, learners' use of assessors' decisions about the level of supervision they need has not been fully investigated.

METHODS

In this study, we conducted semi-structured interviews of clerkship students who participated in the first year of our EPA program to determine how they interpret and use supervision ratings provided in EPA assessments. Content analysis was performed using concept-driven and open coding.

RESULTS

Nine interviews were completed. Twenty-two codes derived from previous work describing factors involved in trust decisions and 12 novel codes were applied to the interview text. Analyses revealed that students focus on written and verbal feedback from assessors more so than on supervision ratings. Axial coding revealed a temporal organization that categorized how students considered the data from EPA assessments. While factors before, during, and after an assessment affected students' use of information, the relationship between the student and the assessor had impact throughout.

CONCLUSIONS

Although students reported varying use of the supervision ratings, their perspectives about how assessors and students interact and/or partner before, during, and after assessments provide insights into the importance of an educational alliance in making a program of assessment meaningful and acceptable to learners.

Numbers Encapsulate, Words Elaborate: Toward the Best Use of Comments for Assessment and Feedback on Entrustment Ratings

The adoption of entrustment ratings in medical education is based on a seemingly simple premise: to align workplace-based supervision with resident assessment. Yet it has been difficult to operationalize this concept. Entrustment rating forms combine numeric scales with comments and are embedded in a programmatic assessment framework, which encourages the collection of a large quantity of data. The implicit assumption that more is better has led to an untamable volume of data that competency committees must grapple with. In this article, the authors explore the roles of numbers and words on entrustment rating forms, focusing on the intended and optimal use(s) of each, with a focus on the words. They also unpack the problematic issue of dual-purposing words for both assessment and feedback. Words have enormous potential to elaborate, to contextualize, and to instruct; to realize this potential, educators must be crystal clear about their use. The authors set forth a number of possible ways to reconcile these tensions by more explicitly aligning words to purpose. For example, educators could focus written comments solely on assessment; create assessment encounters distinct from feedback encounters; or use different words collected from the same encounter to serve distinct feedback and assessment purposes. Finally, the authors address the tyranny of documentation created by programmatic assessment and urge caution in yielding to the temptation to reduce words to numbers to make them manageable. Instead, they encourage educators to preserve some educational encounters purely for feedback, and to consider that not all words need to become data.

Entrustment Unpacked: Aligning Purposes, Stakes, and Processes to Enhance Learner Assessment

Educators use entrustment, a common framework in competency-based medical education, in multiple ways, including frontline assessment instruments, learner feedback tools, and group decision making within promotions or competence committees. Within these multiple contexts, entrustment decisions can vary in purpose (i.e., intended use), stakes (i.e., perceived risk or consequences), and process (i.e., how entrustment is rendered). Each of these characteristics can be conceptualized as having 2 distinct poles: (1) purpose has formative and summative, (2) stakes has low and high, and (3) process has ad hoc and structured. For each characteristic, entrustment decisions often do not fall squarely at one pole or the other, but rather lie somewhere along a spectrum. While distinct, these continua can, and sometimes should, influence one another, and can be manipulated to optimally integrate entrustment within a program of assessment. In this article, the authors describe each of these continua and depict how key alignments between them can help optimize value when using entrustment in programmatic assessment within competency-based medical education. As they think through these continua, the authors will begin and end with a case study to demonstrate the practical application as it might occur in the clinical learning environment.

Communion, Care, and Leadership in Computer-Mediated Learning during the Early Stage of COVID-19

This qualitative inquiry explores how, during the early stage of the COVID-19 pandemic in Lithuania (European Union), the stakeholders in the education system—university teachers, general education teachers, students, and children’s parents—coped with the encountered challenge and what was important to them under the changed conditions. This paper uses a communication management objective to describe how participants in the education system responded to the emerging distance learning situation and its challenges. The phenomenographic research approach was chosen to carry out the qualitative study. The 37 interviews from higher education teachers, university students, school teachers, and parents of minor school-aged children were conducted during the early stage of COVID-19 quarantine. The research allows for concluding that, after a successful transition to distance learning, the dimensions of communion and supportive collaboration acquired importance among stakeholders in education. Starting new activities, a need for communion and mobilization for joint activities under the crisis emerged. The research showed that the adaptation period was necessary at the beginning of these new activities. Competent leadership was expected from the teacher. Additionally, the preparation of all the actors in the education process was needed. The data offer a window into the dynamics of online teaching in crisis and experiences with a new activity that are key to success. Although research on technology-mediated learning has increased in recent years, it still lags behind developments in practice.

Where the rubber meets the road - An integrative review of programmatic assessment in health care professions education

INTRODUCTION

Programmatic assessment was introduced as an approach to design assessment programmes with the aim to simultaneously optimize the decision-making and learning function of assessment. An integrative review was conducted to review and synthesize results from studies investigating programmatic assessment in health care professions education in practice.

METHODS

The authors systematically searched PubMed, Web of Science, and ERIC to identify studies published since 2005 that reported empirical data on programmatic assessment. Characteristics of the included studies were extracted and synthesized, using descriptive statistics and thematic analysis.

RESULTS

Twenty-seven studies were included, which used quantitative methods (n = 10), qualitative methods (n = 12) or mixed methods (n = 5). Most studies were conducted in clinical settings (77.8%). Programmatic assessment was found to enable meaningful triangulation for robust decision-making and used as a catalyst for learning. However, several problems were identified, including overload in assessment information and the associated workload, counterproductive impact of using strict requirements and summative signals, lack of a shared understanding of the nature and purpose of programmatic assessment, and lack of supportive interpersonal relationships. Thematic analysis revealed that the success and challenges of programmatic assessment were best understood by the interplay between quantity and quality of assessment information, and the influence of social and personal aspects on assessment perceptions.

CONCLUSION

Although some of the evidence may seem compelling to support the effectiveness of programmatic assessment in practice, tensions will emerge when simultaneously stimulating the development of competencies and assessing its result. The identified factors and inferred strategies provide guidance for navigating these tensions.