Problems presented by medical students seeking support: a possible intervention framework

Managing assessment during curriculum change: Ottawa Consensus Statement

Curriculum change is relatively frequent in health professional education. Formal, planned curriculum review must be conducted periodically to incorporate new knowledge and skills, changing teaching and learning methods or changing roles and expectations of graduates. Unplanned curriculum evolution arguably happens continually, usually taking the form of "minor" changes that in combination over time may produce a substantially different programme. However, reviewing assessment practices is less likely to be a major consideration during curriculum change, overlooking the potential for unintended consequences for learning. This includes potentially undermining or negating the impact of even well-designed and important curriculum changes. Changes to any component of the curriculum "ecosystem "- graduate outcomes, content, delivery or assessment of learning - should trigger an automatic review of the whole ecosystem to maintain constructive alignment. Consideration of potential impact on assessment is essential to support curriculum change. Powerful contextual drivers of a curriculum include national examinations and programme accreditation, so each assessment programme sits within its own external context. Internal drivers are also important, such as adoption of new learning technologies and learning preferences of students and faculty. Achieving optimal and sustainable outcomes from a curriculum review requires strong governance and support, stakeholder engagement, curriculum and assessment expertise and internal quality assurance processes. This consensus paper provides guidance on managing assessment during curriculum change, building on evidence and the contributions of previous consensus papers.

Using document phenomenology to investigate academic failure among year 1 undergraduate Malaysian medical students

BACKGROUND

Academic failure is common among medical schools worldwide. However, the process behind this failure itself is underexplored. A deeper understanding of this phenomenon may avert the vicious cycle of academic failure. Hence, this study investigated the process of academic failure among medical students in Year 1.

METHODS

This study employed a document phenomenological approach, which is a systematic process to examine documents, interpret them to attain understanding, and develop empirical knowledge of the phenomenon studied. Using document analysis, interview transcripts and reflective essays of 16 Year 1 medical students who experienced academic failure were analysed. Based on this analysis, codes were developed and further reduced into categories and themes. Thirty categories in eight themes were linked to make sense of the series of events leading to academic failure.

RESULTS

One or more critical incidents commenced during the academic year, which led to possible resulting events. The students had poor attitudes, ineffective learning methods, health problems or stress. Students progressed to mid-year assessments and reacted differently to their results in the assessments. Afterwards, the students tried different types of attempts, and they still failed the end-of-year assessments. The general process of academic failure is illustrated in a diagram describing chronological events.

CONCLUSION

Academic failure may be explained by a series of events (and consequences) of what students experience and do and how they respond to their experiences. Preventing a preceding event may prevent students from suffering these consequences.

A Simple and Sustainable Exercise to Enhance Student Self-Reflection on Error-Making, Focus Support, and Guide Curricular Design

Problem: Self-reflection is a critical component of professional development and clinical practice, but medical students' ability to self-reflect is typically limited. While inadequate self-reflection impacts future clinical decision-making, it may also adversely impact current learning through an inability to identify learning-behavior deficits. This may be exacerbated by common use of multiple-choice questions (MCQ) where incorrect responses provide less insight than other measures for students, faculty, or academic support. To address this, an Error Reflection Method (ERM) was developed to help students focus on 'why' they got an MCQ wrong rather than 'what' they got wrong, thereby promoting self-reflection and a learning-focus on assessment. Understanding students' learning-behavior deficits could also enrich engagement with academic support services and guide curricular design. Intervention: The ERM is a list of 10 common types of exam errors that were either 'test-taking' (unwitting) errors or 'learning-behavior' errors that reflected learning deficits. The ERM is simple, transferable, and sustainable, allowing longitudinal and regular monitoring of individual and collective error-making to focus support and guide curricular development. Context: Undergraduate medical students at the Virginia Tech Carilion School of Medicine, USA, used the ERM in formative assessment review sessions in pre-clinical years to select an error type that best described the cause of each incorrect response. Impact: Initial findings suggest the ERM is robust and associated with improved student performance and curricular development. Analysis of 3,775 student-identified errors showed the error types in the ERM described 96% of errors students made. Learning-behavior errors were more common (76%), but surprisingly, 19% were test-taking errors, allowing academic support to focus on test-taking skills in a population previously thought of as consummate test-takers. The most common error type reported was 'the content looked familiar but I couldn't answer the question' (32%); which we suggest is consistent with shallow learning. This finding has helped steer recent curricular development toward active and applied learning techniques. Lessons Learned: By formally and regularly identifying learning deficits, students may be more capable of addressing them and improve summative exam performance. As well as focusing academic support, understanding common student errors has been useful in guiding curricular design and content delivery. Further potential of the ERM may be realized in faculty development and directing assessment culture toward a learning focus.

The Impact of Undergraduate Tutor System in Chinese 8-Year Medical Students in Scientific Research

Purpose

This study aimed to evaluate the effect and influence of the undergraduate tutor system on the undergraduate stage of Chinese 8-year medical program students in scientific research.

Methods

We collected related data from 194 medical students in the Xiangya Medical School of Central South University. The questionnaire was composed of three parts, namely, eight questions for basic information about individual and undergraduate tutor system, five questions for the subjective feeling impact of the undergraduate tutor system, and 22 questions for accessing the scientific research ability and academic results. The students were mainly divided into three groups to compare different kinds of undergraduate tutor systems, namely, single tutor for multiple students' system (group A), multiple tutors for multiple students' system (group B), and no tutor system for comparison (group C).

Results

The type of tutorial system, the frequency of guidance, and the way of guidance were independent influence factors of the view of 8-year medical students on scientific research. Group B behaved better than group C in literature processing (P = 0.012), experimental operation (P < 0.001), statistical analysis (P < 0.001), and manuscript producing (P = 0.019). Group A and B joined in more National college students' innovation and entrepreneurship training programs (P = 0.003, P < 0.001). The most popular types of articles published by students were bioinformatics, meta-analysis, and reviews.

Conclusion

Undergraduate tutor system has made tremendous achievements in cultivating students' scientific research capacity; however, implement improvement should be considered to better educate students.

Why do students struggle in their first year of medical school? A qualitative study of student voices

BACKGROUND

Struggling at medical school incorporates academic failure, course disruption and early course exit. Struggling is usually multi-factorial involving academic, personal, financial and health factors. Struggling students may fail to engage with available support. First year students are particularly susceptible as they transition to university and a professional career.

METHODS

The study aim was to explore medical students' own voices on struggling and assess how they match up to existing literature. During one academic year, all first year medical students at the University of Birmingham (UK) who opted to leave or were required to withdraw (n = 52) were asked to participate in an individual exit interview. Fifteen students responded and fourteen (27%) agreed to be interviewed. Interviews were face to face (n = 10), telephone (n = 3) and via email (n = 1). Interviews were unstructured and led by a general open question. Framework analysis identified key data themes.

RESULTS

Students described year one of medical school as a critical transition. They simultaneously needed to adapt to being a university student, a medical student and a doctor. A six-group typology of students emerged, each of which struggled with one or more of these adaptations. The groups were: wrong degree choice, mental health problems, acute crisis, at capacity, slow starter and family rock. Some students experienced an isolated problem from within this typology. Most had a multi-factorial story of struggling. Mental health problems and acute crises were the most common issues. Early professional identity formation was a key hurdle. Help-seeking behaviours were varied.

CONCLUSIONS

This study explores the narratives of medical students who struggled from an early stage and presents a data-driven typology of their issues. It advances existing qualitative understanding of this topic, which to date is predominantly derived from educator perceptions and not specific to early course issues. Although our results broadly cohere with existing knowledge, we also present novel findings which may reflect our focus on first year students. Issues around early professional identity formation may reflect the increasing emphasis on professionalism in medical school curricula. Listening to these narratives could help university staff to identify students at risk of struggling for targeted support.

A qualitative study on self-regulated learning among high performing medical students

BACKGROUND

Self-regulated learning (SRL) is an important contributing element to the academic success of students. Literature suggests that the understanding of SRL among medical students is obscure as there is still some uncertainty about whether high performing medical students use SRL. This study explored the characteristics of high performing medical students from the SRL perspective to gain a better understanding of the application of SRL for effective learning.

METHODS

Twenty-one students who scored at the 90th percentile in written knowledge-based assessment consented to participate in this study. Each student wrote a guided reflective journal and subsequently attended a semi-structured interview. Students were prompted to explain the rationales for their answers. The data were then analysed using thematic analysis to identify patterns among these students from the SRL perspective. Two coders analysed the data independently and discussed the codes to reach a consensus.

RESULTS

High performing students set goals, made plans, and motivated themselves to achieve the goals. They put consistent efforts into their studies and applied effective learning strategies. They also employed coping mechanisms to deal with challenges. High performing students regularly evaluated their performance and adopted new strategies.

CONCLUSIONS

This study reported that high performing students applied SRL and described the rationales of practice. Medical schools could design SRL-driven interventions to enhance the learning experiences of medical students. Recommendations are made for students on how to apply SRL.

How to identify, address and report students' unprofessional behaviour in medical school

This AMEE guide provides a research overview of the identification of, and responding to unprofessional behaviour in medical students. It is directed towards medical educators in preclinical and clinical undergraduate medical education. It aims to describe, clarify and categorize different types of unprofessional behaviours, highlighting students' unprofessional behaviour profiles and what they mean for further guidance. This facilitates identification, addressing, reporting and remediation of different types of unprofessional behaviour in different types of students in undergraduate medical education. Professionalism, professional behaviour and professional identity formation are three different viewpoints in medical education and research. Teaching and assessing professionalism, promoting professional identity formation, is the positive approach. An inevitable consequence is that teachers sometimes are confronted with unprofessional behaviour. When this happens, a complementary approach is needed. How to effectively respond to unprofessional behaviour deserves our attention, owing to the amount of time, effort and resources spent by teachers in managing unprofessional behaviour of medical students. Clinical and medical educators find it hard to address unprofessional behaviour and turn toward refraining from handling it, thus leading to the 'failure to fail' phenomenon. Finding the ways to describe and categorize observed unprofessional behaviour of students encourages teachers to take the appropriate actions.

Guidelines: The dos, don'ts and don't knows of remediation in medical education

INTRODUCTION

Two developing forces have achieved prominence in medical education: the advent of competency-based assessments and a growing commitment to expand access to medicine for a broader range of learners with a wider array of preparation. Remediation is intended to support all learners to achieve sufficient competence. Therefore, it is timely to provide practical guidelines for remediation in medical education that clarify best practices, practices to avoid, and areas requiring further research, in order to guide work with both individual struggling learners and development of training program policies.

METHODS

Collectively, we generated an initial list of Do's, Don'ts, and Don't Knows for remediation in medical education, which was then iteratively refined through discussions and additional evidence-gathering. The final guidelines were then graded for the strength of the evidence by consensus.

RESULTS

We present 26 guidelines: two groupings of Do's (systems-level interventions and recommendations for individual learners), along with short lists of Don'ts and Don't Knows, and our interpretation of the strength of current evidence for each guideline.

CONCLUSIONS

Remediation is a high-stakes, highly complex process involving learners, faculty, systems, and societal factors. Our synthesis resulted in a list of guidelines that summarize the current state of educational theory and empirical evidence that can improve remediation processes at individual and institutional levels. Important unanswered questions remain; ongoing research can further improve remediation practices to ensure the appropriate support for learners, institutions, and society.

Developing a two-dimensional model of unprofessional behaviour profiles in medical students

Standardized narratives or profiles can facilitate identification of poor professional behaviour of medical students. If unprofessional behaviour is identified, educators can help the student to improve their professional performance. In an earlier study, based on opinions of frontline teachers from one institution, the authors identified three profiles of medical students' unprofessional behaviour: (1) Poor reliability, (2) Poor reliability and poor insight, and (3) Poor reliability, poor insight and poor adaptability. The distinguishing variable was Capacity for self-reflection and adaptability. The current study used Nominal Group Technique and thematic analysis to refine these findings by synthesizing experts' opinions from different medical schools, aiming to develop a model of unprofessional behaviour profiles in medical students. Thirty-one experienced faculty, purposively sampled for knowledge and experience in teaching and evaluation of professionalism, participated in five meetings at five medical schools in the Netherlands. In each group, participants generated ideas, discussed them, and independently ranked these ideas by allocating points to them. Experts suggested ten different ideas, from which the top 3 received 60% of all ranking points: (1) Reflectiveness and adaptability are two distinct distinguishing variables (25%), (2) The term reliability is too narrow to describe unprofessional behaviour (22%), and (3) Profiles are dynamic over time (12%). Incorporating these ideas yielded a model consisting of four profiles of medical students' unprofessional behaviour (accidental behaviour, struggling behaviour, gaming-the-system behaviour and disavowing behaviour) and two distinguishing variables (reflectiveness and adaptability). The findings could advance educators' insight into students' unprofessional behaviour, and provide information for future research on professionalism remediation.

Student response to reports of unprofessional behavior: assessing risk of subsequent professional problems in medical school

BACKGROUND

An early concern note (ECN) program is used by some medical schools to identify, counsel, and intervene when students exhibit unprofessional behavior. Student maturity, insight, propensity for reflection, and receptiveness to feedback have been suggested as predictors of future behavior.

OBJECTIVE

We hypothesized that (a) classifying students with a first ECN based on their response to the report would identify students at risk of repeat ECNs better than the action that prompted it and (b) receipt of multiple ECNs would identify students at risk of adverse academic events.

DESIGN

For this study, 459 ECNs were classified based on students' (1) recognition that their behavior was inappropriate and (2) acceptance of responsibility for the behavior. Student academic progress and receipt of subsequent ECNs were tracked.

RESULTS

Students who recognized their behavior was inappropriate and accepted responsibility after an initial ECN received subsequent ECNs at lower rates (14-19%) than students who disagreed with the significance of their behavior or were resistant to accepting responsibility (36-59%). Students with limited insight and adaptability appeared to be at highest risk. Seventy-one percent of students with three or more ECNs encountered adverse academic events during enrollment.

CONCLUSION

Student reactions to reports of unprofessional behavior may be useful as a tool to help assess risk of recurrent lapses. Students with diminished capacity to recognize behaviors as unprofessional or accept responsibility for them appear to be at highest risk for additional adverse academic and professionalism events while in medical school.

Cross-cultural Differences in Mental Health, Quality of Life, Empathy, and Burnout between US and Brazilian Medical Students

OBJECTIVE

This study aimed to compare mental health, quality of life, empathy, and burnout in medical students from a medical institution in the USA and another one in Brazil.

METHODS

This cross-cultural study included students enrolled in the first and second years of their undergraduate medical training. We evaluated depression, anxiety, and stress (DASS 21), empathy, openness to spirituality, and wellness (ESWIM), burnout (Oldenburg), and quality of life (WHOQOL-Bref) and compared them between schools.

RESULTS

A total of 138 Brazilian and 73 US medical students were included. The comparison between all US medical students and all Brazilian medical students revealed that Brazilians reported more depression and stress and US students reported greater wellness, less exhaustion, and greater environmental quality of life. In order to address a possible response bias favoring respondents with better mental health, we also compared all US medical students with the 50% of Brazilian medical students who reported better mental health. In this comparison, we found Brazilian medical students had higher physical quality of life and US students again reported greater environmental quality of life. Cultural, social, infrastructural, and curricular differences were compared between institutions. Some noted differences were that students at the US institution were older and were exposed to smaller class sizes, earlier patient encounters, problem-based learning, and psychological support.

CONCLUSION

We found important differences between Brazilian and US medical students, particularly in mental health and wellness. These findings could be explained by a complex interaction between several factors, highlighting the importance of considering cultural and school-level influences on well-being.

The Four-Tier Continuum of Academic and Behavioral Support (4T-CABS) Model: An Integrated Model for Medical Student Success

Not all students cope successfully with the demands of medical school, and students' struggles may result in study delay or dropout. To prevent these outcomes, medical schools need to identify students who are experiencing academic difficul ties and provide them with timely interventions through access to support programs. Although the importance of early identification and intervention is well recognized, less is known about successful strategies for identifying and supporting struggling students.Building on the literature and their own empirical findings, the authors propose an integrated, school-wide model for medical student success comprising a continuum of academic and behavioral support. This Four-Tier Continuum of Academic and Behavioral Support (4T-CABS) model focuses on improving both academic and behavioral outcomes by offering support for students at four levels, which range from adequate instruction for all, to targeted small-group interventions, to individualized support, and also include exit support for students who might be better off in another degree program. Additionally, medical schools should provide both academic and behavioral support; set high, yet realistic expectations and clearly communicate these to students; and intervene early, which requires timely identification of at-risk students who would benefit from the different types and tiers of support. Finally, interventions should be evidence based and fit the needs of the identified groups of students. The authors argue that adopting the core principles of the 4T-CABS model will enable medical schools to maximize academic engagement and performance for all students.

A preliminary investigation to explore the cognitive resources of physicians experiencing difficulty in training

BACKGROUND

Treating patients is complex, and research shows that there are differences in cognitive resources between physicians who experience difficulties, and those who do not. It is possible that differences in some cognitive resources could explain the difficulties faced by some physicians. In this study, we explore differences in cognitive resources between different groups of physicians (that is, between native (UK) physicians and International Medical Graduates (IMG); those who continue with training versus those who were subsequently removed from the training programme); and also between physicians experiencing difficulties compared with the general population.

METHODS

A secondary evaluation was conducted on an anonymised dataset provided by the East Midlands Professional Support Unit (PSU). One hundred and twenty one postgraduate trainee physicians took part in an Educational Psychology assessment through PSU. Referrals to the PSU were mainly on the basis of problems with exam progression and difficulties in communication skills, organisation and confidence. Cognitive resources were assessed using the Wechsler Adult Intelligence Scale (WAIS-IV). Physicians were categorised into three PSU outcomes: 'Continued in training', 'Removed from training' and 'Active' (currently accessing the PSU).

RESULTS

Using a one-sample Z test, we compared the referred physician sample to a UK general population sample on the WAIS-IV and found the referred sample significantly higher in Verbal Comprehension (VCI; z = 8.78) and significantly lower in Working Memory (WMI; z = -4.59). In addition, the native sample were significantly higher in Verbal Comprehension than the UK general population sample (VCI; native physicians: z = 9.95, p < .001, d = 1.25), whilst there was a lesser effect for the difference between the IMG sample and the UK general population (z = 2.13, p = .03, d = 0.29). Findings also showed a significant difference in VCI scores between those physicians who were 'Removed from training' and those who 'Continued in training'.

CONCLUSIONS

Our results suggest it is important to understand the cognitive resources of physicians to provide a more focussed explanation of those who experience difficulties in training. This will help to implement more targeted interventions to help physicians develop compensatory strategies.

The validity of student tutors' judgments in early detection of struggling in medical school. A prospective cohort study

Early identification and support of strugglers in medical education is generally recommended in the research literature, though very little evidence of the diagnostic qualities of early teacher judgments in medical education currently exists. The aim of this study was to examine the validity of early diagnosis of struggling in medical school based on informal teacher judgements of in-class behavior. The study design was a prospective cohort study and the outcomes/truth criteria were anatomy failure and medical school drop out. Six weeks into an anatomy course, student tutors attempted to identify medical students, who they reckoned would fail the anatomy course or drop out, based on their everyday experiences with students in a large group educational setting. In addition, they were asked to describe the indicators of struggling they observed. Sixteen student tutors evaluated 429 medical students for signs of struggling. By week six, the student tutors were able to detect approximately 1/4-1/3 of the students who eventually failed or dropped out, and for ¾ of the strugglers they identified, they were correct in their judgments. Informal student tutor's judgements showed incremental validity for both outcomes when controlling for grades obtained in preceeding exams. Lack of participation, lack of commitment, poor academic performance, poor social interactions and general signs of distress were the main indicators of struggling identified. Teachers' informal judgements of in-class behavior may be an untapped source of information in the early identification of struggling medical students with added value above and beyond formal testing.

Residents in difficulty--just slower learners? A case-control study

BACKGROUND

Recent meta-analyses have found small-moderate positive associations between general performance in medical school and postgraduate medical education. In addition, a couple of studies have found an association between poor performance in medical school and disciplinary action against practicing doctors. The aim of this study was to examine if a sample of Danish residents in difficulty tended to struggle already in medical school, and to determine whether administratively observable performance indicators in medical school could predict difficulties in residency.

METHODS

The study design was a cumulative incidence matched case-control study. The source population was all active specialist trainees, who were medical school graduates from Aarhus University, in 2010 to June 2013 in two Danish regions. Cases were doctors who decelerated, transferred, or dropped out of residency. Cases and controls were matched for graduation year. Medical school exam failures, grades, completion time, and academic dispensations as predictors of case status were examined with conditional logistic regression.

RESULTS

In total 89 cases and 343 controls were identified. The total number of medical school re-examinations and the time it took to complete medical school were significant individual predictors of subsequent difficulties (deceleration, transferral or dropout) in residency whereas average medical school grades were not.

CONCLUSIONS

Residents in difficulty eventually reached similar competence levels as controls during medical school; however, they needed more exam attempts and longer time to complete their studies, and so seemed to be slower learners. A change from "fixed-length variable-outcome programmes" to "fixed-outcome variable-length programmes" has been proposed as a way of dealing with the fact that not all learners reach the same level of competence for all activities at exactly the same time. This study seems to support the logic of such an approach to these residents in difficulty.

Twelve tips for providing effective student support in undergraduate medical education

Medical students often require high levels of specialised institutional and personal support to facilitate success. Contributory factors may include personality type, course pressures and financial hardship. Drawing from research literature and the authors' experience, 12 tips are listed under five subheadings: policy and systems; people and resources; students; delivering support; limits of support. The 12 tips provide guidance to organisations and individual providers that encourages implementation of good practice and helps them better visualise their role within the system. By following the tips, medical schools can make more effective provisions for the expected, diverse and sometimes specialist needs of their students. Schools must take a proactive, anticipatory approach to provide appropriately for their entire student body. This ensures that students receive the best quality support, are more likely to succeed and are adequately prepared for their medical careers.

The "problem" learner: whose problem is it? AMEE Guide No. 76

Clinical teachers often work with students or residents whom they perceive as a "problem". For some, it is a knowledge deficit that first alerts them to a problem; for others it is an attitudinal problem or distressing behaviour . And in some cases, it is difficult to know if the learner is, indeed, presenting with a problem. The goal of this Guide is to outline a framework for working with "problem" learners. This includes strategies for identifying and defining learners' problems, designing and implementing appropriate interventions, and assuring due process. The potential stress of medical school and residency training will also be addressed, as will a number of prevention strategies. Identifying learners' problems early - and providing guidance from the outset - can be an important investment in the training and development of future health professionals. It is hoped that this Guide will be of help to clinical teachers, program directors and faculty developers.

Development of a 'toolkit' to identify medical students at risk of failure to thrive on the course: an exploratory retrospective case study

BACKGROUND

An earlier study at Nottingham suggested that 10-15% of the medical student intake was likely to fail completely or have substantial problems on the course. This is a problem for the students, the Faculty, and society as a whole. If struggling students could be identified early in the course and additional pastoral resources offered, some of this wastage might be avoided. An exploratory case study was conducted to determine whether there were common indicators in the early years, over and above academic failure, that might aid the identification of students potentially at risk.

METHODS

The study group was drawn from five successive cohorts. Students who had experienced difficulties were identified in any of four ways: from Minutes of the Academic Progress Committee; by scanning examination lists at key stages (end of the first two years, and finals at the end of the clinical course); from lists of students flagged to the Postgraduate Deanery as in need of extra monitoring or support; and from progress files of those who had left the course prematurely. Relevant data were extracted from each student's course progress file into a customised database.

RESULTS

1188 students were admitted over the five years. 162 (14%) were identified for the study, 75 of whom had failed to complete the course by October 2010. In the 87 who did graduate, a combination of markers in Years 1 and 2 identified over half of those who would subsequently have the most severe problems throughout the course. This 'toolkit' comprised failure of 3 or more examinations per year, an overall average of <50%, health or social difficulties, failure to complete Hepatitis B vaccination on time, and remarks noted about poor attitude or behaviour.

CONCLUSIONS

A simple toolkit of academic and non-academic markers could be used routinely to help identify potential strugglers at an early stage, enabling additional support and guidance to be given to these students.