Voluntary vs. compulsory student evaluation of clerkships: effect on validity and potential bias

Starting from the needs: what are the appropriate sources to co-create innovative solutions for persons with disabilities?

PURPOSE

Technical solutions could facilitate activities and participation in individuals with disabilities. For the development of solutions, hackathons are a method of interdisciplinary collaboration. For hackathon, the definition of pain points that require solutions is crucial. We aimed to determine engineers' preferences and expectations regarding pain point qualities.

METHODS

We used a collaborative approach involving individuals with disability, families, and healthcare professionals to determine pain points for use by engineering students during a disability Hackathon. A pain point bank was built using 3 upstream sources: a survey (350 responses, 20 pain points selected), interviews (8 children, 13 pain points), and a multidisciplinary workshop based on design thinking methods (45 people, 32 pain points). A fourth source was 20 adults with disabilities present during the Hackathon. Engineering students rated pain point qualities from each source in a questionnaire that included closed questions relating to predefined criteria: achievability, specificity, relevance and attractiveness and open questions to collect non-predefined quality criteria.

RESULTS

Pain points from the workshop were most frequently used (48%); followed by on-site discussions with mentors (43%), the survey (38%), and interviews (31%). On-site discussions received the highest quality ratings followed by the workshop, survey, and interviews. Three quality criteria emerged from the responses to open questions: "representative", "empathy", and "real-need".

CONCLUSIONS

To be actionable by engineers, pain points must relate to real needs, be achievable, specific, relevant and attractive but also representative and arouse empathy. We devised a checklist of qualities along with a toolbox of methods to achieve each.

Implications For RehabilitationThe first step of the development of technical solutions for children and individuals with disabilities is the identification of their needs and their adequate formulation to be submitted to technical solutions providers.Daily life needs of individuals with disability were gathered for an engineering hackathon and proposed as pain points to 400 engineering students.To facilitate the development of solutions by engineers, pain points must relate to real needs, be specific, relevant, achievable and attractive; be representative and arouse empathy; a toolbox of needs collection methods is proposed to achieve each of those qualities.Discussions with individuals with disability and health professionals should be provided.

Keys to Successful Survey Research in Health Professions Education

Background

Survey research is well suited to measuring the knowledge, behavior, and attitudes of study participants and has been widely used in medical education and pulmonary and critical care medicine research. Although the ease of survey administration via electronic platforms has led to an increased volume of survey publications, improving the quality of this work remains an important challenge.

Objective

To provide an overview of key steps for rigorous survey design and conduct.

Methods

Narrative review.

Results

Conducting survey research begins with a clearly defined research question pertaining to a specified population that is accessible for sampling. Survey investigators may choose to adapt relevant preexisting survey instruments, an approach with the potential for conducting more valid, generalizable, and comparable studies. If a new survey tool is used, more extensive piloting and psychometric analysis of the survey instruments may be needed to assess if they accurately measure the concepts of interest. When administering the survey, the use of appropriate methods for sample recruitment maximizes the chances of a high response rate in a generalizable study population. Finally, when writing up and disseminating survey research, careful attention to reporting guidelines can increase the clarity of survey reports and assist readers in interpreting the results and conclusions.

Conclusion

With careful attention to study design and conduct, the quality of survey research can be improved and lead to higher impact and more generalizable studies in the fields of medical education and pulmonary and critical care medicine.

Restructuring an internal medicine rotation to take on a fully virtual experience in a resource-limited setting: A mixed-methods study

BACKGROUND

During the coronavirus disease 2019 (COVID-19) pandemic, medical schools in the Philippines accelerated the adoption of virtual learning. Course developers were challenged to provide equal opportunities for clinical exposure given the differential access of students to technology. This study describes the modifications in the course design of an internal medicine rotation for third-year medical students and the perceptions of the faculty and students toward these changes.

MATERIALS AND METHODS

Course evaluations by students and faculty were reviewed. Using a concurrent mixed-methods approach, we analyzed the quantitative and qualitative responses and triangulated the results of the faculty and student surveys.

RESULTS

Shifting to a virtual learning platform decreased the number of student-patient interactions. Observing a telemedicine consultation done by faculty substituted for real patient encounters. In consideration of students with limited Internet access, synchronous activities were made nongraded. The survey response rate was 51% (93/181) for students and 34% (32/94) for faculty. Survey participants indicated high overall satisfaction toward the virtual course with a general agreement between students and faculty respondents in most domains. Recurrent themes were the demand for more patient encounters, more synchronous activities, and better evaluation tools. Only the faculty were critical of technical issues, such as audibility and Internet connectivity.

CONCLUSION

The experiences of a single institution in redesigning and implementing an undergraduate medical course in internal medicine for a fully virtual platform were described. Strategies for augmenting patient exposure and tailored clinical assessment tools are needed to improve stakeholder satisfaction. In resource-limited settings, access to appropriate technology must be considered to ensure equitable learning.

Using standardized patients for undergraduate clinical skills training in an introductory course to psychiatry

BACKGROUND

The goal of this study was to assess the value and acceptance of Standardized or Simulated Patients (SPs) for training clinically inexperienced undergraduate medical students in psychiatric history taking, psychopathological assessment, and communication with psychiatric patients.

METHODS

As part of a newly developed introductory course to psychiatry, pairs of 3rd year medical students conducted psychiatric assessments of SPs, including history and psychopathological state, under the supervision of a clinical lecturer. Prior to the assessment, students attended introductory lectures to communication in psychiatry and psychopathology but were clinically inexperienced. After the interview, the students' summary of their findings was discussed with other students and the lecturer. Students, lecturers, and actors were invited to a survey after the course. Questions for the students included self-reports about perceived learning success and authenticity of the interviews.

RESULTS

41 students, 6 actors and 8 lecturers completed the survey (response rates of 48%, 50%, and 100%, respectively). The survey results indicated that, despite their lack of clinical experience, students learned how to conduct a psychiatric interview, communicate in a non-judgmental and empathetic manner, take a psychiatric history and perform a psychopathological examination. SPs were perceived as authentic. The survey results suggested that this setting allowed for an enjoyable, non-distressful and motivating learning experience within a restricted time frame of just two afternoons.

CONCLUSION

The results indicated that the SP approach presented is useful for teaching clinical skills in psychiatry to students with limited previous clinical experience and knowledge of psychiatry. We argue that SPs can be used to teach practical psychiatric skills already during an early phase of the curriculum. Limitations of our study include a limited sample size, a temporal gap between the course and the survey, reliance on self-reports, and lack of comparison to alternative interventions.

Integrated clinical case discussions - a fully student-organized peer-teaching program on internal medicine

BACKGROUND

In response to students´ poor ratings of emergency remote lectures in internal medicine, a team of undergraduate medical students initiated a series of voluntary peer-moderated clinical case discussions. This study aims to describe the student-led effort to develop peer-moderated clinical case discussions focused on training cognitive clinical skill for first and second-year clinical students.

METHODS

Following the Kern Cycle a didactic concept is conceived by matching cognitive learning theory to the competence levels of the German Medical Training Framework. A 50-item survey is developed based on previous evaluation tools and administered after each tutorial. Educational environment, cognitive congruence, and learning outcomes are assessed using pre-post-self-reports in a single-institution study.

RESULTS

Over the course of two semesters 19 tutors conducted 48 tutorials. There were 794 attendances in total (273 in the first semester and 521 in the second). The response rate was 32%. The didactic concept proved successful in attaining all learning objectives. Students rated the educational environment, cognitive congruence, and tutorials overall as "very good" and significantly better than the corresponding lecture. Students reported a 70%-increase in positive feelings about being tutored by peers after the session.

CONCLUSION

Peer-assisted learning can improve students´ subjective satisfaction levels and successfully foster clinical reasoning skills. This highlights successful student contributions to the development of curricula.

Impact of compulsory participation of medical students in a multiuser online game to learn radiological anatomy and radiological signs within the virtual world Second Life

Competitive game-based learning within Second Life enables effective teaching of basic radiological anatomy and radiological signs to medical students, with good acceptance and results when students participate voluntarily, but unknown in a compulsory context. The objectives of this study were to reproduce a competitive online game based on self-guided presentations and multiple-choice tests in a mandatory format, to evaluate its development and student perceptions compared to a voluntary edition in 2015 (N = 90). In 2016 and 2017, respectively, 191 and 182 third-year medical students participated in the game as a mandatory course activity. The mean (±SD) score of the game was 74.7% (±19.5%) in 2015, 71.2% (±21.5%) in 2016, and 67.5% (±21.5%) in 2017 (P < 0.01). Participants valued positively the organization and educational contents but found the virtual world less attractive and the game less interesting than in the voluntary edition. The experience globally was rated with 8.2 (±1.5), 7.8 (±1.5), and 7.1 (±1.7) mean points (±SD) in a ten-point scale, in the 2015, 2016, and 2017 editions, respectively (P < 0.05). Competitive learning games within virtual worlds like Second Life have great learning potential in radiology, but the mean score in the game decreased, acceptance of virtual world technology was lower, and opinion about the game was worse with a compulsory participation, and even worse when dropouts were not allowed. Under the conditions in which this study was conducted, learning games in three-dimensional virtual environments should be voluntary to maintain adequate motivation and engagement of medical students.

Serious Games in Surgical Medical Education: A Virtual Emergency Department as a Tool for Teaching Clinical Reasoning to Medical Students

Background

Serious games enable the simulation of daily working practices and constitute a potential tool for teaching both declarative and procedural knowledge. The availability of educational serious games offering a high-fidelity, three-dimensional environment in combination with profound medical background is limited, and most published studies have assessed student satisfaction rather than learning outcome as a function of game use.

Objective

This study aimed to test the effect of a serious game simulating an emergency department (“EMERGE”) on students’ declarative and procedural knowledge, as well as their satisfaction with the serious game.

Methods

This nonrandomized trial was performed at the Department of General, Visceral and Cancer Surgery at University Hospital Cologne, Germany. A total of 140 medical students in the clinical part of their training (5th to 12th semester) self-selected to participate in this experimental study. Declarative knowledge (measured with 20 multiple choice questions) and procedural knowledge (measured with written questions derived from an Objective Structured Clinical Examination station) were assessed before and after working with EMERGE. Students’ impression of the effectiveness and applicability of EMERGE were measured on a 6-point Likert scale.

Results

A pretest-posttest comparison yielded a significant increase in declarative knowledge. The percentage of correct answers to multiple choice questions increased from before (mean 60.4, SD 16.6) to after (mean 76.0, SD 11.6) playing EMERGE (P<.001). The effect on declarative knowledge was larger in students in lower semesters than in students in higher semesters (P<.001). Additionally, students’ overall impression of EMERGE was positive.

Conclusions

Students self-selecting to use a serious game in addition to formal teaching gain declarative and procedural knowledge.

Web-Based Immersive Patient Simulator as a Curricular Tool for Objective Structured Clinical Examination Preparation in Surgery: Development and Evaluation

BACKGROUND

Objective Structured Clinical Examination is a standard method of testing declarative and process knowledge in clinical core competencies. It is desirable that students undergo Objective Structured Clinical Examination training before participating in the exam. However, establishing Objective Structured Clinical Examination training is resource intensive and therefore there is often limited practice time. Web-based immersive patient simulators such as ALICE (Artificial Learning Interface of Clinical Education) can possibly fill this gap as they allow for the training of complex medical procedures at the user's individual pace and with an adaptable number of repetitions at home. ALICE has previously been shown to positively influence knowledge gain and motivation.

OBJECTIVE

Therefore, the aim of this study was to develop a Web-based curriculum that teaches declarative and process knowledge and prepares students for a real Objective Structured Clinical Examination station. Furthermore, we wanted to test the influence of ALICE on knowledge gain and student motivation.

METHODS

A specific curriculum was developed in order to implement the relevant medical content of 2 surgical Objective Structured Clinical Examination stations into the ALICE simulator framework. A total of 160 medical students were included in the study, where 100 students had access to ALICE and their performance was compared to 60 students in a control group. The simulator performance was validated on different levels and students' knowledge gain and motivation were tested at different points during the study.

RESULTS

The curriculum was developed according to the Kern cycle. Four virtual clinical cases were implemented with different teaching methods (structured feedback, keynote speech, group discussion, and debriefing by a real instructor) in order to consolidate declarative and process knowledge. Working with ALICE had significant impact on declarative knowledge gain and Objective Structured Clinical Examination performance. Simulator validation was positive for face, content, construct, and predictive validity. Students showed high levels of motivation and enjoyed working with ALICE.

CONCLUSIONS

ALICE offers Web-based training for Objective Structured Clinical Examination preparation and can be used as a selective didactic intervention as it has positive effect on knowledge gain and student motivation.