Narrative review: use of student-generated logbooks in undergraduate medical education

Investigating the advantages and disadvantages of electronic logbooks for education goals promotion in medical sciences students: A systematic review

Background and Aims

Electronic logbook (E-Logbook) is one of the practical software in medical science that serves as an auxiliary tool for comprehensive education, formative evaluation, and student learning documentation in clinical education. E-logbooks are available to people on the Internet without any time or place restrictions. Experts' familiarity with e-logbooks and their advantages and disadvantages can be effective in their better design so professors and students can use their potential benefits. Therefore, this study examines the advantages and disadvantages of an e-logbook.

Methods

This systematic review was conducted until June 13, 2022, by searching relevant keywords such as logbook, e-logbook, and medical students in PubMed, Scopus, and Web of Science databases. Data were extracted using the data extraction form. The contents of the studies were analyzed based on the study's aim. The results of the analyses were presented in the form of descriptive statistics (tables and figures).

Results

Out of 365 retrieved studies, 13 were selected to investigate the advantages and disadvantages of e-logbooks. Most studies were conducted in Pakistan (n = 4) and focused on medical students with different specialties (n = 10). The advantages and disadvantages of e-logbooks were classified into nine and four categories, respectively. Most advantages of e-logbooks were related to monitoring and evaluating the performance of students and instructors (n = 11). Their most disadvantages were associated with hardware and software (n = 8).

Conclusion

According to the results, e-logbooks can improve clinical education, provide feedback to people, control the achievement of educational goals, and increase professor-student interaction. Hence, it is recommended to address their disadvantages and barriers to improve the quality of students' performance.

Electronic logbooks (e-logbooks) for the continuous assessment of medical licentiates and their medical skill development in the low-resource context of Zambia: A mixed-methods study

Background

Countries in sub-Saharan Africa continue to face insufficient health education resources and facilities, as well as a severe shortage of health care professionals. In 2019, the Levy Mwanawasa Medical University (LMMU) in Lusaka was launched to address the shortage of healthcare professionals implementing a decentralized training model utilizing selected regional and district hospitals in Zambia as training sites for various cadres. Decentralization makes it more challenging to monitor the learning process as part of continuous assessment; consequently, adequate approaches are necessary to ensure the quality and quantity of medical skills training. Electronic logbooks (e-logbooks) provide a promising tool for monitoring and evaluation of the medical training process.

Objective

We designed and implemented an e-logbook for Medical Licentiate students based on an existing software system. We evaluated the feasibility of this e-logbook, its acceptability among a cohort of Medical Licentiate students and their mentors, as well as its facilitators and barriers.

Materials and methods

During the course of a five-week-long clinical rotation in a training site in Kabwe, Zambia, two mentors and ten students participated in the pilot study and its evaluation. A mixed-methods approach utilized log-based usage data from the e-logbook web platform and conducted semi-structured in-depth interviews.

Results

Overall, both students and mentors accepted e-logbooks as a means to monitor skills development in this context, indicating that e-logbooks are a feasible tool in this decentralized setting. Feedback pointed out that the design and software-induced terminology of the e-logbook posed usability issues. The complexity and greater time commitment (mentors used a web-based platform instead of an app) limited the e-logbook's potential.

Conclusion

We conclude that there is acceptability of monitoring medical skill development through a tablet-based e-logbook. However, the e-logbook in its current form (based on an existing software system, with limited adaptation possibilities to the local context) was insufficient for the LMMU environment. Given that this was attributable to design flaws rather than technology issues or rejection of the e-logbook as a quality assessment tool in and of itself, we propose that the e-logbook be implemented in a co-design approach to better reflect the needs of students and mentors.

Thoracic ultrasound competence for ultrasound guided pleural procedures: The creation and validation of an assessment tool for use in the certification of basic thoracic ultrasound competence

OBJECTIVES

Focused thoracic ultrasound (TUS) provides an increased safety profile when undertaking invasive pleural procedures. This has led to the requirement for defined curricula, high quality teaching and robust, validated assessment tools among physicians to ensure patient safety and clinical excellence. Current UK practice is based almost exclusively on expert consensus, but assessment methods employed have been shown to have low reliability and validity and are potentially open to bias. As a result, several assessment tools have been developed, although each has its own limitations.

METHODS

This study aimed to develop and validate an assessment tool corresponding to those skills associated with the most basic level of practice, defined recently as an emergency level operator in the British Thoracic Society Training Standards for Thoracic Ultrasound.

RESULTS

A total of 27 candidates were enrolled by two examiners based in Belfast and Oxford over a 10-month period between February and November 2019. Mean score of the inexperienced group was 44.3 (95% CI 39.2-49.4, range 28-54) compared with 74.9 (95% CI 72.8-77, range 64-80) in the experienced group providing an estimated mean difference of 30.7 between the two groups (95% CI 24.7-36.7; p < .001).

CONCLUSIONS

This tool appears to discriminate between trainees with limited experience of TUS performance and those with no experience. It has the potential to form part of the assessment strategy for trainees in the United Kingdom and beyond, alongside well established assessment tools in postgraduate training.

Educational Impact Drives Feasibility of Implementing Daily Assessment in the Workplace

Construct: Authors investigated the perspectives of stakeholders on feasibility elements of workplace-based assessments (WBA) with varying designs. Background: In the transition to competency-based medical education, WBA are taking a more prominent role in assessment programs. However, the increased demand for WBA leads to new challenges for implementing suitable WBA tools with published validity evidence, while also being feasible and useful in practice. Despite the availability of published WBA tools, implementation does not necessarily occur; a more fulsome understanding of the perspectives of stakeholders who are ultimately the end-users of these tools, as well as the system factors that both deter or support their use, could help to explain why evidence-based assessment tools may not be incorporated into residency programs. Approach: We examined the perspectives of two groups of stakeholders, surgical teachers and resident learners, during an assessment intervention that varied the assessment tools while keeping the assessment process constant. We chose diverse exemplars from published assessment tools that each represented a different response format: global rating scales, step-by-step surgical rubrics, and an entrustability scale. The primary purpose was to investigate how stakeholders are impacted by WBA tools with varying response formats to better understand their feasibility for assessment of cataract surgery. Secondarily, we were able to explore the culture of assessment in cataract surgery education including stakeholders' perceptions of WBA unrelated to assessment form design. Semi-structured interviews with teachers and a focus group with the residents enabled discussion of their perspectives on dimensions of the tools such as acceptability, demand, implementation, practicality, adaptation, and integration. Findings: Three themes summarize teachers' and residents' experiences with the assessment tools: (1) Feedback is the priority; (2) Forms informing coaching; and (3) Forcing the conversation. The tools helped to facilitate the feedback conversation by serving as a reminder to initiate the conversation, a framework to structure the conversation, and a memory aid for providing detailed feedback. Surgical teachers preferred the assessment tool with a design that best aligned with their approach to teaching and how they wanted to provide feedback. Orientation to the tools, combined with established remediation pathways, may help preceptors to better use assessment tools and improve their ability to give critical feedback. Conclusions: Feedback, more so than assessment, dominated the comments provided by both teachers and residents after using the various WBA tools. Our typical assessment design efforts focus on the creation or selection of a robust assessment tool according to good design and measurement principles, but the current findings would encourage us to also prioritize the coaching relationship and include efforts to design WBA tools to function as a mediator to augment teaching, learning, and feedback exchange within that relationship in the workplace.

Thoracic ultrasound competence for ultrasound-guided pleural procedures

Focused thoracic ultrasound has become essential in the guidance and direction of pleural interventions to reduce unwanted complications and as a result now forms a crucial component of physician training. Current training standards along with assessment methods vary widely, and are often not robust enough to ensure adequate competence.This review assesses the current state of training and assessment of thoracic ultrasound competence in various settings, allowing comparison with alternative competency based programmes. Future directions for training and assessment of thoracic ultrasound competence are discussed.

Feedback learning opportunities from medical student logs of paediatric patients

BACKGROUND

Feedback can alter medical student logging practices, although most learners feel feedback is inadequate. A varied case mix in rural and urban contexts offers diverse clinical encounters. Logs are an indicator of these clinical experiences, and contain opportunities for feedback, which can greatly influence learning: we labelled these 'feedback learning opportunities' (FLOs). We asked: How often do FLOs occur? What are the case complexities of rural compared to urban paediatric logs? Do more complex cases result in more FLOs?

METHODS

In Western Australia, 25% of medical students are dispersed in a Rural Clinical School (RCSWA) up to 2175 miles (3500 km) from the city. Urban students logged 20 written cases; rural students logged a minimum of 25 paediatric cases electronically. These were reviewed to identify FLOs, using a coding convention. FLO categories provided a structure for feedback: medical, professionalism, insufficient, clinical reasoning, student wellbeing, quality and safety, and sociocultural. Each log was assigned an overall primary, secondary or tertiary case complexity.

RESULTS

There were 76 consenting students in each urban and rural group, providing 3034 logs for analysis after exclusions. FLOs occurred in more than half the logs, with significantly more rural (OR 1.35 95% CI 1.17, 1.56; p < 0.0001). Major FLOs occurred in over a third of logs, but with no significant difference between rural and urban (OR 1.10 95% CI 0.94, 1.28; p = 0.24). Medical FLOs were the most common, accounting for 64.0% of rural and 75.2% of urban FLOs (OR 1.71 95% CI 1.37, 2.12; p < 0.0001). Students logged cases with a variety of complexities. Most cases logged by urban students in a tertiary healthcare setting were of primary and secondary complexity. Major medical FLOs increased with increasing patient complexity, occurring in 32.1% of tertiary complexity cases logged by urban students (p < 0.001).

CONCLUSIONS

Case logs are a valuable resource for medical educators to enhance students' learning by providing meaningful feedback. FLOs occurred often, particularly in paediatric cases with multiple medical problems. This study strengthens recommendations for regular review and timely feedback on student logs. We recommend the FLOs categories as a framework for medical educators to identify FLOs.

Paediatric case mix in a rural clinical school is relevant to future practice

BACKGROUND

Exposure to a representative case mix is essential for clinical learning, with logbooks established as a way of demonstrating patient contacts. Few studies have reported the paediatric case mix available to geographically distributed students within the same medical school. Given international interest in expanding medical teaching locations to rural contexts, equitable case exposure in rural relative to urban settings is topical. The Rural Clinical School of Western Australia locates students up to 3500 km from the urban university for an academic year. There is particular need to examine paediatric case mix as a study reported Australian graduates felt unprepared for paediatric rotations. We asked: Does a rural clinical school provide a paediatric case mix relevant to future practice? How does the paediatric case mix as logged by rural students compare with that by urban students?

METHODS

The 3745 logs of 76 urban and 76 rural consenting medical students were categorised by presenting symptoms and compared to the Australian Institute of Health and Welfare (AIHW) database Major Diagnostic Categories (MDCs).

RESULTS

Rural and urban students logged core paediatric cases, in similar order, despite the striking difference in geographic locations. The pattern of overall presenting problems closely corresponded to Australian paediatric hospital admissions. Rural students logged 91% of cases in secondary healthcare settings; urban students logged 90% of cases in tertiary settings. The top four presenting problems were ENT/respiratory, gastrointestinal/urogenital, neurodevelopmental and musculoskeletal; these made up 60% of all cases. Rural and urban students logged similar proportions of infants, children and adolescents, with a variety of case morbidity.

CONCLUSIONS

Rural clinical school students logged a mix of core paediatric cases relevant to illnesses of Australian children admitted to public hospitals, with similar order and pattern by age group to urban students, despite major differences in clinical settings. Logged cases met the curriculum learning outcomes of graduates. Minor variations were readily addressed via recommendations about logging. This paper provides evidence of the legitimacy of student logs as useful tools in affirming appropriate paediatric case mix. It validates the rural clinical school context as appropriate for medical students to prepare for future clinical paediatric practice.

Twelve tips for successfully implementing logbooks in clinical training

BACKGROUND

Logbooks are widely used to set learning outcomes and to structure and standardize teaching in clinical settings. Experience shows that logbooks are not always optimally employed in clinical training. In this article, we have summarized our own experiences as well as results of studies into twelve tips on how to successfully implement logbooks into clinical settings.

METHODS

We conducted both a workshop concerning the importance of logbook training to exchange experiences in teaching practice, organization, didactic knowledge and a literature research to compare our own experiences and add additional aspects.

RESULTS

Tips include the process of developing the logbook itself, the change-management process, conditions of training and the integration of logbooks into the curriculum.

CONCLUSIONS

Logbooks can be a valuable tool for training in clinical settings, especially when multiple sites are involved, when you take our tips into consideration.

Video-based on-ward supervision for final year medical students

BACKGROUND

Constructive feedback is an essential element of the educational process, helping trainees reach their maximum potential and increasing their skill level. Video-based feedback has been described as highly effective in various educational contexts. The present study aimed to evaluate the feasibility and acceptability of video-based, on-ward supervision for final year students in a clinical context with real patients.

METHODS

Nine final year medical students (three male, six female; aged 25.1 ± 0.7 years) and eight patients (five male, three female; aged 59.3 ± 16.8 years) participated in the pilot study. Final year students performed routine medical procedures at bedside on internal medicine wards at the University of Heidelberg Medical Hospital. Students were filmed and were under supervision. After performing the procedures, an oral feedback loop was established including student, patient and supervisor feedback on communicative and procedural aspects of skills performed. Finally, students watched their video, focusing on specific teachable moments mentioned by the supervisor. Written evaluations and semi-structured interviews were conducted that focused on the benefits of video-based, on-ward supervision. Interviews were analysed qualitatively, using open coding to establish recurring themes and overarching categories to describe patients' and students' impressions. Descriptive, quantitative analysis was used for questionnaire data.

RESULTS

Supervised, self-chosen skills included history taking (n = 6), physical examination (n = 1), IV cannulation (n = 1), and ECG recording (n = 1). The video-based, on-ward supervision was well accepted by patients and students. Supervisor feedback was rated as highly beneficial, with the video material providing an additional opportunity to focus on crucial aspects and to further validate the supervisor's feedback. Students felt the video material would be less beneficial without the supervisor's feedback. The setting was rated as realistic, with filming not influencing behaviour.

CONCLUSION

Video-based, on-ward supervision may be a powerful tool for improving clinical medical education. However, it should be regarded as an additional tool in combination with supervisors' oral feedback. Acceptance was high in both students and patients. Further research should address possibilities of efficiently combining and routinely establishing these forms of feedback in medical education.

Novel Examination for Evaluating Medical Student Clinical Reasoning: Reliability and Association With Patients Seen

Background: Medical students learn clinical reasoning, in part, through patient care. Although the numbers of patients seen is associated with knowledge examination scores, studies have not demonstrated an association between patient problems and an assessment of clinical reasoning. Aim: To examine the reliability of a clinical reasoning examination and investigate whether there was association between internal medicine core clerkship students' performance on this examination and the number of patients they saw with matching problems during their internal medicine clerkship. Methods: Students on the core internal medicine clerkship at the Uniformed Services University students log 11 core patient problems based on the Clerkship Directors in Internal Medicine curriculum. On a final clerkship examination (Multistep), students watch a scripted video encounter between physician and patient actors that assesses three sequential steps in clinical reasoning: Step One focuses on history and physical examination; Step Two, students write a problem list after viewing additional clinical findings; Step Three, students complete a prioritized differential diagnosis and treatment plan. Each Multistep examination has three different cases. For graduating classes 2010–2012 (n = 497), we matched the number of patients seen with the problem most represented by the Multistep cases (epigastric pain, generalized edema, monoarticular arthritis, angina, syncope, pleuritic chest pain). We report two-way Pearson correlations between the number of patients students reported with similar problems and the student's percent score on: Step One, Step Two, Step Three, and Overall Test. Results: Multistep reliability: Step 1, 0.6 to 0.8; Step 2, 0.41 to 0.65; Step 3, 0.53 to 0.78; Overall examination (3 cases): 0.74 to 0.83. For three problems, the number of patients seen had small to modest correlations with the Multistep Examination of Analytic Ability total score (r = 0.27 for pleuritic pain, p &lt; 0.05, n = 81 patients; r = 0.14 for epigastric pain, p &lt; 0.05, n = 324 patients; r = 0.19 for generalized edema, p &lt; 0.05, n = 118 patients). Discussion or Conclusion: Although a reliable assessment, student performance on a clinical reasoning examination was weakly associated with the numbers of patients seen with similar problems. This may be as a result of transfer of knowledge between clinical and examination settings, the complexity of clinical reasoning, or the limits of reliability with patient logs and the Multistep.

How we use patient encounter data for reflective learning in family medicine training

INTRODUCTION

Consulting with patients is the core learning activity of Australian family medicine (general practice/GP) training, providing a rich source of reflective learning for trainees. We have developed a reflective learning program for postgraduate vocational trainees based on clinical encounters.

METHODS

The Registrar Clinical Encounters in Training (ReCEnT) program is an educational program documenting GP trainees' consultations in five Australian GP training providers. Trainees record patient demographics, consultation details, problems managed, management practices and educational factors from sixty consecutive consultations per six-month training term. Trainees receive a detailed feedback report comparing individual data to aggregated trainee data and national GP data.

RESULTS

The patient encounter system provides multiple opportunities for reflective learning across a number of domains of exposure and practice. Reflection can occur during completion of the encounter form; as self-reflection on the feedback report; as facilitated reflection with the GP trainer and medical educator; and as part of integration of data into teaching. We have identified areas for further development, including enhancing the reflective skills of trainees and trainers.

CONCLUSION

The ReCEnT patient encounter program provides a rich platform for reflective learning for vocational trainees and supports development of skills in lifelong learning.

Could clinical experience during clerkship enhance students' clinical performance?

BACKGROUND

Medical students learn and practice various clinical skills during clinical clerkship. Patient encounters are important for developing clinical thinking, communication skills, and professional attitude. We investigated whether the amount of clinical experience during clerkship correlated with students' clinical competency and students' perception of effectiveness of their clerkship on it.

METHODS

Fourth year medical students undertook the Objective Structured Clinical Examinations (OSCE) in August 2012. Students provided the number of patients for whom they took medical histories or performed physical examinations during clerkship and provided feedback as to whether or not the clinical clerkship was helpful in preparing OSCE. The correlation between the OSCE score and number of patients was analyzed.

RESULTS

One hundred thirty students completed the questionnaire (86.6%). OSCE scores correlated with the total number of patients encountered for physical examinations (correlation coefficient, 0.274; p = 0.0105). Cumulative 3-year GPAs were positively correlated with OSCE scores (correlation coefficient, 0.330; p = 0.0001). Most (92.3%) answered that their clinical clerkship was helpful in preparing them for the OSCE; however, only 20% felt that their clinical clerkship was most helpful. Others felt that role playing (38.46%) or the guide book (33.84%) was most helpful.

CONCLUSIONS

The amount of clinical experience during the students' clerkship had a small but positive relationship with students' clinical performance. Further research to elucidate the influence of clinical experience on clinical competency is needed.

Benefits and limitations of an employer-led, structured logbook to promote self-directed learning in the clinical workplace

A structured logbook, consisting of a competency log and a learning contract, was designed and implemented as part of a two-week structured work placement for final-year veterinary students to help them become more self-directed in the workplace. The competency log encompassed 48 core skills and, along with the learning contract, was reviewed at the start and end of the placement. To assess their perceptions of the logbook in promoting self-directed learning, students and supervisors were asked to complete a questionnaire pre- and post-placement and to participate in focus groups (students) and interviews (supervisors) after the placement. The study found significant increases pre- to post-placement in students' perceived levels of competence in all 48 skills and their confidence in being self-directed. However, student attitudes toward the logbook significantly decreased in terms of it encouraging supervisors to take a clearly designed role in structuring learning and facilitating alignment of employer and student expectations. Although supervisors were generally positive about the logbook, some had not been able to review it with their students, which affected students' perceptions of the logbook's usefulness. Some supervisors felt they had not received enough training, and most, erroneously, believed the logbook to be an external research initiative rather than having been designed by the head of their own organization. This study demonstrated that a structured logbook may be useful in helping students become more self-directed; however, supervisor support for the logbook is critical. To facilitate this, supervisors require training and support from senior management.

Evaluation of Canadian family practice anesthesia training programs: can the Resident Logbook help?

PURPOSE

Without a core curriculum for the training and evaluation of Canada's family practice anesthetists, little is known regarding the training process of these physicians. This article offers a description of the variety of cases and procedures experienced by family practice anesthesiology (FPA) residents during their training year based on records in the Resident Logbook.

METHODS

Data were extracted from the Resident Logbook, an online program wherein FPA residents across Canada record their daily activities. Data were extracted for four residents from 2009/2010 and four from 2010/2011 who had recorded the largest number of patient encounters for the academic year. Medians were calculated for cases and procedures relevant to the practice of a family practice anesthetist.

RESULTS

Residents in FPA use the Resident Logbook inconsistently. The trainees we selected entered a median of 1,418 encounters for 2009/2010 and 1,074 for 2010/2011. Residents appear to have most of their clinical encounters with cases and procedures relevant to FPA. There is still a need, though, to improve areas previously cited as requiring augmentation, such as trauma management and peripheral nerve blocks.

CONCLUSIONS

Although FPA residents use the Resident Logbook inconsistently, the data obtained offer an initial description of the composition of the FPA training year. We believe that the Resident Logbook offers an excellent tool for furthering the goal of a standardized curriculum and assessment program for FPA training.

Relationship between clinical experiences and internal medicine clerkship performance

OBJECTIVES

This study was conducted to assess the associations between several clerkship process measures and students' clinical and examination performance in an internal medicine clerkship.

METHODS

We collected data from the internal medicine clerkship at one institution over a 3-year period (classes of 2010-2012; n = 507) and conducted correlation and multiple regression analyses. We examined the associations between clerkship process measures (student-reported number of patients evaluated, percentage of core problems encountered, total number of core problems encountered, total number of clinics attended) and four clerkship outcomes (clinical points [a weighted summation of a student's clinical grade recommendations], ambulatory clinical points [the out-patient portion of clinical points], examination points [a weighted summation of scores on three clerkship examinations], and National Board of Medical Examiners examination score).

RESULTS

After controlling for pre-clerkship ability and gender, percentage of core problems was significantly associated with ambulatory clinical points (b = 3.84, total model R(2) = 0.14). Further, number of patients evaluated was significantly associated with clinical points (b = 0.19, total model R(2) = 0.22), but only for students who undertook first-quarter clerkships, who reported higher numbers of patients.

CONCLUSIONS

Notwithstanding a few positive (but small) associations, the results from this study suggest that clinical exposure is, at best, weakly associated with internal medicine clerkship performance.

Study protocol: the Registrar Clinical Encounters in Training (ReCEnT) study

BACKGROUND

Patient encounters are the core learning activity of Australian general practice (family practice) training. Exposure to patient demographics and presentations may vary from one general practice registrar (vocational trainee) to another. This can affect comprehensiveness of training. Currently, there is no mechanism to systematically capture the content of GP registrar consultations. The aim of the Registrar Clinical Encounters in Training (ReCEnT) study is to document longitudinally the nature and associations of consultation-based clinical and educational experiences of general practice registrars.

METHODS/DESIGN

This is an ongoing prospective multi-site cohort study of general practice registrars' consultations, entailing paper-based recording of consultation data. The study setting is general practices affiliated with three geographically-based Australian general practice regional training providers. Registrars record details of 60 consecutive consultations. Data collected includes registrar demographics, details of the consultation, patient demographics, reasons for encounter and problems managed. Problems managed are coded with the International Classification of Primary Care (second edition) classification system. Additionally, registrars record educational factors related to the encounter. The study will follow the clinical exposure of each registrar six-monthly over the 18 months to two years (full-time equivalent) of their general practice training program.

CONCLUSIONS

The study will provide data on a range of factors (patient, registrar and consultation factors). This data will be used to inform a range of educational decisions as well as being used to answer educational research questions. We plan to use ReCEnT as a formative assessment tool for registrars and help identify and address educational needs. The study will facilitate program evaluation by the participating training providers and thus improve articulation of educational programs with practice experience. From the research point of view it will address an evidence gap - the in-practice clinical and educational experience of general practice trainees, determinants of these experiences, and the determinants of registrars' patterns of practice (for example, prescribing practice) over the course of their training.

Reaching the limits of mandated self-reporting: clinical logbooks do not predict clerkship performance

BACKGROUND

Logbooks are used by US medical schools to evaluate curricular objectives and meet accreditation requirements, but research supporting their utility is conflicting.

AIMS

The goal of our study was to examine the relationship between volume of clinical rotation experiences as reported in a logbook and clerkship grades within a longitudinal integrated clerkship.

METHODS

We conducted a retrospective cohort study of third-year (clinical) medical students during academic year 2008-2009. We tracked student entries in a pocket-sized logbook (number of clinical encounters per clerkship, total number of exams, and procedures over the academic year). We performed correlation analyses between logbook entries and clerkship grades.

RESULTS

We enrolled 36 students, who reported a total of 2992 encounters, 2262 exams, and 2342 procedures. Correlation coefficients between volume of clinical experience and clerkship grades were less than 0.4, indicating low correlation. We found borderline statistical significance for the Neurology, OB/Gyn, and Surgery clerkships. Sensitivity analyses showed little correlation between low-reporting activity and clerkship grades.

CONCLUSIONS

Even within an integrated longitudinal clerkship framework, our findings are consistent with previous studies showing a negligible relationship between logbooks as an educational process measure and how they relate to educational outcomes.

Associations between medical student log data and clerkship learning outcomes

BACKGROUND

Largely due to regulatory requirements, medical students increasingly document patient encounters in logs. Prior studies demonstrated value in achieving course objectives but not regarding objective learning outcomes.

PURPOSE

The purpose of this article is to assess the associations between students' log data and clerkship outcomes.

METHODS

We assessed ambulatory, internal medicine-pediatrics clerkship logs. Students recorded patients' diagnoses, preceptor's teaching content and performing histories and physicals (H&P). We examined associations between log data and National Board of Medical Examiners Subject Exams (NBME), Objective Structured Clinical Exams (OSCE), and clinical evaluations.

RESULTS

All 272 students completed logs (M diagnoses = 146.5, SD = 24.9). In univariate analysis number of diagnoses, preceptors' teaching patient management and performing H&P independently correlated with OSCE scores (r = .13-.24, p < .05). NBME scores correlated with diagnoses, performing H&P independently (r = .13-.18, p < .05). Teaching patient management, diagnoses, performing H&P independently were associated with clinical evaluations (all ps < .05). Regressions demonstrated that performing physicals independently was associated with NBME and OSCE scores (p < .05, R (2) = .03 and .05, respectively). Teaching disease management and pediatric diagnoses were associated with clinical evaluations (p < .05) in medicine and pediatrics, respectively.

CONCLUSIONS

Aspects of teaching, increased clinical involvement, and student autonomy were associated with clerkship performance.

Understanding the educational value of first-year medical students' patient encounter data

BACKGROUND

Many medical schools use patient-encounter logs to track students' clinical experiences, but few have investigated the student-user perspective to understand how students enter, review, and use patient-encounter information.

AIMS

This study examines first-year medical students' use of a web-based tracking system and whether students thought logging patient-encounters was educationally valuable.

METHODS

This mixed-methods study uses data from student encounter log entries, a focus group with first-year students, and a survey. Data analysis involved descriptive statistics for quantitative data, qualitative content analysis of students' free-response entries in the logs, and thematic analysis for focus group and survey responses.

RESULTS

Most students logged at least one encounter (90%), but used the system substantially less than expected. Focus group and survey data indicated that students found minimal educational value in the encounter tracking system as designed, but identified several ways in which the system could be improved to better support their learning. Suggestions included clearer guidelines for use, better integration into the curriculum, a mentoring process, and provision of benchmarks or target number of specific encounter types.

CONCLUSION

Student-user perspectives are crucial in optimizing information collected through patient-encounter tracking systems, and can improve both the functionality and the educational value of such systems.

Using patient encounter logs for mandated clinical encounters in an internal medicine clerkship

BACKGROUND

Patient encounter logs help assess a student's educational experience. The use of a grading incentive linked to the mandatory documentation of prespecified clinical encounters has been insufficiently studied.

PURPOSES

Given this, our objectives were to determine (a) if mandating student exposure to patients with 18 key training problems leads to the successful documentation of these encounters, (b) the degree of difficulty students and clerkship directors experienced in meeting these mandates, (c) the accuracy of mandated log entries, and (d) how often the log entries were questioned or rejected by preceptors.

METHODS

Ninety-two 3rd-year internal medicine students and 6 internal medicine clerkship directors at 9 geographically dispersed hospitals and 30 ambulatory sites participated in the study. Over a single academic year, we directly measured the completion rate of mandated logs, the degree of their accuracy as judged against faculty-generated logs of two required videotaped standardized patient encounters, and the percentage of logs that were not validated. We surveyed students and directors on the perceived degree of difficulty in meeting mandated requirements.

RESULTS

Ninety-eight percent of students met our mandated requirements and 93.8% of students found it "easy" or "very easy" to meet this requirement. The amount of estimated time spent by clerkship directors helping students meet mandated requirements for the entire year was 4.5 hr. The accuracy of submitted logs was 77%; however, almost all inaccurate log entries were "validated" by preceptors.

CONCLUSIONS

Mandating encounters is effective in assuring that students document encounters with patients who present with specific clinical problems. The accuracy of our students' mandated logs is similar to previously published data. However, even inaccurate logs were rarely questioned or rejected by preceptors.

Internal medicine core clerkships experience with core problem lists: results from a national survey of clerkship directors in internal medicine

BACKGROUND

The Liaison Committee on Medical Education (LCME) requires clinical clerkships in North American Medical Schools to define and monitor core problems and clinical conditions for medical students and adjust the clerkships to ensure that all students meet those objectives at all instructional sites. Clinical clerkships usually use medical student generated logbooks to meet these requirements. It is not clear what clinical clerkship directors are doing to meet these standards.

PURPOSE

To determine how internal medicine clerkship directors develop problem lists, whether and how they verify student problem list entry, and how missed core problems are covered.

METHODS

The Clerkship Directors in Internal Medicine (CDIM) organization conducts an annual survey of its institutional members (one member per medical school). In 2007, 75% of North American medical schools (82 of 110) responded to the survey, which included questions on core problem lists and medical student logbooks.

RESULTS

Ninety-four percent of responding medical schools had a core problem list for the core internal medicine clerkship. Most schools used an electronic logbook and verified student exposure to core problems (76%). The clerkship director usually did the verification (63%). Clerkships offered a variety of options to remedy lack of exposure to core problems and usually offered more than one option.

CONCLUSIONS

Clerkship directors in internal medicine develop, verify, and ensure exposure to core problems in internal medicine. Most clerkship directors used the CDIM recommended core problem list as a basis for their local core problem list. Emerging computer-assisted learning options may simplify clerkship adjustments for lack of exposure to core problems. It appears that clerkship directors are using logbooks appropriately to meet the LCME charge to monitor core problems and clinical conditions.

Demystifying "learning" in clinical rotations: do immersive patient encounters predict achievement on the clinical performance examination (CPX)?

BACKGROUND

Although the Liaison Committee on Medical Education required U.S. medical schools to quantify students' clinical encounters, optimum patient exposure needed to predict performance has been elusive. This study explored the relationship between comprehensive patient encounters logged on personal digital assistants (PDAs) during three medicine clerkships, to performance on a clinical performance examination (CPX).

METHOD

PDA log data for 166 medical students were used to identify educationally "rich" patient encounters, where students were assigned full responsibility for patient care.

RESULTS

Univariate regression analyses predicting the effect of immersive patient encounters on CPX case scores did not show statistical significance.

CONCLUSIONS

Amount of patient exposure defined by the richness of student-patient interaction did not reflect on the CPX performance for six selected cases. Further research should examine qualitatively different learning experiences occurring with patient encounters and a higher volume of exposure to predict outcomes.

Accuracy of medical student electronic logbook problem list entry

BACKGROUND

The accuracy of medical student logbooks has not been extensively studied.

PURPOSE

The purpose of this study was to determine accuracy of student entry of core problems and completeness of patient entry in an electronic logbook.

METHODS

Third-year internal medicine clerkship students entered patient encounters as required by the clerkship. Experts entered information from the same encounters.

RESULTS

A total of 1,440 patient entries generated by 37 consecutive students were compared to expert entries. Sensitivity (core problem underreporting) was low (60%, SD=22%). Percent agreement (87%, SD=7%), kappa (0.46, SD=0.19), and specificity (core problem overreporting; 95%, SD=5%) were good to excellent. Students both omitted (underreported 14%, SD=12%) and overreported (23%, sd 17%) patients.

CONCLUSIONS

Under ideal study circumstances, there was significant underreporting of core problems by students. Although the high specificity, meaning that students are not reporting problems they have not encountered, is reassuring, logbook sensitivity in this study was not good enough for high-stakes evaluations of students or for medical school licensing.

A national survey of pediatric clerkship directors' approaches to meeting the LCME ED-2 requirement for quantified patient criteria for medical students

BACKGROUND

The Liaison Committee on Medical Education (LCME) recently reinterpreted ED-2, their quantified patient criteria requirement.

PURPOSE

The purpose is to describe pediatric clerkship directors' response to ED-2.

METHOD

We used a survey of U.S. and Canadian pediatric clerkship directors.

RESULTS

Survey response rate was 76% (108/142). The most frequent categories that clerkship directors used to describe quantified patient criteria were age group (77%), organ systems (65%), or symptoms (61%). Computer simulations were the alternative patient experience in 83% of clerkships. Patient logs (90%) and checklists (31%) tracked student completion of ED-2. Thirty-two respondents had an LCME visit since ED-2 was reinterpreted, and 69% reported that LCME had concerns about their clerkships' fulfillment of ED-2. There was no difference in method of ED-2 implementation between schools who were cited by LCME and those who were not.

CONCLUSIONS

Clerkship directors have used a variety of approaches to specify and monitor quantified patient criteria.