The reliability of logbook data of medical students: an estimation of interobserver agreement, sensitivity and specificity

Validation of a Novel Statistical Method to Identify Aberrant Patient Logging: A Multi-Institutional Study

INTRODUCTION

Student patient encounter logging informs the quality of supervised clinical practice experiences (SCPEs). Yet, it is unknown whether logs accurately reflect patient encounters, and the faculty resources necessary to review for potential aberrant logging are significant. The purpose of this study was to identify a statistical method to identify aberrant logging.

METHODS

A multi-institutional (n = 6) study examined a statistical method for identifying potentially aberrant logging behavior. An automated statistical Mahalanobis Distance (MD) measurement was used to categorize student logs as aberrant if they were identified as probable multivariate outliers. This approach was validated using a gold standard for aberrant logging behavior with manual review by 4 experienced faculty ("faculty consensus") and then comparing interrater agreement between faculty and MD-based categorization. In secondary analyses, we compared the relative accuracy of MD-based categorization to individual faculty categorizing data from their own program ("own program" categorization).

RESULTS

323 student logging records from 6 physician assistant (PA) programs were included. Compared to "faculty consensus" (the gold standard), MD-based categorization was highly sensitive (0.846, 95% CI: 0.650, 1.000) and specific (0.766, 95% CI: 0.645, 0.887). Additionally, there was no significant difference in sensitivity, specificity, positive predictive value, or negative predictive value between MD-based categorization and "own program" categorization.

DISCUSSION

The MD-based method of identifying aberrant and nonaberrant student logging compared favorably to the more traditional, faculty-intensive approach of reviewing individual student logging records. This supports MD-based screening as a less labor-intensive alternative to individual faculty review to identify aberrant logging. Identification of aberrant logging may facilitate early intervention with students to improve clinical exposure logging during their SCPEs.

A paediatric logbook: Millstone or milestone?

Logbooks are ubiquitous in undergraduate and postgraduate medical education. Here, two alumni who are in the early phase of their career, reflect on their experience in their undergraduate paediatric term and how this was shaped by their mandatory paediatric logbook.

Logging in: a comparative analysis of electronic health records versus anesthesia resident-driven logbooks

PURPOSE

Resident logbooks (RLBs) documenting clinical case exposure are widespread in medical education despite evidence of poor accuracy. Electronic health records (e.g., anesthesia information management systems [AIMS]) may provide advantages for auditing longitudinal case exposure. We evaluated the agreement between AIMS and RLBs for tracking case exposure during anesthesiology residency.

METHODS

We performed a historical cohort study with anesthesiology residents (2011-2018, all of whom used a RLB contemporaneously with AIMS) working in a multisite academic health sciences network. The primary outcome was total case-load logging; secondary outcomes were volumes for seven surgical specialties (general, gynecology, neuro, orthopedic, thoracic, urology, and vascular surgery). Correlation of case numbers tracked by AIMS vs RLB was assessed using Pearson correlation; agreement was determined using Bland-Altman plots and intraclass correlation coefficients (ICC).

RESULTS

Data from 27 anesthesiology residents were collected. Overall, mean (standard deviation) case numbers were generally greater with AIMS vs RLB [649 (103) vs 583 (191); P = 0.049). Total case volumes between systems had moderate correlation (r = 0.50) and agreement (intraclass correlation coefficient [ICC], 0.42; 95% CI, 0.34 to 0.59). Bland-Altman plots showed variable agreement between AIMS and RLB data [mean (SD) bias = 66 (166) cases]. For general, gynecology, neuro, orthopedic, thoracic, urology, and vascular surgery, there was a range of poor to moderate agreement (ICC, 0.23-0.57) between AIMS and RLB.

CONCLUSION

For anesthesiology resident case-logging, the number of cases logged in an AIMS was higher with lower variance compared with RLBs. Anesthesia information management systems vs RLB data showed low-moderate correlation and agreement. Given the additional time and resources required for RLBs, AIMS may be a superior method for tracking cases where available.

Feedback learning opportunities from medical student logs of paediatric patients

BACKGROUND

Feedback can alter medical student logging practices, although most learners feel feedback is inadequate. A varied case mix in rural and urban contexts offers diverse clinical encounters. Logs are an indicator of these clinical experiences, and contain opportunities for feedback, which can greatly influence learning: we labelled these 'feedback learning opportunities' (FLOs). We asked: How often do FLOs occur? What are the case complexities of rural compared to urban paediatric logs? Do more complex cases result in more FLOs?

METHODS

In Western Australia, 25% of medical students are dispersed in a Rural Clinical School (RCSWA) up to 2175 miles (3500 km) from the city. Urban students logged 20 written cases; rural students logged a minimum of 25 paediatric cases electronically. These were reviewed to identify FLOs, using a coding convention. FLO categories provided a structure for feedback: medical, professionalism, insufficient, clinical reasoning, student wellbeing, quality and safety, and sociocultural. Each log was assigned an overall primary, secondary or tertiary case complexity.

RESULTS

There were 76 consenting students in each urban and rural group, providing 3034 logs for analysis after exclusions. FLOs occurred in more than half the logs, with significantly more rural (OR 1.35 95% CI 1.17, 1.56; p < 0.0001). Major FLOs occurred in over a third of logs, but with no significant difference between rural and urban (OR 1.10 95% CI 0.94, 1.28; p = 0.24). Medical FLOs were the most common, accounting for 64.0% of rural and 75.2% of urban FLOs (OR 1.71 95% CI 1.37, 2.12; p < 0.0001). Students logged cases with a variety of complexities. Most cases logged by urban students in a tertiary healthcare setting were of primary and secondary complexity. Major medical FLOs increased with increasing patient complexity, occurring in 32.1% of tertiary complexity cases logged by urban students (p < 0.001).

CONCLUSIONS

Case logs are a valuable resource for medical educators to enhance students' learning by providing meaningful feedback. FLOs occurred often, particularly in paediatric cases with multiple medical problems. This study strengthens recommendations for regular review and timely feedback on student logs. We recommend the FLOs categories as a framework for medical educators to identify FLOs.

Students' Perspectives of a Community-Based Medical Education Programme in a Rural District Hospital

BACKGROUND

The adoption of community-based medical education (CoBME) into the undergraduate medical curriculum is in line with the SPICE model for medical curriculum proposed by Harden and colleagues. Students are the consumers of medical education and are, thus, the ideal evaluators of the efficacy of their own course and learning environment. To evaluate the quality of the CoBME programme in Botshabelo District Hospital (BDH), this study investigated student's perceptions of their experience during their CoBME training at BDH. In addition, suggestions on how to enrich students' experience during the CoBME posting were obtained from the participants.

METHODS

This research was designed as a qualitative (ethnographic) study that used a structured questionnaire, to obtain written statements from 120 fourth-year undergraduate medical students describing their experience during their CoBME training at BDH. The structured questionnaire in the form of an evaluation form was self-administered, consisted of only open-ended questions grouped into 4 main sections and was distributed manually (hard copy) to the participants.

RESULTS

Of the 120 questionnaires distributed, 84 were returned, giving a response rate of 70.0%. When asked to indicate what they liked or disliked about their CoBME training, 'Good educators/staff' and the 'Poor attitude of some doctors' were the themes that scored highly (25.1% and 19.4%) in the 'likes' and 'dislikes' category, respectively. Some of the major challenges faced during the CoBME training at BDH included: exposure to new learning environment (14.2%), clinical practice context (12.6%), and language barrier (7.2%). Participants stated that they gained knowledge of how to perform certain clinical procedures and acquired core clinical skills in the areas of formulating management and managing some medical emergencies during their training at BDH. Increasing the duration of training (25.6% coverage) was suggested as a major way to enrich students' experience during the training at BDH.

CONCLUSION

Findings by this study reveal that CoBME is a valuable pedagogical tool to enhance learning in undergraduate medical education and that more work is required to improve the quality of CoBME training in BDH. We believe that the findings by this study will inform future planning of CoBME training programmes in BDH.

Twelve tips for successfully implementing logbooks in clinical training

BACKGROUND

Logbooks are widely used to set learning outcomes and to structure and standardize teaching in clinical settings. Experience shows that logbooks are not always optimally employed in clinical training. In this article, we have summarized our own experiences as well as results of studies into twelve tips on how to successfully implement logbooks into clinical settings.

METHODS

We conducted both a workshop concerning the importance of logbook training to exchange experiences in teaching practice, organization, didactic knowledge and a literature research to compare our own experiences and add additional aspects.

RESULTS

Tips include the process of developing the logbook itself, the change-management process, conditions of training and the integration of logbooks into the curriculum.

CONCLUSIONS

Logbooks can be a valuable tool for training in clinical settings, especially when multiple sites are involved, when you take our tips into consideration.

Comparison of students' encountered diseases and available diseases at clerkship sites by exploratory multivariate analysis: Are encountered diseases predictable?

BACKGROUND

Students in clerkship are expected to gain clinical expertise by interacting with real patients in clinical situations. Monitoring and predicting the students' encounter diseases (EDs) is important for providing an optimal experience. EDs should be compared with the available diseases (ADs) at the clerkship site and with the required diseases described in some guidelines for the clinical curriculum.

AIMS

To explore the differences in ADs as learning resources among different types of clerkship sites and to investigate discrepancies between EDs and ADs.

METHOD

A retrospective observational study used secondary data from government statistics to compare ADs of various types of observable clerkship sites by biplot analyses, which allowed multivariate comparisons. EDs collected from logbooks during clerkships at a university hospital were also compared with ADs across sites.

RESULTS

The distributions of ADs differed according to institutional type, and EDs at Kyushu University Hospital were similar to the ADs for the category of hospitals in which it was placed.

CONCLUSION

EDs at a clerkship site may be predictable to some extent by analysing the site's distribution of ADs, but further study is needed. Biplot is useful for visualising these types of statistical similarity.

An electronic portfolio for quantitative assessment of surgical skills in undergraduate medical education

BACKGROUND

We evaluated a newly designed electronic portfolio (e-Portfolio) that provided quantitative evaluation of surgical skills. Medical students at the University of Seville used the e-Portfolio on a voluntary basis for evaluation of their performance in undergraduate surgical subjects.

METHODS

Our new web-based e-Portfolio was designed to evaluate surgical practical knowledge and skills targets. Students recorded each activity on a form, attached evidence, and added their reflections. Students self-assessed their practical knowledge using qualitative criteria (yes/no), and graded their skills according to complexity (basic/advanced) and participation (observer/assistant/independent). A numerical value was assigned to each activity, and the values of all activities were summated to obtain the total score. The application automatically displayed quantitative feedback. We performed qualitative evaluation of the perceived usefulness of the e-Portfolio and quantitative evaluation of the targets achieved.

RESULTS

Thirty-seven of 112 students (33%) used the e-Portfolio, of which 87% reported that they understood the methodology of the portfolio. All students reported an improved understanding of their learning objectives resulting from the numerical visualization of progress, all students reported that the quantitative feedback encouraged their learning, and 79% of students felt that their teachers were more available because they were using the e-Portfolio. Only 51.3% of students reported that the reflective aspects of learning were useful. Individual students achieved a maximum of 65% of the total targets and 87% of the skills targets. The mean total score was 345 ± 38 points. For basic skills, 92% of students achieved the maximum score for participation as an independent operator, and all achieved the maximum scores for participation as an observer and assistant. For complex skills, 62% of students achieved the maximum score for participation as an independent operator, and 98% achieved the maximum scores for participation as an observer or assistant.

CONCLUSIONS

Medical students reported that use of an electronic portfolio that provided quantitative feedback on their progress was useful when the number and complexity of targets were appropriate, but not when the portfolio offered only formative evaluations based on reflection. Students felt that use of the e-Portfolio guided their learning process by indicating knowledge gaps to themselves and teachers.

Steering the patient mix of GP trainees: results of a randomized controlled intervention

BACKGROUND

In studies exploring the patient mixes of general practitioner (GP) trainees, gaps were repeatedly found, as there were disparities between the patient mixes of GP trainers and trainees. This reduces the opportunities of trainees to acquire enough competence.

AIMS

To investigate whether steering the patient mix can be effectuated by instructing medical receptionist, trainer and trainee, and to study the effects of this intervention on trainee's self-efficacy (SE) and knowledge.

METHOD

Randomized Controlled Trial (RCT). After a six-month basic registration period, 73 trainees were randomized. Patients with skin conditions and psychosocial conditions were actively assigned to trainees in the intervention group (n=35) during two successive periods of three months. The patient mix was measured by extracting data from electronic patient records. Learning outcomes were measured by SE questionnaires and by a knowledge test.

RESULTS

No increase was found in patient volume and diversity of the steered conditions in the intervention group as compared to the control group. However, the percentual increase of exposure to skin conditions was greater in the intervention group. No difference in skin SE and psychiatric knowledge was found. The increase of psychosocial SE was greater in the intervention group. In a regression analysis, patient volume was a significant predictor of both skin and psychosocial SE.

CONCLUSIONS

Despite the difficulty in implementing steering in daily practice, tailoring the patient mix to the individual learning needs of trainees could be considered.

Benefits and limitations of an employer-led, structured logbook to promote self-directed learning in the clinical workplace

A structured logbook, consisting of a competency log and a learning contract, was designed and implemented as part of a two-week structured work placement for final-year veterinary students to help them become more self-directed in the workplace. The competency log encompassed 48 core skills and, along with the learning contract, was reviewed at the start and end of the placement. To assess their perceptions of the logbook in promoting self-directed learning, students and supervisors were asked to complete a questionnaire pre- and post-placement and to participate in focus groups (students) and interviews (supervisors) after the placement. The study found significant increases pre- to post-placement in students' perceived levels of competence in all 48 skills and their confidence in being self-directed. However, student attitudes toward the logbook significantly decreased in terms of it encouraging supervisors to take a clearly designed role in structuring learning and facilitating alignment of employer and student expectations. Although supervisors were generally positive about the logbook, some had not been able to review it with their students, which affected students' perceptions of the logbook's usefulness. Some supervisors felt they had not received enough training, and most, erroneously, believed the logbook to be an external research initiative rather than having been designed by the head of their own organization. This study demonstrated that a structured logbook may be useful in helping students become more self-directed; however, supervisor support for the logbook is critical. To facilitate this, supervisors require training and support from senior management.

Opening the black box: the patient mix of GP trainees

BACKGROUND

The variety of health problems (patient mix) that medical trainees encounter is presumed to be sufficient to master the required competencies.

AIM

To describe the patient mix of GP trainees, to study differences in patient mix between first-year and third-year GP trainees, and to investigate differences in exposure to sex-specific diseases between male and female trainees.

DESIGN AND SETTING

Prospective cohort study in Dutch primary care.

METHOD

During a 6-month period, aggregated data about International Classification of Primary Care diagnosis codes, and data on the sex and age of all contacts were collected from the electronic patient record (EPR) system.

RESULTS

Seventy-three trainees participated in this study. The mean coding percentage was 86% and the mean number of face-to-face consultations per trimester was 450.0 in the first year and 485.4 in the third year, indicating greater variance in the number of patient contacts among third-year trainees. Diseases seen most frequently were: musculoskeletal (mean per trimester = 89.2 in the first year/91.0 in the third year), respiratory (98.2/92.7) and skin diseases (89.5/96.0). Least often seen were diseases of the blood and blood-forming organs (5.3/7.2), male genital disorders (6.1/7.1), and social problems (4.3/4.2). The mean number of chronic diseases seen per trimester was 48.0 for first-year trainees and 62.4 for third-year trainees. Female trainees saw an average of 39.8 female conditions per trimester--twice as many as male trainees (mean = 21.3).

CONCLUSION

Considerable variation exists trainees in the number of patient contacts. Differences in patient mix between first- and third-year trainees seem at least partly related to year-specific learning objectives. The use of an EPR-derived educational instrument provides insight into the trainees' patient mix at both the group and the individual level. This offers opportunities for GP trainers, trainees, and curriculum designers to optimise learning when exposure may be low.

Mixed messages in learning communication skills? Students comparing role model behaviour in clerkships with formal training

BACKGROUND

Medical students learn professional communication through formal training and in clinical practice. Physicians working in clinical practice have a powerful influence on student learning. However, they may demonstrate communication behaviours not aligning with recommendations in training programs.

AIMS

This study aims to identify more precisely what differences students perceive between role model communication behaviour during clerkships and formal training.

METHOD

In a cross-sectional study, data were collected about physicians' communication performance as perceived by students. Students filled out a questionnaire in four different clerkships in their fourth and fifth year.

RESULTS

Just over half of the students reported communication similar to formal training. This was especially true for students in the later clerkships (paediatrics and primary care). Good examples were seen in providing information corresponding to patients' needs and in shared decision making, although students often noted that in fact the doctor made the decision. Bad examples were observed in exploring cognitions and emotions, and in providing information meeting patient's pace.

CONCLUSIONS

Further study is needed on actual physician behaviour in clinical practice. From our results, we conclude that students need help in reflecting on and learning from the gap in communication patterns they observe in training versus clinical practice.

Exploring differences in patient mix in a cohort of GP trainees and their trainers

Background During specialty training for general practice, trainees acquire the required competencies through work-based learning. Previous small-scale and older studies suggest that the patient mix of general practitioner (GP) trainees differs from that of their trainers: trainees are exposed to more minor illnesses, and fewer chronic diseases and severe conditions, which may influence the development of their competency. Research question What are the differences in the patient mix between trainees and trainers? Methods 49 first- and 24 third-year trainees and their trainers (n=114) were included in the study. International Classification of Primary Care (ICPC) contact and diagnosis codes were extracted from electronic patient records over 6 months. Results Trainers had double the number of face-to-face consultations, and treble the number of telephone consultations compared with trainees. The trainees' patient mix consisted of significantly more patients with eye diseases, ear diseases, respiratory diseases, skin diseases and minor illnesses compared with their trainers. Trainers encountered significantly more patients with circulatory diseases, psychiatric diseases, metabolic diseases, male genital conditions, social problems, and chronic and oncological diseases. Female trainers and trainees encountered almost twice the number of female conditions compared with their male counterparts, while for male conditions, the opposite was found. Discussion Considerable differences between the patient mix of trainers and trainees were found. Specialty trainers and teachers must be aware of areas of low exposure. Trainers should ensure trainees handle more chronic, complex, psychosocial and circulatory conditions.

Ethical issues confronted by medical students during clinical rotations

OBJECTIVES

This study aimed to examine the most common and important ethical issues confronting medical students during clinical rotations so that ethics-related topics can be prioritised according to students' needs and this information used to develop a curriculum for the ethics course.

METHODS

In a cross-sectional approach, we reviewed the medical ethics-related cases recorded in the logbooks of all medical students (n=241) at Tehran University of Medical Sciences who attended the medical ethics course during October 2006 to July 2007. As part of a graded assignment, each student was required to record three encounters with ethics-related issues in his or her logbook. A total of 713 cases were assessed. Information related to the ethical issues and the conditions in which ethical issues arose was extracted and recorded by two experts, whose analysis showed agreement of kappa 0.77. In cases of discrepancy, both experts reviewed and discussed the record until they achieved agreement.

RESULTS

A total of 713 cases were analysed. The most common issues reported by students related to ethics in medical education (20.1%, n=143), professionalism (18.8%, n=134), confidentiality (7.6%, n=54), the doctor-patient relationship (7.3%, n=52), informed consent (7.0%, n=50) and the doctor-peer relationship (7.0%, n=50). After adjusting for length of rotation, the highest numbers of ethics-related incidents were reported from urology, general surgery, orthopaedics, internal medicine, neurology, and obstetrics and gynaecology wards.

CONCLUSIONS

The results of this study indicate that professionalism and related elements represent one of the most important areas of concern that need to be addressed when planning courses for medical students. The other significant area of concern is that of ethics in medical education, which, although the subject is not considered essential for medical practitioners, should be taught and respected so that student sensitivity to medical ethics is maintained and even increased.

Using patient encounter logs for mandated clinical encounters in an internal medicine clerkship

BACKGROUND

Patient encounter logs help assess a student's educational experience. The use of a grading incentive linked to the mandatory documentation of prespecified clinical encounters has been insufficiently studied.

PURPOSES

Given this, our objectives were to determine (a) if mandating student exposure to patients with 18 key training problems leads to the successful documentation of these encounters, (b) the degree of difficulty students and clerkship directors experienced in meeting these mandates, (c) the accuracy of mandated log entries, and (d) how often the log entries were questioned or rejected by preceptors.

METHODS

Ninety-two 3rd-year internal medicine students and 6 internal medicine clerkship directors at 9 geographically dispersed hospitals and 30 ambulatory sites participated in the study. Over a single academic year, we directly measured the completion rate of mandated logs, the degree of their accuracy as judged against faculty-generated logs of two required videotaped standardized patient encounters, and the percentage of logs that were not validated. We surveyed students and directors on the perceived degree of difficulty in meeting mandated requirements.

RESULTS

Ninety-eight percent of students met our mandated requirements and 93.8% of students found it "easy" or "very easy" to meet this requirement. The amount of estimated time spent by clerkship directors helping students meet mandated requirements for the entire year was 4.5 hr. The accuracy of submitted logs was 77%; however, almost all inaccurate log entries were "validated" by preceptors.

CONCLUSIONS

Mandating encounters is effective in assuring that students document encounters with patients who present with specific clinical problems. The accuracy of our students' mandated logs is similar to previously published data. However, even inaccurate logs were rarely questioned or rejected by preceptors.

Internal medicine core clerkships experience with core problem lists: results from a national survey of clerkship directors in internal medicine

BACKGROUND

The Liaison Committee on Medical Education (LCME) requires clinical clerkships in North American Medical Schools to define and monitor core problems and clinical conditions for medical students and adjust the clerkships to ensure that all students meet those objectives at all instructional sites. Clinical clerkships usually use medical student generated logbooks to meet these requirements. It is not clear what clinical clerkship directors are doing to meet these standards.

PURPOSE

To determine how internal medicine clerkship directors develop problem lists, whether and how they verify student problem list entry, and how missed core problems are covered.

METHODS

The Clerkship Directors in Internal Medicine (CDIM) organization conducts an annual survey of its institutional members (one member per medical school). In 2007, 75% of North American medical schools (82 of 110) responded to the survey, which included questions on core problem lists and medical student logbooks.

RESULTS

Ninety-four percent of responding medical schools had a core problem list for the core internal medicine clerkship. Most schools used an electronic logbook and verified student exposure to core problems (76%). The clerkship director usually did the verification (63%). Clerkships offered a variety of options to remedy lack of exposure to core problems and usually offered more than one option.

CONCLUSIONS

Clerkship directors in internal medicine develop, verify, and ensure exposure to core problems in internal medicine. Most clerkship directors used the CDIM recommended core problem list as a basis for their local core problem list. Emerging computer-assisted learning options may simplify clerkship adjustments for lack of exposure to core problems. It appears that clerkship directors are using logbooks appropriately to meet the LCME charge to monitor core problems and clinical conditions.

The Learning of 7th Year Medical Students at Internal Medical – Evaluation by Logbooks

Introduction: The purpose of this study was to understand the learning of internal medicine of 7th year medical students through records of the “Learning Passport”. Materials and Methods: Between June 2005 and June 2006, data from the learning passport (a type of logbook) of 207 7th year medical students at the Department of Internal Medicine, National Taiwan University Hospital (NTUH) were collected. Results: Among the 19 symptoms/signs listed in the logbook, a large number of students did not learn well on low back pain, skin rash, oedema, oliguria and anxiety/depression; only a few students rated themselves as knowledgeable about anxiety/depression, malaise, skin rash, headache and anorexia. Among the 16 diseases listed, a large number of students did not learn well on chronic obstructive pulmonary disease, stroke, hypertension, coronary artery disease and cirrhosis; only a few students rated themselves as knowledgeable about shock, respiratory failure, consciousness disturbance, sepsis and renal failure. Among the 21 physical examination skills listed, a large number of students did not learn well on the male genitalia, eyes, cognitive status, mental state and the digital rectal examination; only a few students rated themselves as fully competent about cognitive status, mental state, eyes, neurology examination and ENT examination. Among the 11 laboratory skills and image interpretation skills listed, a large number of students did not learn well on blood smear, Gram’s stain and specimen sampling; only a few students rated themselves as fully competent about the interpretation of brain computed tomographic (CT) scan, blood smear and Gram’s stain. Among the 12 procedures and therapeutic skills listed, a large number of students did not learn well on observation of lumbar puncture, basic cardiopulmonary resuscitation (CPR) and aseptic procedure; only a few students rated themselves as fully competent about basic CPR and transfusion management. Conclusions: The weak points of intern training conducted by the Department of Internal Medicine, NTUH were revealed by analysis obtained from their logbooks. Thus, we need to strengthen the learning of the interns in these specific parts and assess their performance based on the use of portfolios. Key words: Clinical skills, Evaluation, Interns

Successful accomplishment of educational goals with clinical experience at public primary care facilities

GOAL

To compare the spectrum of clinical encounters experienced by medical students at the primary level of care in six urban public health units, and to determine the extent to which these educational experiences were sufficient to meet learning objectives proposed for a teaching module.

METHOD

During the 4th year of a new six- year curriculum, 113 students cared for adults, the elderly, women and children. They were supervised by faculty and trained supervisors during three 4-hours periods a week, every other week, from January to October at six primary health units.

RESULTS

There were 7198 clinical encounters (2493 for adults, 2440 for women, and 2302 for children), during a total of 37 periods, averaging 1.8 cases/student per period. The top five primary diagnoses, similar at all primary health units, included: for adults--hypertension, diabetes, upper respiratory diseases, anxiety/depression, and obesity; for children--first-year follow up, upper respiratory diseases, dermatological, and infectious diseases; for women--antenatal care, vaginal discharge, cervical cancer screening, climacteric symptoms/menstrual disorders, and family planning.

CONCLUSIONS

Students were exposed to and cared for the most common conditions observed at the primary level of care, with a sufficient homogeneous clinical spectrum among six primary health units, meeting essential learning objectives related to ambulatory care.

Clinical experience and examination performance: is there a correlation?

CONTEXT

The Liaison Committee on Medical Education (LCME) requires there to be: '...comparable educational experiences and equivalent methods of evaluation across all alternative instructional sites within a given discipline'. It is an LCME accreditation requirement that students encounter similar numbers of patients with similar diagnoses. However, previous empirical studies have not shown a correlation between the numbers of patients seen by students and performance on multiple-choice examinations.

OBJECTIVE

This study examined whether student exposure to patients with specific diagnoses predicts performance on multiple-choice examination questions pertaining to those diagnoses.

METHODS

The Department of Pediatrics at the University of Nebraska Medical Center has collected patient logbooks from clerks since 1994. These contain information on patient demographics and students' roles in patient care. During week 7 of an 8-week course, students took an examination intended to help them prepare for their final examination. Logbooks and pre-examination questions were coded using standard ICD-9 codes. Data were analysed using Minitab statistical software to determine dependence between patient encounters and test scores. Subjects comprised a convenience sample of students who completed the clerkship during 1997-2000.

RESULTS

Our analysis indicates that performance on a multiple-choice examination is independent of the number of patients seen.

CONCLUSIONS

Our data suggest knowledge-based examination performance cannot be predicted by the volume of patients seen. Therefore, too much emphasis on examination performance in clinical courses should be carefully weighed against clinical performance to determine the successful completion of clerkships.

Accuracy of medical student electronic logbook problem list entry

BACKGROUND

The accuracy of medical student logbooks has not been extensively studied.

PURPOSE

The purpose of this study was to determine accuracy of student entry of core problems and completeness of patient entry in an electronic logbook.

METHODS

Third-year internal medicine clerkship students entered patient encounters as required by the clerkship. Experts entered information from the same encounters.

RESULTS

A total of 1,440 patient entries generated by 37 consecutive students were compared to expert entries. Sensitivity (core problem underreporting) was low (60%, SD=22%). Percent agreement (87%, SD=7%), kappa (0.46, SD=0.19), and specificity (core problem overreporting; 95%, SD=5%) were good to excellent. Students both omitted (underreported 14%, SD=12%) and overreported (23%, sd 17%) patients.

CONCLUSIONS

Under ideal study circumstances, there was significant underreporting of core problems by students. Although the high specificity, meaning that students are not reporting problems they have not encountered, is reassuring, logbook sensitivity in this study was not good enough for high-stakes evaluations of students or for medical school licensing.

Influence of clerkship experiences on clinical competence

BACKGROUND

Clerkship experiences are considered crucial for the development of clinical competence. Yet whether there is a direct relationship between the nature and volume of patient encounters and learning outcomes is far from clear. Some evidence in the literature points towards the importance of clinical supervision on student learning, but the relationship between clinical supervision, patient encounters and student competence remains unclear.

OBJECTIVES

This study aimed firstly to determine the variation in students' clinical experiences within and across sites; secondly, to identify the causes of this variation, and thirdly, to investigate the consequences of this variation on students' competence.

METHODS

Clerkship students at 12 hospital sites recorded their patient encounters in logbooks. Site characteristics that might influence the variation in patient encounters were collected. Student competence was determined by 3 independent indicators: a practical end-of-clerkship examination; a theoretical end-of-clerkship examination, and an evaluation of professional performance. A model was developed to test the available clerkship data using structural equation modelling (SEM) software.

RESULTS

Analysis of the logbooks revealed a large variation in the number of patients encountered by students. The average length of patient stay, number of patients admitted, and quality of supervision accounted partly for this variation. An increased number of patient encounters did not directly lead to improved competence. Quality of supervision turned out to be crucially important because it directly impacted student learning and also positively influenced the number of patient encounters.

CONCLUSION

Monitoring the effectiveness of clerkship by merely asking students to keep a tally of the problems and diseases they encounter, without paying attention to the quality of supervision, does not contribute towards improving student learning.

Narrative review: use of student-generated logbooks in undergraduate medical education

BACKGROUND

Logbooks are used by clinical clerkships in undergraduate medical education as tools for individual student guidance, programmatic evaluation, and Liaison Committee on Medical Education (LCME) accreditation. The purpose of this narrative review was to summarize the published literature on the form and function of logbooks and to review logbook validity and reliability. We performed a literature search from 1980 through 2004 and reviewed 50 articles on logbook use during clinical clerkships.

SUMMARY

Articles were categorized into 5 themes: description and feasibility of logbooks (27 articles), accuracy and completeness of logbook entries (14 articles), utility to student education (11 articles), utility to program evaluation (26 articles), and connecting logbook process measures to clerkship outcomes (2 articles).

CONCLUSIONS

A feasible and acceptable logbook system is an attainable goal, although students usually did not complete logbooks unless required. The available literature does not establish that logbooks currently in use have sufficient reliability or validity to allow for the routine use of the information for program change or accreditation purposes. The ideal logbook should be inexpensive, feasible, and acceptable to students and should allow rapid collation of accurate, relevant data for timely analysis and feedback to the student and clerkship director.

The impact of multifaceted educational structuring on learning effectiveness in a surgical clerkship

INTRODUCTION

Various measures have been introduced to enhance learning experiences in clerkships, generally with limited success. This study evaluated the impact of a multifaceted approach on the effectiveness of learning in a surgical clerkship. In accordance with results obtained in continuing medical education, several interventions were introduced simultaneously. We compared students' evaluations of the traditional surgical clerkship with those of the restructured clerkship.

METHODS

Two consecutive cohorts of students were asked to complete a questionnaire about the quality and quantity of their learning experiences. Cohort 1 (n = 28) undertook the traditional clerkship and cohort 2 (n = 72) the restructured clerkship. A Mann-Whitney test was used to compare outcomes between the 2 cohorts.

RESULTS

There were few statistically significant differences between cohorts 1 and 2. Overall, quality indicators did not differ between the 2 cohorts.

DISCUSSION

A short-term multifaceted intervention led to a slight increase in the performance of clinical skills and a slight decrease in time spent on activities of limited educational value. The intervention may have been too brief to produce substantial effects. Future interventions should also target teachers, including trainees, in order to assess their opinions and address their educational needs.