Resident self-assessment and self-reflection: University of Wisconsin-Madison's Five-Year Study

Adapting and applying student-centered learning in a perfusion clinical rotation

While the process of teaching student perfusionists has been in development since the 1950s, the publication of the processes to improve perfusion clinical education has been largely lacking. Publications regarding education from other allied health and medical fields have shown the value of student-centered learning. The use of reflective practice to move perfusion students from thinking about actions after cardiopulmonary bypass (CPB) to reflecting and reacting on actions during CPB is better encouraged by moving from a teacher-centered to a student-centered clinical model. Our institution's teaching process has developed into a multi-point procedure to make our students into reflective practicing clinicians. Student preceptor evaluations were reversed to allow the students to evaluate themselves first, with feedback from the preceptor given subsequently. Additionally, a biweekly student educational session, where the student chooses a topic and reviews current evidence-based practice, was instituted. The clinical program director serves as the moderator and clinical expert to facilitate problem-based learning during the sessions. Students were also given three skill/experience levels with goals to reach and move through during the rotation. These student levels were also helpful to our preceptors in knowing what each student's skill level was throughout their rotation. Overall, moving from a teacher-centered to a student-centered clinical rotation has helped make students familiar with reflective practice, self-evaluation, evidence-based practice, and problem-based learning. The incorporation of these processes will hopefully lead students to become lifelong reflective perfusionists.

A Mixed-Methods Program Evaluation of a Self-directed Learning Panel Management Curriculum in an Internal Medicine Residency Clinic

BACKGROUND

Panel management (PM) curricula in internal medicine (IM) residency programs often assign performance measures which may not address the varied interests or needs of resident-learners.

AIM

To evaluate a self-directed learning (SDL)-based PM curriculum.

SETTING

University-based primary care practice in Burlington, Vermont.

PARTICIPANTS

Thirty-five internal medicine residents participated.

PROGRAM DESCRIPTION

Residents completed a PM curriculum that integrated SDL, electronic health record (EHR)-driven performance feedback, mentorship, and autonomy to set learning and patient care goals.

PROGRAM EVALUATION

Pre/post-curricular surveys assessed EHR tool acceptability, weekly curricular surveys and post-curricular focus groups assessed resident perceptions and goals, and an interrupted time series analysis of care gap closure rates was used to compare the pre-intervention and intervention periods. Majority of residents (28-32 or 80-91%) completed the surveys and focus groups. Residents found the EHR tools acceptable and valued protected time, mentorship, and autonomy to set goals. A total of 13,313 patient visits were analyzed. There were no significant differences between rates between the pre-intervention period and the first intervention period (p=0.44).

DISCUSSION

A longitudinal PM curriculum that incorporated SDL and goal setting with EHR-driven performance feedback was well-received by residents, however did not significantly impact the rate of care gap closure.

Continuity Clinic Practice Feedback Curriculum for Residents: A Model for Ambulatory Education

BACKGROUND

There is an unmet need for formal curricula to deliver practice feedback training to residents.

OBJECTIVE

We developed a curriculum to help residents receive and interpret individual practice feedback data and to engage them in quality improvement efforts.

METHODS

We created a framework based on resident attribution, effective metric selection, faculty coaching, peer and site comparisons, and resident-driven goals. The curriculum used electronic health record-generated resident-level data and disease-specific ambulatory didactics to help motivate quality improvement efforts. It was rolled out to 144 internal medicine residents practicing at 1 of 4 primary care clinic sites from July 2016 to June 2017. Resident attitudes and behaviors were tracked with presurveys and postsurveys, completed by 126 (88%) and 85 (59%) residents, respectively. Data log-ins and completion of educational activities were monitored. Group-level performance data were tracked using run charts.

RESULTS

Survey results demonstrated significant improvements on a 5-point Likert scale in residents' self-reported ability to receive (from a mean of 2.0 to 3.3, P < .001) and to interpret and understand (mean of 2.4 to 3.2, P < .001) their practice performance data. There was also an increased likelihood they would report that their practice had seen improvements in patient care (13% versus 35%, P < .001). Run charts demonstrated no change in patient outcome metrics.

CONCLUSIONS

A learner-centered longitudinal curriculum on ambulatory patient panels can help residents develop competency in receiving, interpreting, and effectively applying individualized practice performance data.

Internal Medicine Resident Experiences With a 5-Month Ambulatory Panel Management Curriculum

BACKGROUND

Panel management is emphasized as a subcompetency in internal medicine graduate medical education. Despite its importance, there are few published curricula on population medicine in internal medicine residency programs.

OBJECTIVE

We explored resident experiences and clinical outcomes of a 5-month diabetes and obesity ambulatory panel management curriculum.

METHODS

From August through December 2016, internal medicine residents at the University of Vermont Medical Center reviewed registries of their patients with diabetes, prediabetes, and obesity; completed learning modules; coordinated patient outreach; and updated gaps in care. Resident worksheets, surveys, and reflections were analyzed using descriptive and thematic analyses. Before and after mean hemoglobin A1c results were obtained for patients in the diabetic group.

RESULTS

Most residents completed the worksheet, survey, and reflection (93%-98%, N = 42). The worksheets showed 70% of participants in the diabetic group had appointments scheduled after outreach, 42% were offered referrals to the Community Health Team, and 69% had overdue laboratory tests ordered. Residents reported they worked well with staff (95%), were successful in coordinating outreach (67%), and increased their sense of patient care ownership (66%). In reflections, identified successes were improved patient care, teamwork, and relationship with patients, while barriers included difficulty ensuring follow-up, competing patient priorities, and difficulty with patient engagement. Precurricular mean hemoglobin A1c was 7.7%, and postcurricular was 7.6% (P = .41).

CONCLUSIONS

The curriculum offered a feasible, longitudinal model to introduce residents to population health skills and interdisciplinary care coordination. Although mean hemoglobin A1c did not change, residents reported improved patient care. Identified barriers present opportunities for resident education in patient engagement.

Ready for Residency: A Bloomian Analysis of Competency-Based Osteopathic Medical Education

Context

Bloom's Taxonomy is a widely accepted tool for analyzing learning objectives, creating assessment materials, and ensuring that students move progressively through various levels of knowledge and cognition. Competency-based osteopathic medical education has, to the authors' knowledge, yet to be subjected to systematic bloomian analysis.

Objective

To advance a bloomian analysis of competencies published by the American Association of Colleges of Osteopathic Medicine (AACOM) and draw consequences of that analysis for undergraduate osteopathic medical education.

Methods

Content analysis of the verbs and contexts used in the AACOM competencies was conducted, followed by categorization by the various cognitive and knowledge dimensions as specified by the Revised Bloom's Taxonomy.

Results

The majority of AACOM competencies gravitate toward the taxonomy's apply and procedural cognitive and knowledge dimensions, respectively. In addition, 58% of the top 10 most-used verbs in the document were identified to be in the apply cognitive dimension. An analysis of the competencies revealed that 69% can be categorized under the procedural knowledge dimension. Of 657 code co-occurances, 45% were from the apply cognitive and procedural knowledge dimensions. These figures suggest a skewing toward application-based cognitive and procedural-based knowledge over other types of cognitive and knowledge dimensions.

Conclusion

Bloom's Taxonomy offers a useful framework for understanding how competency-based osteopathic medical education is organized. Such analysis underscores the importance of attending to the language and construction of competencies to better understand how the language of competencies shapes pedagogic practices and begin a broader conversation about the appropriate use and distribution of lower- and higher-order cognitive processes within competency-based osteopathic medical education.

The impact of a continuing training program on the perceived improvement in quality of health care delivered by health care professionals

There is abundant scientific literature concerning factors that affect patients' perceptions of the quality of health care. However, there are few published works that consider the opinions of health care professionals. This article aims to conjointly analyse two organisational strategies that determine professional health care practice: continuous training and quality of care. The objective is to examine the opinions of physicians and nurses on the improvement of the quality of care after a 'learning by doing' program. An evaluation method was designed that integrates the main variables that intervene in quality of care. An online questionnaire was utilised for collecting opinions on the effects of the training program. A total of 184 nurses and 180 other medical professionals participated in the program and all of them were asked to complete the questionnaire. A descriptive, and inferential statistical analysis was undertaken and results showed that there is a direct relationship between perceptions about: satisfaction, professional competence, training modality, optimisation of health resources and quality of care.

What Happened to My Patient? An Educational Intervention to Facilitate Postdischarge Patient Follow-Up

BACKGROUND

Following up on patients' clinical courses after hospital discharge may enhance physicians' learning and care of future patients. Barriers to this practice for residents include time constraints, discontinuous training environments, and difficulty accessing patient information.

OBJECTIVE

We designed an educational intervention facilitating informed self-assessment and reflection through structured postdischarge follow-up of patients' longitudinal clinical courses. We then examined the experience of interns who received this intervention in a mixed methods study.

METHODS

Internal medicine interns on a 4-week patient safety rotation received lists of hospitalized patients they had cared for earlier in the year. They selected patients for chart review and completed a guided reflection worksheet for each patient reviewed. Interns then discussed lessons learned in a faculty-led group debrief session.

RESULTS

Of 62 eligible interns, 62 (100%) participated in this intervention and completed 293 reflection worksheets. We analyzed worksheets and transcripts from 6 debrief sessions. Interns reported that postdischarge patient follow-up was valuable for their professional development, and helped them understand the natural history of disease and patients' illness experiences. After reviewing their patients' clinical courses, interns stated that they would advocate for earlier end-of-life counseling, improve care transitions, and adjust their clinical decision-making for similar patients in the future.

CONCLUSIONS

Our educational intervention created the time, space, and structure for postdischarge patient follow-up. It was well received by participants, and is an opportunity for experiential learning.

A 15-year review of the Stanford Internal Medicine Residency Program: predictors of resident satisfaction and dissatisfaction

INTRODUCTION

Satisfaction with training and with educational experiences represents important internal medicine (IM) programmatic goals. Graduates from IM residency programs are uniquely poised to provide insights into their educational and training experiences and to assess whether these experiences were satisfactory and relevant to their current employment.

METHODS

We surveyed former IM residents from the training program held during the years 2000-2015 at the Department of Medicine, Stanford University. The first part of the survey reviewed the IM residency program and the second part sought identifying data regarding gender, race, ethnicity, work, relationships, and financial matters. The primary outcome was satisfaction with the residency experience.

RESULTS

Of the 405 individuals who completed the Stanford IM residency program in the study period, we identified 384 (95%) former residents with a known email address. Two hundred and one (52%) former residents responded to the first part and 185 (48%) answered both the parts of the survey. The mean age of the respondents was 36.9 years; 44% were female and the mean time from IM residency was 6.1 (±4.3) years. Fifty-eight percent reported extreme satisfaction with their IM residency experience. Predictors associated with being less than extremely satisfied included insufficient outpatient experience, insufficient international experience, insufficient clinical research experience, and insufficient time spent with family and peers.

CONCLUSION

The residents expressed an overall high satisfaction rate with their IM training. The survey results provided insights for improving satisfaction with IM residency training that includes diversifying and broadening IM training experiences.

Reflection as a Learning Tool in Graduate Medical Education: A Systematic Review

BACKGROUND

Graduate medical education programs employ reflection to advance a range of outcomes for physicians in training. However, the most effective applications of this tool have not been fully explored.

OBJECTIVE

A systematic review of the literature examined interventions reporting the use of reflection in graduate medical education.

METHODS

The authors searched Medline/PubMed, Embase, Cochrane CENTRAL, and ERIC for studies of reflection as a teaching tool to develop medical trainees' capacities. Key words and subject headings included reflection, narrative, residents/GME, and education/teaching/learning. No language or date limits were applied. The search yielded 1308 citations between inception for each database and June 15, 2015. A total of 16 studies, encompassing 477 residents and fellows, met eligibility criteria. Study quality was assessed using the Critical Appraisal Skills Programme Qualitative Checklist. The authors conducted a thematic analysis of the 16 articles.

RESULTS

Outcomes studied encompassed the impact of reflection on empathy, comfort with learning in complex situations, and engagement in the learning process. Reflection increased learning of complex subjects and deepened professional values. It appears to be an effective tool for improving attitudes and comfort when exploring difficult material. Limitations include that most studies had small samples, used volunteers, and did not measure behavioral outcomes.

CONCLUSIONS

Critical reflection is a tool that can amplify learning in residents and fellows. Added research is needed to understand how reflection can influence growth in professional capacities and patient-level outcomes in ways that can be measured.

Social worker assessment of bad news delivery by emergency medicine residents: a novel direct-observation milestone assessment

The skill of delivering bad news is difficult to teach and evaluate. Residents may practice in simulated settings; however, this may not translate to confidence or competence during real experiences. We investigated the acceptability and feasibility of social workers as evaluators of residents' delivery of bad news during patient encounters, and assessed the attitudes of both groups regarding this process. From August 2013 to June 2014, emergency medicine residents completed self-assessments after delivering bad news. Social workers completed evaluations after observing these conversations. The Assessment tools were designed by modifying the global Breaking Bad News Assessment Scale. Residents and social workers completed post-study surveys. 37 evaluations were received, 20 completed by social workers and 17 resident self-evaluations. Social workers reported discussing plans with residents prior to conversations 90 % of the time (18/20, 95 % CI 64.5, 97.8). Social workers who had previously observed the resident delivering bad news reported that the resident was more skilled on subsequent encounters 90 % of the time (95 % CI 42.2, 99). Both social workers and residents felt that prior training or experience was important. First-year residents valued advice from social workers less than advice from attending physicians, whereas more experienced residents perceived advice from social workers to be equivalent with that of attending physicians (40 versus 2.9 %, p = 0.002). Social worker assessment of residents' abilities to deliver bad news is feasible and acceptable to both groups. This formalized self-assessment and evaluation process highlights the importance of social workers' involvement in delivery of bad news, and the teaching of this skill. This method may also be used as direct-observation for resident milestone assessment.

The Effects of Resident Peer- and Self-Chart Review on Outpatient Laboratory Result Follow-up

PURPOSE

Performing and teaching appropriate follow-up of outpatient laboratory results (LRs) is a challenge. The authors tested peer-review among residents as a potentially valuable intervention.

METHOD

Investigators assigned residents to perform self-review (n = 27), peer-review (n = 21), or self- + peer-review (n = 30) of outpatient charts. They also compared residence performance with that of historical controls (n = 20). In September 2012, residents examined 10 LRs from April 2012 onward. A second review in November 2012 ascertained whether performing chart review improved residents' practice behaviors.

RESULTS

Initially, the least-square (LS) mean number of LRs without documentation of follow-up per resident in the self-, peer-, and self- + peer-review group was, respectively, 0.5 (SD 1.0), 1.0 (SD 1.7), and 0.9 (SD 1.3), and post intervention, this was 1.0 (SD 0.2), 0.3 (SD 0.2), and 0.6 (SD 0.2) (self- versus peer-review P = .03). Initially the LS mean follow-up time per resident in the self-, peer-, and self- + peer-review group was, respectively, 4.2 (SD 1.2), 6.9 (SD 1.4), and 5.9 (SD 1.2) days, and after the intervention, LS mean time was 5.0 (SD 0.5), 2.5 (SD 0.6), and 3.9 (SD 0.5) days (self- versus peer-review P < .01). Self-review was not associated with significant improvements in practice.

CONCLUSIONS

In this comparison of self- and peer-review, only residents who performed peer-review demonstrated significant improvements in their documentation practices. These findings support the use of resident peer-review in improving LR follow-up, and potentially, in other, broader resident quality improvement initiatives.

Qualitative Analysis of Written Reflections during a Teaching Certificate Program

OBJECTIVE

To evaluate the success of a teaching certificate program by qualitatively evaluating the content and extent of participants' reflections.

METHODS

Two investigators independently identified themes within midpoint and final reflection essays across six program years. Each essay was evaluated to determine the extent of reflection in prompted teaching-related topic areas (strengths, weaknesses, assessment, feedback).

RESULTS

Twenty-eight themes were identified within 132 essays. Common themes encompassed content delivery, student assessment, personal successes, and challenges encountered. Deep reflection was exhibited, with 48% of essays achieving the highest level of critical reflection. Extent of reflection trended higher from midpoint to final essays, with significant increases in the strengths and feedback areas.

CONCLUSION

The teaching certificate program fostered critical reflection and self-reported positive behavior change in teaching, thus providing a high-quality professional development opportunity. Such programs should strongly consider emphasizing critical reflection through required reflective exercises at multiple points within program curricula.

Verification of accurate technical insight: a prerequisite for self-directed surgical training

Simulation-based surgical skills training during preclinical education is a persistent challenge due to time constraints of trainees and instructors alike. Self-directed practice is resource-efficient and flexible; however, insight into technical proficiency among trainees is often lacking. The purpose of this study is to prospectively assess the accuracy of self-assessments among medical students learning basic surgical suturing. Over seven weekly practice sessions, preclinical medical students performed serial repetitions of a simulation-based suturing task under one-on-one observation by one of four trainers. Following each task repetition, self- and trainer-assessments (SA-TA) were performed using a 36-point weighted checklist of technical standards developed a priori by expert consensus. Upon study completion, agreement between SA and TA was measured using weighted Cohen's kappa coefficients. Twenty-nine medical students each performed a median of 25 suture task repetitions (IQR 21.5-28). Self-assessments tended to overestimate proficiency during the first tertile of practice attempts. Agreement between SA and TA improved with experience, such that the weighted kappa statistics for the two-handed and instrument ties were >0.81 after 18-21 task attempts. Inexperienced trainees frequently overestimate technical proficiency through self-assessments. However, this bias diminishes with repetitive practice. Only after trainees have attained the capacity to accurately self-assess can effective self-directed learning take place.

Informal learning from error in hospitals: what do we learn, how do we learn and how can informal learning be enhanced? A narrative review

Learning from error is not just an individual endeavour. Organisations also learn from error. Hospitals provide many learning opportunities, which can be formal or informal. Informal learning from error in hospitals has not been researched in much depth so this narrative review focuses on five learning opportunities: morbidity and mortality conferences, incident reporting systems, patient claims and complaints, chart review and prospective risk analysis. For each of them we describe: (1) what can be learnt, categorised according to the seven CanMEDS competencies; (2) how it is possible to learn from them, analysed against a model of informal and incidental learning; and (3) how this learning can be enhanced. All CanMEDS competencies could be enhanced, but there was a particular focus on the roles of medical expert and manager. Informal learning occurred mostly through reflection and action and was often linked to the learning of others. Most important to enhance informal learning from these learning opportunities was the realisation of a climate of collaboration and trust. Possible new directions for future research on informal learning from error in hospitals might focus on ways to measure informal learning and the balance between formal and informal learning. Finally, 12 recommendations about how hospitals could enhance informal learning within their organisation are given.

Residents' reflections on quality improvement: temporal stability and associations with preventability of adverse patient events

PURPOSE

Residents' reflections on quality improvement (QI) opportunities are poorly understood. The authors used the Mayo Evaluation of Reflection on Improvement Tool (MERIT) to measure residents' reflection scores across three years and to determine associations between reflection scores and resident and adverse patient event characteristics.

METHOD

From 2006 to 2009, 48 Mayo Clinic internal medicine residents completed biannual reflections on adverse events and classified event severity and preventability. Faculty assessed residents' reflections using MERIT, which contains 18 Likert-scaled items and measures three factors-personal reflection, systems reflection, and event merit. ANOVA was used to identify changes in MERIT scores across three years of training and among factors, paired t tests were used to identify differences between MERIT factor scores, and generalized estimating equations were used to examine associations between MERIT scores and resident and adverse event characteristics.

RESULTS

The residents completed 240 reflections. MERIT reflection scores were stable over time. Individual factor scores differed significantly (P < .0001), with event merit being the highest and systems reflection the lowest. Event preventability was significantly associated with MERIT factor scores and overall scores (beta = 0.415; CI = 0.186-0.643; P = .0004). No significant associations between MERIT scores and resident characteristics or event severity were identified.

CONCLUSIONS

Residents' reflections on adverse events remained constant over time, were lowest for systems factors, and were associated with adverse event preventability. Future research should explore learners' emphasis on systems aspects of QI and the relationship between QI and event preventability.

Pilot study evaluating a practice-based learning and improvement curriculum focusing on the development of system-level quality improvement skills

BACKGROUND

We developed a practice-based learning and improvement (PBLI) curriculum to address important gaps in components of content and experiential learning activities through didactics and participation in systems-level quality improvement projects that focus on making changes in health care processes.

METHODS

We evaluated the impact of our curriculum on resident PBLI knowledge, self-efficacy, and application skills. A quasi-experimental design assessed the impact of a curriculum (PBLI quality improvement systems compared with non-PBLI) on internal medicine residents' learning during a 4-week ambulatory block. We measured application skills, self-efficacy, and knowledge by using the Systems Quality Improvement Training and Assessment Tool. Exit evaluations assessed time invested and experiences related to the team projects and suggestions for improving the curriculum.

RESULTS

The 2 groups showed differences in change scores. Relative to the comparison group, residents in the PBLI curriculum demonstrated a significant increase in the belief about their ability to implement a continuous quality improvement project (P  =  .020), comfort level in developing data collection plans (P  =  .010), and total knowledge scores (P < .001), after adjusting for prior PBLI experience. Participants in the PBLI curriculum also demonstrated significant improvement in providing a more complete aim statement for a proposed project after adjusting for prior PBLI experience (P  =  .001). Exit evaluations were completed by 96% of PBLI curriculum participants who reported high satisfaction with team performance.

CONCLUSION

Residents in our curriculum showed gains in areas fundamental for PBLI competency. The observed improvements were related to fundamental quality improvement knowledge, with limited gain in application skills. This suggests that while heading in the right direction, we need to conceptualize and structure PBLI training in a way that integrates it throughout the residency program and fosters the application of this knowledge and these skills.

Development and preliminary evaluation of a practice-based learning and improvement tool for assessing resident competence and guiding curriculum development

BACKGROUND

Although practice-based learning and improvement (PBLI) is now recognized as a fundamental and necessary skill set, we are still in need of tools that yield specific information about gaps in knowledge and application to help nurture the development of quality improvement (QI) skills in physicians in a proficient and proactive manner. We developed a questionnaire and coding system as an assessment tool to evaluate and provide feedback regarding PBLI self-efficacy, knowledge, and application skills for residency programs and related professional requirements.

METHODS

Five nationally recognized QI experts/leaders reviewed and completed our questionnaire. Through an iterative process, a coding system based on identifying key variables needed for ideal responses was developed to score project proposals. The coding system comprised 14 variables related to the QI projects, and an additional 30 variables related to the core knowledge concepts related to PBLI. A total of 86 residents completed the questionnaire, and 2 raters coded their open-ended responses. Interrater reliability was assessed by percentage agreement and Cohen κ for individual variables and Lin concordance correlation for total scores for knowledge and application. Discriminative validity (t test to compare known groups) and coefficient of reproducibility as an indicator of construct validity (item difficulty hierarchy) were also assessed.

RESULTS

Interrater reliability estimates were good (percentage of agreements, above 90%; κ, above 0.4 for most variables; concordances for total scores were R  =  .88 for knowledge and R  =  .98 for application).

CONCLUSION

Despite the residents' limited range of experiences in the group with prior PBLI exposure, our tool met our goal of differentiating between the 2 groups in our preliminary analyses. Correcting for chance agreement identified some variables that are potentially problematic. Although additional evaluation is needed, our tool may prove helpful and provide detailed information about trainees' progress and the curriculum.

Relationship between critical reflection and quality improvement proposal scores in resident doctors

OBJECTIVES

transformative learning theory supports the idea that reflection on quality improvement (QI) opportunities and the ability to develop successful QI projects may be fundamentally linked. We used validated methods to explore associations between resident doctors' reflections on QI opportunities and the quality of their QI project proposals.

METHODS

eighty-six residents completed written reflections on practice improvement opportunities and developed QI proposals. Two faculty members assessed residents' reflections using the 18-item Mayo Evaluation of Reflection on Improvement Tool (MERIT), and assessed residents' QI proposals using the seven-item Quality Improvement Project Assessment Tool (QIPAT-7). Both instruments have been validated in previous work. Associations between MERIT and QIPAT-7 scores were determined. Internal consistency reliabilities of QIPAT-7 and MERIT scores were calculated.

RESULTS

there were no significant associations between MERIT overall and domain scores, and QIPAT-7 overall and item scores. The internal consistency of MERIT and QIPAT-7 item groups were acceptable (Cronbach's α 0.76-0.94).

CONCLUSIONS

the lack of association between MERIT and QIPAT-7 scores indicates a distinction between resident doctors' skills at reflection on QI opportunities and their abilities to develop QI projects. These findings suggest that practice-based reflection and QI project development are separate constructs, and that skilful reflection may not predict the ability to design meaningful QI initiatives. Future QI curricula should consider teaching and assessing QI reflection and project development as distinct components.

Perspective: Transformative learning: a framework using critical reflection to link the improvement competencies in graduate medical education

Quality improvement (QI) in health care involves activities ranging from enhancing personal practice to reforming the larger health care system. The Accreditation Council for Graduate Medical Education recognizes this broad definition of QI in its requirement that physicians-in-training demonstrate competence in practice-based learning and improvement (PBLI) and systems-based practice (SBP). Creative metaphors have been used to teach the PBLI and SBP competencies, but conceptual frameworks describing the relationship between these competencies are needed. Transformative learning is an adult education theory that states individuals must critically reflect on life events in order to change their beliefs or behaviors. The authors propose that critical reflection during transformative learning can conceptually link PBLI and SBP. Reflection on personal experience with suboptimal patient care leads to recognizing personal or system limitations. Addressing personal limitations improves individual practice (PBLI), whereas applying QI methodologies leads to large-scale improvements (SBP). Educators who adopt the transformative learning framework should be able to design meaningful QI curricula that encourage residents to be reflective and empower them with QI skills.

Validation of an online assessment of orthopedic surgery residents' cognitive skills and preparedness for carpal tunnel release surgery

BACKGROUND

Surgical competence requires both cognitive and technical skills. Relatively little is found in the literature regarding the value of Web-based assessments to measure surgery residents' mastery of the competencies.

OBJECTIVE

To determine the validity and reliability of 2 online instruments for predicting the cognitive preparedness of residents for performing carpal tunnel release surgery.

METHOD

Twenty-eight orthopedic residents and 2 medical school students responded to an online measure of their perception of preparedness and to an online cognitive skills assessment prior to an objective structured assessment of technical skills, in which they performed carpal tunnel release surgery on cadaveric specimens and received a pass/fail assessment. The 2 online assessments were analyzed for their internal reliability, external correlation with the pass/fail decision, and construct validity.

RESULTS

The internal consistency of the perception of preparedness measure was high (α  =  .92) while the cognitive assessment was less strong (α  =  .65). Both instruments demonstrated moderately strong correlations with the pass/fail decision, with Spearman correlation of .606 (P  =  .000) and .617 (P  =  .000), respectively. Using logistic regression to analyze the predictive strength of each instrument, the perception of preparedness measure demonstrated a 76% probability (η(2)  =  .354) and the cognitive skills assessment a 73% probability (η(2)  =  .381) of correctly predicting the pass/fail decision. Analysis of variance modeling resulted in significant differences between levels at P < .005, supporting good construct validity.

CONCLUSIONS

The online perception of preparedness measure and the cognitive skills assessment both are valid and reliable predictors of readiness to successfully pass a cadaveric motor skills test of carpal tunnel release surgery.

Creating champions for health care quality and safety

Patient safety and quality of care are public concerns that demand personal responsibility at all levels of the health care organization. Senior residents in our graduate medical education program took responsibility for a capstone quality improvement project designed to transform them into champions for health care quality. Residents (n = 26) participated alone or in pairs in a 1-month faculty-mentored rotation at the Veterans Administration Hospital during the 2007-2008 academic year. They completed a Web-based curriculum, identified a quality-of-care issue, applied Plan-Do-Study-Act cycles, authored a report, and engaged colleagues in their innovations during a department-wide presentation. Results indicated that residents demonstrated significantly enhanced knowledge and attitudes about patient safety and quality improvement and provided consistently positive faculty and rotation evaluations. In addition, residents generated 20 quality improvement project proposals with a 50% rate of hospital-wide implementation, leading to meaningful changes in the systems that affect patient care.

Achieving the AAAs of Ambulatory Care: Aptitude, Appeal, and Appreciation

BACKGROUND

In the current health care environment more patient care has moved from in-hospital care to the ambulatory primary care settings; however, fewer internal medicine residents are pursuing primary care careers. Barriers to residents developing a sense of competency and enjoyment in ambulatory medicine include the complexity of practice-based systems, patients with multiple chronic diseases, and the limited time that residents spend in the outpatient setting.

OBJECTIVE

In an effort to accelerate residents' ambulatory care competence and enhance their satisfaction with ambulatory practice, we sought to change the learning environment. Interns were provided a series of intensive, focused, ambulatory training sessions prior to beginning their own continuity clinic sessions. The sessions were designed to enable them to work confidently and effectively in their continuity clinic from the beginning of the internship year, and it was hoped this would have a positive impact on their perception of the desirability of ambulatory practice.

METHODS

Improvement needs assessment after a performance, so we developed a structured, competency-based, multidisciplinary curriculum for initiation into ambulatory practice. The curriculum focused on systems-based practice, patient safety, quality improvement, and collaborative work while emphasizing the importance of continuity of care and long-term doctor-patient relationships. Direct observation of patient encounters was done by an attending physician to evaluate communication and physical examination skills. Systems of care commonly used in the clinic were demonstrated. Resources for practice-based learning were used.

CONCLUSION

The immersion of interns in an intensive, hands-on experience using a structured ambulatory care orientation curriculum early in training may prepare the intern to be a successful provider and learner in the primary care ambulatory setting.