Peer chart audits: a tool to meet Accreditation Council on Graduate Medical Education (ACGME) competency in practice-based learning and improvement

Reducing the Gap in Knowledge and Expectations between Clinicians and People with Polycystic Ovary Syndrome or Adrenal Conditions: Simulation via Instant Messaging-Birmingham Advance: Patient and Public Involvement (SIMBA-PPI) Study

BACKGROUND

To evaluate the efficacy of SIMBA as an educational intervention for both HCPs and people with either PCOS or adrenal conditions and to study the change in knowledge of people with PCOS or adrenal conditions about the conditions and expectations from the HCPs involved in their  care following SIMBA-PPI sessions.

METHODS

Two SIMBA-PPI sessions (SIMBA-PPI Polycystic ovary syndrome (SIMBA-PCOS) and SIMBA-PPI Adrenal conditions (SIMBA-Adrenal conditions)) were conducted in September 2021 and March 2022. In both sessions, HCPs interacted with moderators on patient management through WhatsApp. Patients with respective conditions underwent workshop-style learning in the same cases. SIMBA-PCOS transcripts were also translated into Brazilian Portuguese and workshops were held in both Brazilian Portuguese and English. The two groups (HCPs and patients) were then brought together to discuss exploring gaps in knowledge and expectations. The Wilcoxon Signed-Rank test compared differences in pre- and post-SIMBA self-reported confidence levels in HCPs and patients. Qualitative data from the online recordings were transcribed and analysed with inductive thematic analysis to identify gaps in knowledge and expectations from managing the cases.

RESULTS

48 HCPs and 25 patients participated in our study. When compared to pre-SIMBA confidence levels, SIMBA-PPI sessions effectively improved clinicians' confidence in managing PCOS (40.5%, p < .001) and adrenal conditions (23.0%, p < .001) post-SIMBA. Patient participants' confidence in HCPs significantly increased in the PCOS session (SIMBA-PCOS: 6.25%, p = 0.01).

CONCLUSIONS

Integration of PPI into SIMBA improved HCPs' confidence in managing PCOS and adrenal conditions. SIMBA-PPI also improved patients' confidence in HCPs. Our findings suggest that participating in SIMBA-PPI sessions can reduce the gap in knowledge and expectations between patients and HCPs involved in their care.

Resident Perceptions of Continuity Clinic Patient Metrics Differ From EHR Data: Pilot Use of Population Health Dashboards

BACKGROUND AND OBJECTIVES

Population health management (PHM) dashboards using electronic health record (EHR) data can teach trainees about the population they serve while also delivering data on their clinical practice. Yet, few studies have demonstrated their use. In this pilot study, we assessed baseline resident perceptions of population health metrics for continuity clinic panels by comparing resident estimates with EHR-reported values delivered by individualized PHM dashboards.

METHODS

A descriptive, comparative study was conducted at a primary continuity clinic site for pediatric residents in January 2018. Residents were surveyed about population health metrics for their patient panels, including demographics, utilization, and medical diagnoses. We compared resident estimates to corresponding EHR-reported values using 2-tailed paired t tests.

RESULTS

A total of 42 out of 55 eligible residents (76%) completed the survey. Compared with EHR-reported values, residents estimated higher percentages of emergency department utilization (22.1% vs 10.3%, P < .01) and morbidity, including medical complexity (15.6% vs 5.9%, P < .01), overweight (38.1% vs 11.7%, P < .01), obesity (20.5% vs 15.8%, P = .02), and asthma (34.6% vs 21.4%, P < .01).

CONCLUSIONS

In this pilot study of PHM dashboards, resident perceptions of continuity clinic population health metrics did not align with EHR data. Estimates were higher for measures of utilization and morbidity. PHM dashboards may help trainees better understand their patient populations and serve as a consistent source of objective practice data. However, further research and investment is needed to evaluate dashboard implementation and impact on trainee and patient outcomes.

SIMBA: using Kolb's learning theory in simulation-based learning to improve participants' confidence

BACKGROUND

Simulation via Instant Messaging- Birmingham Advance (SIMBA) delivers simulation-based learning (SBL) through WhatsApp® and Zoom® based on Kolb's experiential learning theory. This study describes how Kolb's theory was implemented in practice during SIMBA adrenal session.

METHODS

SIMBA adrenal session was conducted for healthcare professionals and replicated Kolb's 4-stage cycle: (a) concrete experience-online simulation of real-life clinical scenarios, (b) reflective observation-discussion and Q&A following simulation, (c) abstract conceptualisation-post-session MCQs, and (d) active experimentation-intentions to implement the acquired knowledge in future practice. Participants' self-reported confidence levels for simulated and non-simulated cases pre- and post-SIMBA were analysed using Wilcoxon Signed-Rank test. Key takeaway and feedback were assessed quantitatively and qualitatively in a thematic analysis.

RESULTS

Thirty-three participants were included in the analysis. A Wilcoxon signed-rank test showed that the SIMBA session elicited a statistically significant change in participants' self-reported confidence in their approach to Cushing's syndrome (Z = 3.873, p = 0.0001) and adrenocortical carcinoma (Z = 3.970, p < 0.0001). 93.9% (n = 31/33) and 84.8% (n = 28/33) strongly agreed/agreed the topics were applicable to their clinical practice and accommodated their personal learning style, respectively. 81.8% (n = 27/33) reported increase in knowledge on patient management, and 75.8% (n = 25/33) anticipated implementing learning points in their practice.

CONCLUSIONS

SIMBA effectively adopts Kolb's theory to provide best possible experience to learners, highlighting the advantages of utilising social media platforms for SBL in medical education. The ability to conduct SIMBA sessions at modest cost internationally paves way to engage more healthcare professionals worldwide.

[Implementation of Recommendations and Clinical Practice Guidelines by residents: the role of the Multiprofessional Family and Community Care Teaching Units in Spain]

OBJECTIVE

To analyze the role of Family and Community Care Trainig Units as facilitators of the implementation of Clinical Practice Guidelines (CPG) and the factors associated with a greater effort in this task.

MATERIAL AND METHODS

Design: Cross-sectional descriptive study with analytical approach.

PARTICIPANTS

Training Units in Spain (N=94).

MAIN MEASUREMENTS

Variables were collected through a self-completed survey into five domains: characteristics of Training Units, training activity directed at evidence-based clinical practice (EBPP), importance attributed to this activity, responsibility for EBPP implementation, perception of barriers and facilitators to its use. Descriptive and multivariate analysis with the dependent variable being the perceived effort of the training unit to implement CPG.

RESULTS

45 Training Units responded (47.9%). 42.2%(CI 95%: 27.8-56.6) of their coordinators have directed research projects and 31.1% (CI 95%:17.6-44.6) have participated in elaborating CPG. They organized an average of 51hours (SD 47.2) of training in PCBP. 97.7% (CI95%:93.3-100) considered it fundamental that the residents ow and apply PCBP and 93.3% (CI95%:86.0-100) considered that tutors are responsible for the implementation. The participation of the coordinator in CPG (coef: 0.58; IC 95%: 0.00-1.16), awareness of how important is that residents know about CPG (coef: 0.89; IC 95%: 0.24-1.54) and that CPG appear to be widely applicable. applicable (coef: 0.35; IC 95%: -0.01-0.70) were related to a greater effort by the training units.

CONCLUSIONS

The training units recognize the importance of CPGs and consider that tutors are responsible for their implementation. Training Units effort to implement CPG was related to unit coordinators previous experience, the perception of applicability and residents needs.

Signed, Sealed, Delivered: Increasing Patient Notification of Test Results in an Internal Medicine Resident Continuity Clinic

OBJECTIVES

Informing patients of their test results is an important patient safety issue, yet many physicians perform dismally in this regard. Residents often face additional barriers to communicating test results to patients. We wanted to determine whether streamlining the notification process, communicating expectations, and having residents audit their performance would increase result notification rates.

METHODS

We used a quasi-experimental design, and a single-group before-and-after intervention. Our multifold intervention consisted of development and standardization of a notification process in the electronic medical record, an education component, and a self-audit component. During a 15-minute session, we educated residents on the use of the new process. We also restated expectations regarding notifying patients of their results. Residents audited their own charts for a period before the intervention and during a second, postintervention period. An independent review of notification rates took place simultaneously as well as during an additional period several months later.

RESULTS

In total, 87 residents were eligible for participation. All 87 completed the project, giving a 100% participation rate. Resident-reported laboratory test notification rates increased from 16% to 91%; other test result rates increased from 33% to 84%. The three independent reviews showed laboratory test notification rates increased from 18.5% to 71.7% to 87.1%, and notification of other test results increased from 23.5% to 66.7% to 91.7%.

CONCLUSIONS

Baseline rates of notification for diagnostic tests results were low, but streamlining the notification process, clearly stating expectations for using it, and using resident self-audit can improve notification rates.

Effectiveness of a strategy that uses educational games to implement clinical practice guidelines among Spanish residents of family and community medicine (e-EDUCAGUIA project): a clinical trial by clusters

Background

Clinical practice guidelines (CPGs) have been developed with the aim of helping health professionals, patients, and caregivers make decisions about their health care, using the best available evidence. In many cases, incorporation of these recommendations into clinical practice also implies a need for changes in routine clinical practice. Using educational games as a strategy for implementing recommendations among health professionals has been demonstrated to be effective in some studies; however, evidence is still scarce. The primary objective of this study is to assess the effectiveness of a teaching strategy for the implementation of CPGs using educational games (e-learning EDUCAGUIA) to improve knowledge and skills related to clinical decision-making by residents in family medicine. The primary objective will be evaluated at 1 and 6 months after the intervention. The secondary objectives are to identify barriers and facilitators for the use of guidelines by residents of family medicine and to describe the educational strategies used by Spanish teaching units of family and community medicine to encourage implementation of CPGs.

Methods/design

We propose a multicenter clinical trial with randomized allocation by clusters of family and community medicine teaching units in Spain. The sample size will be 394 residents (197 in each group), with the teaching units as the randomization unit and the residents comprising the analysis unit. For the intervention, both groups will receive an initial 1-h session on clinical practice guideline use and the usual dissemination strategy by e-mail. The intervention group (e-learning EDUCAGUIA) strategy will consist of educational games with hypothetical clinical scenarios in a virtual environment.

The primary outcome will be the score obtained by the residents on evaluation questionnaires for each clinical practice guideline. Other included variables will be the sociodemographic and training variables of the residents and the teaching unit characteristics. The statistical analysis will consist of a descriptive analysis of variables and a baseline comparison of both groups. For the primary outcome analysis, an average score comparison of hypothetical scenario questionnaires between the EDUCAGUIA intervention group and the control group will be performed at 1 and 6 months post-intervention, using 95 % confidence intervals. A linear multilevel regression will be used to adjust the model.

Discussion

The identification of effective teaching strategies will facilitate the incorporation of available knowledge into clinical practice that could eventually improve patient outcomes. The inclusion of information technologies as teaching tools permits greater learning autonomy and allows deeper instructor participation in the monitoring and supervision of residents. The long-term impact of this strategy is unknown; however, because it is aimed at professionals undergoing training and it addresses prevalent health problems, a small effect can be of great relevance.

Trial registration

ClinicalTrials.gov: NCT02210442.

The Effects of Resident Peer- and Self-Chart Review on Outpatient Laboratory Result Follow-up

PURPOSE

Performing and teaching appropriate follow-up of outpatient laboratory results (LRs) is a challenge. The authors tested peer-review among residents as a potentially valuable intervention.

METHOD

Investigators assigned residents to perform self-review (n = 27), peer-review (n = 21), or self- + peer-review (n = 30) of outpatient charts. They also compared residence performance with that of historical controls (n = 20). In September 2012, residents examined 10 LRs from April 2012 onward. A second review in November 2012 ascertained whether performing chart review improved residents' practice behaviors.

RESULTS

Initially, the least-square (LS) mean number of LRs without documentation of follow-up per resident in the self-, peer-, and self- + peer-review group was, respectively, 0.5 (SD 1.0), 1.0 (SD 1.7), and 0.9 (SD 1.3), and post intervention, this was 1.0 (SD 0.2), 0.3 (SD 0.2), and 0.6 (SD 0.2) (self- versus peer-review P = .03). Initially the LS mean follow-up time per resident in the self-, peer-, and self- + peer-review group was, respectively, 4.2 (SD 1.2), 6.9 (SD 1.4), and 5.9 (SD 1.2) days, and after the intervention, LS mean time was 5.0 (SD 0.5), 2.5 (SD 0.6), and 3.9 (SD 0.5) days (self- versus peer-review P < .01). Self-review was not associated with significant improvements in practice.

CONCLUSIONS

In this comparison of self- and peer-review, only residents who performed peer-review demonstrated significant improvements in their documentation practices. These findings support the use of resident peer-review in improving LR follow-up, and potentially, in other, broader resident quality improvement initiatives.

Curricula for teaching clinical practice guidelines in US psychiatry residency and child and adolescent fellowship programs: a survey study

OBJECTIVE

To determine the characteristics of curricula for teaching the content of clinical practice guidelines (CPGs) in psychiatric residency and child and adolescent fellowship programs as well as to determine if and how the learning of CPG content is applied in clinical care settings.

METHODS

We conducted a national online survey of directors of general psychiatry residency and child and adolescent fellowship programs in the USA. The survey questionnaire included 13 brief questions about the characteristics used to teach CPGs in the programs, as well as two demographic questions about each program and director. Descriptive statistics were reported for each questionnaire item by program classification (i.e., child and adolescent vs. general psychiatry).

RESULTS

The survey response rate was 49.8% (146 out of 293). Just 23% of programs reported having written goals and objectives related to teaching CPGs. The most frequently taught aspect of CPGs was their content (72% of programs). Didactic sessions were the most frequently employed teaching strategy (79% of programs). Regarding the application of CPG learning in treatment care settings, just 16% of programs applied algorithms in care settings, and 15% performed evaluations to determine consistency between CPG recommendations and care delivery. Only 8% of programs utilized audit and feedback to residents about their adherence to CPGs. Faculty time constraints and insufficient interest were the leading barriers (39% and 33% of programs, respectively) to CPG teaching, although 38% reported no barriers. However, child and adolescent programs less commonly identified insufficient interest among faculty as a barrier to teaching CPGs compared to general programs (20% vs. 43%). Moreover, compared to general programs, child and adolescent fellowship programs taught more aspects of CPGs, used more educational activities to teach the content of specific CPGs, and used more methods to evaluate the teaching of CPGs.

CONCLUSIONS

Although the majority of programs provided some teaching of CPGs, the rigorousness of the teaching approaches was limited, especially attempts to evaluate the extent and effectiveness of their use in clinical care. Child and adolescent fellowship programs provided more extensive teaching and evaluation related to CPGs.

Informal learning from error in hospitals: what do we learn, how do we learn and how can informal learning be enhanced? A narrative review

Learning from error is not just an individual endeavour. Organisations also learn from error. Hospitals provide many learning opportunities, which can be formal or informal. Informal learning from error in hospitals has not been researched in much depth so this narrative review focuses on five learning opportunities: morbidity and mortality conferences, incident reporting systems, patient claims and complaints, chart review and prospective risk analysis. For each of them we describe: (1) what can be learnt, categorised according to the seven CanMEDS competencies; (2) how it is possible to learn from them, analysed against a model of informal and incidental learning; and (3) how this learning can be enhanced. All CanMEDS competencies could be enhanced, but there was a particular focus on the roles of medical expert and manager. Informal learning occurred mostly through reflection and action and was often linked to the learning of others. Most important to enhance informal learning from these learning opportunities was the realisation of a climate of collaboration and trust. Possible new directions for future research on informal learning from error in hospitals might focus on ways to measure informal learning and the balance between formal and informal learning. Finally, 12 recommendations about how hospitals could enhance informal learning within their organisation are given.

The effectiveness of self-directed learning in health professions education: a systematic review

OBJECTIVES

Given the continuous advances in the biomedical sciences, health care professionals need to develop the skills necessary for life-long learning. Self-directed learning (SDL) is suggested as the methodology of choice in this context. The purpose of this systematic review is to determine the effectiveness of SDL in improving learning outcomes in health professionals.

METHODS

We searched MEDLINE, EMBASE, ERIC and PsycINFO through to August 2009. Eligible studies were comparative and evaluated the effect of SDL interventions on learning outcomes in the domains of knowledge, skills and attitudes. Two reviewers working independently selected studies and extracted data. Standardised mean difference (SMD) and 95% confidence intervals (95% CIs) were estimated from each study and pooled using random-effects meta-analysis.

RESULTS

The final analysis included 59 studies that enrolled 8011 learners. Twenty-five studies (42%) were randomised. The overall methodological quality of the studies was moderate. Compared with traditional teaching methods, SDL was associated with a moderate increase in the knowledge domain (SMD 0.45, 95% CI 0.23-0.67), a trivial and non-statistically significant increase in the skills domain (SMD 0.05, 95% CI-0.05 to 0.22), and a non-significant increase in the attitudes domain (SMD 0.39, 95% CI-0.03 to 0.81). Heterogeneity was significant in all analyses. When learners were involved in choosing learning resources, SDL was more effective. Advanced learners seemed to benefit more from SDL.

CONCLUSIONS

Moderate quality evidence suggests that SDL in health professions education is associated with moderate improvement in the knowledge domain compared with traditional teaching methods and may be as effective in the skills and attitudes domains.

Curricula for teaching the content of clinical practice guidelines to family medicine and internal medicine residents in the US: a survey study

BACKGROUND

Teaching the content of clinical practice guidelines (CPGs) is important to both clinical care and graduate medical education. The objective of this study was to determine the characteristics of curricula for teaching the content of CPGs in family medicine and internal medicine residency programs in the United States.

METHODS

We surveyed the directors of family medicine and internal medicine residency programs in the United States. The questionnaire included questions about the characteristics of the teaching of CPGs: goals and objectives, educational activities, evaluation, aspects of CPGs that the program teaches, the methods of making texts of CPGs available to residents, and the major barriers to teaching CPGs.

RESULTS

Of 434 programs responding (out of 839, 52%), 14% percent reported having written goals and objectives related to teaching CPGs. The most frequently taught aspect was the content of specific CPGs (76%). The top two educational strategies used were didactic sessions (76%) and journal clubs (64%). Auditing for adherence by residents was the primary evaluation strategy (44%), although 36% of program directors conducted no evaluation. Programs made texts of CPGs available to residents most commonly in the form of paper copies (54%) while the most important barrier was time constraints on faculty (56%).

CONCLUSION

Residency programs teach different aspects of CPGs to varying degrees, and the majority uses educational strategies not supported by research evidence.

Achieving the AAAs of Ambulatory Care: Aptitude, Appeal, and Appreciation

BACKGROUND

In the current health care environment more patient care has moved from in-hospital care to the ambulatory primary care settings; however, fewer internal medicine residents are pursuing primary care careers. Barriers to residents developing a sense of competency and enjoyment in ambulatory medicine include the complexity of practice-based systems, patients with multiple chronic diseases, and the limited time that residents spend in the outpatient setting.

OBJECTIVE

In an effort to accelerate residents' ambulatory care competence and enhance their satisfaction with ambulatory practice, we sought to change the learning environment. Interns were provided a series of intensive, focused, ambulatory training sessions prior to beginning their own continuity clinic sessions. The sessions were designed to enable them to work confidently and effectively in their continuity clinic from the beginning of the internship year, and it was hoped this would have a positive impact on their perception of the desirability of ambulatory practice.

METHODS

Improvement needs assessment after a performance, so we developed a structured, competency-based, multidisciplinary curriculum for initiation into ambulatory practice. The curriculum focused on systems-based practice, patient safety, quality improvement, and collaborative work while emphasizing the importance of continuity of care and long-term doctor-patient relationships. Direct observation of patient encounters was done by an attending physician to evaluate communication and physical examination skills. Systems of care commonly used in the clinic were demonstrated. Resources for practice-based learning were used.

CONCLUSION

The immersion of interns in an intensive, hands-on experience using a structured ambulatory care orientation curriculum early in training may prepare the intern to be a successful provider and learner in the primary care ambulatory setting.

Measurement of the general competencies of the accreditation council for graduate medical education: a systematic review

PURPOSE

To evaluate published evidence that the Accreditation Council for Graduate Medical Education's six general competencies can each be measured in a valid and reliable way.

METHOD

In March 2008, the authors conducted searches of Medline and ERIC using combinations of search terms "ACGME," "Accreditation Council for Graduate Medical Education," "core competencies," "general competencies," and the specific competencies "systems-based practice" (SBP) and "practice based learning and improvement (PBLI)." Included were all publications presenting new qualitative or quantitative data about specific assessment modalities related to the general competencies since 1999; opinion pieces, review articles, and reports of consensus conferences were excluded. The search yielded 127 articles, of which 56 met inclusion criteria. Articles were subdivided into four categories: (1) quantitative/psychometric evaluations, (2) preliminary studies, (3) studies of SBP and PBLI, and (4) surveys.

RESULTS

Quantitative/psychometric studies of evaluation tools failed to develop measures reflecting the six competencies in a reliable or valid way. Few preliminary studies led to published quantitative data regarding reliability or validity. Only two published surveys met quality criteria. Studies of SBP and PBLI generally operationalized these competencies as properties of systems, not of individual trainees.

CONCLUSIONS

The peer-reviewed literature provides no evidence that current measurement tools can assess the competencies independently of one another. Because further efforts are unlikely to be successful, the authors recommend using the competencies to guide and coordinate specific evaluation efforts, rather than attempting to develop instruments to measure the competencies directly.

Can residents accurately abstract their own charts?

PURPOSE

To assess the accuracy of residents' record review, using trained abstractors as a gold standard comparison.

METHOD

In 2005, the authors asked 74 residents to review their own charts (n = 392) after they received brief instruction on both how to locate data on the medical record and how to use a data abstraction form. Trained abstractors then re-reviewed these charts to assess performance of preventive health care measures in medicine (smoking screening, smoking cessation advice, mammography, colon cancer screening, lipid screening, and pneumonia vaccination) and pediatrics (parent smoking screening, parent smoking cessation advice, car seat safety, car restraint use, eye alignment, and immunizations up to date). The authors then quantified agreement between the two record reviews and assessed the sensitivity and specificity of the residents versus the trained abstractors.

RESULTS

Overall resident-measured performance was similar (within 5%) to that of the trained abstractor for five of six measures in medicine and four of six in pediatrics. For the various measures, sensitivity of resident-measured performance ranged from 100% to 15% and specificity from 100% to 33% compared with the trained abstractors. Relative to the trained abstractor record review, residents did not overestimate their performance. Most residents' (81%) relative performance rankings did not change when the basis for the ranking was resident measured versus trained abstractor measured.

CONCLUSIONS

Residents' self-abstraction can be an alternative to costly trained abstractors. Appropriate use of these data should be carefully considered, acknowledging the limitations.

Pay for performance and medical education: strategies for preparing physicians of the future

Performance-based reimbursement has become an increasingly important topic in the field of medicine and one that has met with significant legislative support. Small- and large-scale pilot programs in the United States and more comprehensive programs implemented abroad have yielded preliminary findings that raise several interesting questions regarding the form that pay-for-performance programs will take and concerns about the unintended and unforeseen consequences of this new reimbursement approach. One important area that has not been explored, however, is the potential implications of pay for performance to "clinician educators"-individuals from diverse health-related disciplines who both provide health care and are responsible for training the next generation of health care professionals. Because the effect of pay for performance is likely to vary by health care discipline, we focus here on potential implications for physician educators. Our objective is to analyze the experience to date with pay for performance, make predictions from these experiences about the potential impact of pay for performance on the education of resident physicians, and propose educational strategies that might be useful in positioning physician trainees for success as they enter the workforce.