Development and Validation of the ACP-CAT for Assessing the Quality of Advance Care Planning Communication

A Novel Serious Illness Communication Curriculum Improves Neurology Residents' Confidence and Skills

Background and Purpose: Competency in serious illness communication is mandated by the Accreditation Council of Graduate Medical Education. Previous efforts to teach communication skills have been hampered by intensive time requirements. In this study, we developed and evaluated a brief goals-of-care communication curriculum for neurology residents. Methods: We developed and implemented a two-part curriculum based on themes identified from a needs assessment: (1) fundamental physician-patient communication skills; and (2) counseling surrogate decision makers and providing neuroprognostication. We used a three-pronged pre-post study design to evaluate the impact of the curriculum: resident self-assessment surveys, direct observations of resident-patient interactions, and patient perception surveys using the Communication Assessment Tool. Results: Residents reported a significant increase in mean scores [standard deviation] of confidence practicing fundamental communication skills, such as offering opportunities for emotion (3.84 [0.9] vs. 4.54 [0.6], p = 0.002), and goals-of-care communication skills, such as using triggers for serious conversations (2.65 [0.7] vs. 3.29 [0.5], p = 0.004). Observed resident-patient interactions showed significant improvement in fundamental communication skills, such as involving the patient in decision making (1.89 [0.6] vs. 4.0 [0.9], p < 0.001). There was no significant impact on patient perception of resident communication skills in the three months following the intervention. Conclusions: A brief, learner-centered curricular intervention improved neurology residents' confidence in serious illness communication and improved their skills as judged by trained observers.

Advance care planning clinic: A structured clinical experience for internal medicine residents

BACKGROUND

Advance care planning (ACP) is an important step to provide medical care consistent with patients' preferences and values. Nationally, rates of ACP completion are low, and internal medicine residency clinics face additional barriers. To address this need, we implemented an ACP clinic for internal medicine residents.

METHODS

An ACP clinical experience was created for PGY2 residents beginning in 2018, with 6 total sessions, consisting of consolidated didactics, protected time to identify, outreach, and schedule patients, and two half days of dedicated ACP visits. Residents were surveyed before (end of PGY1) and after (end of PGY2) the intervention. The preceding residency class, serving as a historic control, only received the curriculum and were surveyed at the end of their PGY2 year. Electronic medical record (EMR) data was accessed to track ACP documentation.

RESULTS

The overall survey response rate was 124/134 (93%). Comparing the intervention cohort before and after the intervention, there was a significant increase in self-assessed confidence in completing ACP (2.1/4.0 vs 3.5/4.0, p < 0.01). Comparing the intervention and historic cohorts (end of PGY2), the intervention was associated with improved confidence in ability to complete ACP for their patients (3.5/4.0 vs 2.7/4.0, p < 0.01). The historic control had no increase in ACP documentation rates over time, while the intervention cohort had a 13.9% absolute increase in ACP documentation for their patients over the course of residency (p < 0.01).

CONCLUSION

The creation of an ACP-specific clinical experience, in conjunction with existing curricula, resulted in significant improvements in knowledge, self-assessed skills and behavior, and EMR documentation.

How to Estimate Absolute-Error Components in Structural Equation Models of Generalizability Theory

Structural equation modeling (SEM) has been proposed to estimate generalizability theory (GT) variance components, primarily focusing on estimating relative error to calculate generalizability coefficients. Proposals for estimating absolute-error components have given the impression that a separate SEM must be fitted to a transposed data matrix. This paper uses real and simulated data to demonstrate how a single SEM can be specified to estimate absolute error (and thus dependability) by placing appropriate constraints on the mean structure, as well as thresholds (when used for ordinal measures). Using the R packages lavaan and gtheory, different estimators are compared for normal and discrete measurements. Limitations of SEM for GT are demonstrated using multirater data from a planned missing-data design, and an important remaining area for future development is discussed.

Continuing Education Outcomes for Advance Care Planning: A Systematic Review of the Literature

INTRODUCTION

Advance care planning (ACP) is a process of considering future health and care needs for a time when a person may be unable to speak for themselves. Health professional continuing education programs have been proposed for facilitating patient participation in ACP; however, their impacts on participants, patient and clinical outcomes, and organizational approaches to ACP are not well understood.

METHODS

This systematic literature review examined interventional studies of education programs conducted with health professionals and care staff across a broad range of settings. Five electronic databases were searched up to June 2020, and a manual search of reference lists was conducted. The quality of studies was appraised by the first, second, and third authors.

RESULTS

Of the 7993 articles identified, 45 articles met the inclusion criteria. Program participants were predominantly medical, nursing, and social work staff, and students. Interventions were reported to improve participants' self-perceived confidence, knowledge, and skills; however, objectively measured improvements were limited. Multimodal programs that combined initial didactic teaching and role-play simulation tasks with additional activities were most effective in producing increased ACP activity in medical records. Evidence for improved clinical outcomes was limited.

DISCUSSION

Further studies that use rigorous methodological approaches would provide further evidence about what produces improved patient and clinical outcomes. Needs analyses and quality indicators could be considered to determine the most appropriate and effective education resources and monitor their impacts. The potential contribution of a broader range of health professionals and interprofessional learning approaches could be considered to ultimately improve patient care.

Serious Illness Conversation-Evaluation Exercise: A Novel Assessment Tool for Residents Leading Serious Illness Conversations

Background/Objectives: The serious illness conversation (SIC) is an evidence-based framework for conversations with patients about a serious illness diagnosis. The objective of our study was to develop and validate a novel tool, the SIC-evaluation exercise (SIC-Ex), to facilitate assessment of resident-led conversations with oncology patients.

Design: We developed the SIC-Ex based on SIC and on the Royal College of Canada Medical Oncology milestones. Seven resident trainees and 10 evaluators were recruited. Each trainee conducted an SIC with a patient, which was videotaped. The evaluators watched the videos and evaluated each trainee by using the novel SIC-Ex and the reference Calgary-Cambridge guide (CCG) at months zero and three. We used Kane's validity framework to assess validity.

Results: Intra-class correlation using average SIC-Ex scores showed a moderate level of inter-evaluator agreement (range 0.523–0.822). Most evaluators rated a particular resident similar to the group average, except for one to two evaluator outliers in each domain. Test–retest reliability showed a moderate level of consistency among SIC-Ex scores at months zero and three. Global rating at zero and three months showed fair to good/very good inter-evaluator correlation. Pearson correlation coefficients comparing total SIC-Ex and CCG scores were high for most evaluators. Self-scores by trainees did not correlate well with scores by evaluators.

Conclusions: SIC-Ex is the first assessment tool that provides evidence for incorporating the SIG guide framework for evaluation of resident competence. SIC-Ex is conceptually related to, but more specific than, CCG in evaluating serious illness conversation skills.