The dialysis orders objective structured clinical examination (OSCE): a formative assessment for nephrology fellows

The peritoneal dialysis orders objective structured clinical examination (OSCE): A formative assessment for nephrology fellows

Background:

Peritoneal dialysis (PD) management is a fundamental nephrology skill, especially with the recent emphasis on home dialysis. We report a prospective multicentre cohort study of a formative objective structured clinical examination (OSCE) assessing competence in managing PD-associated bacterial peritonitis, using the unified model of construct validity.

Methods:

The OSCE was developed by the principal investigators and reviewed by two subject matter experts. The test committee (eight nephrologists and one PD nurse) assessed test item difficulty/relevance and determined passing score. There were 22 test items (7 evidence-based/standard-of-care questions). Passing score was 16/22 (73%). No item had median relevance less than ‘important’, and all were easy to medium difficulty. Content validity index was 0.91. Preliminary validation (16 board-certified volunteers): mean score was 19 ± 2, with 94% (15/16) passing. Kappa = 0.85 [95% confidence interval (CI) 0.77–0.94]. Cronbach’s α = 0.70.

Results:

Eighty-seven fellows (16 programmes) were tested; 67% passed. Fellows scored significantly less than validators: 17 ± 3 versus 19 ± 2, p < 0.001 [95% CI 1.2–3.6]. Eighty-six per cent of evidence-based/standard-of-care questions were answered correctly by validators versus 54% by fellows; p < 0.001. Ninety-three per cent of fellows recognized that sufficient criteria were present to diagnose peritonitis, but only 17% correctly indicated all three. Seventy-seven per cent recognized peritonitis-associated ultrafiltration failure, but only 17% prescribed 21 days of antibiotic treatment for gram-negative peritonitis. Eighty-five per cent of fellows surveyed agreed/strongly agreed that the OSCE was useful in self-assessing proficiency. Second-year in-training examination and OSCE scores were positively correlated (Pearson’s r = 0.57, p < 0.00).

Conclusions:

The OSCE may be used to formatively assess fellow proficiency in managing PD-associated peritonitis.

Great nephrologists begin with great teachers: update on the nephrology curriculum

PURPOSE OF REVIEW

The purpose of this review is to highlight developments and opportunities in the nephrology curriculum from the basic science foundation years through teaching medical students, residents and fellows in the clinical realm.

RECENT FINDINGS

Teaching skills are a vital tool for nephrologists both to promote excellent patient care and attract talented learners to the field. Exposure to dynamic and inspiring nephrologists is one of the main factors given by students and residents for selecting a career in nephrology. Nephrology teaching, including case discussions, problem-based learning, team-based learning and flipped classrooms, provides motivating active learning for medical students and is equally effective for didactics in graduate medical education. Avenues for teaching in the clinical realm include the microskills framework, bedside teaching and grounding in evidence-based medicine. Areas of growth include blended nephrology/subspecialty fields as well as social media applications.

SUMMARY

Medical education is a satisfying and exciting area of growth in the field of nephrology. The recent literature provides a framework for best practices in active learning as well as providing numerous examples of educational interventions and innovations. In addition, this field is ripe for further development and scholarly activity.

The Acute Dialysis Orders Objective Structured Clinical Examination (OSCE): Fellow Performance on a Formative Assessment of Acute Kidney Replacement Therapy Competence

BACKGROUND AND OBJECTIVES

Acute kidney replacement therapy (KRT) prescription is a critical nephrology skill. We administered a formative objective structured clinical examination (OSCE) to nephrology fellows to assess acute KRT medical knowledge, patient care, and systems-based practice competencies.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

Prospective cohort study of an educational test using the unified model of construct validity. We tested 117 fellows: 25 (four programs) in 2016 and 92 (15 programs) in 2017; 51 first-year and 66 second-year fellows. Using institutional protocols and order sets, fellows wrote orders and answered open-ended questions on a three-scenario OSCE, previously validated by board-certified, practicing clinical nephrologists. Outcomes were overall and scenario pass percentage and score; percent correctly answering predetermined, evidence-based questions; second-year score correlation with in-training examination score; and satisfaction survey.

RESULTS

A total of 76% passed scenario 1 (acute continuous RRT): 92% prescribed a ≥20 ml/kg per hour effluent dose; 63% estimated clearance as effluent volume. Forty-two percent passed scenario 2 (maintenance dialysis initiation); 75% correctly prescribed 3-4 mEq/L K+ dialysate and 12% identified the two absolute, urgent indications for maintenance dialysis initiation (uremic encephalopathy and pericarditis). Six percent passed scenario 3 (acute life-threatening hyperkalemia); 20% checked for rebound hyperkalemia with two separate blood draws. Eighty-three percent correctly withheld intravenous sodium bicarbonate for acute hyperkalemia in a nonacidotic, volume-overloaded patient on maintenance dialysis, and 32% passed overall. Second-year versus first-year fellow overall score was 44.4±4 versus 42.7±5 (one-tailed P=0.02), with 39% versus 24% passing (P=0.08). Second-year in-training examination and OSCE scores were not significantly correlated (r=0.15; P=0.26). Seventy-seven percent of fellows agreed the OSCE was useful in assessing "proficiency in ordering" acute KRT. Limitations include lack of a validated criterion test, and unfamiliarity with open-ended question format.

CONCLUSIONS

The OSCE can provide quantitative data for formative Accreditation Council for Graduate Medical Education competency assessments and identify opportunities for dialysis curriculum development.

PODCAST

This article contains a podcast at https://www.asn-online.org/media/podcast/CJASN/2019_08_08_CJASNPodcast_19_09_.mp3.