The pediatric rheumatology objective structured clinical examination: progressing from a homegrown effort toward a reliable and valid national formative assessment

Assessing Pediatric Rheumatology Fellow Competence in the Milestone Era: Past, Present, and Future

Starting in 2015, pediatric rheumatology fellowship training programs were required by the Accreditation Council for Graduate Medical Education to assess fellows' academic performance within 21 subcompetencies falling under six competency domains. Each subcompetency had four or five milestone levels describing developmental progression of knowledge and skill acquisition. Milestones were standardized across all pediatric subspecialties. As part of the Milestones 2.0 revision project, the Accreditation Council for Graduate Medical Education convened a workgroup in 2022 to write pediatric rheumatology-specific milestones. Using adult rheumatology's Milestones 2.0 as a starting point, the workgroup revised the patient care and medical knowledge subcompetencies and milestones to reflect requirements and nuances of pediatric rheumatology care. Milestones within four remaining competency domains (professionalism, interpersonal and communication skills, practice-based learning and improvement, and systems-based practice) were standardized across all pediatric subspecialties, and therefore not revised. The workgroup created a supplemental guide with explanations of the intent of each subcompetency, 25 in total, and examples for each milestone level. The new milestones are an important step forward for competency-based medical education in pediatric rheumatology. However, challenges remain. Milestone level assignment is meant to be informed by results of multiple assessment methods. The lack of pediatric rheumatology-specific assessment tools typically result in clinical competency committees determining trainee milestone levels without such collated results as the foundation of their assessments. Although further advances in pediatric rheumatology fellowship competency-based medical education are needed, Milestones 2.0 importantly establishes the first pediatric-specific rheumatology Milestones to assess fellow performance during training and help measure readiness for independent practice.

Assessment of Resident Knowledge in Pediatric Rheumatology

Pediatric Rheumatic illnesses are complex, chronic, and often multi-systemic. Our goal was to assess the efficacy of 2 standardized pediatric rheumatology lectures, administered to pediatric residents, in improving the pediatric residents’ knowledge and confidence in pediatric rheumatology. Two lectures, 1 hour each, were delivered by 2 residents and given 1 week apart to 28 pediatric residents. Pre- and post-tests assessed knowledge and residents’ self-rated competence. Change in knowledge was assessed using paired t-tests. Twenty-eight residents participated in the study. Sixty-three percent (17/27) had an improvement in score from pre-test to post-test. Thirty-seven percent (10/27) reported increased competence from pre-test to post-test. Ninety-six percent (22/23) of the residents found the lectures to be beneficial. Residents in the third post-graduate year (PGY3) cohort had the most significant improvement in scores. The lectures given to the pediatric residents increased pediatric knowledge and the pediatric residents found these lectures to be beneficial.

Multi-pronged approach to enhance education of children and adolescents with lupus, caregivers, and healthcare providers in New Jersey: Needs assessment, evaluation, and development of educational materials

BACKGROUND

Childhood Systemic Lupus Erythematosus (cSLE) patients are younger at diagnosis and have a more severe disease course compared to adult onset SLE patients and develop significant complications related to disease and or immunosuppression. Moreover, female and minority populations experience higher rates of cSLE, with African American, Afro-Caribbean, and Hispanic populations being at greatest risk and having poor prognosis.

METHODS

The Pediatric Alliance for Lupus initiative addressed the dearth in education and resources in a multi-stage process. First, we conducted a need assessment identifying knowledge gaps among healthcare providers (HCPs), and resources needed to care for cSLE patients and their families. Second, we educated HCPs about the diagnosis and treatment of cSLE by Continuing Medical Education (CME) sessions/webinars (presented here). Third, HCPs participated in a Quality Improvement (QI) program on cSLE approved by the American Board of Pediatrics Maintenance of Certification Part 4. Finally, patients and caregivers were educated through the development of appropriate, culturally and linguistically sensitive cSLE resources. PAL disseminated materials among HCPs and the community to improve the awareness of the availability of these materials.

RESULTS

According to results from the statewide needs assessment (representative of every county throughout NJ), HCPs face significant challenges in providing care to cSLE patients and their families, in part due to the multi-systemic nature of the autoimmune disease.

CONCLUSION

Based on this need, we developed educational sessions, with pre-post comparison data showing a significant increase in knowledge after HCP education. The 15 different materials developed as part of the endeavor is a major contribution to the cSLE community, HCPs and pediatric rheumatologists. Resources are available in multiple formats (PDF and web pages), and are accessible on the National Resource Center on Lupus, the latest web site of the Lupus Foundation of American that houses materials for SLE patients, their families, schools, HCPs, and the community at large.Improving cSLE knowledge will empower the children and adolescents and families by increasing their self-efficacy; and positively impact key health outcomes (transition readiness and HRQOL) that are not optimally addressed with current medical treatment alone.

Turning Objective Structured Clinical Examinations into Reality

Objective structured clinical examinations assess learners "showing how" to perform complex clinical tasks. Devised as summative evaluations, these examinations with immediate feedback are useful formative evaluations to improve learner performance. This review describes how objective structured clinical examinations have been used in rheumatology education. Steps for creating an objective structured clinical examination are discussed. Validity and reproducibility are important considerations, especially for high-stakes summative objective structured clinical examinations. Consideration of the potential benefits in clinical education and their hazards are reviewed. When well-designed, formative objective structured clinical examinations have high educational value for learners and medical educators.