Assessment of competences in rheumatology training: results of a systematic literature review to inform EULAR points to consider

Comparability of Objective Structured Clinical Examinations (OSCEs) and Written Tests for Assessing Medical School Students' Competencies: A Scoping Review

Objective Structured Clinical Examinations (OSCEs) and written tests are commonly used to assess health professional students, but it remains unclear whether the additional human resources and expenses required for OSCEs, both in-person and online, are worthwhile for assessing competencies. This scoping review summarized literature identified by searching MEDLINE and EMBASE comparing 1) OSCEs and written tests and 2) in-person and online OSCEs, for assessing health professional trainees' competencies. For Q1, 21 studies satisfied inclusion criteria. The most examined health profession was medical trainees (19, 90.5%), the comparison was most frequently OSCEs versus multiple-choice questions (MCQs) (18, 85.7%), and 18 (87.5%) examined the same competency domain. Most (77.5%) total score correlation coefficients between testing methods were weak (r < 0.40). For Q2, 13 articles were included. In-person and online OSCEs were most used for medical trainees (9, 69.2%), checklists were the most prevalent evaluation scheme (7, 63.6%), and 14/17 overall score comparisons were not statistically significantly different. Generally low correlations exist between MCQ and OSCE scores, providing insufficient evidence as to whether OSCEs provide sufficient value to be worth their additional cost. Online OSCEs may be a viable alternative to in-person OSCEs for certain competencies where technical challenges can be met.

Points to consider: EULAR-UEMS standards for the training of European rheumatologists

BACKGROUND

Postgraduate rheumatology training programmes are already established at a national level in most European countries. However, previous work has highlighted a substantial level of heterogeneity in the organisation and, in part, content of programmes.

OBJECTIVE

To define competences and standards of knowledge, skills and professional behaviours required for the training of rheumatologists.

METHODS

A European Alliance of Associations for Rheumatology (EULAR) task force (TF) of 23 experts, including two members of the European Union of Medical Specialists (UEMS) section of rheumatology, was convened. The mapping phase consisted of the retrieval of key documents on specialty training in rheumatology and other related specialties across a broad set of international sources. The content of these documents was extracted and represented the foundation for the document draft that underwent several rounds of online discussion within the TF, and afterwards was also distributed to a broad group of stakeholders for collecting feedback. The list of generated competences was voted on during the TF meetings, while the level of agreement (LoA) with each statement was established by anonymous online voting.

RESULTS

A total of 132 international training curricula were retrieved and extracted. In addition to the TF members, 253 stakeholders commented and voted on the competences through an online anonymous survey. The TF developed (1) an overarching framework indicating the areas that should be addressed during training, (2) 7 domains defining broad areas that rheumatology trainees should master by the end of the training programme, (3) 8 core themes defining the nuances of each domain and (4) 28 competences that trainees should acquire to cover each of the areas outlined in the overarching framework. A high LoA was achieved for all competences.

CONCLUSION

These points to consider for EULAR-UEMS standards for the training of European rheumatologists are now defined. Their dissemination and use can hopefully contribute to harmonising training across European countries.

Intra-articular therapies: patient preferences and professional practices in European countries

To assess patient perspective and professional practice of intraarticular therapies (IATs) across Europe, an expert international multidisciplinary panel designed two open web-based surveys: one targeting people who had experienced at least two IATs (44 items); and one targeting health care providers (HCPs) (160 items). Surveys were disseminated via patient and professional associations and social media. A descriptive analysis was performed. The surveys were answered by 200 patients and 186 HCPs from 26 countries, showing that IAT is routinely performed by rheumatologists (97%) and orthopaedic surgeons (89%), with specific training being compulsory in a few countries. The most frequent indications for IAT are arthritis (76%), osteoarthritis (74%), crystal arthritis (71%) and bursitis (70%); the most frequently injected joints are knee (78%) and shoulder (70%); and the most used compounds are glucocorticoids. The majority of HCPs report informing patients about side-effects (73%), benefits (72%), and the nature of the procedure (72%), which coincides with 27% of patients reporting that they had not been informed about benefits or potential complications of IATs; 73% of patients had not been asked whether they wanted an anaesthetic. Few HCPs (10%) obtain written consent (56% get oral consent, being mandatory for 32%), a procedure deemed necessary by 41% of the patients. 50% of patients reported a clear benefit of IAT and 20% experienced complications including pain, impaired mobility, rashes, or swelling. In summary, the practice of IAT is variable across Europe, and although patients perceive it as relatively safe and usually effective procedure, some gaps were identified.

Rheumatology education in India: a survey-based cross sectional study

This study aimed at understanding the perception and perspectives of rheumatology trainees about specialist training in India. Rheumatology trainees (Doctorate of Medicine, Diplomate of National Board) in Indian universities (2010 onwards) were contacted to complete a validated e-survey consisting of 41 questions to evaluate the current rheumatology training in India. Of 53 respondents (M:F 3.4:1, mean age 37 years ± 12.7), 81.1% trained at government hospitals, and 15.1% trained at private hospitals. During training period, 37.5% respondents were exposed to 6–7 h of didactics/week. They treated nearly 175 patients (175 ± 35.4) per week and reported a reasonable level of independence in management of patients with common rheumatic diseases (RDs) during their training (7.5 ± 0.7 SD). However, nearly one-third of the trainees were not exposed to basic immunology and laboratory techniques. Similarly, placement in the radiology department was not a part of the curriculum for nearly half of the trainees, 80% were not confident to manage paediatric RDs and soft tissue rheumatism. Almost 60% did not feel comfortable in addressing ancillary care including patient counselling as they had not received formal training. Among the participants, 59% were not satisfied by the current system of assessment, 86.8% suggested for multiple time point-based assessment systems and 45.3% preferred objective and subjective assessment in final examinations. Rheumatology training in India offers notable exposure to patients and independence in managing cases. However, there is an unmet need for improvement in training in the field of laboratory, radiology and ancillary care, and to overhaul assessment system by including objective evaluation.

EULAR portfolio for Rheumatology training: a EULAR School of Rheumatology initiative

OBJECTIVE

About half of the rheumatology trainees do not use a portfolio. This project was established to reach consensus about the content of a EULAR portfolio for Rheumatology training and subsequently develop portfolio assessment forms.

METHODS

After establishing a portfolio working group (WG), including nine rheumatologists and one educationalist, a systematic literature review (SLR) on the content and structure of portfolios for postgraduate learning was conducted (November 2018). This was followed by a survey among WG members and members of the EMerging EUlar NETwork, inquiring about the content and structure of existing national portfolios. The portfolio WG selected the key components of the portfolio, taking previous experience and feasibility into account. Assessment forms (eg, case-based discussion) were developed and pilot-tested.

RESULTS

13/2034 articles were included in the SLR (12 high/1 moderate risk of bias). Information on procedural skills, personal reflections, learning goals and multisource feedback was most often included a portfolio. Twenty-five respondents completed the survey (response≈50%). Feedback from assessors, reflective writing and formulation of learning goals were considered important dimensions to be covered in a portfolio. Six key components of the portfolio were established: curriculum vitae, personal development plan, clinical work, professional behaviours, education and research activities. Suggested minimal content for each component was formulated. Four assessment forms were successfully pilot-tested by 11 rheumatologists and their trainees.

CONCLUSION

A EULAR portfolio for Rheumatology training and assessment forms were developed. Portfolio implementation, particularly in countries without an existing portfolio, may promote a higher standard of rheumatology training across Europe.

Strategies for the assessment of competences during rheumatology training across Europe: results of a qualitative study

OBJECTIVES

To gain insight into current methods and practices for the assessment of competences during rheumatology training, and to explore the underlying priorities and rationales for competence assessment.

METHODS

We used a qualitative approach through online focus groups (FGs) of rheumatology trainers and trainees, separately. The study included five countries-Denmark, the Netherlands, Slovenia, Spain and the United Kingdom. A summary of current practices of assessment of competences was developed, modified and validated by the FGs based on an independent response to a questionnaire. A prioritising method (9 Diamond technique) was then used to identify and justify key assessment priorities.

RESULTS

Overall, 26 participants (12 trainers, 14 trainees) participated in nine online FGs (2 per country, Slovenia 1 joint), totalling 12 hours of online discussion. Strong nationally (the Netherlands, UK) or institutionally (Spain, Slovenia, Denmark) standardised approaches were described. Most groups identified providing frequent formative feedback to trainees for developmental purposes as the highest priority. Most discussions identified a need for improvement, particularly in developing streamlined approaches to portfolios that remain close to clinical practice, protecting time for quality observation and feedback, and adopting systematic approaches to incorporating teamwork and professionalism into assessment systems.

CONCLUSION

This paper presents a clearer picture of the current practice on the assessment of competences in rheumatology in five European countries and the underlying rationale of trainers' and trainees' priorities. This work will inform EULAR Points-to-Consider for the assessment of competences in rheumatology training across Europe.

Moving towards online rheumatology education in the era of COVID-19

The coronavirus disease-2019 (COVID-19) pandemic has unsettled conventional medical education, hastening a switch to digital platforms and open-access publishing. Rheumatology is a fast evolving academic discipline that stands to gain by this switch. Most rheumatology textbooks are now available in digital formats, and these are complemented with live updating educational hubs such as UpToDate and ClinicalKey. Emerging topics of COVID-19 on these proprietary platforms are now freely available to all specialists. Social media channels, particularly Twitter, are becoming major players in the era of COVID-19 by offering online journal clubs, enabling fast dissemination of influential articles, and facilitating interactive education. Indexed rheumatology journals, in turn, aid online education by opening access to recommendations and other materials that are rapidly changing research and practice worldwide. Research peer review additionally offers learning experience to novice and seasoned researchers and authors. Global rheumatology societies have online learning resources, which are changing their format and geographic reach to meet the changing needs in the times of pandemic. While online teaching lacks emotional connections between mentors and mentees, switch to a more interactive format of education and regular contacts may partly solve the issue. Rheumatologists can take the lead in these challenging times and contribute more to online scholarly activities which are aimed to maintain and enrich education. Key Points • Disparities in rheumatology education are likely to be widened during the COVID-19 pandemic. • Barriers to rheumatology education include limited number of instructors and their limited experience in online teaching. • Online textbooks, didactic materials of indexed rheumatology journals, and frequently updated online educational hubs such as UpToDate serve as a foundation of online rheumatology education. • Online rheumatology education is enriched by peer review and social media activities, which are becoming major players in the time of the COVID-19 pandemic.