On-Line Assessment (OLA) as a preparation for the European Diploma in Anaesthesiology and Intensive Care (EDAIC), a retrospective observational study on the results and the potential impact on the Part-I examination

BACKGROUND

The EDAIC is a diploma of the European Society of Anaesthesiology and Intensive Care (ESAIC). which is obtained after passing two a written MCQ-based (Part1) and a structured oral (part2) examinationIn 2011, a formative On-Line Assessment (OLA) was introduced to help candidates to prepare for EDAIC Part1 examination (EDAIC-I). This retrospective observational study evaluated the results of the OLA and its impact on the EDAIC-1 between 2013 and 2019.

METHODS

After obtaining the authorisation from the ESAIC Examinations Committee, all the results of candidates registered to OLA and/or EDAIC-I between 2013 and 2019 were included. The total number of registrations and the results were analysed and compared for both.

RESULTS

Over 17,000 candidates (17,401) sat any of the written exams of the EDAIC. The overall pass-rate for the EDAIC-1 was 68.95%. The OLA score increased significantly with the number of attempts for Paper A (Basic Science) (p=0.006). Overall success of the EDAIC-I was higher in candidates who took the OLA before (72.9% versus 68.3%; OR: 1.25; 95% CI [1.12; 1.39]; p<0.001). Candidates who failed in their first attempt for EDAIC-I were more likely to sit the exam again if they had performed the OLA before (OR: 1.396, 95% CI [1.237; 1.574]; p<0.001).

CONCLUSION

The OLA was associated with an improvement of the results in basic science and success rate in the EDAIC-I.

Acceptance and validity of the methods used to implement a competency based medical education programme in an Intensive Care Department of a teaching referral center

OBJECTIVES

1. To determine the satisfaction of tutors and residents with a specific methodology used to implement CoBaTrICE. 2. To determine the reliability and validity of the global rating scales designed ad hoc to assess the performance of the residents for training purposes.

DESIGN

Prospective cohort study.

PARTICIPANTS

All the residents and tutors of the ICU Department of the Hospital Universitario y Politécnico la Fe de Valencia.

INTERVENTION

CoBaTrICE implementation started in March 2016, it was based on: (1) Training the tutors in feedback techniques; (2) Performing multiple objective and structured work based assessments to achieve the competences of the program; and (3) The use of an electronic portfolio to promote learning reflection and to collect the evidence that learning was taking place.

METHODS

The acceptance of CoBaTrICE was explored through a satisfaction survey conducted after 9 months of implementation of the training program. The 15 residents and 5 tutors of the ICU Department were asked about the methodology of the formative assessments, the quality of the feedback, self-learning regulation and the electronic portfolio usefulness. The validity of the global rating scales was assessed through the tests alfa de Cronbach, reliability and generalizability indexes, and intraclass correlation coefficient.

RESULTS

The implementation of CoBaTrICE was satisfactory in all the dimensions studied. The global rating scales used for formative purposes showed reliability and validity.

CONCLUSIONS

The methodology used to implement CoBaTrICE was highly valued by tutors and residents. The global rating scales used for formative purposes showed reliability and validity.