Simulation-based summative assessment in healthcare: an overview of key principles for practice

BACKGROUND

Healthcare curricula need summative assessments relevant to and representative of clinical situations to best select and train learners. Simulation provides multiple benefits with a growing literature base proving its utility for training in a formative context. Advancing to the next step, "the use of simulation for summative assessment" requires rigorous and evidence-based development because any summative assessment is high stakes for participants, trainers, and programs. The first step of this process is to identify the baseline from which we can start.

METHODS

First, using a modified nominal group technique, a task force of 34 panelists defined topics to clarify the why, how, what, when, and who for using simulation-based summative assessment (SBSA). Second, each topic was explored by a group of panelists based on state-of-the-art literature reviews technique with a snowball method to identify further references. Our goal was to identify current knowledge and potential recommendations for future directions. Results were cross-checked among groups and reviewed by an independent expert committee.

RESULTS

Seven topics were selected by the task force: "What can be assessed in simulation?", "Assessment tools for SBSA", "Consequences of undergoing the SBSA process", "Scenarios for SBSA", "Debriefing, video, and research for SBSA", "Trainers for SBSA", and "Implementation of SBSA in healthcare". Together, these seven explorations provide an overview of what is known and can be done with relative certainty, and what is unknown and probably needs further investigation. Based on this work, we highlighted the trustworthiness of different summative assessment-related conclusions, the remaining important problems and questions, and their consequences for participants and institutions of how SBSA is conducted.

CONCLUSION

Our results identified among the seven topics one area with robust evidence in the literature ("What can be assessed in simulation?"), three areas with evidence that require guidance by expert opinion ("Assessment tools for SBSA", "Scenarios for SBSA", "Implementation of SBSA in healthcare"), and three areas with weak or emerging evidence ("Consequences of undergoing the SBSA process", "Debriefing for SBSA", "Trainers for SBSA"). Using SBSA holds much promise, with increasing demand for this application. Due to the important stakes involved, it must be rigorously conducted and supervised. Guidelines for good practice should be formalized to help with conduct and implementation. We believe this baseline can direct future investigation and the development of guidelines.

[Reforms of medical / health studies: in the middle of the ford]

IN THE MIDDLE OF THE FORD Medical Education Reform: The main aim is learning of skills, attitudes and behavior of students and graduates which was very often neglected until then. Two problems remains ; first, what are the disponibility and motivation of medical faculties and teachers and second how the students will accept the new form of evaluation to access at post graduate level (specialization cycle).

A novel personalized approach to cardiovascular prevention: The VIVOPTIM programme

BACKGROUND

Cardiovascular diseases are a leading cause of mortality, but a substantial proportion are preventable.

AIMS

The Mutuelle générale de l'éducation nationale (MGEN), a provider of private health insurance in France, has developed the VIVOPTIM programme, a novel digital approach to healthcare based on individualized, multiprofessional, ranked management of cardiovascular risk factors.

METHODS

Between November 2015 and June 2016, eligible individuals (age 30-70 years) from two regions of France were invited to participate. Volunteers completed a questionnaire based on the Framingham Heart Study Risk Score and were assigned to one of three cardiovascular risk levels. VIVOPTIM comprises four components: cardiovascular risk assessment, instruction on cardiovascular diseases and associated risk factors, personalized coaching (telephone sessions with a specially trained healthcare professional to provide information on risk factors and disease management, set individual health targets, monitor progress and motivate participants), and e-Health monitoring.

RESULTS

Data from 2240 participants were analysed. Significant benefits were observed on mean systolic blood pressure (-3.4mmHg), weight (-1.5kg), smoking (-2.2 cigarettes/day) and daily steps (+1726 steps/day (all P<0.0001)), though not on weekly duration of exercise (-0.2hours/week, P=0.619).

CONCLUSION

As a result of the positive mid-to-long-term results of the pilot programme on weight, smoking, blood pressure, and uptake of physical activity, the VIVOPTIM programme was extend to the whole of France in 2018 and has the potential to have a genuine impact on patient care and organization of the healthcare system in France.

[VIVOPTIM: Feedback of an e-Health experimental program of primary prevention of cardiovascular risk on 30 to 70 years old volunteers]

Today by the e-health and the telemedicine, many people are more and more interested by the improvement of disease knowledge on cardiovascular diseases and associated risk factors, personalized self management support follow-up and e-Health monitoring. MGEN is a not-for-profit complementary health insurance gave itself the ways to use the new digital tools in health. MGEN developed an original and personalized program VIVOPTIM for the primary prevention of the cardiovascular risks for their members. The VIVOPTIM Pilot program is based upon digital services and was experimented by November 2015 to December, 2017 with 8000 members of the MGEN, from 30 to 70 years old and resident in two French areas (Occitanie and Bourgogne Franche-Comté). The assessment of the experiment VIVOPTIM e -health program was positive for the personalized cardiovascular support and for their health. Therefore, the MGEN generalized the VIVOPTIM program of cardiovascular prevention, to the whole France on July 11th, 2018.

[Access to graduate medical education and other reforms]

Access to graduate medical education and other reforms. In France, the access to Graduate Medical Education is the key component of medical education. The actual process is not usefull ; specially for the quality of the practical/ theorical education (pregraduate students and residents too). The actual process based on clinical-problem solving with MCQ is not so valid quite insecure and too costly. It is time to redifining the process. Looking to the north america and the « National resident matching Program » could be the solution.

Formative assessment based on an audit and feedback improves nuchal translucency ultrasound image quality

OBJECTIVES

The purpose of this work was to study the impact of an audit and feedback on the quality of routine first-trimester nuchal transparency ultrasound images.

METHODS

Eighty-eight sonographers were each sent 2 different series of 30 consecutive nuchal translucency images at a mean interval of 3 months to a dedicated, protected server for remote double-blind independent analysis based on the new Collège Français d'Echographie Foetale/Centre National de la Recherche Scientifique image-scoring method (https://www.cfef.org/evaluation/ISMCFEFCNRS.pdf). The sonographers were classified as low (score below the median) or high (score above the median) scorers for each series. Before their second evaluation, 73 of the 88 sonographers received a feedback report on their first series of images, whereas the other 15 participants received no feedback. The baseline characteristics of the participants who did and did not receive feedback were comparable.

RESULTS

Participants who received feedback increased their average score significantly, from a mean ± SD of 11.1 ± 1.3 to 13.4 ± 1.4 among low scorers (P < .00001) and from 15.1 ± 1.2 to 16.0 ± 1.4 among high scorers (P < .001), whereas no significant change was seen among participants who received no feedback (low scorers, 10.9 ± 1.5 to 12.1 ± 2.0; P = .11; high scorers, 14.7 ± 1.3 to 14.6 ± 1.3; P = .99). The proportion of satisfactory images increased by 48% among low scorers who received feedback.

CONCLUSIONS

Formative assessment based on a moderately intensive audit and feedback is feasible and effective for improving the quality of routine first-trimester nuchal transparency ultrasound images.