Priorities in surgical simulation research: What do the experts say?

Evaluating user experience with immersive technology in simulation-based education: A modified Delphi study with qualitative analysis

BACKGROUND

Immersive technology is becoming more widespread in simulation-based medical education with applications that both supplement and replace traditional teaching methods. There is a lack of validated measures that capture user experience to inform of the technology utility. We aimed to establish a consensus of items and domains that different simulation experts would include in a measure for immersive technology use.

METHODS

A 3-stage modified Delphi using online software was conducted to support the conceptual framework for the proposed measure. The first round was informed by prior work on immersive technology in simulation. In the first round, participants were asked to describe what we could measure in simulation-based education and technology. Thematic analysis generated key themes that were presented to the participants in the second round. Ranking of importance in round 2 was determined by mean rank scores. The final round was an online meeting for final consensus discussion and most important domains by experts were considered.

RESULTS

A total of 16 simulation experts participated in the study. A consensus was reached on the ideal measure in immersive technology simulation that would be a user questionnaire and domains of interest would be: what was learnt, the degree of immersion experienced, fidelity provided, debrief, psychological safety and patient safety. No consensus was reached with the barriers that this technology introduces in education.

CONCLUSIONS

There is varied opinion on what we should prioritise in measuring the experience in simulation practice. Importantly, this study identified key areas that aids our understanding on how we can measure new technology in educational settings. Synthesising these results in to a multidomain instrument requires a systematic approach to testing in future research.

Consensus priority research questions in gastrointestinal and endoscopic surgery in the year 2020: results of a SAGES Delphi study

INTRODUCTION

The objectives of this study were to identify consensus priority research questions according to members of the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES), and to explore differences in priorities according to specific membership subgroups.

METHODS

A modified Delphi study was conducted including active members of SAGES. An initial list of research questions was compiled by members of 26 SAGES Committees and Task Forces, and was further refined by the SAGES Delphi Task Force. The questions were divided into five research categories: (1) Surgical Outcomes; (2) Education, Training, and Simulation; (3) Health Services Research; (4) New Technology; and (5) Artificial Intelligence. Delphi respondents were asked to rank each question with regards to its importance in the field of gastrointestinal and endoscopic surgery (1-low; 5-high). "Priority" was defined as a single-round mean score of ≥ 3.5, and "consensus" as a single-round standard deviation < 1.0. Subgroup analyses were performed according to a priori selected respondent characteristics.

RESULTS

The total number of respondents for each round was: Round 1 (n = 407); Round 2 (n = 569); Round 3 (n = 273). In each round, the majority of respondents were male (Round 1: 77.4%; Round 2: 77.1%; Round 3: 76.7%), self-identified as academic (vs. community) surgeons (Round 1: 57.1%; Round 2: 61.1%; Round 3: 60.2%), and practiced in North America (Round 1: 71.8%; Round 2: 70.8%; Round 3: 75.9%). A total of 29 out of 122 research questions met criteria for both "priority" and "consensus"-Surgical Outcomes, n = 6; Education, Training, and Simulation, n = 9; Health Services Research, n = 5; New Technology, n = 5; and Artificial Intelligence, n = 4.

CONCLUSIONS

Consensus priority research questions in gastrointestinal and endoscopic surgery were identified across five different research categories. These results can provide direction and areas of interest for funding and investigation for future studies.