The co.LAB Generic Framework for Collaborative Design of Serious Games: Development Study

The validation of a serious game for teaching ultrasound skills

Background

Point-of-care ultrasound (POCUS) is an important bedside diagnostic tool and is being taught in several specialties. However, mastering the required psychomotor skills takes time and learning curves are different between students. Especially learning to make the right probe movements with the corresponding changes of the ultrasound image on screen, and integrating it into a 3D mental model takes time. This precious bedside-time of trainers and physicians may be reduced using other learning methods for mastering the psychomotor skills, for example the use of serious games. Such a game is under development but it needs to be validated before widespread use can be advised. In this article we describe the development and the first three steps in the validation of a serious game for ultrasound skills.

Results

We have included 18 ultrasound experts and 24 ultrasound novices who played the serious game ‘Underwater” and provided feedback. They concluded that “underwater” is fun to play and that movement of the 3D-printed probe resembled real ultrasound probe movements. Participants highly valued the potential of the game for training eye–hand coordination and stability of probe handling, two very important skills in performing ultrasound in real practice. Although we compared several in-game parameters such as distance and speed, no difference was observed between novices and experts. This means that content- and face validity of the serious game is demonstrated but optimal parameters to measure differences between novices and experts still have to be determined.

Conclusions

Our study shows solid content- and face validity of the serious game “UnderWater” for training ultrasound skills, although construct validity could not be demonstrated yet. The game is appreciated as a promising serious game for training eye–hand coordination and learning ultrasound, which may reduce expensive bed-side teaching.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13089-022-00280-8.

Serious game design model for language learning in the cultural context

Digital learning environments have been gaining prominence during the last few years. In particular, the rising usage of mobile devices, including smartphones and tabs, has invited researchers to design and develop learning applications and games for such platforms. Mobile applications and games have been developed for learning languages like many other domains. However, most of these games are fun-based and lack a holistic design and development approach. Therefore, as a principal contribution, this research presents a theoretical model for designing language learning games in a cultural context. The proposed model combines the elements of sociocultural theory with the concepts and elements of gamification, keeping in view the requirements and educational settings, including level and mode of education, etc., to ensure the effectiveness and usability of the developed game. Subsequently, based on the proposed model, a Language Learning Game (LLG) has been designed and developed through a systematic process that involves game design, low-fidelity, and high-fidelity prototyping and its validation. The LLG has been evaluated comprehensively at different stages by incorporating standard methods. Whereby this research augments the existing set of heuristics by proposing a number of specialized heuristics for the evaluation of serious games to gauge their conformance to the cultural context. The evaluation results show that the game has overall usability scores of 90%. While the quasi-experiment-based pre-test and post-test have been conducted, the results reveal that the results obtained by LLG are statistically significantly better than adopted mobile application and traditional group.

Effects of a serious game on knowledge, attitude and practice in vector control and dengue prevention among adults in primary care: A randomised controlled trial

Objective

Dengue is endemic but vaccination against it is optional in tropical Singapore. Despite vector control measures to curb mosquito breeding, dengue infection continues to be prevalent. A serious game has been developed to raise the community awareness of dengue vector control programme among residents in Singapore. The study aimed to evaluate the effectiveness of this serious game on the knowledge, attitude and practice (KAP) in dengue prevention among adults. It also determined their willingness to be vaccinated against dengue.

Methods

A randomised controlled trial was conducted among volunteer adults who were visiting a regional primary care clinic in Sengkang, Singapore. 400 participants were randomly allocated to receive information regarding dengue prevention from either playing a serious game (intervention) or visiting a dengue prevention website (control). Before and after receiving information on dengue prevention, participants completed a self-administered online questionnaire within a two-week interval to assess the KAP score and their interest to vaccinate against dengue. Participants, who played serious game, evaluated the game with the System Usability Scale (SUS).

Results

A total of 374 participants, comprising of 178 participants (89%) from intervention group and 196 participants (98%) from control group, completed both questionnaires. 157 (78.5%) participants in the intervention group completed playing the serious game. Participants in both groups had increased KAP score from baseline, but the mean difference in score (SD) was greater when assessing participants’ daily practice towards dengue prevention in the serious game group compared to the control group (1.0 (2.8) vs 0.3 (1.9), p = .009). There was a positive correlation (rho = 0.275) between mean change in KAP score and highest achievement attained within serious game ( p = .001). The mean SUS (SD) was 61.8 (19.2) among participants who played the serious game. 84.2% of the study population was willing to undertake the dengue vaccination at baseline. Participants in both groups had increased proportion of participants who showed interest in dengue vaccination from baseline but the difference between serious game group and control group were not significant (5.6% vs 2.6%, p = .131).

Conclusions

Serious game is at least equally effective compared to conventional web-based learning in promoting dengue prevention measures and vaccination intention among adults, and may be considered as a feasible alternative to digitally engage local residents.

Nationwide Deployment of a Serious Game Designed to Improve COVID-19 Infection Prevention Practices in Switzerland: Prospective Web-Based Study

Background

Lassitude and a rather high degree of mistrust toward the authorities can make regular or overly constraining COVID-19 infection prevention and control campaigns inefficient and even counterproductive. Serious games provide an original, engaging, and potentially effective way of disseminating COVID-19 infection prevention and control guidelines. Escape COVID-19 is a serious game for teaching COVID-19 infection prevention and control practices that has previously been validated in a population of nursing home personnel.

Objective

We aimed to identify factors learned from playing the serious game Escape COVID-19 that facilitate or impede intentions of changing infection prevention and control behavior in a large and heterogeneous Swiss population.

Methods

This fully automated, prospective web-based study, compliant with the Checklist for Reporting Results of Internet E-Surveys (CHERRIES), was conducted in all 3 main language regions of Switzerland. After creating an account on the platform, participants were asked to complete a short demographic questionnaire before accessing the serious game. The only incentive given to the potential participants was a course completion certificate, which participants obtained after completing the postgame questionnaire. The primary outcome was the proportion of participants who reported that they were willing to change their infection prevention and control behavior. Secondary outcomes were the infection prevention and control areas affected by this willingness and the presumed evolution in the use of specific personal protective equipment items. The elements associated with intention to change infection prevention and control behavior, or lack thereof, were also assessed. Other secondary outcomes were the subjective perceptions regarding length, difficulty, meaningfulness, and usefulness of the serious game; impression of engagement and boredom while playing the serious game; and willingness to recommend its use to friends or colleagues.

Results

From March 9 to June 9, 2021, a total of 3227 accounts were created on the platform, and 1104 participants (34.2%) completed the postgame questionnaire. Of the 1104 respondents, 509 respondents (46.1%) answered that they intended to change their infection prevention and control behavior after playing the game. Among the respondents who answered that they did not intend to change their behavior, 86.1% (512/595) answered that they already apply these guidelines. Participants who followed the German version were less likely to intend to change their infection prevention and control behavior (odds ratio [OR] 0.48, 95% CI 0.24-0.96; P=.04) and found the game less engaging (P<.001). Conversely, participants aged 53 years or older had stronger intentions of changing infection prevention and control behavior (OR 2.07, 95% CI 1.44-2.97; P<.001).

Conclusions

Escape COVID-19 is a useful tool to enhance correct infection prevention and control measures on a national scale, even after 2 COVID-19 pandemic waves; however, the serious game's impact was affected by language, age category, and previous educational training, and the game should be adapted to enhance its impact on specific populations.

Contribution of the co.LAB Framework to the Collaborative Design of Serious Games: Mixed Methods Validation Study

Background

Multidisciplinary collaboration is essential to the successful development of serious games, albeit difficult to achieve. In a previous study, the co.LAB serious game design framework was created to support collaboration within serious game multidisciplinary design teams. Its use has not yet been validated in a real usage context.

Objective

The objective of this study was to perform a first assessment of the impact of the co.LAB framework on collaboration within multidisciplinary teams during serious game design and development.

Methods

A mixed methods study was conducted, based on 2 serious game design projects in which the co.LAB framework was used. The first phase was qualitative and carried out using a general inductive approach. To this end, all members of the first serious game project team who used the co.LAB framework were invited to take part in a focus group session (n=6). In a second phase, results inferred from qualitative data were used to define a quantitative instrument (questionnaire) that was designed according to the Checklist for Reporting Results of Internet E-Surveys. Members of both project teams (n=11) were then asked to answer the questionnaire. Quantitative results were reported as median (Q1, Q3), and appropriate nonparametric tests were used to assess between-group differences. Finally, results gathered through the qualitative and quantitative phases were integrated.

Results

In both phases, the participation rate was 100% (6/6 and 11/11). Verbatim transcripts were classified into 4 high level themes: (1) influence on collaborative dimensions; (2) impact on project course, monitoring, and efficiency; (3) qualitative perceptions of the framework; and (4) influence of team composition on the use of the framework. The web-based questionnaire was then developed according to the 7 dimensions of collaboration by Burkhardt et al. In both projects, the co.LAB framework had a positive impact on most dimensions of collaboration during the multidisciplinary design and development of serious games. When all collaborative dimensions were aggregated, the overall impact of the framework was rated on a scale from –42 to 42 (very negative to very positive). The overall median score was 23 (Q1, Q3: 20, 27), with no significant difference between groups (P=.58). Most respondents also believed that all serious game design teams should include a member possessing significant expertise in serious game design to guide the development process.

Conclusions

The co.LAB framework had a positive impact on collaboration within serious game design and development teams. However, expert guidance seems necessary to maximize development efficiency. Whether such guidance can be provided by means of a collaborative web platform remains to be determined.