Driver Expectations of a Partial Driving Automation System in Relation to Branding and Training

OBJECTIVE

The current study examined whether differences in the branding and description or mode of training materials influence drivers' understanding and expectations of a partial driving automation system.

BACKGROUND

How technology is described might influence consumers' understanding and expectations, even if all information is accurate.

METHOD

Ninety drivers received training about a real partial driving automation system with a fictitious name. Participants were randomly assigned to a branding condition (system named AutonoDrive, training emphasized capabilities; or system named DriveAssist, training emphasized limitations) and training mode (quick-start brochure; video; or in-person demonstration). No safety-critical information was withheld nor deliberately misleading information provided. After training, participants drove a vehicle equipped with the system. Associations of drivers' expectations with branding condition and training mode were assessed using between-subjects comparisons of questionnaire responses obtained pre- and post-drive.

RESULTS

Immediately after training, those who received information emphasizing the system's capabilities had greater expectations of the system's function and crash avoidance capability in a variety of driving scenarios, including many in which the system would not work, as well as greater willingness to utilize the system's workload reduction benefits to take more risks. Most but not all differences persisted after driving the vehicle. Expectations about collision avoidance differed by training mode pre-drive but not post-drive.

CONCLUSION

Training that emphasizes a partial driving automation system's capabilities and downplays its limitations can foster overconfidence.

APPLICATION

Accuracy of technical information does not guarantee understanding; training should provide a balanced view of a system's limitations as well as capabilities.

The Role of Self-Study in Addressing Competency Decline Among Airline Pilots During the COVID-19 Pandemic

OBJECTIVE

This study aimed to investigate the self-study methods used by pilots while grounded and their perception of competence decline and confidence upon their return to flying.

BACKGROUND

Previously, long absences from flying were managed on a case-by-case basis. Thousands of pilots returning to flying as the pandemic eases have burdened airline training systems. Limited research has been conducted on the decline in skills of airline pilots while operationally absent from the cockpit. Few studies have considered this topic in the context of a pandemic.

METHOD

A questionnaire study was conducted with 234 airline pilots who were grounded during the COVID-19 pandemic.

RESULTS

Uncertainty regarding sudden and indefinite periods of grounding made it challenging to maintain motivation to self-study. This matter was aggravated by the additional financial and personal stress caused by the state of the airline industry and the outcomes of the pandemic. The participants anticipated a decline in manual flying skills as the worst outcome after being absent from the flight deck. However, these pilots proved quick to recover these skills when they resumed flying. It took significantly more time for pilots to regain proficiency in applying knowledge, procedures and compliance with regulations, situation awareness and workload management.

CONCLUSION

The study proposes recommendations for pilots and airlines to harness essential self-study practices in competency areas identified to have significantly declined.

APPLICATION

The outcome of this paper guides airlines, pilots and regulators in better understanding how grounded pilots observe skill decline in a broader range of competencies.

Effects of Augmented Reality-, Virtual Reality-, and Mixed Reality-Based Training on Objective Performance Measures and Subjective Evaluations in Manual Assembly Tasks: A Scoping Review

OBJECTIVE

The present scoping review aims to transform the diverse field of research on the effects of mixed reality-based training on performance in manual assembly tasks into comprehensive statements about industrial needs for and effects of mixed reality-based training.

BACKGROUND

Technologies such as augmented and virtual reality, referred to as mixed reality, are seen as promising media for training manual assembly tasks. Nevertheless, current literature shows partly contradictory results, which is due to the diversity of the hardware used, manual assembly tasks as well as methodological approaches to investigate the effects of mixed reality-based training.

METHOD

Following the methodological approach of a scoping review, we selected 24 articles according to predefined criteria and analyzed them concerning five key aspects: (1) the needs in the industry for mixed reality-based training, (2) the actual use and classification of mixed reality technologies, (3) defined measures for evaluating the outcomes of mixed reality-based training, (4) findings on objectively measured performance and subjective evaluations, as well as (5) identified research gaps.

RESULTS

Regarding the improvement of performance and effectiveness through mixed reality-based training, promising results were found particularly for augmented reality-based training, while virtual reality-based training is mostly-but not consistently-as good as traditional training.

APPLICATION

Mixed reality-based training is still not consistently better, but mostly at least as good as traditional training. However, depending on the use case and technology used, the training outcomes in terms of assembly performance and subjective evaluations show promising results of mixed reality-based training.

A methodology for projecting the return on investment of training technologies

Each year significant tax dollars are spent on the development of new technologies to increase efficiency and/or reduce costs of military training. However, there are currently no validated methods or measures to quantify the return on investment for adopting these new technologies for military training. Estimating the return on investment (ROI) for training technology adoption involves 1) developing a methodology or framework, 2) validating measures and methods, and 3) assessing predictive validity. The current paper describes a projective methodology using the Kirkpatrick framework to compare projected tangible and intangible benefits against tangible and intangible costs to estimate future ROI. The use-case involved an advanced technology demonstration in which sixty aircrew participated in a series of live, virtual, and constructive (LVC) exercises over a five-week period. Participants evaluated the technology's potential costs and benefits according to the Kirkpatrick framework of training program evaluation, and analyses resulted in a nominal projection of $488 million dollars saved, significant enhancements in large-force proficiency, and 1.4 lives saved over ten years at an implementation rate of 0.5% of budgeted flight hours. A discussion of theoretical implications, data-based limitations, and recommendations for future research are provided.

The development of the helicopter non-technical skills (HeliNOTS) behavioural marker systems

Behavioural marker systems (observational frameworks geared towards the assessment of non-technical skills by way of behavioural markers) exist across a variety of high-risk occupations, however, no identifiable system currently exists developed from rotary operative data. In this study, discussion groups (n = 9) were undertaken with subject matter experts (n = 20)-including pilots and technical crew operating across search and rescue and offshore transport environments-with the objective of identifying role-specific behavioural markers. Systems were reviewed on an iterative basis by the academic team and received final reviews by additional subject matter experts (n = 6). Two behavioural marker systems were constructed: HeliNOTS (O) for offshore transport pilots and HeliNOTS (SAR) for search and rescue crews; each with domain-specific behavioural markers. Both represent a significant step towards a nuanced approach to training and assessment of helicopter flight crews' non-technical skills and are the first publicly available systems tailored to these distinct mission types.Practitioner summary: There is no publicly available behavioural marker system based on data from rotary operatives. Across this study, two prototype systems were developed: HeliNOTS (SAR) for helicopter search and rescue, and HeliNOTS (O) for helicopter offshore transport. Both HeliNOTS systems represent a nuanced approach towards rotary CRM training and assessment.

Development and Human Factors Evaluation of a Portable Auditory Localization Training System

OBJECTIVE

To design and develop a Portable Auditory Localization Acclimation Training (PALAT) system capable of producing psychoacoustically accurate localization cues; evaluate the training effect against a proven full-scale, laboratory-grade system under three listening conditions; and determine if the PALAT system is sensitive to differences among electronic level-dependent hearing protection devices (HPDs).

BACKGROUND

In-laboratory auditory localization training has demonstrated the ability to improve localization performance with the open (natural) ear, that is, unoccluded, and while wearing HPDs. The military requires a portable system capable of imparting similar training benefits as those demonstrated in laboratory experiments.

METHOD

In a full-factorial repeated measures design experiment, 12 audiometrically normal participants completed localization training and testing using an identical, optimized training protocol on two training systems under three listening conditions (open ear, TEP-100, and ComTac™ III). Statistical tests were performed on mean absolute accuracy score and front-back reversal errors.

RESULTS

No statistical difference existed between the PALAT and laboratory-grade DRILCOM systems on two dependent localization accuracy measurements at all stages of training. In addition, the PALAT system detected the same localization performance differences among the three listening conditions.

CONCLUSION

The PALAT system imparted similar training benefits as the DRILCOM system and was sensitive to HPD localization performance differences.

APPLICATION

The user-operable PALAT system and optimized training protocol can be employed by the military, law enforcement, and various industries, to improve auditory localization performance in conditions where auditory situation awareness is critical to safety.

Supervisor Training: A Promising Approach to Addressing Impairment in the Workplace

OBJECTIVE

This study presents a pilot evaluation of a new online training program created to improve supervisors' ability to protect worker safety by recognizing and responding to signs of impairment from diverse causes.

METHODS

The study design relied on a mixed-methods approach with two waves of surveys and interviews immediately after training completion and after 3 months to gauge changes in participants' perceived knowledge and competencies.

RESULTS

Participants had a positive reaction to training and found it relevant to their job responsibilities and aligned with existing organizational safety programs, practices, and policies. There was statistically significant improvement in perceived impairment knowledge and competencies. The study also provided early indications of training transfer.

CONCLUSIONS

The study offers encouraging results that may be used to improve the impairment recognition and response training for supervisors and support future investigations of its impact.

Identifying what works for whom: Implementation outcomes following iLookOut, a child abuse identification and referral training program

Introduction

iLookOut, a web-based child abuse training for early childcare professionals (ECPs), has been shown to improve knowledge and attitudes related to correctly identifying and reporting suspected cases of child abuse. The overarching goal of the present study is to examine "what works for whom" for iLookOut in order to identify strategies for optimizing learner outcomes.

Methods

This prospective study enrolled 12,705 ECPs who completed iLookOut (November 2014-December 2018). We used structural equation models to test whether learner demographic and professional characteristics were differentially associated with implementation outcomes (i.e., acceptability and appropriateness) and whether these mediated subsequent indicators of training effectiveness (i.e., gains in knowledge).

Results

Consistent with previous research, individuals with lower baseline knowledge scores showed greater knowledge gains (β = -.57; p < .001). Greater knowledge gains were seen for learners who reported higher acceptability (β = .08; p < .001) or appropriateness (β = .14; p < .001). Implementation outcomes strongly associated with knowledge gains included acceptability for female learners and appropriateness for learners who had not completed high school or had >15 years of experience in childcare settings. Where mediation was found, for the majority of groups, appropriateness emerged as the driving mediator.

Conclusion

Implementation outcomes emerged as important drivers of knowledge change for most groups. The iLookOut Core Training's use of a multimedia learning environment, video-based storylines, and game-based techniques were endorsed by learners and correlated with increases in knowledge. Future work should explore why aspects of the iLookOut training are rated as less acceptable or appropriate by some groups and what changes would improve efficacy for low performing learners.

Rapid and sustained contact tracing training for COVID-19 in San Francisco: a training model for developing an emergency public health workforce

The City and County of San Francisco was the first municipality in the United States to institute a COVID-19 contact tracing program. The San Francisco Department of Public Health (SFDPH) and the University of California, San Francisco (UCSF) created an outcome-based fully remote contact tracing curriculum using participatory learning methods to train non-public health emergency workers as contact tracers. Between April and December 2020, we trained over 300 individuals in contact tracing skills and procedures over three training phases. Using iterative curriculum design and Kirkpatrick's evaluation methodology, we aimed to ensure high quality and successful person-centered contact tracing. The resulting curriculum consisted of 24 learning outcomes taught with six participatory skills development activities, asynchronous materials, and one-on-one contact tracer support. We collected more than 700 responses from trainees using various evaluation tools across the training phases, and contact tracers interviewed more than 24,000 contacts after training in our program. Our evaluations showed that knowledge and skills improved for most trainees and demonstrated the utility of the training program in preparing trainees to perform person-centered contact tracing in San Francisco. Local health jurisdictions and state health agencies can use this model of curriculum development and evaluation to rapidly train a non-public health workforce to respond to future public health emergencies.

Acute physiological, perceived exertion and enjoyment responses during a 4-week basketball training: a small-sided game vs. high-intensity interval training

Introduction

Although previous research found that small-sided game (SSG) training was more enjoyable than high-intensity interval training (HIT) in various sports, no data were provided during longer training period in basketball. Furthermore, the comparison of internal loads between the two training approaches needs to be further examined. Thus, this study aimed to examine the acute physiological, perceived exertion and enjoyment responses during 4-week progressive basketball SSG or HIT programs.

Methods

Nineteen female collegiate basketball players were randomly assigned to two groups that performed either HIT (n = 10) or SSG (n = 9) 3 times per week for 4 continuous weeks. Average and percentage of maximal heart rate (HR_mean and %HR_max), rating of perceived exertion (RPE), and physical activity enjoyment (PACES) were determined during each training session.

Results

There was a main group effect in PACES (p < 0.001; ηp2 = 0.44, moderate), and SSG had higher PACES than HIT in each week (p < 0.05). There were no significant interactions or main group effects in HR_mean, %HR_max or RPE, but a main time effect was found in HR_mean (p = 0.004; ηp2 = 0.16, minimum), %HR_max (p < 0.001; ηp2 = 0.25, minimum), and RPE (p < 0.001; ηp2 = 0.31, moderate), respectively. In the SSG group, although no significant differences were found in HR responses, %HR_max was below 90% in week 1 and week 2. Accompanied with changes in %HR_max, RPE in week 1 and week 2 was lower than that in week 3 and week 4 (p < 0.05).

Conclusion

Our findings suggest that SSG and HIT elicit similar acute HR response and RPE level, but SSG is perceived as more enjoyable and therefore it is more likely to increase exercise motivation and adherence comparing to HIT. Moreover, it seems that half-court, 2 vs. 2 SS Gtraining format with modified rules and lasting ≥ 7.5 min should be prescribed as an enjoyable training alternative to provide optimal cardiovascular stimuli (> 90% of HR_max) for female basketball players.

Evaluation of an Online Training Program on COVID-19 for Health Workers in Papua New Guinea

BACKGROUND

Health worker training is an important component of a holistic outbreak response, and travel restrictions resulting from the COVID-19 pandemic have highlighted the potential of virtual training. Evaluation of training activities is essential for understanding the effectiveness of a training program on knowledge and clinical practice. We conducted an evaluation of the online COVID-19 Healthcare E-Learning Platform (CoHELP) in Papua New Guinea (PNG) to assess its effectiveness, measure engagement and completion rates, and determine barriers and enablers to implementation, in order to inform policy and practice for future training in resource-limited settings.

METHODS

The evaluation team conducted a mixed methods evaluation consisting of pre- and post-knowledge quizzes; quantification of engagement with the online platform; post-training surveys; qualitative interviews with training participants, non-participants, and key informants; and audits of six health facilities.

RESULTS

A total of 364 participants from PNG signed up to participate in the CoHELP online training platform, with 41% (147/360) completing at least one module. Of the 24 participants who completed the post-training survey, 92% (22/24) would recommend the program to others and 79% (19/24) had used the knowledge or skills gained through CoHELP in their clinical practice. Qualitative interviews found that a lack of time and infrastructural challenges were common barriers to accessing online training, and participants appreciated the flexibility of online, self-paced learning.

CONCLUSIONS

Initially high registration numbers did not translate to ongoing engagement with the CoHELP online platform, particularly for completion of evaluation activities. Overall, the CoHELP program received positive feedback from participants involved in the evaluation, highlighting the potential for further online training courses in PNG.

Minding the Gap: Effects of an Attention Maintenance Training Program on Driver Calibration

OBJECTIVE

The present study examines the effect of an existing driver training program, FOrward Concentration and Attention Learning (FOCAL) on young drivers' calibration, drivers' ability to estimate the length of their in-vehicle glances while driving, using two different measures, normalized difference scores and Brier Scores.

BACKGROUND

Young drivers are poor at maintaining attention to the forward roadway while driving a vehicle. Additionally, drivers may overestimate their attention maintenance abilities. Driver training programs such as FOCAL may train target skills such as attention maintenance but also might serve as a promising way to reduce errors in drivers' calibration of their self-perceived attention maintenance behaviors in comparison to their actual performance.

METHOD

Thirty-six participants completed either FOCAL or a Placebo training program, immediately followed by driving simulator evaluations of their attention maintenance performance. In the evaluation drive, participants navigated four driving simulator scenarios during which their eyes were tracked. In each scenario, participants performed a map task on a tablet simulating an in-vehicle infotainment system.

RESULTS

FOCAL-trained drivers maintained their attention to the forward roadway more and reported better calibration using the normalized difference measure than Placebo-trained drivers. However, the Brier scores did not distinguish the two groups on their calibration.

CONCLUSION

The study implies that FOCAL has the potential to improve not only attention maintenance skills but also calibration of the skills for young drivers.

APPLICATION

Driver training programs may be designed to train not only targeted higher cognitive skills but also driver calibration-both critical for driving safety in young drivers.

Fostering Self-Regulated Learning in Online Environments: Positive Effects of a Web-Based Training With Peer Feedback on Learning Behavior

Although training in self-regulated learning (SRL) is effective in improving performance, human trainers can reach only a few people at a time. We developed a web-based training for potentially unlimited numbers of participants based on the process model of SRL by Schmitz and Wiese (2006). A prior study (Bellhäuser et al., 2016) observed positive effects on self-reported SRL and self-efficacy. In the present randomized controlled trial, we investigated an improved version of the web-based training, augmented by the application of peer feedback groups. Prospective university students in an online mathematics preparation course were assigned randomly to one of four experimental conditions: Group D (diary), group TD (training + diary), group TDP (training + diary + peer feedback group), and group C (control). Complete data was obtained for 136 participants (78.8% male; M = 19.8 years). The learning diary was intended to trigger goal setting, planning, and self-motivation in the morning and reflection in the evening. The web-based training consisted of three lessons (approximately 90 min each) with videos, presentations, self-tests, and exercises. In the peer feedback condition, participants were randomly assigned to groups of five persons each and used a bulletin board to discuss pre-defined topics related to the content of the web-based training. Outcome measures included a test of declarative SRL knowledge, an SRL questionnaire, a general self-efficacy scale, log file data, and a mathematics test. Results showed positive effects for the web-based training, particularly when combined with peer feedback on both SRL knowledge and SRL questionnaires, self-efficacy, and on objective time-investment, but not on the mathematics test. The learning diary did not exhibit positive effects. We conclude that additional peer-feedback seems to be a useful supplement to web-based trainings with comparably low organizational costs.

Fitness Facility Staff Can Be Trained to Deliver a Motivational Interviewing-Informed Diabetes Prevention Program

Background: Training programs must be evaluated to understand whether the training was successful at enabling staff to implement a program with fidelity. This is especially important when the training has been translated to a new context. The aim of this community case study was to evaluate the effectiveness of the in-person Small Steps for Big Changes training for fitness facility staff using the 4-level Kirkpatrick training evaluation model. Methods: Eight staff were trained to deliver the motivational interviewing-informed Small Steps for Big Changes program for individuals at risk of developing type 2 diabetes. Between August 2019 and March 2020, 32 clients enrolled in the program and were allocated to one of the eight staff. The Kirkpatrick 4-level training evaluation model was used to guide this research. Level one assessed staff satisfaction to the training on a 5-point scale. Level two assessed staff program knowledge and motivational interviewing knowledge/skills. Level three assessed staff behaviors by examining their use of motivational interviewing with each client. Level four assessed training outcomes using clients' perceived satisfaction with their staff and basic psychological needs support both on 7-point scales. Results: Staff were satisfied with the training (M = 4.43; SD = 0.45; range = 3.86-4.71). All learning measures demonstrated high post-training scores that were retained at implementation follow-up. Staff used motivational interviewing skills in practice and delivered the program at a client-centered level (≥6; M = 6.34; SD = 0.83; range = 3.75-7.80). Overall, clients perceived staff supported their basic psychological needs (M = 6.55; SD = 0.64; range = 6.17-6.72) and reported high staff satisfaction scores (M = 6.88; SD = 0.33; range = 6-7). Conclusion: The Small Steps for Big Changes training was successful and fitness facility staff delivered a motivational interviewing-informed program. While not all staff operated at a client-centered level, clients perceived their basic psychological needs to be supported. Findings support the training for future scale-up sites. Community fitness staff represent a feasible resource through which to run evidence-based counseling programs.

Impact of a Pharmacy Management Course for Pharmacists Working Within Brazil's Public Health System

Objective. To assess the impact of a continuing education course that focused on the development of management competencies on pharmacists working in Brazil's public health system.Methods. A specialization blended in-service course (360 hours) entitled Pharmaceutical Service and Access to Medicine Management was offered to pharmacists working within the Brazilian public health system. Data on course outcomes were collected through individual interviews with pharmacists who worked in the Brazilian public health system, as well as from focus groups and records of researchers' observations. The analysis was based on models of learning and training evaluation theory.Results. The findings showed proximal outcomes on students' knowledge, skills, and attitudes; behavioral changes; and programmatic outcomes in the organization and management of health services, promoting the recognition and integration of the "pharmaceutical services division" - a dedicated sector within the health departments in municipalities and states. The inputs (course content, structure, and in-service hands-on activities) were directly linked to the outcomes reported by participants and helped them to overcome some of the barriers to using knowledge and skills in the workplace.Conclusion. A well-structured course including leaning activities to intervene in the workplace had a positive impact on pharmacists' behavior and contributed to the capacity building of the organizations in which they operate.

Training results in increased practitioner confidence and identification of depression in people with low vision: a mixed methods study

PURPOSE

The prevalence of depression in people with low vision is high and often goes undiagnosed. There is the potential for those who provide low vision services to perform concurrent depression screening. However, prior training in depression identification and suitable referral pathways is required. The aims of this study were: (1) to assess the impact of a training programme on practitioners' confidence and behaviour in addressing depression in patients with low vision, and (2) to review the training programme and identify areas for further development.

METHODS

A convergent mixed methods approach was used. Questionnaires were completed by practitioners pre-, immediately post- and 6 months post- training (n = 40) to assess practitioner confidence in approaching depression in patients with low vision. Qualitative interviews were performed with a subset of practitioners 6 months post-training (n = 9). Additionally, routine data from the Low Vision Service Wales (LVSW) database was used to determine the change in the number of practitioners identifying depression in patients, and the change in the number of patients identified at risk of depression 6 months post-training.

RESULTS

Of the 148 practitioners who completed low vision assessments pre- and post-training, 28 (18.9%) documented risk of depression in their patients pre-training, which increased substantially to 65 (43.9%) post-training (p < 0.0001). Mixed methods analysis confirmed increased documentation of depressive symptoms by practitioners. Practitioner confidence increased following training, with 92.3% feeling more confident to approach emotional issues with patients and 92.2% intending to use the recommended screening tool to identify depression. Interviews provided insight into areas where confidence was still lacking. Quantitative questionnaires revealed that training content was considered appropriate by 91% of participants. Interviews confirmed these findings while expanding upon possibilities for programme improvement.

CONCLUSIONS

Training for depression screening was found to be time-efficient and acceptable for LVSW practitioners and shown to increase practitioner confidence in the identification of depression. Additionally, the programme changed behaviour, resulting in an increase in the identification of depression in patients with low vision. However, this is a complex topic and ongoing development is required to embed depression screening as an integral part of low vision services.

Impact of Systematic Tailored Assessment for Responding to Suicidality (STARS) Protocol Training on Mental Health Professionals' Attitudes, Perceived Capabilities, Knowledge, and Reluctance to Intervene

BACKGROUND AND AIMS

Systematic Tailored Assessment for Responding to Suicidality (STARS) protocol and associated training were developed with the key objectives of supporting clinicians to conduct a suicide enquiry, obtaining a comprehensive account of psycho-social factors contributing to suicidality, and collaboratively developing a safety plan with clients. STARS training aims to address knowledge, attitudes and capabilities that influence intervention behavior/skills. This study aimed to examine associations between clinician characteristics and pre-training competencies in suicide risk assessment (SRA), as well as the impact of STARS training workshop on clinician competencies; and to determine the predictors of SRA training outcomes.

METHOD

Australian mental health professionals working with suicidal persons who undertook the STARS 2-day face-to-face workshop between 2018 and 2020 completed an online survey at pre- and post-training. Of the 222 participants who completed the pre-training questionnaire, 144 (64.9%) also completed the post-training questionnaire. Participants were mostly female (75.7%), had completed a university degree (86.4%), had <10 years of experience in suicide prevention (71.7%), and were allied and mental health professionals (78.1%). We used linear mixed-effects regression for statistical analyses.

RESULTS

STARS participants who reported higher perceived capability at baseline had significantly greater formal and informal training, more years of experience in suicide prevention, and were more likely to have experienced client suicide and/or suicide attempt and to report fewer SRA related fears. We found overall significant positive impacts of STARS training on clinician competencies (attitudes, perceived capability, declarative knowledge) from pre- to post-training. The most distinct changes following STARS training were for perceived capability and declarative knowledge. Participants who had more positive attitudes after training were significantly more likely to have had less prior supervision/mentoring. Reluctance to intervene was not found to significantly change after training.

CONCLUSIONS

We found evidence that attitudes, perceived capability and declarative knowledge changed positively from pre- to post-STARS training among mental health professionals. Underpinned by the minimum standardized SRA competencies, STARS training may be critical for informing evidence-based knowledge and skills in SRA and safety planning.

An evaluation of training to prepare nurses in a home-based service to care for children and families

Evaluation of training was conducted for a paediatric hospital-in-the-home service in Sydney, Australia. Community nurses with no paediatric training or experience were employed and undertook a training program. The aim was to assess the degree to which the training had prepared them to care for children and families in their homes. A mixed-methods design was employed. Overall, the following aspects of the training were well received by the community nurses: paediatric resuscitation, growth and development, clinical deterioration and child protection. Each topic provided basic knowledge and skills in the speciality. The participating nurses generally reached a 'competent' level of practice as defined by Benner (2000). Further training and development is recommended. Where paediatric nursing practice is isolated from acute paediatrics services, opportunities must be provided to improve safe levels of practice for children of all ages.

Simulation in perfusion: evaluating the efficacy of a specific training with eye-tracking

BACKGROUND

With the aim of integrating simulation training into the training of perfusionists, we examined whether the participants were able to transfer a specific learning content to the same and different situations and assessed their feedback on the simulation training. Eye-tracking was tested as a measure and supplemented by additional measures.

METHODS

A 2 × 2 mixed design was used, with test time (pre- and post-test) and training group (same and different content training) as factors. In the pre- and post-test, the participant had to handle a critical situation on the cardiopulmonary bypass, namely, a drop in arterial partial oxygen pressure. Between the two test times, the participant practised under guidance the handling of either the same critical situation (Group 1) or a different one, that is, impaired venous return (Group 2). Dependent measures were fixations of the eyes on specific areas of interest on the heart-lung machine, measures of latency and subjective assessments. Moreover, participants gave feedback on the simulation training.

RESULTS

Fixation analyses showed that the training led to an increased gaze on areas of interest relevant to the drop in arterial partial oxygen pressure in both groups, with a significant increase only for Group 1. The surveys revealed a great interest in the integration of simulation training into education.

CONCLUSION

In combination with other measures, eye-tracking is suitable for the evaluation of simulation training. Due to the positive training effects and positive participant feedback, the integration of simulation into the training of perfusionists is advocated. Concerning transfer of learning content, more research is needed.

Evaluation of resident satisfaction and change in knowledge following use of high-fidelity simulation teaching for anaesthesia residents

Background and Aims:

Anaesthesia practice demands medical knowledge and skills as essential components for patient management in peri-operative emergencies. Since all residents are not exposed to such situations during their residency, training them using simulation technology could bridge this knowledge and skill gap. The aim of this study was to train and evaluate residents to manage anaesthesia emergencies on high fidelity simulators.

Methods:

Kirkpatrick model of program evaluation was carried out. Resident reaction was captured using a satisfaction questionnaire and the change in knowledge was assessed using pre-test and post-test Multiple Choice Questions (MCQs). Six scenarios were created and executed on a human patient simulator (HPS). All 22 residents participated in this teaching learning method. The steps of simulation teaching included pre-test, pre-briefing, orientation to manikins, performing/scribe, debriefing, feedback questionnaire, and post-test. The satisfaction questionnaire was administered following the second and fourth scenario.

Results:

95% residents agreed on overall satisfaction, that it helps in building team dynamics and clinical reasoning. All students agreed that this teaching had positive professional impact. 14% residents felt they were anxious during the class. The items in the questionnaire had a Cronbach's α value of 0.9. The mean score for pre-test was 24.22 ± 7 (Mean ± SD) and the post-test was 47.18 ± 5.6, the difference between the scores were statistically significant (P = 0.007).

Conclusion:

The use of high-fidelity simulation to train anaesthesia residents resulted in greater satisfaction scores and improved the residents' reasoning skills.

One-Hour Pilot Training to Prevent Workers From Taking Home Workplace Contaminants

Workers can accidentally transport chemical hazards from the workplace to the home, known as “take-home exposures.” Recent take-home lead-poisoning cases highlight the need for effective prevention training. A one-hour take-home prevention training was developed in partnership with a nonprofit. The training was administered and evaluated during two training sessions with twenty-one trainees. The training was composed of a lecture and interactive activities. An illustrated poster was used with different prevention actions within a story line to reduce take-home exposures under three categories: facilities with formal health and safety programs, small businesses, and outdoor work. The effectiveness and acceptability of the training was measured by a survey and pre- and posttraining exams. The second training exam responses showed a 14 percent (84 percent to 98 percent) increase in take-home prevention knowledge. Community-based prevention training could reduce the burden of chemical exposures on vulnerable workers and their families.

Couples' voluntary HIV counseling and testing provider training evaluation, Zambia

With the expansion of couples' voluntary HIV counseling and testing (CVCT) in urban Zambia, there is a growing need to evaluate CVCT provider trainings to ensure that couples are receiving quality counseling and care. We evaluated provider knowledge scores, pre- and post-training and predictors of pre- and post-training test scores. Providers operating in 67 government clinics in four Copperbelt Province cities were trained from 2008 to 2013 in three domains: counseling, rapid HIV laboratory testing and data management. Trainees received pre- and post-training tests on domain-specific topics. Pre- and post-training test scores were tabulated by provider demographics and training type, and paired t-tests evaluated differences in pre- and post-training test scores. Multivariable ANCOVA determined predictors of pre- and post-training test scores. We trained 1226 providers, and average test scores increased from 68.8% pre-training to 83.8% post-training (p < 0.001). Test scores increased significantly for every demographic group and training type (p < 0.001) with one exception-test scores did not significantly increase for those receiving counseling or data management training who had less than a high school education. In multivariable analysis, higher educational level and having a medical background were predictive of a higher pre-test score; higher pre-test scores and having a medical background were predictive of higher post-test scores. Pre- and post-test assessments are critical to ensure quality services, particularly as task-shifting from medical to lay staff becomes more common. Assessments showed that our CVCT trainings are successful at increasing knowledge, and that those with lower education may benefit from repeat trainings.

Using Predictive Evaluation to Design, Evaluate, and Improve Training for Polio Volunteers

Background

Predictive Evaluation (PE) uses a four-step process to predict results then designs and evaluates a training intervention accordingly. In 2012, the Sustainable Management Development Program (SMDP) at the Centers for Disease Control and Prevention used PE to train Stop Transmission of Polio (STOP) program volunteers.

Methods

Stakeholders defined specific beliefs and practices that volunteers should demonstrate. These predictions and adult learning practices were used to design a curriculum to train four cohorts. At the end of each workshop, volunteers completed a beliefs survey and wrote goals for intended actions. The goals were analyzed for acceptability based on four PE criteria. The percentage of acceptable goals and the beliefs survey results were used to define the quality of the workshop. A postassignment adoption evaluation was conducted for two cohorts, using an online survey and telephone or in-person structured interviews. The results were compared with the end of workshop findings.

Results

The percentage of acceptable goals across the four cohorts ranged from 49% to 85%. In the adoption evaluation of two cohorts, 88% and 94% of respondents reported achieving or making significant progress toward their goal. A comparison of beliefs survey responses across the four cohorts indicated consistencies in beliefs that aligned with stakeholders' predictions.

Conclusions

Goal statements that participants write at the end of a workshop provide data to evaluate training quality. Beliefs surveys surface attitudes that could help or hinder workplace performance. The PE approach provides an innovative framework for health worker training and evaluation that emphasizes performance.

Training Employers to Implement Health Promotion Programs: Results From the CDC Work@Health® Program

PURPOSE

Centers for Disease Control and Prevention (CDC) initiated the Work@Health Program to teach employers how to improve worker health using evidence-based strategies. Program goals included (1) determining the best way(s) to deliver employer training, (2) increasing employers' knowledge of workplace health promotion (WHP), and (3) increasing the number of evidence-based WHP interventions at employers' worksites. This study is one of the few to examine the effectiveness of a program designed to train employers how to implement WHP programs.

DESIGN

Pre- and posttest design.

SETTING

Training via 1 of 3 formats hands-on, online, or blended.

PARTICIPANTS

Two hundred six individual participants from 173 employers of all sizes.

INTERVENTION

Eight-module training curriculum to guide participants through building an evidence-based WHP program, followed by 6 to 10 months of technical assistance.

MEASURES

The CDC Worksite Health ScoreCard and knowledge, attitudes, and behavior survey.

ANALYSIS

Descriptive statistics, paired t tests, and mixed linear models.

RESULTS

Participants' posttraining mean knowledge scores were significantly greater than the pretraining scores (61.1 vs 53.2, P < .001). A year after training, employers had significantly increased the number of evidence-based interventions in place (47.7 vs 35.5, P < .001). Employers' improvements did not significantly differ among the 3 training delivery formats.

CONCLUSION

The Work@Health Program provided employers with knowledge to implement WHP interventions. The training and technical assistance provided structure, practical guidance, and tools to assess needs and select, implement, and evaluate interventions.

Development and Evaluation of a Train-the-Trainer Workshop for Hong Kong Community Social Service Agency Staff

INTRODUCTION

Capacity building approaches are useful in large-scale community-based health promotion interventions. However, models to guide and evaluate capacity building among social service agency staff in community settings are rare in the literature. This paper describes the development and evaluation of a 1-day (7 h) train-the-trainer (TTT) workshop for the "Enhancing Family Well-Being Project". The workshop aimed at equipping staff from different community agencies with the knowledge and skills to design, implement, and evaluate positive psychology-based interventions for their clients in Sham Shui Po, an over-crowded and low-income district in Hong Kong.

METHODS

The current TTT extended and improved on our previous successful model by adding research and evaluation methods (including the Logic Model, process evaluation, and randomized controlled trial), which are important to plan and evaluate the community interventions. Evaluation of the TTT was guided by the Integrated Model of Training Evaluation and Effectiveness (IMTEE), with quantitative and qualitative methods. Quantitative data were collected from pretraining (T1), post-training (T2), and 6-month (T3) and 12-month (T4) follow-up surveys. Qualitative data were collected from four focus groups of agency staff after the intervention.

RESULTS

Ninety-three staff from 30 community agencies attended the training, and 90 completed the baseline survey. Eighty-eight, 63, and 57 staff performed the evaluations at T2, T3, and T4, respectively. Agency staff were satisfied with the TTT. Immediate enhancement of knowledge, self-efficacy, and positive attitudes toward the training content was found at T2 (Cohen's d ranged from 0.24 to 1.22, all p < 0.05). Enhancement of knowledge of all training contents persisted at T3 and T4 (Cohen's d ranged from 0.34 to 0.63, all p < 0.05). Enhancement of self-efficacy in the use of positive psychology in intervention design persisted at T3 (Cohen's d = 0.22, p = 0.04). The skills learned were utilized to plan and develop subsequent interventions. Twenty-nine interventions were successfully designed and implemented by the agency staff, and delivered to 1,586 participants. The agency staff indicated their intention to utilize the skills they had learned for other interventions (score ≥4 out of 6) and to share these skills with their colleagues. Qualitative feedbacks from 23 agency staff supported the quantitative results.

CONCLUSION

Our brief TTT was effectively delivered to a large number of agency staff and showed effects that persisted up to 12 months. Our training and evaluation models may offer a template for capacity building among social service agency staff for community brief, universal family health promotion interventions in diverse settings.

Team Training for Dynamic Cross-Functional Teams in Aviation: Behavioral, Cognitive, and Performance Outcomes

OBJECTIVE

This study evaluates the effectiveness of a training program designed to improve cross-functional coordination in airline operations.

BACKGROUND

Teamwork across professional specializations is essential for safe and efficient airline operations, but aviation education primarily emphasizes positional knowledge and skill. Although crew resource management training is commonly used to provide some degree of teamwork training, it is generally focused on specific specializations, and little training is provided in coordination across specializations.

METHOD

The current study describes and evaluates a multifaceted training program designed to enhance teamwork and team performance of cross-functional teams within a simulated airline flight operations center. The training included a variety of components: orientation training, position-specific declarative knowledge training, position-specific procedural knowledge training, a series of high-fidelity team simulations, and a series of after-action reviews.

RESULTS

Following training, participants demonstrated more effective teamwork, development of transactive memory, and more effective team performance.

CONCLUSION

Multifaceted team training that incorporates positional training and team interaction in complex realistic situations and followed by after-action reviews can facilitate teamwork and team performance.

APPLICATION

Team training programs, such as the one described here, have potential to improve the training of aviation professionals. These techniques can be applied to other contexts where multidisciplinary teams and multiteam systems work to perform highly interdependent activities.

Evaluation of Skill-oriented Training on Enhanced Syndromic Case Management (ESCM) of Reproductive Tract Infections / Sexually Transmitted Infections (RTI/STIs) of Care Providers from Three-tier Health-care System of Gujarat

Background:

Enhanced syndromic case management (ESCM) deals with reproductive tract and sexually transmitted infections. Capacity building of service providers not only boosts the program but also inputs from them improve the quality of services.

Objectives:

To (1) identify problem areas from providers' perspectives and the gaps in knowledge and application and (2) assess the gains (if any) through pre and post-training evaluation.

Materials and Methods:

A total of 121 participants (medical/para medical) from various medical colleges, district/sub-district hospitals/ community health centers, and urban dispensaries across Gujarat were trained at a teaching institute. Trainings were of 2-3 days duration involving different learning methodology. Pre- and post-training evaluation were done on a designed pro forma and data were entered in MS office Excel 2007. Gains in knowledge/skills if any were assessed by comparing pre-/post-evaluation responses and applying test of significance (x² test).

Observations:

Out of total 121 participants, half (60) were doctors and the rest were paramedics [staff nurse (SN) and lab technicians (LT)]. Doctors revealed significant gain in basics of reproductive tract infections (RTI) and sexually transmitted infections (STI), syndrome identification, STI/HIV co-infection, and ESCM and less gain in asymptomatic STI/ complications, vulnerability, male reproductive organs, causes of vaginal/urethral discharge, STI complications, cervical cancer screening, and limitation of syndromic management. Gain was statistically significant in basics of RTI/STI amongst adolescent in paramedics; lab technicians showed significant gain in knowledge of laboratory-related areas.

Conclusion:

Assessment revealed (1) poor baseline knowledge and (2) gains following training sometimes significant and other times not significant even in core areas. Quality monitoring and contents/ methodologies modification are essential for robust trainings. Gains in skills could not be assessed through this evaluation.

Evaluating response shift in training evaluation: comparing the retrospective pretest with an adapted measurement invariance approach in a classroom management training program

BACKGROUND

Response shift (RS) can threaten the internal validity of pre-post designs. As RS may indicate a redefinition of the target construct, its occurrence in training evaluation is rather likely. The most common approach to deal with RS is to implement a retrospective pretest (then-test) instead of the traditional pre-test. In health psychology, an adapted measurement invariance approach (MIad) was developed as an alternative technique to study RS. Results produced by identifying RS with the two approaches were rarely studied simultaneously or within an experimental framework.

OBJECTIVES

To study RS in two different treatment conditions and compare results produced by both techniques in identifying various types of RS. We further studied validity aspects of the then-test.

RESEARCH DESIGN

We evaluated RS by applying the then-test procedure (TP) and the measurement invariance apporach MIad within an experimental design: Participants either attended a short-term or a long-term classroom management training program.

SUBJECTS

Participants were 146 student teachers in their first year of master's study.

MEASURES

Pre (before training), post, and then self-ratings (after training) on classroom management knowledge were administered.

RESULTS

Results indicated that the two approaches do not yield the same results. The MIad identified more and also group-specific RS as opposed to the findings of the TP, which found less and only little evidence for group-specific RS.

CONCLUSIONS

Further research is needed to study the usability and validity of the respective approaches. In particular, the usability of the then-test seems to be challenged.

Standard basic emergency obstetric and neonatal care training in Addis Ababa; trainees reaction and knowledge acquisition

BACKGROUND

In 2010, the Federal Ministry of Health of Ethiopia (FMOH) has developed standard Basic Emergency Obstetric and Neonatal Care (BEmONC) in-service training curricula to respond to the high demand for competency in EmONC. However, the effectiveness of the training curricula has not been well documented. A collaborative intervention project in Addis Ababa has trained providers using the standard BEmONC curricula where this paper presents Krikpartick level 1 and level 2 evaluation of the training.

METHODS

The project has been conducted in 10 randomly selected public health centers (HC) in Addis Ababa. Providers working in the labour wards of the selected HCs have received the standard BEmONC training between May and July 2013. Using standard tools, trainees' reaction to the course and factual knowledge during the immediate post-course and six months after the training were assessed. Descriptive statistics and t-tests were done.

RESULTS

Of the total 82 providers who received the training, 30 (36.6%) were male, 61 (74.4%) were midwives. Providers' work experiences ranged from 1 month to 37 years. Seventy-four (89%) providers reported that the training was appropriate for their work, 95% reported that the training have updated their knowledge & skills, while 27 (32.9%) reported that the training facilities & arrangements were unsatisfactory. The mean immediate post-course knowledge score was 83.5% and 33 (40%) providers did not achieve knowledge-based mastery in their first attempt. The midwives were more likely to achieve knowledge-based mastery than the nurses (p < 0.05). The mean knowledge score six-months post-training was 80.2% and 40% have scored knowledge based mastery.

CONCLUSIONS

Being one of the first papers reporting the implementation of the standard in-service BEmONC training curriculum, we have identified an important limitation on the course evaluations of the curriculum, which need urgent consideration. The majority of the trainees has reported favourable reaction to the training, but many of them did not achieve knowledge-based mastery in the immediate post training although the knowledge retention six months post training was encouraging.

Training for an effective health and safety committee in a small business setting

Health and safety committees are often heralded as a key element of successful health and safety programs, and are thought to represent a means of engaging workers in health and safety efforts. While the understanding of the factors that make these committees effective is growing, there are few resources for how to assist committees in developing these characteristics. This paper describes one approach to creating and implementing a training intervention aimed at improving health and safety committee function at one multilingual worksite. Short-term impacts were evaluated via questionnaire and qualitative observations of committee function. Results indicated high satisfaction with the training as well as modest increases in participation, cooperation, role clarity, and comfort with health and safety skills among committee members. The committee also made considerable achievements in establishing new processes for effective function. Similar interventions may be useful in other workplaces to increase health and safety committee success.

Cost-effectiveness analyses of training: a manager's guide

The evidence on the cost and cost-effectiveness of global training programs is sparse. This manager's guide to cost-effectiveness analysis (CEA) is for professionals who want to recognize and support high quality CEA. It focuses on CEA of training in the context of program implementation or rapid program expansion. Cost analysis provides cost per output and CEA provides cost per outcome. The distinction between these two analyses is essential for making good decisions about value. A hypothetical example of a cost analysis compares the cost per trainee of a computer-based anti-retroviral therapy (ART) training to a more intensive ART training. In a CEA of the same example, cost per trainee who met ART clinical performance standards is compared. The cost analysis is misleading when the effectiveness differs across trainings. Two additional hypothetical examples progress from simple to more complex costs and from a narrow to a broader scope: 1) CEA of the cost per ART patient with 95% adherence that compares the performance of doctors to counselors who attend additional training, and 2) CEA of the cost per infant HIV infection averted for a Prevention of Mother to Child Transmission program that compares the current program to one with additional training. To create an evidence base on CEA of training, more well-designed analyses and data on the cost of training are needed. Analysts should understand more about how capacity is built, how quality is improved within a health facility, and the costs associated with them. Considering the life of an investment in training, evaluations are needed on how many trainees apply the skills taught, how long trainees continue to apply them, and how long the content of the training conforms to national or international guidelines. Better data on effectiveness of training is also needed. It is feasible to measure effectiveness by clinical performance standards, or intermediate outcomes and coverage. Intermediate outcomes and coverage can also be combined with published estimates on health outcomes.

The video assessment of simulated encounters-revised (VASE-R): reliability and validity of a revised measure of motivational interviewing skills

The video assessment of simulated encounters-revised (VASE-R) is a video-based method, administered in individual or group settings, for assessing motivational interviewing (MI) skills. The 18-item instrument includes three video-based vignettes, in which actors portray substance abusers, with each vignette followed by questions that prompt examinees to write responses that are then scored against MI standards. The VASE-R was administered to two independent samples: (1) substance abuse practitioners participating in a study of MI training methods, and (2) MI training facilitators with a high level of MI skill and expertise. This multi-study report describes basic VASE-R psychometric properties -- including scoring reliability, internal consistency, concurrent validity, and sensitivity to the effects of training -- and then presents proficiency standards based on administration to a sample of MI training facilitators (MI Experts). The findings indicate excellent inter-rater reliability using intra-class correlations for the full-scale score (.85) and acceptable levels for subscales (.44 to .73). The instrument displayed strong concurrent validity with the Helpful Responses Questionnaire (HRQ) and a behavioral sample of clinician behavior with a standardized patient scored using the MI Treatment Integrity (MITI) system, as well as good sensitivity to improvement in MI skill as a result of training. The findings provide an empirical basis for suggesting VASE-R benchmarks for beginning proficiency and expert MI practice.

Impacts of health and safety education: comparison of worker activities before and after training

BACKGROUND

The International Chemical Workers Union Council (ICWUC) Center for Worker Health and Safety Education in Cincinnati, Ohio, trains workers to protect themselves from hazards due to chemical spills and other chemical exposures. We evaluated whether the ICWUC Hazardous Waste Worker Training Program affects the attitudes and post-training activities, of trained union workers.

METHODS

Detailed survey questionnaires were administered to 55 workers prior to and 14-18 months following training. Surveys queried trainees' interest and involvement in safety and health, use of information resources, training activities at their worksite, and their attempts and successes at making worksite improvements.

RESULTS

Post-training, the study population showed an increase in training of other workers, use of resources, attempts at improvements, success rates for those attempting change, and overall success at making improvements. Self-reported interest decreased, and self reported involvement in health and safety did not significantly change.

CONCLUSION

The study demonstrates that workers are more willing to attempt to change worksite conditions following training, and that their efficacy at making changes is substantially greater than before they were trained. The study confirms earlier work and strengthens these conclusions by using statistically tested comparisons of impact measures pre- and post-training.