A process evaluation of a mobile cooperation intervention: A mixed methods study

Optimizing short-term in-class process evaluation: analyzing the effectiveness of teaching interventions in pharmaceutical education using repeated measures analysis

BACKGROUND

The assessment of the effectiveness of teaching interventions in enhancing students' understanding of the Pharmaceutical Care Network Europe (PCNE) Classification System is crucial in pharmaceutical education. This is especially true in regions like China, where the integration of the PCNE system into undergraduate teaching is limited, despite its recognized benefits in addressing drug-related problems in clinical pharmacy practice. Therefore, this study aimed to evaluate the effectiveness of teaching interventions in improving students' understanding of the PCNE Classification System in pharmaceutical education.

METHODS

Undergraduate pharmacy students participated in a series of sessions focused on the PCNE system, including lectures (t1), case analyses (t2), and practical implementation (t3). The levels of understanding were evaluated using time-course questionnaires. Initially, paired samples t-Tests were used to compare understanding levels between different time points. Subsequently, Repeated Measures Analysis (RMA) was employed. Pearson correlation analysis was conducted to examine the relationship between understanding levels and the usability and likelihood of using the PCNE system, as reported in the questionnaires.

RESULTS

The paired samples t-Tests indicated insignificant differences between t2 and t3, suggesting limited improvement following the practical implementation of the PCNE system. However, RMA revealed significant time effects on understanding levels in effective respondents and the focused subgroup without prior experience (random intercept models: all p < 0.001; random slope models: all p < 0.001). These results confirmed the effectiveness of all three teaching interventions. Pearson correlation analysis demonstrated significant positive correlations between understanding levels and the usability and likelihood of using the PCNE system at all examined time points. This finding highlighted the reliability of the understanding levels reported in the questionnaires. The homework scores were used as external calibration standards, providing robust external validation of the questionnaire's validity.

CONCLUSION

The implementation of RMA provided robust evidence of the positive impact of time on understanding levels. This affirmed the effectiveness of all teaching interventions in enhancing students' comprehension of the PCNE Classification System. By utilizing RMA, potential errors inherent in common statistical methods, such as t-Tests, were mitigated. This ensured a more comprehensive and accurate assessment of the effectiveness of the teaching interventions.

Connecting Actors With the Introduction of Mobile Technology in Health Care Practice Placements (4D Project): Protocol for a Mixed Methods Study

BACKGROUND

The learning process in clinical placements for health care students is a multifaceted endeavor that engages numerous actors and stakeholders, including students, clinical tutors, link teachers, and academic assessors. Successfully navigating this complex process requires the implementation of tasks and mentorships that are synchronized with educational and clinical processes, seamlessly embedded within their respective contexts. Given the escalating number of students and the rising demand for health care services from the general population, it becomes imperative to develop additional tools that support the learning process. These tools aim to simplify day-to-day clinical practice, allowing a concentrated focus on value-based activities. This paper introduces a project funded by the European Commission that involves 5 European countries. The project's objective is to comprehensively outline the entire process of development and ultimately implement mobile technology in practice placements. The project tackles the existing gap by constructing tailored mobile apps designed for students, teachers, tutors, and supervisors within each participating organization. This approach leverages practice-based learning, mobile technology, and technology adoption to enhance the overall educational experience.

OBJECTIVE

This study aims to introduce mobile technology in clinical practice placements with the goal of facilitating and enhancing practice-based learning. The objective is to improve the overall effectiveness of the process for all stakeholders involved.

METHODS

The "4D in the Digitalization of Learning in Practice Placement" (4D Project) will use a mixed methods research design, encompassing 3 distinct study phases: phase 1 (preliminary research), which incorporates focus groups and a scoping review, to define the problem, identify necessities, and analyze contextual factors; phase 2 (collaborative app development), which involves researchers and prospective users working together to cocreate and co-design tailored apps; and phase 3, which involves feasibility testing of these mobile apps within practice settings.

RESULTS

The study's potential impact will primarily focus on improving communication and interaction processes, fostering connections among stakeholders in practice placements, and enhancing the assessment of training needs. The literature review and focus groups will play a crucial role in identifying barriers, facilitators, and factors supporting the integration of mobile technology in clinical education. The cocreation process of mobile learning apps will reveal the core values and needs of various stakeholders, including students, teachers, and health care professionals. This process also involves adapting and using mobile apps to meet the specific requirements of practice placements. A pilot study aimed at validating the app will test and assess mobile technology in practice placements. The study will determine results related to usability and design, learning outcomes, student engagement, communication among stakeholders, user behavior, potential issues, and compliance with regulations.

CONCLUSIONS

Health care education, encompassing disciplines such as medicine, nursing, midwifery, and others, confronts evolving challenges in clinical training. Essential to addressing these challenges is bridging the gap between health care institutions and academic settings. The introduction of a new digital tool holds promise for empowering health students and mentors in effectively navigating the intricacies of the learning process.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/53284.

The role of the teacher and learning in clinical practicum scales: A psychometric testing of the Finnish versions

AIM

To test the psychometric properties of the further developed Role of the Teacher (Tc2) and the new Learning in Clinical Practicum (LCP) scales in a sample of Finnish healthcare students.

DESIGN

Psychometrical testing of the Tc2 and LCP scales.

METHODS

A sample consisting of 1133 healthcare students completed the further developed Tc2 scale and the new LCP scale between January and June 2020. Statistical data analysis was performed using explorative factor analysis and Cronbach's alpha coefficient. The STROBE checklist for cross-sectional studies was applied in reporting.

RESULTS

Both the further developed Tc2 and the new LCP scales were shown to be valid and reliable. Explorative factor analysis confirmed a two-factor solution for the Tc2 scale with a 76.3% explained variance and a one-factor solution for the LCP scale with a 58.0% explained variance. The scales' internal consistency reliabilities were high (0.9).

Nurses' Roles in mHealth App Development: Scoping Review

BACKGROUND

Although mobile health (mHealth) apps for both health consumers and health care providers are increasingly common, their implementation is frequently unsuccessful when there is a misalignment between the needs of the user and the app's functionality. Nurses are well positioned to help address this challenge. However, nurses' engagement in mHealth app development remains unclear.

OBJECTIVE

This scoping review aims to determine the extent of the evidence of the role of nurses in app development, delineate developmental phases in which nurses are involved, and to characterize the type of mHealth apps nurses are involved in developing.

METHODS

We conducted a scoping review following the 6-stage methodology. We searched 14 databases to identify publications on the role of nurses in mHealth app development and hand searched the reference lists of relevant publications. Two independent researchers performed all screening and data extraction, and a third reviewer resolved any discrepancies. Data were synthesized and grouped by the Software Development Life Cycle phase, and the app functionality was described using the IMS Institute for Healthcare Informatics functionality scoring system.

RESULTS

The screening process resulted in 157 publications being included in our analysis. Nurses were involved in mHealth app development across all stages of the Software Development Life Cycle but most frequently participated in design and prototyping, requirements gathering, and testing. Nurses most often played the role of evaluators, followed by subject matter experts. Nurses infrequently participated in software development or planning, and participation as patient advocates, research experts, or nurse informaticists was rare.

CONCLUSIONS

Although nurses were represented throughout the preimplementation development process, nurses' involvement was concentrated in specific phases and roles.

Design, Development, and Evaluation of the Blood Collection Management Workstation

Purpose

To design and develop a blood collection management workstation with high usability to reduce the risk of preanalytical errors and improve patient safety.

Methods

A five-phase mobile application development lifecycle model (MADLC) and experience-based co-design (EBCD) were used for design and development. Subsequently, the blood collection management workstation was evaluated using the Chinese System Usability Scale (SUS) in a general ward setting from January to June 2021.

Results

It was used on 2593 in-patients who underwent phlebotomy with 12,378 tubes being labeled. The rate of errors and meantime for blood sampling were decreased compared with the same period in the previous year. A total of 14 nurses agreed to participate in the evaluation, and the overall raw SUS score was 69.26 ± 10.39, which indicated above average results.

Conclusion

The blood collection management workstation has shown the potential to decrease errors and improve working efficiency in a clinical setting. The study also identified some weaknesses, which will be amended in the future.

Usability Methods and Attributes Reported in Usability Studies of Mobile Apps for Health Care Education: Scoping Review

BACKGROUND

Mobile devices can provide extendable learning environments in higher education and motivate students to engage in adaptive and collaborative learning. Developers must design mobile apps that are practical, effective, and easy to use, and usability testing is essential for understanding how mobile apps meet users' needs. No previous reviews have investigated the usability of mobile apps developed for health care education.

OBJECTIVE

The aim of this scoping review is to identify usability methods and attributes in usability studies of mobile apps for health care education.

METHODS

A comprehensive search was carried out in 10 databases, reference lists, and gray literature. Studies were included if they dealt with health care students and usability of mobile apps for learning. Frequencies and percentages were used to present the nominal data, together with tables and graphical illustrations. Examples include a figure of the study selection process, an illustration of the frequency of inquiry usability evaluation and data collection methods, and an overview of the distribution of the identified usability attributes. We followed the Arksey and O'Malley framework for scoping reviews.

RESULTS

Our scoping review collated 88 articles involving 98 studies, mainly related to medical and nursing students. The studies were conducted from 22 countries and were published between 2008 and 2021. Field testing was the main usability experiment used, and the usability evaluation methods were either inquiry-based or based on user testing. Inquiry methods were predominantly used: 1-group design (46/98, 47%), control group design (12/98, 12%), randomized controlled trials (12/98, 12%), mixed methods (12/98, 12%), and qualitative methods (11/98, 11%). User testing methods applied were all think aloud (5/98, 5%). A total of 17 usability attributes were identified; of these, satisfaction, usefulness, ease of use, learning performance, and learnability were reported most frequently. The most frequently used data collection method was questionnaires (83/98, 85%), but only 19% (19/98) of studies used a psychometrically tested usability questionnaire. Other data collection methods included focus group interviews, knowledge and task performance testing, and user data collected from apps, interviews, written qualitative reflections, and observations. Most of the included studies used more than one data collection method.

CONCLUSIONS

Experimental designs were the most commonly used methods for evaluating usability, and most studies used field testing. Questionnaires were frequently used for data collection, although few studies used psychometrically tested questionnaires. The usability attributes identified most often were satisfaction, usefulness, and ease of use. The results indicate that combining different usability evaluation methods, incorporating both subjective and objective usability measures, and specifying which usability attributes to test seem advantageous. The results can support the planning and conduct of future usability studies for the advancement of mobile learning apps in health care education.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/19072.

The nurse teacher's pedagogical cooperation with students, the clinical learning environment and supervision in clinical practicum: a European cross-sectional study of graduating nursing students

BACKGROUND

A supportive clinical practicum experience may enhance the successful transition and socialization to working life of graduating nursing students. Nurse teachers have the main responsibility of supporting and guiding nursing students with their pedagogical expertise during the students' clinical practicum. Thus, the clinical role of nurse teachers is seen as an essential part of a high-quality clinical practicum. Nursing students appreciate the nurse teacher's cooperation with students, but it is often reported to be unattainable. The aim of this study was to explore and compare graduating nursing students' experiences of the nurse teacher's pedagogical cooperation with students, the clinical learning environment and supervision in their final clinical practicum, and to analyze factors associated with these experiences in six European countries.

METHODS

A cross-sectional comparative international survey design was used. The modified Clinical Learning Environment, Supervision and Nurse Teacher (CLES+T) Scale, with a new subscale measuring the nurse teacher's pedagogical cooperation with students, was used. A convenience sample of graduating nursing students in Finland, Germany, Iceland, Ireland, Lithuania and Spain completed the online survey in 2018-2019. The data were analyzed using a Chi-Square test, Pearson's correlation coefficients, and linear models.

RESULTS

A total of 1796 (response rate 49%) nursing students completed the survey. Overall, students had positive experiences of the nurse teacher's pedagogical cooperation, the clinical learning environment and supervision in their final clinical practicum. Students in Spain had the most positive experiences. Educational background factors appeared to be associated with the students' experiences of the nurse teacher's pedagogical cooperation with students, the clinical learning environment and supervision. The relationships between the subscale Nurse teacher's pedagogical cooperation with students and the Clinical Learning Environment and Supervision Scale were perceived as weak to strong depending on the country.

CONCLUSIONS

This study reveals that nurse teachers play an essential role in supporting and guiding nursing students' final clinical practicum. In this light, researchers, educators, and leaders should collaborate seamlessly between educational institutions and healthcare organizations to establish the nurse teachers' pedagogical cooperation role within the clinical learning environment.

Promoting the health and wellbeing of children: A feasibility study of a digital tool among professionals

The foundations of children’s health and wellbeing are laid in early childhood. A gamified app (EmpowerKids tool) was designed to support professionals to have discussions with 6- to 12-year-olds from low-income families about their health and wellbeing. The aim of this feasibility study was to evaluate the usability and acceptability of the tool from the perspective of professionals in social, health and education settings. The study was conducted using a one-group post-test-only design. The usability data were collected using System Usability Scale and the acceptability data were collected using an open-ended questionnaire distributed to professionals (n = 24) in Estonia, Finland and Latvia. The data were collected during two phases. The tool was modified further on the basis of the results. The total usability scores were 82/100 (first testing) and 84/100 (second testing), indicating excellent usability. The answers related to acceptability were divided into four categories: suitability for the context; satisfaction and quality; attractiveness; modification needs. The professionals perceived that the tool helped them to build an overall picture of a child’s health and wellbeing, and to gain information about the child’s individual needs. The requirements for modification detected during the first testing were mostly related to difficulties with textual expressions and graphics. No major modification requirements were expressed during the second testing. The tool is considered feasible and may be used by professionals from different settings to support children’s health and wellbeing. Further studies are needed to evaluate the effectiveness of the tool from the perspective of child outcomes.

Usability Evaluation by Nurses of a Knowledge-Based System of Care Plans for People Hospitalized by COVID-19

The aim of this study is to analyze the usability by nurses of the Knowledge-Based System "Diagnostics care for COVID-19." A convenience sample of 16 nurses was selected, among hospital workers and external experts. The group was divided into three subgroups intentionally to obtain different usability perspectives. Usability was evaluated by the System Usability Scale questionnaire. The participants completed the questionnaire on general usability, data inputs, and information output, after completing a minimum of 12 care plans. The first subgroup used real cases and the "think aloud" technique, the second simulated cases from the same hospital, and the third subgroup performed the external simulation. The highest scores were obtained in data inputs (94.38-97.50); and the lowest, in general usability (90.00-95.00). The subgroup of external experts scored the highest (93.13-95.63), and the first subgroup, which carried out real cases, gave the lowest score (90.00-94.38). The "think aloud" technique found an improvement in including more diagnoses and being able to carry out several plans for one person at the same time. The usability obtained was "excellent" in all subgroups and questionnaires, although the application showed limitations related to its characteristics imposed in the requirements specification.

Human Connection and Technology Connectivity: A Systematic Review of Available Telehealth Survey Instruments

CONTEXT

The current upsurge in telehealth use in palliative and hospice care warrants consideration of patient, family caregiver, and interdisciplinary palliative perspectives on telehealth modality and communication experiences. Currently, telehealth experiences and encounters are being described but not yet extensively evaluated by palliative care teams.

OBJECTIVES

To locate survey instruments available to assess telehealth interactions, to determine the content and constructs covered by the available instruments, and to describe the patient populations previously surveyed by the existing instruments.

METHODS

This study and its reporting followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines with the protocol registered in The International Prospective Register of Systematic Reviews. Three databases were searched with over 3100 articles analyzed for use of a telehealth survey instrument.

RESULTS

Twelve telehealth communication assessment instruments were identified with a mean length of 20 questions, primarily Likert-scale responses with one inclusive of free text and one qualitative inquiry survey. Three inquired only into modality, four queried communication, and five studied both modality and communication experience. Existing telehealth survey instruments are unidirectional in exploring patient or family experience, with two inclusive of provider perspectives. Participant demographics are notably underreported in telehealth experience studies with a frank lack of diversity in ethnic/racial, geographic, age, educational, and income representativeness in current telehealth survey instrument respondents.

CONCLUSION

Palliative care teams may consider familiarity with telehealth survey instrument as an essential component to progress from description of telehealth use to evaluation of telehealth encounters. Current survey instrument outcome reports do not represent inclusivity or diversity, although telehealth is now being clinically applied across settings.