Using cloud-based mobile technology for assessment of competencies among medical students

Using mobile technology in assessment of entrustable professional activities in undergraduate medical education

BACKGROUND

The adoption of competency-based medical education requires objective assessments of a learner's capability to carry out clinical tasks within workplace-based learning settings. This study involved an evaluation of the use of mobile technology to record entrustable professional activity assessments in an undergraduate clerkship curriculum.

APPROACH

A paper-based form was adapted to a mobile platform called eClinic Card. Students documented workplace-based assessments throughout core clerkship and preceptors confirmed accuracy via mobile phones. Assessment scores for the 2017-2018 academic year were collated and analyzed for all core rotations, and preceptors and students were surveyed regarding the mobile assessment experience.

EVALUATION

The mobile system enabled 80 students and 624 preceptors to document 6850 assessment submissions across 47 clinical sites over a 48-week core clerkship curriculum. Students' scores demonstrated progressive improvement across all entrustable professional activities with stage-appropriate levels of independence reported by end of core clerkship. Preceptors and students were satisfied with ease of use and dependability of the mobile assessment platform; however, students felt quality of formative coaching feedback could be improved.

REFLECTION

Our preliminary evaluation suggests the use of mobile technology to assess entrustable professional activity achievement across a core clerkship curriculum is a feasible and acceptable modality for workplace-based assessment. The use of mobile technology supported a programmatic assessment approach. However, meaningful coaching feedback, as well as faculty development and support, emerged as key factors influencing successful adoption and usage of entrustable professional activities within an undergraduate medical curriculum.

Assessment Tools for Feedback and Entrustment Decisions in the Clinical Workplace: A Systematic Review

BACKGROUND: Entrustable Professional Activities (EPAs) combine feedback and evaluation with a permission to act under a specified level of supervision and the possibility to schedule learners for clinical service. This literature review aims to identify workplace-based assessment tools that indicate progression toward unsupervised practice, suitable for entrustment decisions and feedback to learners. METHODS: A systematic search was performed in the PubMed, Embase, ERIC, and PsycINFO databases. Based on title/abstract and full text, articles were selected using predetermined inclusion and exclusion criteria. Information on workplace-based assessment tools was extracted using data coding sheets. The methodological quality of studies was assessed using the medical education research study quality instrument (MERSQI). RESULTS: The search yielded 6,371 articles (180 were evaluated in full text). In total, 80 articles were included, identifying 67 assessment tools. Only a few studies explicitly mentioned assessment tools used as a resource for entrustment decisions. Validity evidence was frequently reported, and the MERSQI score was 10.0 on average. CONCLUSIONS: Many workplace-based assessment tools were identified that potentially support learners with feedback on their development and support supervisors with providing feedback. As expected, only few articles referred to entrustment decisions. Nevertheless, the existing tools or the principals could be used for entrustment decisions, supervision level, or autonomy.

A digital peer-to-peer learning platform for clinical skills development

BACKGROUND

Due to constraints in time and resources, medical curricula may not provide adequate opportunities for pre-clerkship students to practice clinical skills. To address this, medical students at the University of Alberta developed a digital peer-to-peer learning initiative. The initiative assessed if students can learn clinical skills from their peers in co-curricular practice objective structured clinical exams (OSCEs).

METHODS

A total of 144 first-year medical students participated. Students wrote case scenarios that were reviewed by physicians. Students enacted the cases in practice OSCEs, acting as the patient, physician, and evaluator. Verbal and electronic evaluations were completed. A digital platform was used to automate the process. Surveys were disseminated to assess student perceptions of their experience.

RESULTS

Seventy-five percent of participants said they needed opportunities to practice patient histories and physical exams in addition to those provided in the medical school curriculum. All participants agreed that the co-curricular practice OSCEs met this need. The majority of participants also agreed that the digital platform was efficient and easy to use.

CONCLUSION

Students found the practice OSCEs and digital platform effective for learning clinical skills. Thus, peer-to-peer learning and computer automation can be useful adjuncts to traditional medical curricula.

Medical Student Perceptions of Learner-Initiated Feedback Using a Mobile Web Application

Feedback, especially timely, specific, and actionable feedback, frequently does not occur. Efforts to better understand methods to improve the effectiveness of feedback are an important area of educational research. This study represents preliminary work as part of a plan to investigate the perceptions of a student-driven system to request feedback from faculty using a mobile device and Web-based application. We hypothesize that medical students will perceive learner-initiated, timely feedback to be an essential component of clinical education. Furthermore, we predict that students will recognize the use of a mobile device and Web application to be an advantageous and effective method when requesting feedback from supervising physicians. Focus group data from 18 students enrolled in a 4-week anesthesia clerkship revealed the following themes: (1) students often have to solicit feedback, (2) timely feedback is perceived as being advantageous, (3) feedback from faculty is perceived to be more effective, (4) requesting feedback from faculty physicians poses challenges, (5) the decision to request feedback may be influenced by the student's clinical performance, and (6) using a mobile device and Web application may not guarantee timely feedback. Students perceived using a mobile Web-based application to initiate feedback from supervising physicians to be a valuable method of assessment. However, challenges and barriers were identified.

Ready to work or not quite? Self-perception of practical skills among medical students from Serbia ahead of graduation

Aim

To assess final year medical students’ self-perception of their practical skills.

Methods

The study was conducted at the Faculty of Medicine in Belgrade during compulsory practical sessions in the period December 2-9, 2013 and 390 students agreed to participate (response rate 77.8%). The questionnaire included questions on demographic characteristics, 21 questions on students' self-perception of their practical skills, and 1 question on students’ self-perceived readiness to start working with patients.

Results

Cronbach’s α for the entire scale was 0.891. Students felt most confident about measuring arterial pulse and blood pressure and taking patients' history (average score 10 for all three skills) and least confident about placing a urinary catheter (average score 1) and suturing a wound (average score 2). They rated their readiness to work with patients with 5.0 out of 10.0 points. The total score did not correlate with students’ average mark (Spearman's ρ = 0.039; P = 0.460) and the average mark did not correlate with the self-perceived readiness to work with patients (Spearman's ρ = -0.048; P = 0.365).

Conclusion

Our study suggests that medical students lack confidence to perform various clinical procedures, particularly those related to surgical interventions. To improve students’ confidence, clinical curriculum should include either more hours of practical work or ensure closer supervision of practical training in wards.

Use of mobile learning technology among final year medical students in Kenya

Introduction

Mobile phone penetration has increased exponentially over the last decade as has its application in nearly all spheres of life including health and medical education. This study aimed at assessing the use of mobile learning technology and its challenges among final year undergraduate students in the College of Health sciences, University of Nairobi.

Methods

This was a cross-sectional descriptive study conducted among final year undergraduate students at the University of Nairobi, College of Health Sciences. Self-administered, anonymous questionnaires were issued to all final year students in their lecture rooms after obtaining informed consent. Data on demographics, mobile device ownership and mobile learning technology use and its challenges was collected. Data entry and analysis was done using SPSS^®. Chi-square and t-test were used for bivariate analysis.

Results

We had 292 respondents; 62% were medical students, 16% were nursing students, 13% were pharmacy students and 9% were dental surgery students. The majority were female (59%) and the average age was 24 years. Eighty eight percent (88%) of the respondents owned a smart device and nearly all of them used it for learning. 64% of the respondents used medical mobile applications. The main challenges were lack of a smart device, lack of technical know-how in accessing or using apps, sub-optimal internet access, cost of acquiring apps and limited device memory.

Conclusion

Mobile learning is increasingly popular among medical students and should be leveraged in promoting access and quality of medical education.

A scoping review of cloud computing in healthcare

BACKGROUND

Cloud computing is a recent and fast growing area of development in healthcare. Ubiquitous, on-demand access to virtually endless resources in combination with a pay-per-use model allow for new ways of developing, delivering and using services. Cloud computing is often used in an "OMICS-context", e.g. for computing in genomics, proteomics and molecular medicine, while other field of application still seem to be underrepresented. Thus, the objective of this scoping review was to identify the current state and hot topics in research on cloud computing in healthcare beyond this traditional domain.

METHODS

MEDLINE was searched in July 2013 and in December 2014 for publications containing the terms "cloud computing" and "cloud-based". Each journal and conference article was categorized and summarized independently by two researchers who consolidated their findings.

RESULTS

102 publications have been analyzed and 6 main topics have been found: telemedicine/teleconsultation, medical imaging, public health and patient self-management, hospital management and information systems, therapy, and secondary use of data. Commonly used features are broad network access for sharing and accessing data and rapid elasticity to dynamically adapt to computing demands. Eight articles favor the pay-for-use characteristics of cloud-based services avoiding upfront investments. Nevertheless, while 22 articles present very general potentials of cloud computing in the medical domain and 66 articles describe conceptual or prototypic projects, only 14 articles report from successful implementations. Further, in many articles cloud computing is seen as an analogy to internet-/web-based data sharing and the characteristics of the particular cloud computing approach are unfortunately not really illustrated.

CONCLUSIONS

Even though cloud computing in healthcare is of growing interest only few successful implementations yet exist and many papers just use the term "cloud" synonymously for "using virtual machines" or "web-based" with no described benefit of the cloud paradigm. The biggest threat to the adoption in the healthcare domain is caused by involving external cloud partners: many issues of data safety and security are still to be solved. Until then, cloud computing is favored more for singular, individual features such as elasticity, pay-per-use and broad network access, rather than as cloud paradigm on its own.

A real-time mobile web-based module promotes bidirectional feedback and improves evaluations of the surgery clerkship

BACKGROUND

We implemented a real-time mobile web-based reporting module for students in our surgery clerkship and evaluated its effect on student satisfaction and perceived abuse.

METHODS

Third-year medical students in the surgery clerkship received surveys regarding intimidation, perceived abuse, satisfaction with clerkship resources, and interest in a surgical career. Survey data were analyzed to assess differences after implementing the mobile reporting system and to identify independent predictors of perceived abuse.

RESULTS

With the reporting module, students perceived less intimidation by residents (P < .001) and by faculty (P = .008), greater satisfaction reporting feedback (P < .001), and greater interest in surgical careers (P = .003). Perceived abuse decreased without reaching statistical significance (P = .331). High ratings of intimidation by faculty independently predicted perceived abuse (odds ratio = 1.3), and satisfaction with anonymous reporting was a negative predictor (odds ratio = .2).

CONCLUSIONS

A mobile web-based system for real-time reporting fosters open communication and bidirectional feedback and promotes greater satisfaction with the surgery clerkship and interest in a surgical career.