Direct observation of procedural skills (DOPS) evaluation method: Systematic review of evidence

The Effect of Direct Observation of Procedural Skills on Learning and Satisfaction of Students in Community Dentistry Course

Introduction

Direct Observation of Practical Skills (DOPS) tests is a valuable method for clinical assessment. This study aimed to implement the DOPS test to assess the procedural skills of community dentistry courses and its effects on mastery learning and satisfaction of professors and students at Tabriz faculty of dentistry in 2021-2022.

Methods

In a quasi-experimental study, 60 dentistry students of a class were assigned into two study (n=30) and control (n=30) groups by Permuted block randomization. In the case group, the skills were related to Fluoride therapy, fissure sealant therapy, and health education evaluated by DOPS. In the control group, these skills were evaluated by traditional evaluation methods. Each test was repeated three times. Finally, the satisfaction of students in the case group was assessed by a questionnaire. The chi-square test was used to compare qualitative variables. Repeated measure ANOVA test was used to compare the mean scores in three stages and two groups. P value less than 0.05 was considered significant. Data were analyzed using SPSS 16 software.

Results

A significant difference in the mean score of Fluoride therapy, pit and fissure sealant therapy, and health education was seen between the case and control groups (P<0.001). Also a significant increase in these skills in the third stage of assessment in the case group was observed (P<0.001). The professors and students' satisfaction was considerably high on the DOPS test.

Conclusion

The DOPS method had more impact on Fluoride therapy, pit and fissure sealant therapy, and health education's learning process in dentistry students than the conventional evaluation. The professors and students' satisfaction level was high regarding DOPS. The advantages of the DOPS method are student-centeredness, objectivity, and appropriate feedback.

Application of direct observation of operational skills in nursing skill evaluation of pressure injury: A randomized clinical trial

This was a non-blinded, single-centre, randomized, controlled clinical trial that compared the effectiveness of direct observation of procedural skills (DOPSs)with traditional assessment methods in pressure injury (PI) care skills. The study population included 82 nursing professionals randomly assigned to the study group (n = 41) and the control group (n = 41). Both groups of nurses underwent a 6-month training in PI care skills and were subsequently evaluated. The main outcome variables were the PI skill operation scores and theoretical scores. Secondary outcome variables included satisfaction and critical thinking abilities. Independent sample t-tests and chi-square tests were used to assess differences between the two groups of nurses. The results showed no statistically significant difference in PI skill operation scores between the two groups of nurses (p > 0.05). When comparing the PI theoretical scores, the study group scored higher than the control group, and this difference was statistically significant (p < 0.05). In terms of satisfaction assessment, the study group and the control group showed differences in improving self-directed learning, enhancing communication skills with patients, improving learning outcomes and increasing flexibility in clinical application (p < 0.05). When comparing critical thinking abilities between the two groups of nurses, there was no statistically significant difference at the beginning of the training, but after 3 months following the training, there was a statistically significant difference between the two groups (p < 0.01).The results indicated that the DOPS was effective in improving PI theoretical scores, increasing nurse satisfaction with the training and enhancing critical thinking abilities among nurses.

Self-perceived competence in managing obstetric emergencies among recently graduated physicians from Lima, Peru

OBJECTIVE

This study aimed to examine the self-perception of competencies in obstetric emergencies among recently graduated physicians from universities in Lima, Peru; and to identify its associated factors.

METHODS

An analytical study was conducted, with the study population comprising newly graduated doctors who attended the "VI SERUMS National Convention" in 2017. We used Poisson regressions to assess the factors associated with the self-perception of competencies in obstetric emergencies, calculating prevalence ratios (PR) and their 95% confidence intervals (95% CI).

RESULTS

We analyzed a population of 463 newly graduated physicians (mean age: 25.9 years), of which 33.3% reported feeling competent in obstetric emergencies. In the adjusted analyses, we found that having a previous health career (PR: 1.77, 95% CI: 1.12-2.81), having completed the internship in EsSalud hospitals (PR: 1.48, 95% CI: 1.31-1.68), and completing a university externship (PR: 1.62, 95% CI: 1.34-1.96) were associated with a higher prevalence of self-perceived competence in obstetric emergencies.

CONCLUSION

Our findings suggest that certain academic factors, such as completing an externship and internship in specific hospital settings, may enhance the competencies or competence self-perception of recently graduated physicians in obstetric emergencies. Further studies are needed to confirm these results and identify other factors that may impact physicians' competencies in this field.

Assessing operative competence in core surgical training: A reliability analysis

BACKGROUND

This study quantifies the number of observations required to reliably assess the operative competence of Core Surgical Trainees (CSTs) in Ireland, using the Supervised Structured Assessment of Operative Performance (SSAOP) tool.

METHODS

SSAOPs (April 2016-February 2021) were analysed across a mix of undifferentiated procedures, as well as for three commonly performed general surgery procedures in CST: appendicectomy, abdominal wall hernia repair, and skin/subcutaneous lesion excision. Generalizability and Decision studies determined the number of observations required to achieve dependability indices ≥0.8, appropriate for use in high-stakes assessment.

RESULTS

A total of 2,294 SSAOPs were analysed. Four assessors, each observing 10 cases, can generate scores sufficiently reliable for use in high-stakes assessments. Focusing on a selection of core procedures yields more favourable reliability indices.

CONCLUSION

Trainers should conduct repeated assessments across a smaller number of procedures to improve reliability. Programs should increase the assessor mix to yield sufficient dependability indices for high-stakes assessment.

Designing and implementing an OMMID midwifery professional competence mixed test: A multimethod, multiphasic study

The assessment of professional competence is essential to ensure the achievement of competence standards. The goal of this study is to design and implement a professional competence test model to make the test unified, comprehensive, and fair. This research was a multimethod, multiphasic study. The first qualitative phase of the nominal group technique was conducted to design the test model with specific guidelines. The second phase was a mixed-method parallel field trial conducted on 161 senior midwifery bachelor students in universities of Iran. The test was conducted following the traditional method in the control group and OMMID model in the intervention group. This model consists of three parts: the MCQs (multiple choice questions), OSCE (objective structured clinical examination), and clinically-oriented tests, which included Mini-CEX (mini-clinical evaluation exercise) and DOPS (direct observation of procedural skills). Data were collected using questionnaires and interviews, and the results were assessed qualitatively and quantitatively. The OMMID model and the associated guideline were designed. Qualitative data analysis resulted in six main themes, including organizing, structure, tension due to change, fairness, unification, and outcome. The OMMID model did not increase students' anxiety and stress and did not decrease their satisfaction. The merits of this model include having a centralized guideline, using multiple evaluation methods, comprehensive evaluation of necessary skills, promotion of fairness, and increased student satisfaction.

Perception of the Use of the Direct Observation of Procedural Skills in Occupational Therapy Postgraduate Year Training in Taiwan: Survey of the Perspectives of Trainees and Supervisors

Introduction

Assessment of clinical competence is a significant part of the training for young occupational therapists (OTs). Objective and systematic assessment allows both supervisors and trainees to be aware of the training objectives and monitor the progress. The direct observation of procedural skills (DOPS) is a work-based assessment to evaluate professional knowledge, skills, and attitude in clinical training. This study investigated the perspectives of OT educators and trainees on using DOPS and their discrepancy for OT postgraduate year (PGY) training.

Methods

This study used a quantitative online survey. Eighty-six supervisors and 41 trainees of OT PGY training programs from 95 hospitals returned the questionnaire (a 90.5% return rate), and 64 supervisors and 30 trainees who used DOPS were analyzed. Outcomes included the practicality in using the DOPS in clinical settings, the ease of rating the DOPS, and advantages and the disadvantages of the DOPS.

Results

Most respondents reported that completing one DOPS required at least 11 minutes for direct observation (11-40 minutes: teacher 92.2%; trainee 80.6%). Most respondents (teacher 96.9%, trainee 96.8%) had feedback after direct observation of DOPS, and about half of the feedback assessments took 5 to 10 minutes (teacher 53.1%, trainee 48.4%). Most OT educators and trainees agreed that clinical resources were sufficient and that DOPS matched with OT training goals, benefited OT competence training, and had a fair, objective, and consistent scoring system. Significantly higher percentages of OT trainees felt stressed in and satisfied with the DOPS assessment than trainers. Differences between teachers and trainees regarding easiness of rating DOPS items were not significant.

Conclusion

Most OT educators and trainees agreed that DOPS was a practical and appropriate assessment for OT PGY training.

Development and Integration of DOPS as Formative Tests in Head and Neck Ultrasound Education: Proof of Concept Study for Exploration of Perceptions

In Germany, progress assessments in head and neck ultrasonography training have been carried out mainly theoretically and lack standardisation. Thus, quality assurance and comparisons between certified courses from various course providers are difficult. This study aimed to develop and integrate a direct observation of procedural skills (DOPS) in head and neck ultrasound education and explore the perceptions of both participants and examiners. Five DOPS tests oriented towards assessing basic skills were developed for certified head and neck ultrasound courses on national standards. DOPS tests were completed by 76 participants from basic and advanced ultrasound courses (n = 168 documented DOPS tests) and evaluated using a 7-point Likert scale. Ten examiners performed and evaluated the DOPS after detailed training. The variables of "general aspects" (6.0 Scale Points (SP) vs. 5.9 SP; p = 0.71), "test atmosphere" (6.3 SP vs. 6.4 SP; p = 0.92), and "test task setting" (6.2 SP vs. 5.9 SP; p = 0.12) were positively evaluated by all participants and examiners. There were no significant differences between a basic and advanced course in relation to the overall results of DOPS tests (p = 0.81). Regardless of the courses, there were significant differences in the total number of points achieved between individual DOPS tests. DOPS tests are accepted by participants and examiners as an assessment tool in head and neck ultrasound education. In view of the trend toward "competence-based" teaching, this type of test format should be applied and validated in the future.

Perception and Satisfaction of Undergraduate Medical Students of the Mini Clinical Evaluation Exercise Implementation in Orthopedic Outpatient Setting

PURPOSE

The Mini Clinical Evaluation Exercise (mini-CEX) is a brief and direct observational assessment of trainee-patient interactions that helps to assess several clinical domains. There is limited evidence of mini-CEX implementation in orthopedics and undergraduate perceptions toward such an approach. This study investigated the perception of mini-CEX among undergraduate medical students through a questionnaire-based survey in an orthopedic outpatient setting.

PATIENTS AND METHODS

Undergraduate medical students completing their orthopedic clinical posting were invited to participate in an anonymous, self-administered questionnaire written in English to evaluate their perceptions toward mini-CEX implementation in the orthopedic outpatient setting for the 2016-2017 academic session. The questionnaire comprised 28 closed-ended questions with a five-point Likert rating-scale, and five open-ended questions. The survey responses were analyzed for reliability, validity, and quantitative and qualitative analyses.

RESULTS

A total of 350 students completed the questionnaire; the questionnaire was proven to be valid and reliable. The closed-ended questions were designed to assess the knowledge of the mini-CEX as an assessment tool. The participants demonstrated a satisfactory understanding of the mini-CEX methodology, purpose, clarity, comprehensiveness, and as a self-assessment tool for undergraduate medical students. Instructor support for the implementation of mini-CEX appeared inadequate and was rated with non-confidence among most students. Most participants appreciated better clinical skills, which was reflected through improvements in clinical exam preparation, the Objective Structured Clinical Examination, and clinical judgment.

CONCLUSION

Undergraduate medical students perceived the mini-CEX as an effective tool for clinical teaching in an outpatient orthopedic setting. However, most students indicated suboptimal instructor involvement in the teaching and assessment process; this raises concerns regarding inadequate direct observation and limited feedback for student performance. Additional measures are needed to ensure high quality clinical encounters, teacher training, integration with other assessment tools, and standardized coverage mini-CEX implementation in orthopedics.

The Competency-based Evaluation of Educational Crew of Dental Faculty’s Obstacles in Institutionalizing Performance Assessments

Aims:

This study evaluates the educational crew of dental faculty’s lived experiences facing obstacles and requirements in institutionalizing performance assessments to implement a professional competency-based evaluation system.

Background:

The competency-based evaluation of learning and teaching processes has been adopted as a key policy in the developed world, which indicates the achievement rate of educational goals and the quality of education.

Objective:

The main objective of this study was to evaluate obstacles in institutionalizing performance assessments for the educational crew.

Methods:

This qualitative study used a semi-structured interview in a focus group discussion. The experience of the educational crew regarding the obstacles of using performance assessments and their approaches to conducting a professional competency-based evaluation was assessed. The recruited participants were educational supervisors, professors of orthodontics and prosthodontics, and the medical education department and evaluation committee members of the faculty of dentistry at the University of Tabriz. The purposive sampling technique was used and continued until reaching saturation. Five focus group discussions were conducted with fourteen educational crew and three medical education department members. The data were analysed using thematic analysis.

Results:

The interview analysis results yielded 450 codes in three general categories, including “current condition of clinical education,” “obstacles of implementing new evaluation methods,” and “requirements for effective evaluation of clinical skills.” According to the results, changes in evaluation methods are necessary to respond to community needs. There are also many cultural problems with applying western models in developing countries.

Conclusion:

The medical community should be directed towards a competency-based curriculum, especially in procedure-based fields, such as dentistry.

Other:

They are moving towards altering traditional evaluation methods (the traditional classroom-based lectures). This paradigm change requires support from the department and the provision of infrastructure.

The impact of the evaluations made by Mini-CEX on the clinical competency of nursing students

INTRODUCTION

Clinical competency is defined as one's capabilities in terms of knowledge, attitude and practice, and is a necessity for nursing practice. Evaluation is the inseparable part of the education process, without which the quality of education could not be measured. Considering the characteristics of clinical education and its impact on the clinical competency of students, as well as the importance of making precise evaluations of them using scientific, modern and efficient approaches, this study aims to investigate the impact of the evaluation made using Mini-Clinical Evaluation Exercise on the clinical competency of the nursing students of the School of Nursing and Midwifery Shahid Behesti University.

METHODS

This is a quasi-experimental study with a control group conducted on students who were taking courses "Nursing care for a sick child". The students were classified into the intervention and control groups using complete enumeration. The evaluations were made using Mini-Clinical Evaluation Exercise in the intervention group and the portfolio approach in the control group. The skills regarding patient (mother-child) education, IV therapy and medication were evaluated by checklists.

FINDINGS

The results showed that the mean score of clinical competency in the intervention group was significantly higher than that of the control group.

CONCLUSION

Considering the improvement of clinical competency in the intervention group, it is recommended to use Mini-Clinical Evaluation Exercise for the evaluation of students.

Comparison of the effect of 360° versus two-dimensional virtual reality video on history taking and physical examination skills learning among undergraduate medical students: a randomized controlled trial

Before caring for patients, video instruction is commonly used for undergraduate medical students, and 360° virtual reality (VR) videos have gained increasing interest in clinical medical education. Therefore, the effect of immersive 360° VR video learning compared with two-dimensional (2D) VR video learning in clinical skills acquisition should be evaluated. This randomized, intervention-controlled clinical trial was aimed to assess whether immersive 360° VR video improves undergraduate medical students' learning effectiveness and reduces the cognitive load in history taking and physical examination (H&P) training. From May 1 2018 to October 30 2018, 64 senior undergraduate medical students in a tertiary academic hospital were randomized to receive a 10-min immersive 360° (360° VR video group; n = 32) or 2D VR instructional video (2D VR video group; n = 32), including essential knowledge and competency of H&P. The demographic characteristics of the two groups were comparable for age, sex, and cognitive style. The total procedure skill score, physical examination score, learner's satisfaction score, and total cognitive load in the 360° VR video group were significantly higher than those in the 2D VR video group (effect sizes [95% confidence interval]: 0.72 [0.21-1.22], 0.63 [0.12-1.13], 0.56 [0.06-1.06], and 0.53 [0.03-1.03], respectively). This study suggested that a10-minute 360° VR video instruction helped undergraduate medical students perform fundamental H&P skills as effectively as 2D VR video. Furthermore, the 360° VR video might result in significantly better procedural metrics of physical examinations with higher learner satisfaction despite the higher cognitive load.

Supplementary Information

The online version contains supplementary material available at 10.1007/s10055-022-00664-0.

To evaluate the effect of small-group demonstration-based training module in improving clinical skills in undergraduate medical students

BACKGROUND

In today's modern era, the two most important aspects of medical education are evaluation- and outcome-based learning. Directly Observed Procedural Skills (DOPSs ) is a well-known method of evaluation which constitutes a direct observation of the practical skills being performed by the student and simultaneous written feedback by the teacher.

METHOD

A total of 40 undergraduate students were taught by a module based on DOPS. A pre-test and post-test was conducted on DOPS examination pattern and was compared by the Cochran's Q test.

RESULTS

In this study, a total of 40 medical undergraduate students and 10 teachers participated. Each student was given one pre-test and five post-test with ultrasound proven mild to moderate splenomegaly. Each student was individually assessed on a module based on DOPS and was given direct feedback by the teachers. A total of 34 (85%) students strongly agreed that they felt comfortable and confident with this methodology.

CONCLUSION

The results of this study revealed that DOPS tests can be used as an effective evaluation method to assess medical students because of its appropriate validity and reliability, positive impact on learning, and high satisfaction level amongst students. However, special attention needs to be given to the quality of these tests.

Clinical nursing competency assessment: a scoping review

Objective

To ensure that only competent graduates are licensed to practice nursing, councils conduct licensing examinations, which may include among others clinical competency assessment. This review explored current practices in clinical competency assessment of nursing students as part of a larger study aimed at developing an evidence-based, context-specific framework for clinical competency assessment in a sub-Saharan African (SSA) country.

Methods

A scoping guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) was conducted.

Results

Findings from 28 out of 1151 studies identified from Scopus, PubMed, CINAHL, Wiley Online Library, and ProQuest were included and synthesized. Results show that a good assessment system must be valid, reliable, transparent, feasible, fair, objective, and must provide feedback and continually improve to have an educational impact. Clinical competency assessment systems must be developed on sound empirical evidence, pilot tested, and involve thorough training and evaluation of the examiners. Continuous evaluation of the assessment system is also essential to ensure the quality and relevance of the assessment system. Only one of the included studies was conducted in Africa.

Conclusions

The paucity of clinical competency assessment research in sub-Saharan Africa may lead to benchmarking assessment systems on research conducted outside the context. Sub-Saharan Africa has a set of circumstances that demand a context-specific clinical competency assessment framework to guide clinical competency assessment.

The effect of positive psychology intervention on quality of life among women with unintended pregnancy

BACKGROUND

Studies have shown that quality of life in women with unintended pregnancy is significantly less than the ones with wanted gestation. Therefore, this study was aimed to determine the effect of intervention based on the positive psychology interventions on women's QoL with unintended pregnancies in Kermanshah, Iran.

MATERIALS AND METHODS

In this randomized clinical trial, 40 women with unintended pregnancy met the eligibility criteria and were randomized into experimental (n = 20) or control (n = 20) groups. The Short-form 36-item Questionnaire (SF-36) was administered at pretest, post-test, and six weeks after the intervention. Positive psychology interventions were performed in each session once a week for 10 weeks, with a duration of 90 minutes. The Friedman test and Mann-Whitney U-test were used to analyze QoL and all eight domains for within-group and between-group comparisons, respectively. The level of confidence was set at 0.05 significant.

RESULTS

Distribution of matched variables was not significantly different between the two trial groups. Over the intervention period, the mean of total scores of QoL and seven dimensions showed significant improvement among the experimental group, whereas in the control group, it decreased significantly (P < 0.05). In addition, in the posttest and follow-up stages, the mean score of six dimensions of QoL was higher than the control group (P < 0.05).

CONCLUSION

According to the results of the present study, unintended pregnancy has side effects on QoL in women. Positive counseling interventions can improve the quality of life in women with unintended pregnancy.

Introduction of direct observation of procedural skills (DOPS) as a formative assessment tool during postgraduate training in anaesthesiology: Exploration of perceptions

Background and Aims:

Formative assessment of procedural skills of Anaesthesiology postgraduate (PG) students is not conducted conventionally. Direct observation of procedural skills (DOPS) helps to identify gaps in performance and provides structured feedback. The present study was taken to explore perceptions of PG students and faculty about DOPS.

Methods:

This mixed design interventional study was conducted on 12 PGs and 10 faculty members in Department of Anaesthesiology. After conducting DOPS, a pre-validated questionnaire was given to them about perceptions using 3-point Likert's scale along with open ended questions. Statistical analysis was done using descriptive statistics of perception to calculate percentages and themes were identified for qualitative data.

Results:

Responses of students were positive about skill improvement (83%), time provided (75%), feedback (100%), interaction (83%), motivation (83%), satisfaction (83%), effectiveness (83%) and opportunity creation (92%). Faculty responded positively regarding change in attitude (100%), effectiveness (100%), scope (90%), feasible application (90%), ease (90%), opportunity creation (80%), gap identification (100%), satisfaction (80%). However, 60% felt training was required, 50% thought more time and commitment was required. Themes identified were DOPS is comprehensive, interactive, student-friendly, good teaching-learning tool, identifies gaps, provides focus for learning, provides systematic constructive feedback, improves skills, prepares for future practice, requires planning, may not reflect competence, has assessor variability and can be included in PG curriculum.

Conclusion:

DOPS was perceived as an effective assessment and teaching-learning tool by PG students as well as faculty.

Determinants of Nursing Students’ Confidence in Peripheral Intravenous Catheter Insertion and Management

BACKGROUND: Peripheral intravenous catheter (PIVC) is one of the medical devices commonly inserted in hospitalized patients, both adults and pediatrics. PIVC has crucial functions for delivering drugs, fluids, blood transfusions, and diagnostic tests for patients. Thus, nursing students must be confident in terms of insertion and management of this device. However, studies assessing nursing students’ confidence and its determinants are still limited. AIM: This study aims to examine the internship nursing student’s confidence in PIVC insertion and management in adult patients and its contributing factors. METHODS: A cross-sectional study was conducted to measure internship nursing students’ confidence in PIVC cannulation and management in adult patients as well as its contributing factors. Purposive samples of 100 nursing students in Yogyakarta were recruited, and a 5-point Likert scale questionnaire consisting of 19 questions was used. Kruskal–Wallis test was utilized to investigate the association of the internship nursing student confidence on PIVC insertion and maintenance with the factors. A general linear regression analysis was performed to obtain adjusted estimates of the potential factors with students’ confidence. RESULTS: Results indicated that the internship nursing student’s confidence score ranged from 57 to 95, with a mean value of 75 (±8.1). Among determinants of the student’s confidence investigated in this study, t-test analysis showed that the students’ confidence was associated with their participation in expert lecture, bedside teaching, and direct observation of procedural skill assessment of PIVC insertion and care (p < 0.05). The general linear analysis showed that only bedside teaching and interaction of bedside teaching and direct observation procedural skill assessment were significant predictors of the internship nursing student’s confidence on PIVC insertion and care (β = 10.99, 95% confidence interval [CI] 2.00–20.00 and β = 13.15, 95% CI 1.20–25.15, p < 0.05, respectively). CONCLUSION: This result indicated that nursing students need direct simulation and assessment of PIVC insertion and care to the patients to improve their confidence in PIVC insertion and management in adult patients.

The effect of direct observation of procedural skills/mini-clinical evaluation exercise on the satisfaction and clinical skills of nursing students in dialysis

BACKGROUND

Since the purpose of medicine science is health promotion, education of this group is very important. The use of new evaluation methods is one of the first educational needs. Given that many conventional clinical evaluation methods are not able to fully evaluation students in the clinical settings and only evaluate limited information, This study is designed to evaluate the impact of direct observation of procedural skills (DOPS) and mini-clinical evaluation exercise (mini-CEX) on nursing students and their clinical satisfaction skills.

MATERIALS AND METHODS

This quasi-experimental study was conducted in 2018 to evaluate the dialysis unit of the 6^th semester nursing students of Babol University of Medical Sciences in Iran. Samples were selected by the census method and then random allocation. Data collection was performed using two questionnaires and two checklists to evaluate the clinical skills and satisfaction of the two groups of testing and control. Moreover, the data were analysed with using the SPSS software (version 18, IBM SPSS Inc., Chicago, IL, USA) through the descriptive and analytical statistics (Chi-square test and t-test).

RESULTS

The results showed that the intervention group compared with the control evaluation score of 13/73 ± 2/44, 11/74 ± 2/43, P < 0.002, respectively. Furthermore, the mean score of satisfaction of the intervention group with the DOPS method compared to the traditional method of the control group was 72/50 ± 7/31, 63/48 ± 9/31, P < 0.001, respectively. As well as, the mean score of satisfaction with the mini-CEX method of the intervention group compared to the traditional method of the control group was 73/28 ± 7/78, 63/48 ± 9/31, P < 0.002, respectively.

CONCLUSIONS

The results of this study showed that students who were evaluated by DOPS and mini-CEX methods had a higher score of clinical performance evaluation and higher level of satisfaction. Nursing professors can improve the learning process and the satisfaction of nursing students by using direct feedback on such evaluation methods.

A study of cardiopulmonary resuscitation literacy among the personnel of universities of medical sciences based in Kermanshah and Khuzestan provinces based on the latest 2015 cardiopulmonary resuscitation guidelines

INTRODUCTION

Cardiopulmonary resuscitation (CPR) is regarded as the most important skill of the medical staff who is required to be aware of the latest changes to the CPR guidelines so that they can take the most effective actions in the critical conditions of CPR. Therefore, the present study aimed to determine the levels of CPR literacy among the personnel of universities of medical sciences based in Kermanshah and Khuzestan provinces based on the latest 2015 CPR guidelines in 2019.

MATERIALS AND METHODS

In this descriptive, analytical, cross-sectional study, 525 subjects were selected as the sample population using the two-stage cluster sampling. For data collection, a researcher-made questionnaire was used, whose content validity and reliability were confirmed (r = 0.71). The study screened the data received and analyzed valid data set through the ttest and Spearman's correlation coefficient by incorporating SPSS Statistics software version 23.0. In addition, P < 0.05 was considered statistically significant.

RESULTS

The 2015 CPR literacy levels of the samples were as follows: excellent (85 subjects or 16.2%), good (404 subjects or 77%), and average (36 subjects or 6.9%). The results of Pearson's correlation coefficient revealed a weak and inverse relationship between the levels of CPR literacy and the age of samples (r = -0.092) and work experience (-0.029), which were statistically significant. In addition, the results of Mann-Whitney U-test demonstrated that the level of CPR literacy among the personnel of Ahwaz University of Medical Sciences exceeded that among the personnel of Kermanshah University of Medical Sciences (P < 0.001).

CONCLUSION

It is suggested that in retraining the nursing and paramedical personnel, CPR be carried out with more emphasis on the changes introduced in this guideline compared to that in 2010, including esophageal tracheal airway, reasons for the cessation of CPR, intraosseous infusion, and induced hypothermia.

The Effect of Repeated Direct Observation of Procedural Skills (R-DOPS) Assessment Method on the Clinical Skills of Anesthesiology Residents

Background

The ultimate result of patient care is one of the most important outcomes in medical education. Several methods, including the direct observation of procedural skills (DOPS), have been proposed to assess professional competencies in clinical practice.

Objectives

This study aimed to assess the effects of the Repeated DOPS (R-DOPS) method on the performance of procedural skills in anesthesiology residents.

Methods

The procedural skill performance of anesthesiology residents was assessed using a standard DOPS protocol from May to October 2019. Their scores were then objectively recorded, and the satisfaction rates regarding the 2 DOPS exams were assessed.

Results

We found a considerable improvement in anesthesiology residents’ procedural skill performance, especially in the anesthesiology residency curriculum’s basic items. Besides, anesthesiology residents’ satisfaction was significantly improved after the 2nd DOPS.

Conclusions

R-DOPS leads to improved training outcomes, including assessing the procedural skills, time to feedback to trainees, and trainee satisfaction.

Effects of a mobile phone application for graduate nurses to improve central venous catheter care: A randomized controlled trial

AIM

To develop and test a mobile phone application (app) for graduate nurses on the use and care of central venous catheters.

DESIGN

A randomized controlled trial was conducted at a teaching hospital in the central east coast of Taiwan.

METHODS

Recruitment occurred from 1 August 2019 -31 October 2019. All graduates (N = 90) attending a 2-week induction program attended a lecture and completed a 10-item questionnaire on central venous catheter assessment and care at the end of Week 1 (Time 1). Volunteers were then randomly allocated to receive a link to the learning app on their mobile phone (n = 39 Group A) or control condition (N = 40 Group B). One week later, all graduates completed the knowledge assessment (Time 2) and a simulated clinical assessment with a mannequin. Skills were assessed using an 11-item, direct observation of procedural skills form. Only data from consenting participants were analysed.

RESULTS

Compared with controls, nurses receiving the intervention reported significantly better knowledge (t = -7.98, p < .001, CI = 20.9~34.8) and skill scores (t = 2.83, p = .006, CI = 1.14~6.61). More frequent use of the mobile phone app was associated with higher knowledge (r = 0.39, p = .02, CI = 0.11~0.99) and skills (r = 0.42, p = .008, CI = 0.17~1.03).

CONCLUSION

Mobile app instruction for graduate nurses on central venous catheter care increased specific knowledge and skills compared with conventional methods of instruction.

IMPACT STATEMENT

Implementation of mobile phone application technology can be considered a feasible means to proactively provide training and education. Mobile phone apps could be developed for a range of clinical procedures and various settings. Future studies with a larger sample and a longitudinal follow-up are warranted to confirm results.

Teaching adults how to prevent COVID-19 infection by health workers: The application of intervention mapping approach

INTRODUCTION

Education is one of the most important approaches to preventing infectious diseases at the time of the pandemic. The purpose of the study was to develop an intervention-training program using an intervention mapping approach (IMA) to prevent COVID-19 infection in adults at the time of the pandemic by health workers.

MATERIALS AND METHODS

The present study was a study protocol where IMA was used as a planning framework for developing an intervention-training program to prevent COVID-19 infection in adults at the time of the pandemic by health workers in Ardabil city. Six intervention mapping (IM) steps have been described in this protocol. As the first step, needs assessment was performed by reviewing the studies, qualitative evaluation, and interviews. In the second step, the matrix of change objectives was designed from the intersection of performance goals and determinants. Later on, after designing the program and planning the program implementation, the program evaluation plan was developed.

RESULTS

IMA guided us in designing and implementing a control-oriented training program with the participation of the participants along with the definition of outcomes, performance goals and determinants, theoretical methods and practical applications, intervention program, implementation, and step by step assessment.

CONCLUSION

IM is a control-oriented, systematic, participation-based approach to design and implement targeted and on-going health promotion programs based on the needs of the target group at the time of the pandemic by health workers.

Can We Use Peer-Assisted Learning to Teach Basic Surgical Skills?

Background

It is reported that medical students do not receive adequate opportunities to learn surgical skill and are at risk of being unable to perform simple surgical procedures safely. The usefulness of peer-assisted learning (PAL) as a tool to assist in delivering surgical skills training is worth exploring.

Methods

This is a randomised single blinded controlled trial. Fourth-year students from the university’s Surgical Society were asked to volunteer as peer tutors and those in 3rd-year were asked to undertake surgical skills training. A cohort of 35 students were selected and randomised to receive basic surgical skills training conducted either by faculty members or peers. The students’ performance of basic suturing skills was assessed using a checklist, through directly observed procedural skills (DOPS) technique. The assessment was conducted by faculty blinded to the training. Students’ perception to surgical skills training was assessed using a questionnaire survey.

Results

The suturing and knotting skills of students learned from their peers was comparable to that acquired from faculty. The students’ perceived that their peers could conduct surgical skills training similar to their faculty.

Conclusion

PAL approach for basic surgical skills training is as effective as faculty-led training. PAL has the potential to optimise the delivery of surgical skills training in undergraduate medical education.

Education in Academic Emergency Medicine During the COVID-19 Pandemic - Our Experience From an Ongoing Crisis

Background: The COVID-19 pandemic has resulted in the suspension of the entire teaching program at the Medical University of Vienna till the end of the summer semester. As the department that is responsible for emergency medicine teaching, we adapted the program to continue the courses and maintain the learning progress. Our objective is to evaluate the number of courses conducted and report the methods used. Methods: Teaching was measured as credit hours per week (CHW) in accordance with the university's prospectus. One CHW represents 15 academic hours (45 min) in one semester. Webinars were conducted using the CISCO Webex Events®, Webex Training, and ZOOM®. The Moodle® was utilized for resuscitation courses. Results: Courses and clerkships equivalent to 80.2 out of 101.4 CHW (79.1%) could be held during the ongoing crisis in the summer semester. Courses in the winter semester were all completed. In the human medicine curriculum, 73.7 out of 94.9 CHW (77.7%) could be conducted. In the case of emergency lectures for the dentistry curriculum, all courses were conducted through webinars (6.5 CHW, 100%). After calculating the exact number of students in each class, it has been determined that courses and clerkships equivalent to 78.7% could be conducted. Conclusion: Despite the challenge of preparing for the treatment of numerous patients during the ongoing pandemic, we could shoulder a majority of our teaching responsibilities. Although sufficient skill training could not be imparted under these circumstances, we could provide sufficient theoretical knowledge to allow students to continue studies.

A guide to curriculum inquiry for brachytherapy simulation-based medical education

PURPOSE

Brachytherapy is a required clinical competency within radiation oncology training but decreased utilization and volume have limited trainee exposure. Simulation-based medical education is an established educational tool that allows learners to engage in higher-order learning in a safe space and has unique application in brachytherapy instruction.

METHODS AND MATERIALS

We reviewed best practices in curriculum development for simulation-based medical education in brachytherapy, current works in brachytherapy simulation, and identify areas for future development.

RESULTS

The systematic curriculum inquiry method as it relates to brachytherapy was described using the "six-step approach" of problem identification and general needs assessment, targeted needs assessment, goals and objectives, educational strategies, implementation, and evaluation and feedback. Best practices in simulation identified several features of effective learning, including feedback, repetitive practice, and curriculum integration. A review of current simulation-based medical education brachytherapy publications revealed five manuscripts to date with a focus on feasibility across a variety of disease sites, including the prostate, cervical, breast, and head and neck. Potential areas of future development include curricular quality improvement, long-term outcomes, objectives that scale to the learner's competencies, and expansion beyond psychomotor skills.

CONCLUSIONS

Brachytherapy is an essential modality in radiation oncology. Simulation-based medical education provides a powerful opportunity to improve brachytherapy training effectively.

A global snapshot of endourology residency training

BACKGROUND

Urology has become more complex over the last decades with surgical sophisticated technologies such as endoscopy, laparoscopy and robotic surgery. As these minimally invasive methods gain popularity throughout the world, this has led in some countries to a serious training gap as compared to other countries, and between generations of surgeons within national training systems. There is a huge heterogeneity in urological training between countries, whether developed or developing. This paper attempts to shed some light onto global urological training, comparing a significant number of various national systems, and to outline global tendencies in urological training. It will enable interested readers to see where their own system stands in international comparison, and hopefully enable them to identify training needs to achieve global quality standards.

MATERIALS AND METHODS

This is a questionnaire-based assessment which was sent to 240 members of U-merge from 62 countries. In addition, there is ample literature on the requirements of structured training programs and assessments, and we have tried to briefly outline the key points in this paper.

RESULTS

We received responses from 32 countries Urology residency training is hugely heterogenous between countries. Only 44% of nations use a structured training program with assessments. Others use the Halstedian apprenticeship approach. Notably, some developing countries do use modern teaching and assessment methods, whereas some developed countries still use the outmoded apprenticeship model. For the interested reader, results have been tabled in detail, and training systems described country by country.

CONCLUSIONS

Our results have shown a huge heterogeneity in quality urology training between countries and within continents. In systems without national structure of training, it can be assumed that such differences exist even between hospitals/ training institutions. There is no doubt in times of globalization with resident and doctor migration and exchanges that training needs structure and standardization. The still huge gap in developing countries to catch up and be able to afford latest surgical and learning technologies need to be addressed with the help of responsible outreach programs.

Effectiveness of simulation based teaching of ventilatory management among non-anaesthesiology residents to manage COVID 19 pandemic - A Quasi experimental cross sectional pilot study

BACKGROUND AND AIMS

Simulation is one of the important learning tools when it comes to skill acquisition and as a supplemental tool for training in high stake situations like COVID-19. The aim of this study is to meet the global requirements of knowledge on ventilatory management, prepare and to evaluate the effectiveness of the teaching module for non-anesthesiology trainees on COVID-19 patients.

METHODS

Quasi experimental cross sectional pilot study was conducted with a sample of twenty-six trainees. A teaching module was prepared and validated which consisted of lectures, audio-video sessions, demonstrations with hands-on training, debriefing, analytical-phase and reflection. Pre and Post evaluations from student t-test and direct observation of procedural skills (DOPS) were used for knowledge and skill assessment respectively and feedback obtained from Likert's score.

RESULTS

Pre- and Post-tests had a mean score of 7.42 ± 2.12 and 14.92 ± 2.9 respectively (P value 0.00001). DOPS included 16 point score, in which 23 trainees (88.4%) met the expectations and above expectations as per training objectives. A five-point Likert's score feedback revealed satisfactory and highly satisfactory scores of 100% (ABG), 96.1% (mechanical ventilation), and 84.6% (ventilation in COVID-19 patients). Overall satisfaction for the workshop among respondents was 100 per cent. Confidences of handling scores were 84.5% (interpreting ABG), 65.3% (maneuvering mechanical ventilation), and 96.15% (intubation in COVID-19 patients).

CONCLUSION

A planned teaching module in ventilation management helps to train non-anaesthesiologists more effectively as a part of COVID-19 preparedness. Simulation with debriefing based training to the medical fraternity is the best alternative in the present pandemic and it will also ensure the safety of health care professionals.

Clinical procedural skills assessment during internship in ophthalmology

INTRODUCTION

Directly observed procedural skills (DOPS) is a unique method for assessment since it tests the trainee's ability to apply his knowledge and skills in performing a particular procedure and provides an assessment of the practical work performed by the trainee on a 'real' patient under supervision of an experienced faculty. The study aims to make use of DOPS rating for assessment and further improvement in procedural skills in interns in Ophthalmology rotational posting.

METHODS

A prospective study was planned and 15 interns on 2 weeks' rotational posting in the department of Ophthalmology were included by purposive sampling over a duration of 2 months. Four clinical procedural skills were identified and in the second week of posting, the interns were assessed by three DOPS encounters at an interval of 2 days for each clinical skill. The DOPS ratings were analyzed quantitatively using R-statistical software by repeated measure ANOVA and Banfuroni test.

RESULTS

A total of 180 DOPS were undertaken for 15 interns in 4 core areas of ophthalmic examination. The mean overall DOPS rating for DOPS-1 was 3.70±0.82, DOPS-2, 3.83±1.82 and DOPS-3, 4.93±1.65; the difference in DOPS rating between the first and second encounter was not statistically significant (p=0.497), between the second and the third and between the first and the third were statistically significant (p=0.000 in both cases) using Banfuroni test. The overall difference was also statistically significant (p=0.000) using repeated measure ANOVA. Both the assessor and intern satisfaction increased significantly from the first to the third DOPS, but not statistically significant between the first and the second and between the second and the third DOPS.

CONCLUSIONS

We found significant improvement in interns' clinical skills through repeated DOPS and the method was well accepted by both the students and the faculty. Internship period can be well utilized for improving clinical skills and novel performance assessment methods like DOPS might prove to be highly beneficial in ensuring adequacy of learning during internship and also to assess their readiness for accepting professional responsibilities in future.