Assessment of clinical competencies using clinical images and videos "CIVA"

Application of objective structured clinical examination (OSCE) for the evaluation of Kampo medicine training

BACKGROUND

The purpose of this study was to develop an objective, content-valid, and reliable assessment method for Kampo medicine using an objective structured clinical examination (OSCE) for the assessment of clinical competence in Kampo medicine.

METHODS

We developed a blueprint followed by a list of 47 assessment items and three task scenarios related to clinical competence in Kampo medicine. An eight-member test committee checked the relevance of the assessment items on a Likert scale. We calculated a content validity index and content validity ratio, and used the Angoff method to set the passing threshold. We trained a total of nine simulated patients with three assigned to each scenario. We conducted an OSCE for 11 candidates with varying medical abilities, and conducted three stations per person, which were evaluated by one evaluator in one room by direct observation. We used video recordings to test the inter-rater reliability of the three raters. We used the test results to verify the reliability of the assessment chart.

RESULTS

The inter-rater reliability (intraclass correlation coefficient [2,1]) was 0.973. The reliability of the assessment chart for each scenario (Cronbach's α) was 0.86, 0.89, and 0.85 for Scenarios 1, 2, and 3, respectively. The reliability of the assessment chart for the whole OSCE (Cronbach's α) was 0.90.

CONCLUSIONS

We developed a content-valid new OSCE assessment method for Kampo medicine and obtained high inter-rater and test reliabilities. Our findings suggest that this is one of the most reliable evaluation methods for assessing clinical competence in Kampo medicine.

Training Residents in the Early Identification of Anxiety Disorders: Feasibility and Preliminary Efficacy of a Novel Training Program

Pediatric anxiety disorders are highly prevalent, but tend to go undetected as pediatricians often lack relevant training. We developed a brief, video-based training program for pediatric residents aimed at improving early identification of child anxiety disorders. The novel training was completed in a group-based format or via an online, asynchronous training program. Pediatric residents from 2 residency programs (n = 63) participated and completed pre- and posttraining surveys evaluating attitudes about previous training, knowledge about child anxiety, perceived evaluation skills, and responses to clinical vignettes. Most residents (81%) reported they did not receive enough prior training in the presentation of anxiety disorders in young children. Residents' knowledge and perceived evaluation skills increased posttraining. On the vignette-based assessment, residents demonstrated increased sensitivity with regard to interference, diagnosis, and referral urgency. Despite some challenges with participation, results provide preliminary evidence that brief training programs could be an effective way to improve resident education.

The Video Interaction Guidance approach applied to teaching communication skills in dentistry

PURPOSE

To examine dentists' views of a novel video review technique to improve communication skills in complex clinical situations.

MATERIALS AND METHODS

Dentists (n = 3) participated in a video review known as Video Interaction Guidance to encourage more attuned interactions with their patients (n = 4). Part of this process is to identify where dentists and patients reacted positively and effectively. Each dentist was presented with short segments of video footage taken during an appointment with a patient with intellectual disabilities and communication difficulties. Having observed their interactions with patients, dentists were asked to reflect on their communication strategies with the assistance of a trained VIG specialist.

RESULTS

Dentists reflected that their VIG session had been insightful and considered the review process as beneficial to communication skills training in dentistry. They believed that this technique could significantly improve the way dentists interact and communicate with patients. The VIG sessions increased their awareness of the communication strategies they use with their patients and were perceived as neither uncomfortable nor threatening.

DISCUSSION

The VIG session was beneficial in this exploratory investigation because the dentists could identify when their interactions were most effective. Awareness of their non-verbal communication strategies and the need to adopt these behaviours frequently were identified as key benefits of this training approach. One dentist suggested that the video review method was supportive because it was undertaken by a behavioural scientist rather than a professional counterpart.

CONCLUSION

Some evidence supports the VIG approach in this specialist area of communication skills and dental training.