Adolescent Depression Curriculum Impact on Pediatric Residents' Knowledge and Confidence to Diagnose and Manage Depression

What Type of Self-Assessment Is Best for Your Educational Activity? A Review of Pre-Post, Now-Then, and Post-Only Designs

In medical education, learner self-assessments are standard methods used to evaluate the impact of curricula and workshops. Due to the subjective nature of self-assessment, these measures are prone to known biases including framing, recall, social desirability, and response-shift bias. These biases can contribute to floor and ceiling effects of measurement, which can lead to false conclusions about whether the intended learning objectives were achieved. Ideal assessments of skills-based educational activities would include standardized tests and structured observations of learners demonstrating skill use before and after the educational intervention. However, educators often lack the necessary resources, time, and expertise to routinely conduct these appraisals and rely on self-assessment as a pragmatic approach to obtaining curriculum feedback and evaluation data. In this review, we describe three common designs for self-assessments: the pre-post, now-then, and post-only designs. We then give recommendations for choosing between each design to minimize bias. The choice of the best design is based on alignments with four considerations: (1) the educational objectives (e.g., demonstrate skill competency and/or change in skill level); (2) participants' prior experience and shared understanding of levels of skill performance; (3) the nature of the educational activity; and (4) available resources. For each design, we review strengths, weaknesses, and known biases and discuss examples to highlight trade-offs between options. We also discuss the use of control groups and follow-up surveys to measure retention over time as additional methods to address bias and related confounding. The guidance presented here is intended to raise educators' awareness of common pitfalls in self-assessment; minimize the impact of known biases when possible; provide evidence, examples, and rationales for optimal design choices; and increase the rigor of self-assessment evaluations.

Effectiveness of a Curriculum for Pediatric Residents to Address Adolescent Nonsuicidal Self-Injury

OBJECTIVE

The present study assessed changes in pediatric resident competence in addressing adolescent nonsuicidal self-injury (NSSI) following a brief, single-session didactic training during the educational unit in adolescent medicine. This study also evaluated knowledge acquisition and retention over time, overall helpfulness of the training, and changes in the perception of how important residents believe that physicians receive training in NSSI.

METHODS

Pediatric residents participated in a single-session NSSI curriculum delivered as part of a monthlong clinical rotation from August 2016 to February 2019. Participants completed pretest, posttest, and 6-month follow-up surveys, and t tests and analyses of variance with post hoc comparisons evaluated changes in variables across the three time points.

RESULTS

One hundred twenty-seven of 162 eligible residents participated (78.4% response rate), and 69 completed the 6-month follow-up (54.3% retention rate). Prior to the NSSI didactic, 107 (85.6%) reported feeling not at all competent or only somewhat competent in responding to patients with a history of NSSI. Residents' self-perceived competence increased significantly (p < .001) from pretest to posttest and was sustained longitudinally. A measure of objective knowledge also showed retention over time. Residents were significantly (p < .001) more likely after the didactic to believe that training in NSSI for physicians is very important compared to before the training.

CONCLUSIONS

Following NSSI training, residents demonstrated increased knowledge and subjective competence in caring for adolescents who self-injure. It is feasible and important to integrate a brief, single-session NSSI curriculum into pediatric residency programs, such as in the adolescent medicine rotation.

Evaluating the effectiveness of a single-day simulation-based program in psychiatry for medical students: a controlled study

BACKGROUND

Training in psychiatry requires specific knowledge, attitudes, and skills that are obtainable by simulation, of which the use is only recent and still needs further development. Evidence is accumulating on its effectiveness but requires further validation for medical students. We aimed to evaluate the effectiveness of a single-day optional teaching program in psychiatry by simulation for medical students and validate a scale measuring Confidence in Psychiatric Clinical Skills (CPCQ), as part of the assessment.

METHODS

This was a controlled study in a French University that compared (using paired-sample Student t-tests) knowledge and attitudes (university grades and CPCQ scores) before, just after teaching with simulated patients, and 2 months later. Satisfaction with the program (including the quality of the debriefing) was also investigated. The CPCQ scale was validated by assessing the factor structure, internal consistency, and test-retest reliability. Finally, a comparison was run with a control group who received the usual psychiatric instruction using covariance analyses.

RESULTS

Twenty-four medical students were included in the simulation group and 76 in the control group. Just after the simulation, knowledge and attitudes increased significantly in the simulation group. Satisfaction with the training and debriefing was very high. The CPCQ scale showed good psychometric properties: a single-factor structure, acceptable internal consistency (α = 0.73 [0.65-0.85]), and good test-retest reliability (ICC = 0.71 [0.35-0.88]). Two months after the simulation, knowledge and attitudes were significantly higher in the simulation group than the control group, despite a lack of difference in knowledge before the simulation.

CONCLUSIONS

Adding a simulation program in psychiatry to the usual teaching improved the knowledge and confidence of medical students. The CPCQ scale could be used for the evaluation of educational programs.

Factor Analysis of the Prediction of the Postpartum Depression Screening Scale

Postpartum depression (PPD), a severe form of clinical depression, is a serious social problem. Fortunately, most women with PPD are likely to recover if the symptoms are recognized and treated promptly. We designed two test data and six classifiers based on 586 questionnaires collected from a county in North Carolina from 2002 to 2005. We used the C4.5 decision tree (DT) algorithm to form decision trees to predict the degree of PPD. Our study established the roles of attributes of the Postpartum Depression Screening Scale (PDSS), and devised the rules for classifying PPD using factor analysis based on the participants’ scores on the PDSS questionnaires. The six classifiers discard the use of PDSS Total and Short Total and make extensive use of demographic attributes contained in the PDSS questionnaires. Our research provided some insightful results. When using the short form to detect PPD, demographic information can be instructive. An analysis of the decision trees established the preferred sequence of attributes of the short form of PDSS. The most important attribute set was determined, which should make PPD prediction more efficient. Our research hopes to improve early recognition of PPD, especially when information or time is limited, and help mothers obtain timely professional medical diagnosis and follow-up treatments to minimize the harm to families and societies.