Development and Evaluation of a Component Level Implementation Fidelity Rating System for the GenerationPMTO Intervention

The Effectiveness of GenerationPMTO During Sustained Implementation in the Public Mental Health System: A Single-Arm Open Trial Evaluation

To support families and reduce the burden of child mental, emotional, and behavioral problems, evidence-based parenting interventions must remain effective in real-world service delivery contexts. The purpose of the current study was to evaluate the effectiveness of the GenerationPMTO (GenPMTO) intervention during sustained implementation in the Michigan public mental health system using a single-arm open trial (pre-post) design. We also examined potential predictors of treatment response. A multilevel analysis framework was utilized to evaluate data from 365 caregivers who completed GenPMTO treatment. Results revealed significant positive improvements from pretest to posttest in all four outcome domains (i.e., parenting confidence, parenting practices, caregiver depressive symptoms, child behavior problems). When compared to group-based GenPMTO delivery, the individual delivery format was associated with significantly greater improvements in overall effective parenting practices, as well as in the subdomain of skill encouragement. Caregiver gender, caregiver educational level, and child age were all implicated as predictors of GenPMTO outcomes. These findings add to the literature by supporting the effectiveness of the GenPMTO intervention when fully integrated into mental health care practice and can inform continued efforts to provide families with evidence-based services in community settings.

Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation

BACKGROUND

Implementing evidence-based programs in community service settings introduces the challenge of ensuring sustained fidelity to the original program. We employ a fidelity measure based on direct observation of practitioners' competence and adherence to the evidence-based parenting program (EBPP) GenerationPMTO following installation in national and international sites. Fidelity monitoring is crucial, especially when the program purveyor transfers administration of the program to the community as was done in this case. In previous studies, the Fidelity of Implementation rating system (FIMP) was used to evaluate practitioners' fidelity to the GenerationPMTO intervention in six countries following implementation showing high levels of adherence up to 17 years post certification. Other studies showed FIMP to have predictive validity. The present study provides inter-rater reliability data for this fidelity tool across teams of the purveyor, Implementation Sciences International, Inc./ISII, and national and international sites over a five-year period.

METHODS

Data assess inter-rater reliability in terms of percent agreement and intraclass correlation (ICC) for the purveyor's two fidelity teams and the fidelity teams in seven implementation sites.

RESULTS

Results report stable good to excellent levels of inter-rater reliability and ICCs as well as good attendance at fidelity meetings for all fidelity teams.

CONCLUSIONS

This observational method of assessing fidelity post implementation is a promising approach to enable EBPPs to be transferred safely from purveyors to communities while maintaining reliable fidelity to the intervention.

Investigating active ingredients of the GenerationPMTO intervention: Predictors of postintervention change trajectories in parenting practices

Empirically determining the active ingredients of evidence-based parenting interventions is a promising means for strengthening interventions and enhancing their public health impact. This study aimed to determine which distinct ingredients of the GenerationPMTO (GenPMTO) intervention were associated with subsequent changes in parenting practices. Using a sample of 153 participants randomly assigned to the GenPMTO condition, we employed multilevel modeling to identify intervention ingredients empirically linked with change trajectories in parenting practices observed across the 2 years following intervention exposure. Coercive parenting and positive parenting outcomes were examined. Study results indicated that emotion regulation, effective communication, problem solving, and monitoring each demonstrated a significant pattern of findings for coercive parenting. Differential exposure to each of these ingredients significantly predicted the level of coercive parenting immediately postintervention and/or trajectories of change in coercive parenting across the subsequent 2-year period, controlling for coercive parenting at baseline. No significant predictors were found for positive parenting trajectories. Our findings suggest four components as active ingredients of the GenPMTO intervention for coercive parenting. Identification of these active ingredients may lead to strengthening future iterations of GenPMTO by expanding the set of core components specified in the model, which may further improve public health benefits. Implications for further understanding change stemming from evidence-based parenting interventions are also discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).