Translating evidence-based parenting programs for primary care: Stakeholder recommendations for sustainable implementation

Predictors of the sustainability for an evidence-based eating disorder prevention program delivered by college peer educators

BACKGROUND

Despite ongoing efforts to introduce evidence-based interventions (EBIs) into mental health care settings, little research has focused on the sustainability of EBIs in these settings. College campuses are a natural place to intervene with young adults who are at high risk for mental health disorders, including eating disorders. The current study tested the effect of three levels of implementation support on the sustainability of an evidence-based group eating disorder prevention program, the Body Project, delivered by peer educators. We also tested whether intervention, contextual, or implementation process factors predicted sustainability.

METHODS

We recruited 63 colleges with peer educator programs and randomly assigned them to (a) receive a 2-day Train-the-Trainer (TTT) training in which peer educators were trained to implement the Body Project and supervisors were taught how to train future peer educators (TTT), (b) TTT training plus a technical assistance (TA) workshop (TTT + TA), or (c) TTT plus the TA workshop and quality assurance (QA) consultations over 1-year (TTT + TA + QA). We tested whether implementation support strategies, perceived characteristics of the intervention and attitudes towards evidence-based interventions at baseline and the proportion of completed implementation activities during the implementation year predicted three school-level dichotomous sustainability outcomes (offering Body Project groups, training peer educators, training supervisors) over the subsequent two-year sustainability period using logistic regression models.

RESULTS

Implementation support strategies did not significantly predict any sustainability outcomes, although a trend suggested that colleges randomized to the TTT + TA + QA strategy were more likely to train new supervisors (OR = 5.46, 95% CI [0.89-33.38]). Colleges that completed a greater proportion of implementation activities were more likely to offer Body Project groups (OR = 1.53, 95% CI [1.19-1.98]) and train new peer educators during the sustainability phase (OR = 1.39, 95% CI [1.10-1.74]). Perceived positive characteristics of the Body Project predicted training new peer educators (OR = 18.42, 95% CI [1.48-299.66]), which may be critical for sustainability in routine settings with high provider turnover.

CONCLUSIONS

Helping schools complete more implementation activities and increasing the perceived positive characteristics of a prevention program may result in greater sustainment of prevention program implementation.

TRIAL REGISTRATION

This study was preregistered on 12/07/17 with ClinicalTrials.gov, ID NCT03409809, https://clinicaltrials.gov/ct2/show/NCT03409809 .

Implementing Parenting Programs in Primary Care: A Framework and a Call for Action

Healthy parent-child relationships are clearly critical to healthy child development. Parenting programs develop caregivers' skills to support the health and well-being of children. Rigorous evidence has demonstrated the efficacy of these programs. Rising rates of child and youth depression, anxiety, grief, and suicide, both prior to and compounded by the COVID-19 pandemic, provide further reasons to implement parenting programs that support all parents in their essential roles. Parents can act as a buffer to stressors and support for children's well-being when they have the knowledge and skills to do so. Pediatric primary care practices are a natural setting for parenting programs, but challenges, including stigma, technology, workflow issues, and funding, have prevented their broad dissemination, implementation, and sustainability. In this article, we develop a framework for implementing parenting programs in primary care and present key considerations for selecting programs that fit the needs of parents, providers, patients, and the practice. We offer lessons from our experiences in overcoming these challenges, using the updated Consolidated Framework for Implementation Research to structure our discussion. We also provide an initial stepwise process which readers may use to plan their own parenting program implementation. Pediatric clinicians and practices can use this article and associated resources to plan, implement, and evaluate parenting programs in their practices as a strategy to help address the growing youth mental health crisis. Improving parenting behaviors can reduce the need for current or future mental health interventions by supporting optimal child development, emotional regulation, and parent-child relationships.

A budget impact analysis of cost to implement a whole child health focused, family-based intervention in primary care for children with elevated BMI

BACKGROUND

Although the cost of implementing evidence-based interventions (EBIs) is a key determinant of adoption, lack of cost information is widespread. We previously evaluated the cost of preparing to implement Family Check-Up 4 Health (FCU4Health), an individually tailored, evidence-based parenting program that takes a whole child approach, with effects on both behavioral health and health behavior outcomes, in primary care settings. This study estimates the cost of implementation, including preparation.

METHODS

We assessed the cost of FCU4Health across the preparation and implementation phases spanning 32 months and 1 week (October 1, 2016-June 13, 2019) in a type 2 hybrid effectiveness-implementation study. This family-level randomized controlled trial took place in Arizona with n = 113 predominantly low-income, Latino families with children ages > 5.5 to < 13 years. Using electronic cost capture and time-based activity-driven methods, budget impact analysis from the perspective of a future FCU4Health adopting entity-namely, ambulatory pediatric care clinicians-was used to estimate the cost of implementation. Labor costs were based on 2021 Bureau of Labor Statistics Occupational Employment Statistics, NIH-directed salary cap levels or known salaries, plus fringe benefits at a standard rate of 30%. Non-labor costs were based on actual amounts spent from receipts and invoices.

RESULTS

The cost of FCU4Health implementation to 113 families was $268,886 ($2380 per family). Actual per family cost varied widely, as individual tailoring resulted in families receiving a range of 1-15 sessions. The estimated cost of replicating implementation for future sites ranged from $37,636-$72,372 ($333-$641 per family). Using our previously reported preparation costs (i.e., $174,489; $1544 per family), with estimated replication costs of $18,524-$21,836 ($164-$193 per family), the total cost of delivering FCU4Health was $443,375 ($3924 per family), with total estimated replication costs of $56,160-$94,208 ($497-$834 per family).

CONCLUSIONS

This study provides a baseline for costs associated with implementation of an individually tailored parenting program. Results provide critical information for decision makers and a model for future economic analysis and can be used to inform optimization thresholds for implementation and, when necessary, benchmarks for program adaptation to promote scale-up.

TRIAL REGISTRATION

This trial was prospectively registered on January 6, 2017, at ClinicalTrials.gov (NCT03013309).

Baseline Targeted Moderation in a Trial of the Family Check-Up 4 Health: Potential Explanations for Finding Few Practical Effects

Not all participants will benefit equally from even well-established, evidence-based prevention programs. For this reason, the field of prevention science is beginning to embrace individual tailoring of interventions. The Family Check-Up was among the first prevention programs to tailor at the family level as opposed to the more prevalent focus on adapting programs for different cultures, genders, and other immutable participant characteristics. Despite tailoring, families with lower levels of stress and parental mental health issues, children with lower baseline conduct problems, and families living in an extremely deprived neighborhood benefitted less from the Family Check-Up. This study examined baseline targeted moderation (BTM) within a trial of the Family Check-Up 4 Health (FCU4Health) program, an adaptation of the Family Check-Up for primary care delivery and explicit targeting of obesogenic behaviors. Ethnically diverse, low-income families (N = 240) with children ages 5.5 to 12 years identified in pediatric primary care with elevated body mass index (BMI) were enrolled and randomized to FCU4Health or usual care. Few BTM effects were found using single-variable-as-moderator and latent-class-as-moderator analytic approaches across the primary (child BMI, body composition) and secondary outcomes (family health routines; child eating behaviors, food choices, emotional problems, problem behaviors, quality of life; caregiver BMI and body composition), as well as hypothesized mediators (child self-regulation, parenting skills). The high-risk nature of the sample and the FCU4Health being individually tailored might have mitigated finding BTM effects. This trial was prospectively registered (NCT03013309 ClinicalTrials.gov).

A machine learning approach to improve implementation monitoring of family-based preventive interventions in primary care

Background

Evidence-based parenting programs effectively prevent the onset and escalation of child and adolescent behavioral health problems. When programs have been taken to scale, declines in the quality of implementation diminish intervention effects. Gold-standard methods of implementation monitoring are cost-prohibitive and impractical in resource-scarce delivery systems. Technological developments using computational linguistics and machine learning offer an opportunity to assess fidelity in a low burden, timely, and comprehensive manner.

Methods

In this study, we test two natural language processing (NLP) methods [i.e., Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers (BERT)] to assess the delivery of the Family Check-Up 4 Health (FCU4Health) program in a type 2 hybrid effectiveness-implementation trial conducted in primary care settings that serve primarily Latino families. We trained and evaluated models using 116 English and 81 Spanish-language transcripts from the 113 families who initiated FCU4Health services. We evaluated the concurrent validity of the TF-IDF and BERT models using observer ratings of program sessions using the COACH measure of competent adherence. Following the Implementation Cascade model, we assessed predictive validity using multiple indicators of parent engagement, which have been demonstrated to predict improvements in parenting and child outcomes.

Results

Both TF-IDF and BERT ratings were significantly associated with observer ratings and engagement outcomes. Using mean squared error, results demonstrated improvement over baseline for observer ratings from a range of 0.83-1.02 to 0.62-0.76, resulting in an average improvement of 24%. Similarly, results demonstrated improvement over baseline for parent engagement indicators from a range of 0.81-27.3 to 0.62-19.50, resulting in an approximate average improvement of 18%.

Conclusions

These results demonstrate the potential for NLP methods to assess implementation in evidence-based parenting programs delivered at scale. Future directions are presented.

Trial registration

NCT03013309 ClinicalTrials.gov.

Multi-sector stakeholder's perceptions of determinants of successful implementation of a pediatric weight management intervention

Background

Multi-sector stakeholder engagement is essential in the successful implementation, dissemination, and sustainability of pediatric weight management interventions (PWMI), particularly in low-income settings where sustainability relies on external policies and reimbursement. The objective of this study was to engage stakeholders (1) to inform the creation of the intervention with adaptations needed for a successful PWMI in a primary care and community setting and (2) to identify barriers and facilitators to implementation and dissemination.

Methods

We sought to examine the perspectives of local, state, and national clinic and community stakeholders during the pre-implementation period of a two-arm, randomized trial of a Health Weight Clinic PWMI conducted in two health centers and a modified-Healthy Weight and Your Child PWMI at two local YMCAs that serve a predominantly lower income, Hispanic community. The Consolidated Framework for Implementation Research interview guide served as a template for the study but was modified to fit the PWMIs and the various professional roles. Interviews were transcribed and analyzed using the framework analysis approach and themes were linked to the CFIR domains and constructs.

Results

Twenty-six stakeholders perceived the following as needed components of a PWMI: a formal curriculum with illustrative examples, a patient- and family-centered program, group visits, and high-quality multidisciplinary personnel. These findings led to the creation of a group visit curriculum, implementation trainings and cross-site collaborative technical assistance. Additionally, creating partnerships between community and clinical organizations, and addressing patient barriers and unmet social needs (i.e., transportation, food) were identified as facilitators to successful implementation. These results led to the creation of community resource guides, connections to community organizations, and screening and referring for unmet social needs. Perceived facilitators of dissemination included proving cost-effectiveness of the PWMI to inform insurance reimbursement for long-term sustainability. Therefore, we collected cost data and engaged with Medicaid officials to discuss reimbursement.

Conclusion

Findings highlight the importance of engaging multi-sector stakeholders pre-implementation to ensure the components valued are included, ensuring the program minimizes barriers to participation, considering how staff training can improve implementation and how collected outcomes can inform sustainability and dissemination of PWMIs in clinic and community settings.

Don't Get Lost in Translation: Integrating Developmental and Implementation Sciences to Accelerate Real-World Impact on Children's Development, Health, and Wellbeing

Translation of developmental science discoveries is impeded by numerous barriers at different stages of the research-to-practice pipeline. Actualization of the vast potential of the developmental sciences to improve children's health and development in the real world is imperative but has not yet been fully realized. In this commentary, we argue that an integrated developmental-implementation sciences framework will result in a translational mindset essential for accelerating real world impact. We delineate key principles and methods of implementation science of salience to the developmental science audience, lay out a potential synthesis between implementation and developmental sciences, provide an illustration of the Mental Health, Earlier Partnership (MHE-P), and set actionable steps for realization. Blending these approaches along with wide-spread adoption of the translational mindset has transformative potential for population-level impact of developmental science discovery.

Health behaviour outcomes of a family based intervention for paediatric obesity in primary care: A randomized type II hybrid effectiveness-implementation trial

BACKGROUND

Paediatric obesity is a multifaceted public health problem. Family based behavioural interventions are the recommended approach for the prevention of excess weight gain in children and adolescents, yet few have been tested under "real-world" conditions.

OBJECTIVES

To evaluate the effectiveness of a family based intervention, delivered in coordination with paediatric primary care, on child and family health outcomes.

METHODS

A sample of 240 families with racially and ethnically diverse (86% non-White) and predominantly low-income children (49% female) ages 6 to 12 years (M = 9.5 years) with body mass index (BMI) ≥85th percentile for age and gender were identified in paediatric primary care. Participants were randomized to either the Family Check-Up 4 Health (FCU4Health) program (N = 141) or usual care plus information (N = 99). FCU4Health, an assessment-driven individually tailored intervention designed to preempt excess weight gain by improving parenting skills was delivered for 6 months in clinic, at home and in the community. Child BMI and body fat were assessed using a bioelectrical impedance scale and caregiver-reported health behaviours (eg, diet, physical activity and family health routines) were obtained at baseline, 3, 6 and 12 months.

RESULTS

Change in child BMI and percent body fat did not differ by group assignment. Path analysis indicated significant group differences in child health behaviours at 12 months, mediated by improved family health routines at 6 months.

CONCLUSION

The FCU4Health, delivered in coordination with paediatric primary care, significantly impacted child and family health behaviours that are associated with the development and maintenance of paediatric obesity. BMI did not significantly differ.

Motivational Interviewing and Caregiver Engagement in the Family Check-Up 4 Health

Motivational interviewing (MI) is a therapeutic style in which a provider elicits client motivation and helps strengthen commitment to change (Miller and Rollnick 2002). The original Family Check-Up (FCU; Dishion and Stormshak 2007)-and the adapted version for improving health behaviors in primary care, the Family Check-Up 4 Health (FCU4Health; Smith et al. 2018a)-are brief, assessment-driven, and family-centered preventive interventions that use MI to improve parent engagement in services to improve parenting and prevent negative child outcomes. This study examines the role of MI in the Raising Healthy Children project, a randomized trial to test the effectiveness of the FCU4Health for the prevention of obesity in pediatric primary care, with data from the 141 families assigned to receive the FCU4Health. Families were eligible for the study if the child was between 5.5 and 12 years of age at the time of identification and had a BMI ≥ 85th percentile for age and gender at the most recent visit to their primary care provider. MI skills at the first session predicted caregiver in-session active engagement, attendance at follow-up parenting sessions, and improvements in motivation to address child health and behavior goals. Baseline characteristics of the family (i.e., child health diagnosis, caregiver baseline depression, motivation, and Spanish language preference) had differential associations with responsiveness and MI skills. This study has implications for program development, provider training, and fidelity monitoring.

Effects of the Family Check-Up 4 Health on Parenting and Child Behavioral Health: A Randomized Clinical Trial in Primary Care

The Family Check-Up 4 Health (FCU4Health) is an adaptation of the Family Check-Up (FCU) for delivery in primary care settings. While maintaining the original FCU's focus on parenting and child behavioral health, we added content targeting health behaviors. This study evaluated whether the adapted FCU maintained positive effects on parenting (positive behavior support, limit setting, parental warmth) and child behavioral health (self-regulation, conduct problems, emotional problems). Pediatric (6-12 years) primary care patients with a BMI ≥ 85^th%ile (n = 240) were recruited from primary care clinics in Phoenix. Children were 75% Latino, 49% female, and 73% Medicaid recipients. This type 2 effectiveness-implementation hybrid trial compared families randomized to FCU4Health (n = 141) or usual care (n = 99). FCU4Health was delivered over a period of 6 months. This study focuses on a priori secondary outcomes included parenting and child behavioral health targets of the original FCU, assessed at baseline and 3, 6, and 12 months. Significant improvements were found for the FCU4Health condition, compared to usual care, in parenting from baseline to the 3-month assessment [β = .17 (.01; .32)]. Parenting predicted improvements in child self-regulation at 6-months [β = .17 (.03; .30)], which in turn predicted reductions in conduct problems [β = - .38 (- .51; - .23)] and emotional problems [β = - .24 (- .38; - .09)] at 12 months. Ethnicity and language of delivery (English or Spanish) did not moderate these effects. The FCU4Health can improve parenting and child behavioral health outcomes when delivered in primary care.Trial Registration Trial registration number: NCT03013309 ClinicalTrials.gov.

The comprehensive framework for integrated healthcare implementation: a realist evaluation of positive parenting in rural primary care in the US

Purpose

This study aimed to develop a comprehensive framework that encompasses contextual determinants of integrated care implementation. The initial framework was developed based on literature and was refined based on contexts and mechanisms that facilitated or interfered with integrating a pilot parenting intervention, Behavior Checker (BC), into the routine service delivery in a rural primary care clinic in the USA.

Design/methodology/approach

This study was a single organizational case study informed by the realist evaluation methodology. Data collection methods included key informant interviews, healthcare staff surveys, a focus group and direct observation of meetings. Data analysis focused on identifying the context-mechanism-outcome (CMO) frameworks, a heuristic used in a realist evaluation to map pathways of the interactions between program inputs and contextual conditions leading to intended and/or unintended outcomes.

Findings

The identified CMOs and antecedent contexts identified informed the process of revising the initial Comprehensive Framework for Integrated Healthcare Development and Implementation (CF-IHDI). The revised CF-IHDI contained six parent domains of outer setting, basic structures, people and value, intervention characteristics, organizational capabilities for change and key processes.

Research limitations/implications

The CMOs and core characteristics of contextual conditions that activated facilitating mechanisms can inform future studies examining healthcare integration efforts.

Practical implications

The CF-IHDI can guide primary care clinics in identifying factors and strategies to consider when integrating parenting or other psychosocial interventions into primary care routine service delivery.

Originality/value

The CF-IHDI developed in this study contributes to the knowledge of contexts and mechanisms that facilitate and interfere with integrated care implementation.

A Systematic Review of External Validity in Pediatric Integrated Primary Care Trials

OBJECTIVE

We used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework to conduct a systematic review of external validity reporting in integrated primary care (IPC) interventions for mental health concerns.

METHODS

We searched Medline, CINAHL, PsycINFO, the Cochrane Center Register of Controlled Trials, and relevant literature to identify publications from 1998 to 2018 reporting on open, randomized, or quasi-randomized trials of IPC interventions that targeted child (ages 0-18 years) psychological symptoms. For each publication, we extracted the information reported in each RE-AIM domain and calculated the proportion of the total studies reviewed.

RESULTS

Thirty-nine publications describing 25 studies were included in the review. Publications rarely reported some indicators of external validity, including the representativeness of participants (12%), rate of adoption clinics or providers (16%), cost of implementation (8%), or evidence of maintenance (16%). Few studies reported on key pragmatic factors such as cost or organizational change processes related to implementation and maintenance. Strengths of some studies included comparisons of multiple active treatments, use of tailorable interventions, and implementation in "real world" settings.

CONCLUSIONS

Although IPC interventions appear efficacious under research conditions, there are significant knowledge gaps regarding the degree to which they reach and engage target recipients, what factors impact adoption and implementation of IPC interventions by clinicians, how fidelity can be maintained over time, and cost-effectiveness. Pediatric IPC researchers should embrace dissemination and implementation science methods to balance internal and external validity concerns moving forward.

The Family Check-Up 4 Health: Study protocol of a randomized type II hybrid effectiveness-implementation trial in integrated primary care (the healthy communities 4 healthy students study)

BACKGROUND

Parenting interventions like the Family Check-Up have demonstrated effects on child physical and behavioral health outcomes. However, access to these programs is limited, particularly for populations experiencing health disparities. Primary care settings have become recognized as a potential delivery system in which these programs may be implemented at scale. The purpose of this trial is to test the effectiveness of the Family Check-Up 4 Health (FCU4Health) program, an adaptation of the FCU for primary care, and assess program implementation in an integrated primary care setting.

METHODS

We will conduct a hybrid type 2 effectiveness-implementation trial in partnership with a primary care clinic in a low-income, majority Latino community. Families with 2- to 5-year-old children will be eligible to participate. Families will be randomized to receive the intervention (n = 130) or services as usual (n = 70) and will be assessed annually over three years. Outcomes are informed by the RE-AIM framework (i.e., reach, effectiveness, adoption, implementation, and maintenance). Effectiveness outcomes include child health behaviors (e.g., Dietary Screener Questionnaire), behavioral health (e.g., Strengths and Difficulties Questionnaire), and parenting (e.g., Proactive Parenting). Early stage implementation outcomes are also included (e.g., cost, acceptability, appropriateness, and feasibility). Effectiveness outcomes will be assessed via intent-to-treat (ITT) analyses. Implementation outcomes will be primarily descriptive with comparisons to prior trials of FCU4Health and the original FCU.

PROJECTED OUTCOMES

This trial will provide evidence related to the potential of integrated primary care settings to deliver evidence-based preventive interventions with a dual focus on behavioral and physical health.