"Home Practice Is the Program": Parents' Practice of Program Skills as Predictors of Outcomes in the New Beginnings Program Effectiveness Trial

A call for early, effective, and scalable parent education programs for high-conflict separated/divorcing parents: A synthesized perspective from prevention science and family law

Conflict between parents is stressful and disruptive to children living in the midst of parental separation or divorce. Although some level of post-separation/divorce conflict is understandable in an emotionally-charged separation/divorce, it undermines the extent to which parents protect their children from short- and long-term problems. In this article, we weave together a synthesized perspective informed by our respective training and experience in prevention science and family law on the role of parent education programs for high-conflict separating/divorcing parents. To do so, we first describe the research on the effects of high interparental conflict on children's outcomes and then discuss current approaches and challenges to reducing these negative effects by offering parent education programs for high-conflict separating/divorcing parents. Then, we propose and describe a new model for early, effective, and scalable parent education programs with the ultimate goal of protecting children after separation/divorce.

The Prospective Effects of Caregiver Parenting on Behavioral Health Outcomes for Children with Incarcerated Parents: a Family Resilience Perspective

Rates of parental incarceration in the USA have increased dramatically over the past four decades. The Adverse Childhood Experiences study identified parental incarceration as one of several risk factors related to multiple health outcomes during childhood and adulthood. Parents and other caregivers are widely regarded as sources of resilience for children experiencing adversity, yet few studies have examined caregivers' parenting practices as sources of resilience for children with incarcerated parents. This study used secondary data from a longitudinal randomized controlled trial of the prison-based parent management training program Parenting Inside Out (PIO). Specifically, it included 149 caregivers (i.e., the non-incarcerated parent, extended family member, or other adult who provides the day-to-day caretaking of a child during parental incarceration) of children aged 2-14 years whose incarcerated parents were randomly assigned to receive PIO or the control condition. Path analysis was used to examine associations between caregivers' parenting, social support, self-efficacy, and change in child internalizing and externalizing symptoms across a 6-month period. Direct effects of caregivers' parenting were found on improvements in child behavioral health from baseline (conducted when the parent was incarcerated) to the 6-month follow-up (conducted after most parents had been released). Indirect effects were found for caregiver social support and self-efficacy. The findings highlight the importance of caregivers' adaptive parenting as a protective resource for children who experience parental incarceration and have implications for the design of preventive interventions for this underserved population.

Tiered Universal and Targeted Early Childhood Interventions: Enhancing Attendance Across Families with Varying Needs

This study examined whether a two-tiered parenting program, which provides universal primary prevention along with targeted secondary prevention only for families with increased needs, would have mutually beneficial impacts on attendance across two program components. A secondary analysis of the Smart Beginnings (SB) randomized controlled trial was conducted. SB takes place from birth to age 3 and combines universal delivery of the Video Interaction Project (VIP) with targeted delivery of the Family Check-Up (FCU) for families identified as having increased risks following yearly screening. The current study analyzed whether attendance in VIP in the first six months was associated with FCU attendance for eligible families at six months, and whether FCU attendance at six and 18 months was associated with subsequent VIP attendance. Analyses included logistic and mixed-effects Poisson regression, as well as group-based trajectory analysis. VIP attendance predicted later FCU attendance (AOR = 5.43, p < .01), and FCU attendance predicted later VIP attendance (IRR = 1.35, p < .01) and a high-stable VIP attendance trajectory (AOR=14.98, p < .01). Findings provide strong support for the ability of tiered models to engage parents, to promote effective and efficient service delivery to reduce disparities in school readiness, and their potential to overcome common barriers to attendance and scaling by addressing the heterogeneity of risk among low-income families.

A machine learning approach to improve implementation monitoring of family-based preventive interventions in primary care

Background

Evidence-based parenting programs effectively prevent the onset and escalation of child and adolescent behavioral health problems. When programs have been taken to scale, declines in the quality of implementation diminish intervention effects. Gold-standard methods of implementation monitoring are cost-prohibitive and impractical in resource-scarce delivery systems. Technological developments using computational linguistics and machine learning offer an opportunity to assess fidelity in a low burden, timely, and comprehensive manner.

Methods

In this study, we test two natural language processing (NLP) methods [i.e., Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers (BERT)] to assess the delivery of the Family Check-Up 4 Health (FCU4Health) program in a type 2 hybrid effectiveness-implementation trial conducted in primary care settings that serve primarily Latino families. We trained and evaluated models using 116 English and 81 Spanish-language transcripts from the 113 families who initiated FCU4Health services. We evaluated the concurrent validity of the TF-IDF and BERT models using observer ratings of program sessions using the COACH measure of competent adherence. Following the Implementation Cascade model, we assessed predictive validity using multiple indicators of parent engagement, which have been demonstrated to predict improvements in parenting and child outcomes.

Results

Both TF-IDF and BERT ratings were significantly associated with observer ratings and engagement outcomes. Using mean squared error, results demonstrated improvement over baseline for observer ratings from a range of 0.83-1.02 to 0.62-0.76, resulting in an average improvement of 24%. Similarly, results demonstrated improvement over baseline for parent engagement indicators from a range of 0.81-27.3 to 0.62-19.50, resulting in an approximate average improvement of 18%.

Conclusions

These results demonstrate the potential for NLP methods to assess implementation in evidence-based parenting programs delivered at scale. Future directions are presented.

Trial registration

NCT03013309 ClinicalTrials.gov.

Lessons for a COVID-19 era: Barriers and facilitators to court ordered online parenting programs for divorcing and separating parents

Family courts are increasingly interested in online parenting programs for divorcing and separating parents, particularly during the COVID-19 pandemic. To our knowledge, no previous study has evaluated the barriers to and facilitators of parent participation in these programs for family law cases. We interviewed 61 parents in the midst of family law cases regarding their perspectives. While many parents viewed online parent programs positively (e.g., convenient), they also reported barriers to participation (e.g., technology problems). We offer recommendations (e.g., communication about program benefits) to support courts as they decide whether to continue ordering online parent programs following the pandemic.

Caregiver Participation Engagement in Child Mental Health Prevention Programs: a Systematic Review

Prevention programs are a key method to reduce the prevalence and impact of mental health disorders in childhood and adolescence. Caregiver participation engagement (CPE), which includes caregiver participation in sessions as well as follow-through with homework plans, is theorized to be an important component in the effectiveness of these programs. This systematic review aims to (1) describe the terms used to operationalize CPE and the measurement of CPE in prevention programs, (2) identify factors associated with CPE, (3) examine associations between CPE and outcomes, and (4) explore the effects of strategies used to enhance CPE. Thirty-nine articles representing 27 unique projects were reviewed. Articles were included if they examined CPE in a program that focused to some extent on preventing child mental health disorders. There was heterogeneity in both the terms used to describe CPE and the measurement of CPE. The majority of projects focused on assessment of caregiver home practice. There were no clear findings regarding determinants of CPE. With regard to the impact of CPE on program outcomes, higher levels of CPE predicted greater improvements in child and caregiver outcomes, as well as caregiver-child relationship quality. Finally, a small number of studies found that motivational and behavioral strategies (e.g., reinforcement, appointment reminders) were successful in promoting CPE. This review highlights the importance of considering CPE when developing, testing, and implementing prevention programs for child mental health disorders. Increased uniformity is needed in the measurement of CPE to facilitate a better understanding of determinants of CPE. In addition, the field would benefit from further evaluating strategies to increase CPE as a method of increasing the potency of prevention programs.

The Development, Evaluation, and Implementation of Parenting-focused Prevention Programs in Collaboration with Family Court

Promoting the well-being and best interests of children in separated and divorcing families is a shared value among family court professionals and prevention scientists who develop and evaluate intervention programs. This article chronicles the development, evaluation, and implementation of two programs - the New Beginnings Program (NBP), a parenting intervention for separated/divorcing parents and the Family Transitions Guide (FTG), an intervention designed to motivate high conflict separated/divorcing parents to attend the NBP. The development and evaluation of these programs was facilitated by a long-standing collaboration with Maricopa Family Court. We discuss the process of developing these programs, their underlying small theories, and the evaluation of their effects in randomized trials. We also describe our collaboration with the family court and ways that the court promoted the development and evaluation of these programs. Finally, we summarize lessons learned and discuss future directions to bolster the public health impact of evidence-based programs for separated/divorcing families.

Implementation methodology from a social systems informatics and engineering perspective applied to a parenting training program

OBJECTIVE

For implementation of an evidence-based program to be effective, efficient, and equitable across diverse populations, we propose that researchers adopt a systems approach that is often absent in efficacy studies. To this end, we describe how a computer-based monitoring system can support the delivery of the New Beginnings Program (NBP), a parent-focused evidence-based prevention program for divorcing parents.

METHOD

We present NBP from a novel systems approach that incorporates social system informatics and engineering, both necessary when utilizing feedback loops, ubiquitous in implementation research and practice. Examples of two methodological challenges are presented: how to monitor implementation, and how to provide feedback by evaluating system-level changes due to implementation.

RESULTS

We introduce and relate systems concepts to these two methodologic issues that are at the center of implementation methods. We explore how these system-level feedback loops address effectiveness, efficiency, and equity principles. These key principles are provided for designing an automated, low-burden, low-intrusive measurement system to aid fidelity monitoring and feedback that can be used in practice.

DISCUSSION

As the COVID-19 pandemic now demands fewer face-to-face delivery systems, their replacement with more virtual systems for parent training interventions requires constructing new implementation measurement systems based on social system informatics approaches. These approaches include the automatic monitoring of quality and fidelity in parent training interventions. Finally, we present parallels of producing generalizable and local knowledge bridging systems science and engineering method. This comparison improves our understanding of system-level changes, facilitates a program's implementation, and produces knowledge for the field. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

What got in the way? Caregiver-reported challenges to home practice of assigned intervention skills

Background

Evidence-based intervention (EBI) effectiveness is hindered by low rates of caregiver home practice, or caregiver rehearsal of intervention skills at home. Although home practice is essential to intervention success, we know little about what makes home practice difficult to complete. We sought to identify the challenges caregivers face when attempting home practice assignments within the context of community implementation of a family-based preventative EBI. We also aimed to identify barriers, perceived need, and implementation-related factors that may contribute to frequency of home practice challenges.

Methods

The family-based preventative intervention Bridges was delivered to caregivers and adolescents in three public middle schools in North America within a randomized efficacy-effectiveness hybrid trial. Caregivers (N = 233) completed worksheets in which they reported challenges to home practice of intervention skills. Caregiver-endorsed challenges were coded for emergent themes and categorized into higher-order home practice challenge domains. Additional data was gathered through multiple methods. In addition to descriptive analyses, multilevel linear regressions were conducted to identify factors associated with frequency of home practice challenge domains.

Results

Over 85% of caregivers reported one or more home practice challenges. Challenges fell into six domains: daily stresses, cognitions/beliefs about the intervention/practice, caregiver wellness/regulation, adolescent responsiveness, relevance/acceptability of intervention or assignment, and external barriers to practice. Adolescent externalizing symptoms, academic performance, and caregiver participation in the English-language intervention group (compared to Spanish-language delivery) were associated with greater frequency of reported challenges.

Conclusions

Among caregivers already attending intervention sessions, specific support to overcome challenges in daily stresses and beliefs about the intervention/practice can support greater home practice compliance, which in turn may impact overall intervention effectiveness. Findings suggest that providing services in caregivers' preferred language may avert potential challenges. Further research into home practice challenges is necessary to identify appropriate home practice implementation supports.

Plain Language Summary

Low rates and poor quality of caregiver home practice of intervention skills hampers intervention effectiveness, but no prior studies have identified the difficulties caregivers face when attempting home practice of intervention skills and their rates of occurrence. Within a community implementation of a family-based preventative intervention, we found that difficulties were common and largely fell in the domains theorized to impact engagement, such as daily stresses and beliefs about the intervention/practice. Findings also suggest that providing services and materials in caregivers' preferred language may avert potential home practice challenges. By identifying common home practice challenges and correlates of different challenges, our findings point to a need to develop appropriate and effective home practice implementation support.

Sociodemographic and Psychosocial Predictors of VIP Attendance in Smart Beginnings Through 6 Months: Effectively Targeting At-Risk Mothers in Early Visits

Past research on predictors of participation in early childhood parenting programs suggest that families experiencing higher levels of sociodemographic adversity (e.g., younger maternal age, single parenthood, lower income or education) are less likely to participate in parenting programs. This is significant, as it may indicate that those most in need of additional support are the least likely to receive it. Data from a randomized control trial (RCT) of Smart Beginnings, an integrated, tiered model for school readiness, were used to explore predictors of attendance in Video Interaction Project (VIP) through 6 months. VIP is a primary preventive intervention delivered in tandem with pediatric well-child visits, aimed at reducing income-based disparities in early child development through promotion of responsive parent-child interactions. Using Poisson distribution models (N = 403; treatment arm, n = 201), we find that demographic, socioeconomic status (SES), and psychosocial variables are associated with program attendance but not always in the expected direction. While analyses show that first-time mothers have higher levels of program attendance as expected, we find that less-educated mothers and those with lower parenting self-efficacy have higher levels of attendance as well. The latter findings may imply that the VIP intervention is, by some indicators, effectively targeting families who are more challenging to engage and retain. Implications for pediatric-based interventions with population-level accessibility are discussed.

Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings

The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the "New Beginnings Program" serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.

A Population-Level, Randomized Effectiveness Trial of Recruitment Strategies for Parenting Programs in Elementary Schools

Objective: A population-level, randomized controlled trial was conducted to test the effectiveness of a parent recruitment package for increasing initial engagement into a school-based parenting program and to identify strategies responsible for effects.Method: Participants were caregivers of kindergarten- to third-grade students (N = 1,276) attending one of five schools serving ethnically diverse families living in mostly low-income, urban conditions. First, families were randomly assigned to be recruited for research surveys or not, and then to a parenting program recruitment condition: 1) Engagement-as-usual (EAU) informational flyer; 2) EAU + testimonial booklet; 3) EAU + teacher endorsement; 4) EAU + recruitment call; or 5) all strategies (full package). Caregivers were offered a free parenting program at their child's school. Primary dependent variables were parenting program enrollment and attending at least one session (initiation). Exploratory analyses were conducted on program completion, attendance across sessions, homework completion, and in-session participation.Results: In the population-level sample, enrollment and initiation were higher for the full package compared to all other conditions except the recruitment call condition. Enrollment, initiation, and program completion were higher for the recruitment call and full package conditions compared to the EAU condition. In the subsample of initiators, parents in the full package condition attended fewer parenting sessions than in the EAU condition. Controlling for attendance across sessions, there were no condition effects on homework completion or in-session participation.Conclusions: The recruitment call can increase the public health impact of evidence-based parenting programs by improving enrollment, initiation, and program completion.

The Cascading Effects of Multiple Dimensions of Implementation on Program Outcomes: a Test of a Theoretical Model

This study tests a theoretical cascade model in which multiple dimensions of facilitator delivery predict indicators of participant responsiveness, which in turn lead to improvements in targeted program outcomes. An effectiveness trial of the 10-session New Beginnings Program for divorcing families was implemented in partnership with four county-level family courts. This study included 366 families assigned to the intervention condition who attended at least one session. Independent observers provided ratings of program delivery (i.e., fidelity to the curriculum and process quality). Facilitators reported on parent attendance and parents' competence in home practice of program skills. At pretest and posttest, children reported on parenting and parents reported child mental health. We hypothesized effects of quality on attendance, fidelity and attendance on home practice, and home practice on improvements in parenting and child mental health. Structural Equation Modeling with mediation and moderation analyses were used to test these associations. Results indicated quality was significantly associated with attendance, and attendance moderated the effect of fidelity on home practice. Home practice was a significant mediator of the links between fidelity and improvements in parent-child relationship quality and child externalizing and internalizing problems. Findings provide support for fidelity to the curriculum, process quality, attendance, and home practice as valid predictors of program outcomes for mothers and fathers. Future directions for assessing implementation in community settings are discussed.

Predictors of Participation in the Family Check-Up Program: a Randomized Trial of Yearly Services from Age 2 to 10 Years

A key challenge of community-based prevention programs is engaging families in the context of services settings involving children and families. The Family Check-Up (FCU) program is designed to engage families in parenting support appropriate to their level of need by use of assessment-enhanced motivational interviewing. This study involved families screened for risk who were seeking services at women, infant, and children's offices in three geographical regions (N = 731). Families in the randomized intervention group (N = 367) were offered the FCU yearly, from age 2 through 10. The results of multivariate modeling indicated that caregivers reporting high levels of perceived caregiving stress (i.e., depression, low parenting satisfaction, daily hassles) participated at a higher rate in two critical components (feedback and follow-up support interventions) of the FCU program over the 8-year trial period than caregivers reporting lesser degrees of stress. The implications of these findings are discussed in the context of family-centered programs for the prevention of child behavior problems and directions for future research.

Commentary on Participation in Preventive Interventions

In this commentary on the special issue, Preventive Parenting Interventions: Advancing Conceptualizations and Enhancing Reach, operationalization of participation, who engages in prevention interventions, and common themes from the studies are discussed. The papers in the special issue illustrate the importance of studying factors that increase participation in prevention and point out the need for additional research to understand these factors. These papers highlight the diverse conceptualizations of participation as well as a need to move toward standardization of methods to assess related constructs. Research is needed to elucidate the potential of enhanced participation and engagement in interventions to meaningfully impact intervention outcomes and ultimately improve the population-level benefit attributable to prevention efforts. Prevention scientists should attend to factors influencing participation in prevention as early in the intervention research process as possible in order to increase the likelihood that the target population will utilize evidence-based prevention interventions in real-world settings, under real-world conditions.

The Closing Digital Divide: Delivery Modality and Family Attendance in the Pathways for African American Success (PAAS) Program

Although family-focused, evidence-based programs (EBPs) have the potential to reduce disparities in health and behavioral outcomes for youth, access to such programs is severely limited in the most affected areas, including African American communities in the rural South. As expanding the reach of EBPs is the primary goal of translational research, interest is growing in the potential of technology as a viable platform to disseminate services to areas with limited resources. To test whether African American families in the rural South would be willing to engage in a technology-based family-focused EBP to prevent adolescent risk behavior, we examined attendance using data from two arms of a three-arm community-based trial of the Pathways for African American Success (PAAS) program. In the overall study, sixth graders (N = 412) and their primary caregivers were randomly assigned to the following conditions: (a) in-person, small group sessions led by facilitators; (b) self-directed, technology-based sessions; or (c) a literature control with home-mailed educational materials. Results indicated that attendance was higher in the technology condition than in the small group condition. Parental age, education, and socioeconomic status did not limit attendance in the technology condition. We conclude from these results that the use of technology can be an acceptable strategy for disseminating parenting EBPs to African American families in the rural South.

Randomized Effectiveness Trial of the New Beginnings Program for Divorced Families with Children and Adolescents

This study presents findings from a randomized effectiveness trial of the New Beginnings Program (NBP), which has demonstrated efficacy in 2 prior randomized efficacy trials. Family courts in 4 counties facilitated recruitment of divorcing and separating parents, and providers in community agencies delivered the program. Participants were 830 parents of children ages 3-18 who were randomized to receive either the 10-session NBP or an active 2-session comparison condition in which parents learned about the same parenting skills but did not complete home practice of these skills. Parents were ethnically diverse (59.4% non-Hispanic White, 31.4% Hispanic, 9.2% other race or ethnicity). Multiple rater assessments of parenting, interparental conflict, and child mental health problems were conducted at pretest, posttest, and 10-month follow-up. The results indicated positive moderated effects of the NBP as compared with the active control condition to strengthen parenting at posttest and to reduce child mental health problems at posttest and 10 months. Many of these moderated effects showed positive benefits for non-Hispanic White families but not for Hispanic families. The findings indicate support for the effectiveness of the NBP when delivered by community-based agencies but also indicate the need for further adaptations to make the program effective for Hispanic parents.

New Avenues for Promoting Mindfulness in Adolescence using mHealth

There is a large evidence base supporting the efficacy of mindfulness interventions in adulthood, and growing support for the efficacy of these interventions in adolescence. Historically mindfulness interventions have been delivered in person and in groups, with recommendations for home practice being a critical part of the intervention. However, compliance with these practice recommendations in adolescence is very poor. Past studies indicate that using mobile technology to promote skill transfer to real life can be an effective strategy, particularly when used as a supplement to an in-person intervention strategy. To date, however, mobile technology has largely been used to create stand-alone mindfulness interventions. The goals of the current paper are to discuss the potential opportunities and challenges with a mobile-technology-enhanced mindfulness intervention, and to present the rationale that such an approach is not only theoretical and empirically sound but also a critical next step to increase the efficacy and developmental appropriateness of mindfulness interventions for adolescents. This discussion is grounded in a specific example of a mindfulness intervention supplemented by momentary interventions we are developing.

"Scaling-out" evidence-based interventions to new populations or new health care delivery systems

Background

Implementing treatments and interventions with demonstrated effectiveness is critical for improving patient health outcomes at a reduced cost. When an evidence-based intervention (EBI) is implemented with fidelity in a setting that is very similar to the setting wherein it was previously found to be effective, it is reasonable to anticipate similar benefits of that EBI. However, one goal of implementation science is to expand the use of EBIs as broadly as is feasible and appropriate in order to foster the greatest public health impact. When implementing an EBI in a novel setting, or targeting novel populations, one must consider whether there is sufficient justification that the EBI would have similar benefits to those found in earlier trials.

Discussion

In this paper, we introduce a new concept for implementation called “scaling-out” when EBIs are adapted either to new populations or new delivery systems, or both. Using existing external validity theories and multilevel mediation modeling, we provide a logical framework for determining what new empirical evidence is required for an intervention to retain its evidence-based standard in this new context. The motivating questions are whether scale-out can reasonably be expected to produce population-level effectiveness as found in previous studies, and what additional empirical evaluations would be necessary to test for this short of an entirely new effectiveness trial. We present evaluation options for assessing whether scaling-out results in the ultimate health outcome of interest.

Conclusion

In scaling to health or service delivery systems or population/community contexts that are different from the setting where the EBI was originally tested, there are situations where a shorter timeframe of translation is possible. We argue that implementation of an EBI in a moderately different setting or with a different population can sometimes “borrow strength” from evidence of impact in a prior effectiveness trial. The collection of additional empirical data is deemed necessary by the nature and degree of adaptations to the EBI and the context. Our argument in this paper is conceptual, and we propose formal empirical tests of mediational equivalence in a follow-up paper.

Directions for the Advancement of Culturally Adapted Preventive Interventions: Local Adaptations, Engagement, and Sustainability

To advance the implementation and dissemination of culturally adapted interventions to diverse populations, greater attention should be devoted to three underdeveloped topics: (a) local adaptations of interventions when they are implemented in community settings, (b) participant engagement, and (c) the sustainability of adapted interventions. Several typologies have been developed for studying local adaptations, and some research indicates that such adaptations might add to intervention effectiveness. There is suggestive evidence of ethnocultural group disparities in intervention engagement and in the success of efforts to boost engagement. Theory and limited data indicate that interventions' flexibility and fit with organizational culture and resources can be achieved through cultural adaptations. Furthermore, those adaptations should be associated with sustainability, but research has yet to test that hypothesis adequately. Several recommendations are made for advancing culturally adapted interventions through additional research on local adaptations during implementation, the many facets of participant engagement, and sustainability.