Decision-making in the multiphase optimization strategy: Applying decision analysis for intervention value efficiency to optimize an information leaflet to promote key antecedents of medication adherence

Advances in the multiphase optimization strategy (MOST) have suggested a new approach, decision analysis for intervention value efficiency (DAIVE), for selecting an optimized intervention based on the results of a factorial optimization trial. The new approach opens possibilities to select optimized interventions based on multiple valued outcomes. We applied DAIVE to identify an optimized information leaflet intended to support eventual adherence to adjuvant endocrine therapy for women with breast cancer. We used empirical performance data for five candidate leaflet components on three hypothesized antecedents of adherence: beliefs about the medication, objective knowledge about AET, and satisfaction with medication information. Using data from a 25 factorial trial (n = 1603), we applied the following steps: (i) We used Bayesian factorial analysis of variance to estimate main and interaction effects for the five factors on the three outcomes. (ii) We used posterior distributions for main and interaction effects to estimate expected outcomes for each leaflet version (32 total). (iii) We scaled and combined outcomes using a linear value function with predetermined weights indicating the relative importance of outcomes. (iv) We identified the leaflet that maximized the value function as the optimized leaflet, and we systematically varied outcome weights to explore robustness. The optimized leaflet included two candidate components, side-effects, and patient input, set to their higher levels. Selection was generally robust to weight variations consistent with the initial preferences for three outcomes. DAIVE enables selection of optimized interventions with the best-expected performance on multiple outcomes.

Intervention Optimization: A Paradigm Shift and Its Potential Implications for Clinical Psychology

To build a coherent knowledge base about what psychological intervention strategies work, develop interventions that have positive societal impact, and maintain and increase this impact over time, it is necessary to replace the classical treatment package research paradigm. The multiphase optimization strategy (MOST) is an alternative paradigm that integrates ideas from behavioral science, engineering, implementation science, economics, and decision science. MOST enables optimization of interventions to strategically balance effectiveness, affordability, scalability, and efficiency. In this review we provide an overview of MOST, discuss several experimental designs that can be used in intervention optimization, consider how the investigator can use experimental results to select components for inclusion in the optimized intervention, discuss the application of MOST in implementation science, and list future issues in this rapidly evolving field. We highlight the feasibility of adopting this new research paradigm as well as its potential to hasten the progress of psychological intervention science. Expected final online publication date for the Annual Review of Clinical Psychology, Volume 20 is May 2024. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Using decision analysis for intervention value efficiency to select optimized interventions in the multiphase optimization strategy

OBJECTIVE

Optimizing multicomponent behavioral and biobehavioral interventions presents a complex decision problem. To arrive at an intervention that is both effective and readily implementable, it may be necessary to weigh effectiveness against implementability when deciding which components to select for inclusion. Different components may have differential effectiveness on an array of outcome variables. Moreover, different decision-makers will approach this problem with different objectives and preferences. Recent advances in decision-making methodology in the multiphase optimization strategy (MOST) have opened new possibilities for intervention scientists to optimize interventions based on a wide variety of decision-maker preferences, including those that involve multiple outcome variables. In this study, we introduce decision analysis for intervention value efficiency (DAIVE), a decision-making framework for use in MOST that incorporates these new decision-making methods. We apply DAIVE to select optimized interventions based on empirical data from a factorial optimization trial.

METHOD

We define various sets of hypothetical decision-maker preferences, and we apply DAIVE to identify optimized interventions appropriate to each case.

RESULTS

We demonstrate how DAIVE can be used to make decisions about the composition of optimized interventions and how the choice of optimized intervention can differ according to decision-maker preferences and objectives.

CONCLUSIONS

We offer recommendations for intervention scientists who want to apply DAIVE to select optimized interventions based on data from their own factorial optimization trials. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Optimizing Interventions for Equitability: Some Initial Ideas

Interventions (including behavioral, biobehavioral, biomedical, and social-structural interventions) hold tremendous potential not only to improve public health overall but also to reduce health disparities and promote health equity. In this study, we introduce one way in which interventions can be optimized for health equity in a principled fashion using the multiphase optimization strategy (MOST). Specifically, we define intervention equitability as the extent to which the health benefits provided by an intervention are distributed evenly versus concentrated among those who are already advantaged, and we suggest that, if intervention equitability is acknowledged to be a priority, then equitability should be a key criterion that is balanced with other criteria (effectiveness overall, as well as affordability, scalability, and/or efficiency) in intervention optimization. Using a hypothetical case study and simulated data, we show how MOST can be applied to achieve a strategic balance that incorporates equitability. We also show how the composition of an optimized intervention can differ when equitability is considered versus when it is not. We conclude with a vision for next steps to build on this initial foray into optimizing interventions for equitability.

Operationalizing Primary Outcomes to Achieve Reach, Effectiveness, and Equity in Multilevel Interventions

When intervention scientists plan a clinical trial of an intervention, they select an outcome metric that operationalizes their definition of intervention success. The outcome metric that is selected has important implications for which interventions are eventually supported for implementation at scale and, therefore, what health benefits (including how much benefit and for whom) are experienced in a population. Particularly when an intervention is to be implemented in a population that experiences a health disparity, the outcome metric that is selected can also have implications for equity. Some outcome metrics risk exacerbating an existing health disparity, while others may decrease disparities for some but have less effect for the larger population. In this study, we use a computer to simulate implementation of a hypothetical multilevel, multicomponent intervention to highlight the tradeoffs that can occur between outcome metrics that reflect different operationalizations of intervention success. In particular, we highlight tradeoffs between overall mean population benefit and the distribution of health benefits in the population, which has direct implications for equity. We suggest that simulations like the one we present can be useful in the planning of a clinical trial for a multilevel and/or multicomponent intervention, since simulated implementation at scale can illustrate potential consequences of candidate operationalization of intervention success, such that unintended consequences for equity can be avoided.

A posterior expected value approach to decision-making in the multiphase optimization strategy for intervention science

In current practice, intervention scientists applying the multiphase optimization strategy (MOST) with a 2k factorial optimization trial use a component screening approach (CSA) to select intervention components for inclusion in an optimized intervention. In this approach, scientists review all estimated main effects and interactions to identify the important ones based on a fixed threshold, and then base decisions about component selection on these important effects. We propose an alternative posterior expected value approach based on Bayesian decision theory. This new approach aims to be easier to apply and more readily extensible to a variety of intervention optimization problems. We used Monte Carlo simulation to evaluate the performance of a posterior expected value approach and CSA (automated for simulation purposes) relative to two benchmarks: random component selection, and the classical treatment package approach. We found that both the posterior expected value approach and CSA yielded substantial performance gains relative to the benchmarks. We also found that the posterior expected value approach outperformed CSA modestly but consistently in terms of overall accuracy, sensitivity, and specificity, across a wide range of realistic variations in simulated factorial optimization trials. We discuss implications for intervention optimization and promising future directions in the use of posterior expected value to make decisions in MOST. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Multiphase optimization strategy: How to build more effective, affordable, scalable and efficient social and behavioural oral health interventions

This commentary introduces the field of social behavioural oral health interventions to the multiphase optimization strategy (MOST). MOST is a principled framework for the development, optimization and evaluation of multicomponent interventions. Drawing from the fields of engineering, behavioural science, economics, decision science and public health, intervention optimization requires a strategic balance of effectiveness with affordability, scalability and efficiency. We argue that interventions developed using MOST are more likely to maximize the public health impact of social behavioural oral health interventions.

One view of the next decade of research on behavioral and biobehavioral approaches to cancer prevention and control: intervention optimization

As a new decade begins, we propose that the time is right to reexamine current methods and procedures and look for opportunities to accelerate progress in cancer prevention and control. In this article we offer our view of the next decade of research on behavioral and biobehavioral interventions for cancer prevention and control. We begin by discussing and questioning several implicit conventions. We then briefly introduce an alternative research framework: the multiphase optimization strategy (MOST). MOST, a principled framework for intervention development, optimization, and evaluation, stresses not only intervention effectiveness, but also intervention affordability, scalability, and efficiency. We review some current limitations of MOST along with future directions for methodological work in this area, and suggest some changes in the scientific environment we believe would permit wider adoption of intervention optimization. We propose that wider adoption of intervention optimization would have a positive impact on development and successful implementation of interventions for cancer prevention and control and on intervention science more broadly, including accumulation of a coherent base of knowledge about what works and what does not; establishment of an empirical basis for adaptation of interventions to different settings with different levels and types of resources; and, in the long run, acceleration of progress from Stage 0 to Stage V in the National Institutes of Health Model of Stages of Intervention Development.

Using factorial mediation analysis to better understand the effects of interventions

To improve understanding of how interventions work or why they do not work, there is need for methods of testing hypotheses about the causal mechanisms underlying the individual and combined effects of the components that make up interventions. Factorial mediation analysis, i.e., mediation analysis applied to data from a factorial optimization trial, enables testing such hypotheses. In this commentary, we demonstrate how factorial mediation analysis can contribute detailed information about an intervention's causal mechanisms. We briefly review the multiphase optimization strategy (MOST) and the factorial experiment. We use an empirical example from a 25 factorial optimization trial to demonstrate how factorial mediation analysis opens possibilities for better understanding the individual and combined effects of intervention components. Factorial mediation analysis has important potential to advance theory about interventions and to inform intervention improvements.

The multiphase optimization strategy (MOST) in child maltreatment prevention research

Each year hundreds of thousands of children and families receive behavioral interventions designed to prevent child maltreatment; yet rates of maltreatment have not declined in over a decade. To reduce the prevalence and prevent the life-long negative consequences of child maltreatment, behavioral interventions must not only be effective, but also affordable, scalable, and efficient to meet the demand for these services. An innovative approach to intervention science is needed. The purpose of this article is to introduce the multiphase optimization strategy (MOST) to the field of child maltreatment prevention. MOST is an engineering-inspired framework for developing, optimizing, and evaluating multicomponent behavioral interventions. MOST enables intervention scientists to empirically examine the performance of each intervention component, independently and in combination. Using a hypothetical example of a home visiting intervention and artificial data, this article demonstrates how MOST may be used to optimize the content of a parent-focused in-home intervention and the engagement strategies of an intervention to increase completion rate to identify an intervention that is effective, efficient, economical, and scalable. We suggest that MOST will ultimately improve prevention science and hasten the progress of translational science to prevent child maltreatment.

Lead exposure and the 2010 achievement test scores of children in New York counties

BACKGROUND

Lead is toxic to cognitive and behavioral functioning in children even at levels well below those producing physical symptoms. Continuing efforts in the U.S. since about the 1970s to reduce lead exposure in children have dramatically reduced the incidence of elevated blood lead levels (with elevated levels defined by the current U.S. Centers for Disease Control threshold of 10 μg/dl). The current study examines how much lead toxicity continues to impair the academic achievement of children of New York State, using 2010 test data.

METHODS

This study relies on three sets of data published for the 57 New York counties outside New York City: school achievement data from the New York State Department of Education, data on incidence of elevated blood lead levels from the New York State Department of Health, and data on income from the U.S. Census Bureau. We studied third grade and eighth grade test scores in English Language Arts and mathematics. Using the county as the unit of analysis, we computed bivariate correlations and regression coefficients, with percent of children achieving at the lowest reported level as the dependent variable and the percent of preschoolers in the county with elevated blood lead levels as the independent variable. Then we repeated those analyses using partial correlations to control for possible confounding effects of family income, and using multiple regressions with income included.

RESULTS

The bivariate correlations between incidence of elevated lead and number of children in the lowest achievement group ranged between 0.38 and 0.47. The partial correlations ranged from 0.29 to 0.40. The regression coefficients, both bivariate and partial (both estimating the increase in percent of children in the lowest achievement group for every percent increase in the children with elevated blood lead levels), ranged from 0.52 to 1.31. All regression coefficients, when rounded to the nearest integer, were approximately 1. Thus, when the percent of children showing elevated lead increases by one percent, the percent of children in the lowest achievement group, according to the regression equations generated, also increases by about one percent. All associations were significant at the 0.05 level.

CONCLUSION

Despite public health advances, and despite the imprecision of measures, an association between the incidence of elevated blood lead and achievement in New York counties is still apparent, not attributable to confounding by income. Efforts to reduce lead exposure should persist with vigor.

Martial arts as a mental health intervention for children? Evidence from the ECLS-K

BACKGROUND

Martial arts studios for children market their services as providing mental health outcomes such as self-esteem, self-confidence, concentration, and self-discipline. It appears that many parents enroll their children in martial arts in hopes of obtaining such outcomes. The current study used the data from the Early Childhood Longitudinal Study, Kindergarten class of 1998-1999, to assess the effects of martial arts upon such outcomes as rated by classroom teachers.

METHODS

The Early Childhood Longitudinal Study used a multistage probability sampling design to gather a sample representative of U.S. children attending kindergarten beginning 1998. We made use of data collected in the kindergarten, 3rd grade, and 5th grade years. Classroom behavior was measured by a rating scale completed by teachers; participation in martial arts was assessed as part of a parent interview. The four possible combinations of participation and nonparticipation in martial arts at time 1 and time 2 for each analysis were coded into three dichotomous variables; the set of three variables constituted the measure of participation studied through regression. Multiple regression was used to estimate the association between martial arts participation and change in classroom behavior from one measurement occasion to the next. The change from kindergarten to third grade was studied as a function of martial arts participation, and the analysis was replicated studying behavior change from third grade to fifth grade. Cohen's f2 effect sizes were derived from these regressions.

RESULTS

The martial arts variable failed to show a statistically significant effect on behavior, in either of the regression analyses; in fact, the f2 effect size for martial arts was 0.000 for both analyses. The 95% confidence intervals for regression coefficients for martial arts variables have upper and lower bounds that are all close to zero. The analyses not only fail to reject the null hypothesis, but also render unlikely a population effect size that differs greatly from zero.

CONCLUSION

The data from the ECLS-K fail to support enrolling children in martial arts to improve mental health outcomes as measured by classroom teachers.

Religiosity and teen birth rate in the United States

BACKGROUND

The children of teen mothers have been reported to have higher rates of several unfavorable mental health outcomes. Past research suggests several possible mechanisms for an association between religiosity and teen birth rate in communities.

METHODS

The present study compiled publicly accessible data on birth rates, conservative religious beliefs, income, and abortion rates in the U.S., aggregated at the state level. Data on teen birth rates and abortion originated from the Center for Disease Control; on income, from the U.S. Bureau of the Census, and on religious beliefs, from the U.S. Religious Landscape Survey carried out by the Pew Forum on Religion and Public Life. We computed correlations and partial correlations.

RESULTS

Increased religiosity in residents of states in the U.S. strongly predicted a higher teen birth rate, with r = 0.73 (p < 0.0005). Religiosity correlated negatively with median household income, with r = -0.66, and income correlated negatively with teen birth rate, with r = -0.63. But the correlation between religiosity and teen birth rate remained highly significant when income was controlled for via partial correlation: the partial correlation between religiosity and teen birth rate, controlling for income, was 0.53 (p < 0.0005). Abortion rate correlated negatively with religiosity, with r = -0.45, p = 0.002. However, the partial correlation between teen birth rate and religiosity remained high and significant when controlling for abortion rate (partial correlation = 0.68, p < 0.0005) and when controlling for both abortion rate and income (partial correlation = 0.54, p = 0.001).

CONCLUSION

With data aggregated at the state level, conservative religious beliefs strongly predict U.S. teen birth rates, in a relationship that does not appear to be the result of confounding by income or abortion rates. One possible explanation for this relationship is that teens in more religious communities may be less likely to use contraception.