A feasibility randomised controlled trial of the New Orleans intervention for infant mental health: a study protocol

Framework for the development and evaluation of complex interventions: gap analysis, workshop and consultation-informed update

BACKGROUND

The Medical Research Council published the second edition of its framework in 2006 on developing and evaluating complex interventions. Since then, there have been considerable developments in the field of complex intervention research. The objective of this project was to update the framework in the light of these developments. The framework aims to help research teams prioritise research questions and design, and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods.

METHODS

There were four stages to the update: (1) gap analysis to identify developments in the methods and practice since the previous framework was published; (2) an expert workshop of 36 participants to discuss the topics identified in the gap analysis; (3) an open consultation process to seek comments on a first draft of the new framework; and (4) findings from the previous stages were used to redraft the framework, and final expert review was obtained. The process was overseen by a Scientific Advisory Group representing the range of relevant National Institute for Health Research and Medical Research Council research investments.

RESULTS

Key changes to the previous framework include (1) an updated definition of complex interventions, highlighting the dynamic relationship between the intervention and its context; (2) an emphasis on the use of diverse research perspectives: efficacy, effectiveness, theory-based and systems perspectives; (3) a focus on the usefulness of evidence as the basis for determining research perspective and questions; (4) an increased focus on interventions developed outside research teams, for example changes in policy or health services delivery; and (5) the identification of six 'core elements' that should guide all phases of complex intervention research: consider context; develop, refine and test programme theory; engage stakeholders; identify key uncertainties; refine the intervention; and economic considerations. We divide the research process into four phases: development, feasibility, evaluation and implementation. For each phase we provide a concise summary of recent developments, key points to address and signposts to further reading. We also present case studies to illustrate the points being made throughout.

LIMITATIONS

The framework aims to help research teams prioritise research questions and design and conduct research with an appropriate choice of methods, rather than to provide detailed guidance on the use of specific methods. In many of the areas of innovation that we highlight, such as the use of systems approaches, there are still only a few practical examples. We refer to more specific and detailed guidance where available and note where promising approaches require further development.

CONCLUSIONS

This new framework incorporates developments in complex intervention research published since the previous edition was written in 2006. As well as taking account of established practice and recent refinements, we draw attention to new approaches and place greater emphasis on economic considerations in complex intervention research. We have introduced a new emphasis on the importance of context and the value of understanding interventions as 'events in systems' that produce effects through interactions with features of the contexts in which they are implemented. The framework adopts a pluralist approach, encouraging researchers and research funders to adopt diverse research perspectives and to select research questions and methods pragmatically, with the aim of providing evidence that is useful to decision-makers.

FUTURE WORK

We call for further work to develop relevant methods and provide examples in practice. The use of this framework should be monitored and the move should be made to a more fluid resource in the future, for example a web-based format that can be frequently updated to incorporate new material and links to emerging resources.

FUNDING

This project was jointly funded by the Medical Research Council (MRC) and the National Institute for Health Research (Department of Health and Social Care 73514).

Meta-Analyses of the Associations Between Disinhibited Social Engagement Behaviors and Child Attachment Insecurity or Disorganization

Children with disinhibited social engagement disorder show reduced reticence with strangers, do not check back with their caregiver after venturing away, and may willingly leave with an unfamiliar adult. The recent DSM-5 has moved away from an attachment framework to understand disinhibited social engagement behavior (DSEB) due to studies indicating its presence in previously institutionalized children even after these children are adopted and show a selective, more secure attachment with their substitute caregiver (e.g. Chisholm et al., 1998). This meta-analysis aims to clarify the size of the associations between DSEB and attachment insecurity or disorganization. It also examines whether studies effect sizes differ according to various moderators (e.g., child age, type of attachment and DSEB measures). The results (k = 24) showed that the associations between DSEB and attachment insecurity (d = 0.48) or attachment disorganization (d = 0.47) were of small magnitude. There were no publication biases. As for moderator analyses on both attachment insecurity and disorganization, the effect sizes in studies using DSEB observational measures (respectively d = 0.63 and 0.57) were of moderate magnitude and stronger than those in studies not using an observational component (respectively d = 0.28 and 0.32). Given these small-to-moderate associations, attachment can be considered a relationship process associated with DSEB, and attachment-informed interventions could be potential tools to reduce DSEB in children. Nevertheless, given the sizable unshared portion of variance between DSEB and child attachment, future studies should examine other variables related to caregiving and noncaregiving contexts to further understand DSEB.

Clout or doubt? Perspectives on an infant mental health service for young children placed in foster care due to abuse and neglect

Despite knowledge about the profound effects of child abuse and neglect, we know little about how best to assess whether maltreated children should return home. The effectiveness of the New Orleans Intervention Model (NIM) is being tested in a randomized controlled trial where the comparison is social work 'services as usual.' The future trial results will tell us which approach produces the best outcomes for children; meanwhile qualitative process evaluation is generating intriguing findings about the perceived impact of NIM on decision-making about childrens' futures. Interviews and focus groups were conducted with social workers, foster carers, legal decision-makers and the NIM team (n=63). Data were analysed thematically. Findings suggest that NIM is seen as bringing greater influence ('clout') to decision-making due to its depth of focus, provision of treatment for the family, health professional input and perceived objectivity. Simultaneously, the NIM approach and the detailed information it produces potentially throws judgments into doubt in the legal system. Clout/doubt perceptions permeate opinions about NIM and are inter-related with a historical discourse about 'health versus social' models of information gathering, with implications for assessment of child abuse and neglect that extend beyond the study context. The juxtaposition of 'clout versus doubt' both highlights and is strengthened by an intense focus among social workers and legal professionals on how evidence will be regarded within legal fora when making decisions about children. There is continuing uncertainty in the child welfare system about the best ways of assessing maltreated children, underscoring a continued need for the trial.

Does Quantitative Research in Child Maltreatment Tell the Whole Story? The Need for Mixed-Methods Approaches to Explore the Effects of Maltreatment in Infancy

Background and Aims. Research on child maltreatment has largely overlooked the under-five age group and focuses primarily on quantitative measurement. This mixed-methods study of maltreated children (N = 92) entering care (age 6–60 months) combines a quantitative focus on the associations between care journey characteristics and mental health outcomes with a qualitative exploration of maltreatment in four different families. Methods. Care journey data was obtained from social care records; mental health and attachment assessments were carried out following entry to care; qualitative data comprised semistructured interviews with professionals, foster carers, and parents. Results. Significant associations were found between suspected sexual abuse and increased DAI inhibited attachment symptoms (p = 0.001) and between reported domestic violence and decreased DAI inhibited (p = 0.016) and disinhibited (p = 0.004) attachment symptoms. Qualitative results: two themes demonstrate the complexity of assessing maltreatment: (1) overlapping maltreatment factors occur in most cases and (2) maltreatment effects may be particularly challenging to isolate. Conclusions. Qualitative exploration has underscored the complexity of assessing maltreatment, indicating why expected associations were not found in this study and posing questions for the quantitative measurement of maltreatment in general. We therefore suggest a new categorisation of maltreatment and call for the complimentary research lenses of further mixed-methods approaches.

Challenges of Assessing Maltreated Children Coming into Foster Care

Children who have experienced early adversity have been known to be at risk of developing cognitive, attachment, and mental health problems; therefore, it is crucial that children entering foster care can be properly assessed as early as possible. There are known difficulties in assessing children in foster care, for example, in finding a reliable informant. An ongoing randomised controlled trial in Glasgow, Scotland, recruiting infants entering foster care, provides a unique opportunity to explore some of the issues which need to be considered when assessing these children. The assessment data of 70 infants entering care is described while exploring the reliability of foster carers as informants and the importance of infant engagement with tasks. This group of infants was shown to be having more problems than children from the general population. While correlations were found between a carer's level of concern about a child and the severity of a child's problem, there were still a number of children displaying worrying problem scores whom foster carers did not report concern. The child's engagement in the cognitive task showed associations with the child's attainment on the task. Findings emphasise the importance of a holistic assessment for these children and all should be considered as potential cases with Maltreatment-Associated Psychiatric Problems (MAPP).

Development of a radical foster care intervention in Glasgow, Scotland

Services for maltreated children are inadequate and lack infant mental health input in many parts of the world. A recent audit of Glasgow services revealed that children frequently 'revolve' between maltreating birth parents and various temporary foster placements for many years. Addressing infant mental health in this population will require radical change to current services. The New Orleans programme developed by the Tulane Infant Team in Louisiana is one such radical programme. Prior to the design of a randomized controlled trial (RCT) to test this programme in Glasgow, it was essential that policy-makers had some insight into the local model of service delivery and how a New Orleans model could impact. This article explores the structure and costs of the current Glasgow system and the potential costs and consequence impact of implementing a New Orleans model in Glasgow, using data obtained from the research literature, Glasgow City Council audit data and expert's opinion. A New Orleans-Glasgow model would likely shift resources from social services on to the NHS. The resource intensive nature of this model could increase the cost of an episode in care from £66 300 in the current system to £86 070; however, the probability of repeated episodes in care is likely to fall substantially, making the cost per child fall from £95 500 in the current system to £88 600. This study informed the design of a phase II explorative RCT, identified appropriate outcomes for measurement and areas of uncertainty for further research.