Using flawed, uncertain, proximate and sparse (FUPS) data in the context of complexity: learning from the case of child mental health

Evaluating Aboriginal and Torres Strait Islander Social and Emotional Wellbeing services: A collective case study in Far North Queensland

BACKGROUND

Access to a coordinated range of strengths-based, culturally appropriate community-led primary mental health and Social and Emotional Wellbeing services is critical to the mental health and wellbeing of young Aboriginal and Torres Strait Islander people, and is a policy commitment of the Australian government. However, complex and fragmented service networks and a lack of standardised service data are barriers in identifying what services are available and what care they provide.

METHOD

A standardised service classification tool was used to assess the availability and characteristics of Social and Emotional Wellbeing services for young Aboriginal and Torres Strait Islander people in two regions in Queensland, Australia.

RESULTS

We identified a complex pattern of service availability and gaps in service provision. Non-Indigenous non-governmental organisations provided a significant proportion of services, particularly 'upstream' support, while Aboriginal Community Controlled Organisations were more likely to provide 'downstream' crisis type care. Most services provided by the public sector were through Child Safety and Youth Justice departments.

CONCLUSIONS

Our findings demonstrate the complexity of current networks, and show that non-Indigenous organisations are disproportionately influential in the care received by young Aboriginal and Torres Strait Islander people, despite community goals of self-determination, and government commitment to increasing capacity of Aboriginal Community Controlled Organisations to support their local communities. These findings can be used to support decision making and planning.

Policy action points and approaches to promote fertility care in The Gambia: Findings from a mixed-methods study

INTRODUCTION

In the Global South, (in)fertility care is scarcely recognized as a priority, yet the government of The Gambia has recently included it as one of the key priorities in its reproductive health strategic plan. This inclusion appears to be the result of years of engagement between policy actors, academic researchers, and activists in the field of reproductive health and specifically of infertility. However, the operationalization of the strategic plan may be hampered by multiple factors. The research aims to identify and analyze challenges that may impede the effective implementation of the strategic plan, thereby providing policy action points and practical guidance into the operationalization of (in)fertility care in the context of The Gambia's health system.

METHODS

This is a mixed-methods study with data from a survey and semi-structured interviews collected between 2020 and 2021 in The Gambia that were separately published. In this paper, we present the triangulation of quantitative and qualitative data using a convergence coding matrix to identify relevant policy action points.

RESULTS

Six fertility care policy action points, driven by data, arose from the triangulation and interpretation process, specifically: (i) establishing and maintaining political commitment and national priority for fertility care; (ii) creating awareness and increasing the involvement of men in SRH and fertility; (iii) ensuring data-driven health policymaking; (iv) offering and regulating affordable IVF alternatives; (v) improving knowledge of and means for fertility care provision; and (vi) enhancing the collaboration among stakeholders and building links with the private healthcare sector.

CONCLUSION

This study found the implementation of the fertility care-related activities in the reproductive health strategic plan may face challenges that require careful mitigation through a holistic approach. Such an approach conceptualizes infertility not just as a biomedical issue but as a broader one that incorporates educational and socio-emotional aspects, including male and (not only) female involvement in sexual and reproductive health. Moreover, it is supported by a comprehensive health management information system that includes capturing data on the demand for, and access to, infertility services in The Gambia health system.

Virtually delivered guided self-help for binge eating disorder and bulimia nervosa: findings from a service evaluation

BACKGROUND

Timely intervention is beneficial to the effectiveness of eating disorder (ED) treatment, but limited capacity within ED services means that these disorders are often not treated with sufficient speed. This service evaluation extends previous research into guided self-help (GSH) for adults with bulimic spectrum EDs by assessing the feasibility, acceptability, and preliminary effectiveness of virtually delivered GSH using videoconferencing.

METHOD

Patients with bulimia nervosa (BN), binge eating disorder (BED) and other specified feeding and eating disorders (OSFED) waiting for treatment in a large specialist adult ED out-patient service were offered virtually delivered GSH. The programme used an evidence-based cognitive behavioural self-help book. Individuals were supported by non-expert coaches, who delivered the eight-session programme via videoconferencing.

RESULTS

One hundred and thirty patients were allocated to a GSH coach between 1 September 2020 and 30 September 2022; 106 (82%) started treatment and 78 (60%) completed treatment. Amongst completers, there were large reductions in ED behaviours and attitudinal symptoms, measured by the ED-15. The largest effect sizes for change between pre- and post-treatment were seen for binge eating episode frequency (d = -0.89) and concerns around eating (d = -1.72). Patients from minoritised ethnic groups were over-represented in the non-completer group.

CONCLUSIONS

Virtually delivered GSH is feasible, acceptable and effective in reducing ED symptoms amongst those with bulimic spectrum disorders. Implementing virtually delivered GSH reduced waiting times, offering a potential solution for long waiting times for ED treatment. Further research is needed to compare GSH to other brief therapies and investigate barriers for patients from culturally diverse groups.

Using Electronic Health Record Data for Occupational Therapy Health Services Research: Invited Commentary

Health services research (HSR) is a field of study that examines how social factors, financing systems, organizational structures and processes, health technologies, and personal behaviors affect access to health care, the quality and cost of health care, and health and well-being. HSR approaches can help build the occupational therapy evidence base, particularly in relation to population health. Data from electronic health record (EHR) systems provide a rich resource for applying HSR approaches to examine the value of occupational therapy services. Transparency about data preparation procedures is important for interpreting results. Based on our findings, we describe a six-step cleaning protocol for preparing EHR and billing data from an inpatient rehabilitation facility for research and provide recommendations for the field based on our experience. Using and reporting similar strategies across studies will improve efficiency and transparency, and facilitate comparability of results.

Ethnic differences in treatment outcome for children and young people accessing mental health support

Children and Young People (CYP) from minoritized ethnic backgrounds experience structural inequalities in Children and Young People's Mental Health Settings (CYPMHS). This mixed methods study explores whether CYP's ethnicity is associated with their treatment outcomes (operationalised as 'measurable change') from CYPMHS. A multilevel multi-nominal regression analysis, controlling for age, gender, referral source, presenting difficulty, case closure reason, suggests that CYP from Asian backgrounds (OR = 0.82, CI [0.70, 0.96]) and Mixed-race (odds ratio (OR) = 0.80; 95% CI [0.69, 0.92]) are less likely to report measurable improvement in mental health difficulties compared to White British CYP. Three themes from a thematic analysis of semi-structured interviews with 15 CYP from minoritized ethnic backgrounds focused on views and experiences of ending mental health support are also presented. CYP view personalised support and the right therapist as conducive to good endings and valued a range of outcomes pertaining to empowerment. Experiences of stigma and inequalities may begin to explain the less positive outcomes experienced by Asian and Mixed-race CYP found in the regression analysis. The implications of these findings and future areas of research are suggested.

Mental Health and Well-being Measures for Mean Comparison and Screening in Adolescents: An Assessment of Unidimensionality and Sex and Age Measurement Invariance

Adolescence is a period of increased vulnerability for low well-being and mental health problems, particularly for girls and older adolescents. Accurate measurement via brief self-report is therefore vital to understanding prevalence, group trends, screening efforts, and response to intervention. We drew on data from the #BeeWell study (N = 37,149, aged 12-15) to consider whether sum-scoring, mean comparisons, and deployment for screening were likely to show bias for eight such measures. Evidence for unidimensionality, considering dynamic fit confirmatory factor models, exploratory graph analysis, and bifactor modeling, was found for five measures. Of these five, most showed a degree of non-invariance across sex and age likely incompatible with mean comparison. Effects on selection were minimal, except sensitivity was substantially lower in boys for the internalizing symptoms measure. Measure-specific insights are discussed, as are general issues highlighted by our analysis, such as item reversals and measurement invariance.

Shifting Mindsets: The Impact of a Patient Portal on Functioning and Recovery in a Mental Health Setting

OBJECTIVE

This study aims to understand whether higher use of a patient portal can have an impact on mental health functioning and recovery.

METHOD

A mixed methods approach was used for this study. In 2019-2021, patients with mental health diagnoses at outpatient clinics in an academic centre were invited to complete World Health Organization Disability Assessment Scale 12 (WHODAS-12) and Mental Health Recovery Measure surveys at baseline, 3 months, and 6 months after signing up for the portal. At the 3-month time point, patients were invited to a semistructured interview with a member of the team to contextualize the findings obtained from the surveys. Analytics data was also collected from the platform to understand usage patterns on the portal.

RESULTS

Overall, 113 participants were included in the analysis. There was no significant change in mental health functioning and recovery scores over the 6-month period. However, suboptimal usage was observed as 46% of participants did not complete any tasks within the portal. Thirty-five participants had low use of the portal (1-9 interactions) and 18 participants had high usage (10+ interactions). There were also no differences in mental health functioning and recovery scores between low and high users of the portal. Qualitative interviews highlighted many opportunities where the portal can support overall functioning and mental health recovery.

CONCLUSIONS

Collectively, this study suggests that higher use of a portal had no impact, either positive or negative, on mental health outcomes. While it may offer convenience and improved patient satisfaction, adequate support is needed to fully enable these opportunities for patient care. As the type of interaction with the portal was not specifically addressed, future work should focus on looking at ways to support patient engagement and portal usage throughout their care journey.

Critical analyses of Latina mortality: disentangling the heterogeneity of ethnic origin, place, nativity, race, and socioeconomic status

Despite the significant body of research on social determinants of health (SDH) and mortality, limited knowledge is available on the epidemiology of aggregated Latino health overall, and by women and subgroups. In population health studies, U.S. Latinos often are considered a monolithic population and presented as an aggregate, obscuring the diversity and variations within and across Latino subgroups, contributing to missed opportunities to identify SDH of health outcomes, and limiting the understanding of health differences. Given diverse environmental, racial, class, and geographic factors, a specific focus on women facilitates a more in-depth view of health disparities. This paper provides a scoping review of current gaps in research that assesses the relationships between SDH and mortality rates for the five leading causes of chronic-disease related deaths among Latinas by ethnic origin, place, race, and SES. We analyzed 2020 national mortality statistics from the CDC WONDER Online database jointly with reviews of empirical articles on Latina health, employing the EBSCOhost MEDLINE databases. These findings challenge the phenomenon of the Hispanic paradox that identified Latinos as a relatively healthy population compared to non-Hispanic White populations despite their lower economic status. The findings confirm that prior research on Latino women had methodological limitations due to the exclusion of SDH and an overemphasis on culturalist perspectives, while overlooking the critical role of socioeconomic impacts on health. Findings indicate major knowledge gaps in Latina mortality by SDH and subgroups that may undermine surveillance efforts and treatment efficacy. We offer forward-looking recommendations to assure the inclusion of key SDH associated with Latina mortality by subgroup as essential to inform future studies, intervention programs, and health policy.

User satisfaction in child and adolescent mental health service: Comparison of background, clinical and service predictors for adolescent and parent satisfaction

BACKGROUND AND OBJECTIVE

To improve quality, child and adolescent mental health services (CAMHS) are expected to quantify families' views on healthcare with user satisfaction measures. As little is known about what influences satisfaction in CAMHS, this study aimed to examine predictors of adolescents' and parents' user satisfaction.

METHODS

Data from 231 adolescents and 495 parents in treatment at an outpatient clinic who returned a user satisfaction measure, the Experience of Service Questionnaire (ESQ), was analyzed. Registry data on background, clinical and service characteristics were predictors for the ESQ factors general satisfaction, satisfaction with care and satisfaction with environment.

RESULTS

In regression models, satisfaction with care for adolescents (r2  = .12) was significant and was predicted by low parent-self-reported mental health burden and low clinician-rated overall symptom burden at intake. For parents, regression models for general satisfaction (r2  = .07), satisfaction with care (r2  = .06) and satisfaction with environment (r2  = .08) were significant. Parents general satisfaction was predicted by higher levels of hyperactivity, less family stress and longer travelling distances to the service. Satisfaction with care for parents was predicted by higher levels of hyperactivity at intake and longer travelling distances. Satisfaction with environment for parents was more likely if the adolescents was a boy, with low levels of family stress and longer travelling distances.

CONCLUSION

Predictors for adolescent and parent user satisfaction in CAMHS differ. Hence, to improve quality CAMHS should enhance focus on collaborative practice with parents, and person-centred care for adolescents with moderate to severe mental health illness.

PATIENT OR PUBLIC CONTRIBUTION

Representatives from the hospitals' youth panel and the non-governmental organization called The Change Factory have been consulted regarding study design and results.

Impact of the Care and Coercion Act on recorded involuntary care in intellectual disability care: a time-series analysis

BACKGROUND

On 1 January 2020, the Care and Coercion Act came into effect in the Netherlands, subjecting involuntary care to more strict regulations and monitoring. This study tested changes in recordings of involuntary care during the transitional year of 2020 and after full implementation in 2021, which coincided with the first severe test of the new regulations, when COVID-19 lockdown measures were taken on 16 March 2020.

METHODS

Data consisted of weekly counts of involuntary care from 1 January 2017 to 31 December 2021, taken from the care data of more than 3000 clients with intellectual disabilities and challenging behaviour of 's Heeren Loo, a large long-term care organisation in the Netherlands. An interrupted time series design was used to compare the period under the former law with the period under the new law and to the period during and after implementation, taking into account the impact of the COVID-19 lockdown measures on recordings of involuntary care.

RESULTS

Under the new act in Week 1 of 2020, a statistically significant drop occurred in involuntary care counts, after which these counts gradually decreased. The start of 2021, the year in which the act was fully implemented, showed an initial increase in counts of involuntary care, followed by a decrease (all Ps < .001). The introduction of the COVID-19 lockdown measures did not statistically affect the weekly counts of involuntary care.

CONCLUSIONS

The decrease in registered involuntary care after the Care and Coercion Act came into effect is a first indication of the efficacy of this new law that requires careful multidisciplinary consultations around the right of clients to respect their self-determination. Follow-up research should examine whether the impact of the new law aligns with clients' experiences of self-determination.

Predictors of amounts of child and adolescent mental health service use

The aim of this study was to build evidence about how to tailor services to meet the individual needs of young people by identifying predictors of amounts of child and adolescent mental health service use. We conducted a secondary analysis of a large administrative dataset from services in England was conducted using the Mental Health Services Data Set (years 2016-17 and 2017-18). The final sample included N = 27,362 episodes of care (periods of service use consisting of at least two attended care contacts and less than 180 days between care contacts) from 39 services. There were 50-10,855 episodes per service. The descriptive statistics for episodes of care were: Mage = 13 years, SDage = 4.71, range = 0-25 years; 13,785 or 50% male. Overall, there were high levels of heterogeneity in number of care contacts within episodes of care: M = 11.12, SD = 28.28, range = 2-1529. Certain characteristics predicted differential patterns of service use. For example, young people with substance use (beta = 6.29, 95% CI = 5.06-7.53) or eating disorders (beta = 4.30, 95% CI = 3.29-5.30) were particularly more likely to have higher levels of service use. To build on this, evidence is needed about predictors of child and adolescent mental health treatment outcome and whether the same characteristics predict levels of improvement as well as levels of service use.

Mental health systems modelling for evidence-informed service reform in Australia

Australia's Fifth National Mental Health Plan required governments to report, not only on the progress of changes to mental health service delivery, but to also plan for services that should be provided. Future population demand for treatment and care is challenging to predict and one solution involves modelling the uncertain demands on the system. Modelling can help decision-makers understand likely future changes in mental health service demand and more intelligently choose appropriate responses. It can also support greater scrutiny, accountability and transparency of these processes. Australia has an emerging national capacity for systems modelling in mental health which can enhance the next phase of mental health reform. This paper introduces concepts useful for understanding mental health modelling and identifies where modelling approaches can support health service planners to make evidence-informed decisions regarding planning and investment for the Australian population.

The influence of minority stress-related experiences on mental wellbeing for trans/gender-diverse and cisgender youth: a comparative longitudinal analysis

Trans and gender-diverse (TGD) adolescents are likely to experience poorer mental health and wellbeing than their cisgender peers. Minority stress theory has developed as a possible explanation for some of this disadvantage: factors such as increased bullying and discrimination lead to excess stress and reduced wellbeing. However, the evidence base remains limited. This study drew on secondary data analysis of the #BeeWell longitudinal cohort over 2 years (N = 26 042, aged 12-13 at time one, T1). We report two unregistered hypotheses relating to T1 (autumn 2021) data which was available at the time of stage-one submission: H1, mean differences in T1 wellbeing; H2, mean differences in T1 minority-related stressors. These are followed by two registered hypotheses relating to T2 (autumn 2022) data: H3, replication of T1 mean differences in T2 wellbeing; H4, predictions were made about the strength of the association between T1 minority-related stressors, controlling for sexuality and T2 wellbeing across T1 gender identity groups. At both time points cis-females, TGD and those who preferred not to say their gender had lower wellbeing than cis-males (CM), with the largest effect evident for the TGD group. TGD adolescents also showed the largest disadvantage (mean difference) compared with CM for minority stressors. Counter to H4 and minority stress theory, gender was not found to moderate the effect of minority stressors on later wellbeing. Our findings highlight the vulnerability of the TGD group in terms of wellbeing and minority stressors and are discussed with relevance for policy and future research.

Strategies to reduce low-value care - An applied behavior analysis using a single-case design

Introduction

Implementation science has traditionally focused on the implementation of evidence-based practices, but the field has increasingly recognized the importance of addressing de-implementation (i.e., the process of reducing low-value care). Most studies on de-implementation strategies have used a combination of strategies without addressing factors that sustain the use of LVC and there is a lack of information about which strategies are most effective and what mechanisms of change might underlie these strategies. Applied behavior analysis is an approach that could be a potential method to gain insights into the mechanisms of de-implementation strategies to reduce LVC. Three research questions are addressed in this study: What contingencies (three-term contingencies or rule-governing behavior) related to the use of LVC can be found in a local context and what strategies can be developed based on an analysis of these contingencies?; Do these strategies change targeted behaviors?; How do the participants describe the strategies' contingencies and the feasibility of the applied behavior analysis approach?

Materials and methods

In this study, we used applied behavior analysis to analyze contingencies that maintain behaviors related to a chosen LVC, the unnecessary use of x-rays for knee arthrosis within a primary care center. Based on this analysis, strategies were developed and evaluated using a single-case design and a qualitative analysis of interview data.

Results

Two strategies were developed: a lecture and feedback meetings. The results from the single-case data were inconclusive but some of the findings may indicate a behavior change in the expected direction. Such a conclusion is supported by interview data showing that participants perceived an effect in response to both strategies.

Conclusion

The findings illustrate how applied behavior analysis can be used to analyze contingencies related to the use of LVC and to design strategies for de-implementation. It also shows an effect of the targeted behaviors even though the quantitative results are inconclusive. The strategies used in this study could be further improved to target the contingencies better by structuring the feedback meetings better and including more precise feedback.

National roll-out of early intervention for eating disorders: Process and clinical outcomes from first episode rapid early intervention for eating disorders

AIM

First Episode Rapid Early Intervention for Eating Disorders (FREED) is an early intervention model for young people with recent-onset eating disorders (ED). Promising results from a previous single-centre study and a four-centre study (FREED-Up) have led to the rapid national scaling of FREED to ED services in England (FREED-4-All). Our aim was to evaluate duration of an untreated ED (DUED), wait time target adherence, and clinical outcomes in FREED-4-All and compare these to the (benchmark) findings of the earlier FREED-Up study.

METHOD

FREED services submit de-identified data to the central FREED team quarterly. The current study covers the period between September 2018 and September 2021. This FREED-4-All dataset includes 2473 patients. These were compared to 278 patients from the FREED-Up study.

RESULTS

DUED was substantially shorter in the FREED-4-All dataset relative to the FREED-Up study (15 vs. 18 months). Adherence to the wait time targets was comparable in both cohorts (~85% of engagement calls attempted in <2 days, ~50%-60% of assessments offered in <14 days, ~40% of treatment offered in <28 days). Patients in the FREED-4-All dataset experienced significant improvements in ED and general psychological symptoms from pre- to post-treatment that were comparable to the FREED-Up study. These findings should be interpreted cautiously as only 6% of FREED-4-All patients had post-treatment data.

CONCLUSIONS

Data from the FREED-4-All evaluation suggest that FREED is replicating at scale. However, these data are flawed, uncertain, proximate, and sparse and should therefore be used carefully alongside other evidence and clinical experience to inform decision making.

Process mining for healthcare decision analytics with micro-costing estimations

Managing constrained healthcare resources is an important and inescapable role of healthcare decision makers. Allocative decisions are based on downstream consequences of changes to care processes: judging whether the costs involved are offset by the magnitude of the consequences, and therefore whether the change represents value for money. Process mining techniques can inform such decisions by quantitatively discovering, comparing and detailing care processes using recorded data, however the scope of techniques typically excludes anything 'after-the-process' i.e., their accumulated costs and resulting consequences. Cost considerations are increasingly incorporated into process mining techniques, but the majority of healthcare costs for service and overhead components are commonly apportioned and recorded at the patient (trace) level, hiding event level detail. Within decision-analysis, event-driven and individual-level simulation models are sometimes used to forecast the expected downstream consequences of process changes, but are expensive to manually operationalise. In this paper, we address both of these gaps within and between process mining and decision analytics, by better linking them together. In particular, we introduce a new type of process model containing trace data that can be used in individual-level or cohort-level decision-analytical model building. Furthermore, we enhance these models with process-based micro-costing estimations. The approach was evaluated with health economics and decision modelling experts, with discussion centred on how the outputs could be used, and how similar information would otherwise be compiled.

The impact of the COVID-19 pandemic on referral numbers, diagnostic mix, and symptom severity in Eating Disorder Early Intervention Services in England

OBJECTIVE

First Episode Rapid Early Intervention for Eating Disorders (FREED) is a service model and care pathway which aims to provide timely, well-coordinated, developmentally informed and evidence-based care for young people with eating disorders (EDs). This article investigates the impact of the COVID-19 pandemic on FREED patient presentations and service provision in England.

METHOD

Data from three services spanning the pre- to post-pandemic period were included (January 2019-September 2021; n = 502 patients). Run charts were created to analyze changes in monthly baseline patient data (e.g., referral numbers, duration of an untreated ED, diagnostic mix, and average body mass index for patients with anorexia nervosa [AN]).

RESULTS

Significant increases in referral numbers were found from September 2020 onward, coinciding with the end of the first UK national lockdown. The percentage of AN presentations significantly increased after the onset of the first national lockdown (April 2020-December 2020). No other significant change patterns were identified.

DISCUSSION

There have been substantial increases in referral numbers and presentations of AN to FREED services whereas illness severity seems largely unchanged. Together, this suggests that increased referrals cannot be attributed to milder presentations being seen. Implications for the implementation, funding, and sustainability of the model are discussed.

PUBLIC SIGNIFICANCE

Our research suggests that early intervention eating disorder services across England faced significant increases in patient referrals and presentations of anorexia nervosa over the COVID-19 pandemic. This increase in referrals is not due to a rise in milder eating disorder cases, as baseline symptom severity remained stable across the pandemic. Investment in early intervention for eating disorders must therefore match increased referral trends.

Patterns of service provision in child and adolescent mental health care in Australia

Standard description of local care provision is essential for evidence-informed planning. This study aimed to map and compare the availability and diversity of current mental health service provision for children and adolescents in Australia. We used a standardised service classification instrument, the Description and Evaluation of Services and DirectoriEs (DESDE) tool, to describe service availability in eight urban and two rural health districts in Australia. The pattern of care was compared with that available for other age groups in Australia. Outpatient care was found to be the most common type of service provision, comprising 212 (81.2%) of all services identified. Hospital care (acute and non-acute) was more available in urban than in rural areas (20 services [9.7%] vs 1 [1.8%]). The level of diversity in the types of care available for children and adolescents was lower than that for the general adult population, but slightly higher than that for older people in the same areas. Standardised comparison of the pattern of care across regions reduces ambiguity in service description and classification, enables gap analysis and can inform policy and planning.

Regional evolution of psychosocial services in Australia before and after the implementation of the National Disability Insurance Scheme

OBJECTIVES

This paper compares the evolution of the psychosocial sector in two Australian regions pre and post introduction of the National Disability Insurance Scheme - a major reform to the financing, planning and provision of disability services in Australia, intended to create greater competition and efficiency in the market, and more choice for service users.

METHODS

We used a standardised service classification instrument based on a health ecosystems approach to assess service availability and diversity of psychosocial services provided by non-government organisations in two Primary Health Network regions.

RESULTS

We identified very different evolutionary pathways in the two regions. Service availability increased in Western Sydney but decreased in the Australian Capital Territory. The diversity of services available did not increase in either Primary Health Network 4 years after the reform. Many services were experiencing ongoing funding uncertainty.

CONCLUSION

Assumptions of increased efficiency through organisational scaling up, and a greater diversity in range of service availability were not borne out.

IMPLICATIONS

This study shows the urgent need for evaluation of the effects of the NDIS on the provision of psychosocial care in Australia. Four years after the implementation of the NDIS at vast expense key objectives not been met for consumers or for the system as a whole, and an environment of uncertainty has been created for providers. It demonstrates the importance of standardised service mapping to monitor the effects of major reforms on mental health care as well as the need for a focus at the local level.

Accuracy of Physician Electronic Health Record Usage Analytics using Clinical Test Cases

Usage log data are an important data source for characterizing the potential burden related to use of the electronic health record (EHR) system. However, the utility of this data source has been hindered by concerns related to the real-world validity and accuracy of the data. While time-motion studies have historically been used to address this concern, the restrictions caused by the pandemic have made it difficult to carry out these studies in-person. In this regard, we introduce a practical approach for conducting validation studies for usage log data in a controlled environment. By developing test runs based on clinical workflows and conducting them within a test EHR environment, it allows for both comparison of the recorded timings and retrospective investigation of any discrepancies. In this case report, we describe the utility of this approach for validating our physician EHR usage logs at a large academic teaching mental health hospital in Canada. A total of 10 test runs were conducted across 3 days to validate 8 EHR usage log metrics, finding differences between recorded measurements and the usage analytics platform ranging from 9 to 60%.

Predictors of child and adolescent mental health treatment outcome

BACKGROUND

To examine the predictors of treatment outcome or improvement in mental health difficulties for young people accessing child and adolescent mental health services.

METHODS

We conducted a secondary analysis of routinely collected data from services in England using the Mental Health Services Data Set. We conducted multilevel regressions on N = 5907 episodes from 14 services (M_age = 13.76 years, SD_age = 2.45, range = 8-25 years; 3540 or 59.93% female) with complete information on mental health difficulties at baseline. We conduct similar analyses on N = 1805 episodes from 10 services (M_age = 13.59 years, SD_age = 2.33, range = 8-24 years; 1120 or 62.05% female) also with complete information on mental health difficulties at follow up.

RESULTS

Girls had higher levels of mental health difficulties at baseline than boys (β = 0.28, 95% CI = 0.24-0.32). Young people with higher levels of mental health difficulties at baseline also had higher levels of deterioration in mental health difficulties at follow up (β = 0.72, 95% CI = 0.67-0.76), and girls had higher levels of deterioration in mental health difficulties at follow up than boys (β = 0.09, 95% CI = 0.03-0.16). Young people with social anxiety, panic disorder, low mood, or self-harm had higher levels of mental health difficulties at baseline and of deterioration in mental health difficulties at follow up compared to young people without these presenting problems.

CONCLUSIONS

Services seeing higher proportions of young people with higher levels of mental health difficulties at baseline, social anxiety, panic disorder, low mood, or self-harm may be expected to show lower levels of improvement in mental health difficulties at follow up.

Are We Comparing Apples with Oranges? Assessing Improvement Across Symptoms, Functioning, and Goal Progress for Adolescent Anxiety and Depression

Strategies for comparing routinely collected outcome data across services or systems include focusing on a common indicator (e.g., symptom change) or aggregating results from different measures or outcomes into a comparable core metric. The implications of either approach for judging treatment success are not fully understood. This study drew on naturalistic outcome data from 1641 adolescents with moderate or severe anxiety and/or depression symptoms who received routine specialist care across 60 mental health services in England. The study compared rates of meaningful improvement between the domains of internalizing symptoms, functioning, and progress towards self-defined goals. Consistent cross-domain improvement was observed in only 15.6% of cases. Close to one in four (24.0%) young people with reliably improved symptoms reported no reliable improvement in functioning. Inversely, one in three (34.8%) young people reported meaningful goal progress but no reliable symptom improvement. Monitoring systems that focus exclusively on symptom change risk over- or under-estimating actual impact, while aggregating different outcomes into a single metric can mask informative differences in the number and type of outcomes showing improvement. A move towards harmonized outcome measurement approaches across multiple domains is needed to ensure fair and meaningful comparisons.

Sociodemographic and clinical characteristics in child and youth mental health; comparison of routine outcome measurements of an Australian and Dutch outpatient cohort

AIMS

Although of great value to understand the treatment results for mental health problems obtained in clinical practice, studies using naturalistic data from children and adolescents seeking clinical care because of complex mental health problems are limited. Cross-national comparison of naturalistic outcomes in this population is seldomly done. Although careful consideration is needed, such comparisons are likely to contribute to an open dialogue about cross-national differences and may stimulate service improvement. The aim of this observational study is to investigate clinical characteristics and outcomes in naturalistic cohorts of specialized child and adolescent mental health outpatient care in two different countries.

METHODS

Routinely collected data from 2013 to 2018 of 2715 outpatients in the Greater Area of Brisbane, Australia (CYMHS) and 1158 outpatients in Leiden, the Netherlands (LUMC-Curium) were analysed. Demographics, clinical characteristics and severity of problems at start and end of treatment were described, using Children's Global Assessment Scale (CGAS), Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) and the parental Strength and Difficulties Questionnaire (SDQ-P).

RESULTS

Routine outcome measures (CGAS, HoNOSCA, SDQ-P) showed moderate to severe mental health problems at start of treatment, which improved significantly over time in both cohorts. Effect sizes ranged between 0.73-0.90 (CYMHS) and 0.57-0.76 (LUMC-Curium). While internalizing problems (mood disorder, anxiety disorder and stress-related disorder) were more prevalent at CYMHS, externalizing developmental problems (ADHD, autism) prevailed at LUMC-Curium. Comorbidity (>1 diagnosis on ICD10/DSM-IV) was relatively similar: 45% at CYMHS and 39 % at LUMC-Curium. In both countries, improvement of functioning was lowest for conduct disorder and highest for somatoform/conversion disorders and obsessive-compulsive disorders (OCD). Overall, 20-40% showed clinically significant improvement (shift from clinical-range at start to a non-clinical-range at the end of treatment), but nearly half of patients still experienced significant symptoms at discharge.

CONCLUSIONS

This large-scale outcome study showed both cohorts from Australia and the Netherlands improve during the course of treatment on clinician- and parent-reported measures. Although samples were situated within different contexts and differed in patient profiles, they showed similar trends in improvement per diagnostic group. While 20-40% showed clinically significant change, many patients experienced residual symptoms reflecting increased risk for negative outcome into adulthood. We emphasize cross-national comparison of naturalistic outcomes faces challenges, although it can similarly reveal trends in treatment outcome providing direction for future research: what factors determine discharge from specialized services; and how to improve current treatments in this severely affected population.

Cancer-related mortality among people with intellectual disabilities: A nationwide population-based cohort study

Background

Concerns have been raised about access to cancer screening and the timely receipt of cancer care for people with an intellectual disability (ID). However, knowledge about cancer mortality as a potential consequence of these disparities is still limited. This study, therefore, compared cancer‐related mortality patterns between people with and without ID.

Methods

A historical cohort study (2015‐2019) linked the Dutch adult population (approximately 12 million people with an ID prevalence of 1.45%) and mortality registries. Cancer‐related mortality was identified by the underlying cause of death (according to the chapter on neoplasms in the International Classification of Diseases, Tenth Revision). Observed mortality and calculated age‐ and sex‐standardized mortality ratios (SMRs) with 95% confidence intervals (CIs) were reported.

Results

There were 11,102 deaths in the ID population (21.7% cancer‐related; n = 2408) and 730,405 deaths in the general population (31.2%; n = 228,120) available for analysis. Cancer was noted as the cause of death more often among people with ID in comparison with the general population (SMR, 1.48; 95% CI, 1.42‐1.54), particularly in the young age groups. High‐mortality cancers included cancers within the national screening program (SMRs, 1.43‐1.94), digestive cancers (SMRs, 1.24‐2.56), bladder cancer (SMR, 2.07; 95% CI, 1.61‐2.54), and cancers of unknown primary (SMR, 2.48; 95% CI, 2.06‐2.89).

Conclusions

Cancer was reported as the cause of death approximately 1.5 times more often in people with ID compared with the general population. This mortality disparity may indicate adverse effects from inequalities in screening and cancer care experienced by people with ID.

Lay Summary

People with an intellectual disability (ID) may find it challenging to participate in cancer screening or to receive timely cancer care.

To understand potential consequences in terms of mortality, this study compared cancer‐related mortality between people with and without ID in the Netherlands.

Cancer was reported as the cause of death approximately 1.5 times more often among people with ID than others.

Because large differences were found that were related to screening cancers and cancers for which the primary tumor was unknown, this study's results raise concerns about equality in screening practices and cancer care for people with ID.

Cancer is reported as the cause of death approximately 1.5 times more often in people with an intellectual disability compared with those without one. Differences are particularly noted for deaths related to cancers within the national screening program and cancers of unknown primary.

Telling the story of complex change: an Impact Framework for the real world

Background

In the National Health Service (NHS) in England, traditional approaches to evidencing impact and value have an important role to play but are unlikely to demonstrate the full value of national quality improvement programmes and large-scale change initiatives in health and care. This type of work almost always takes place in complex and complicated settings, in that it involves multiple players, numerous interventions and a host of other confounding factors. Improvement work is usually emergent, with cause and effect only understood in hindsight; challenges around contribution and attribution can lead the key players to question how they can be certain that the described or observed changes are due to their intervention and would not have happened without them. In this complex environment, there is a risk of oversimplifying the observed impact and focusing instead on those things that are easier to measure, missing that which is important but more difficult to evidence.

Methods

Between 2016 and 2019, an action-orientated approach, drawing on realist and development evaluation approaches, was taken to designing and testing the Impact Framework. First, we undertook a pragmatic review of tools and approaches used by others to capture and demonstrate their impact both within and outside the health and care environment. Following the identification and review of these tools and approaches, and in consultation with national improvement teams in England about their evaluation challenges and aspirations, we developed a set of underpinning principles to inform the design and build of the framework. The principles were informed and finessed following conversations with improvement teams and programme leads in NHS England with respect to the challenges that they were facing and their aspirations in terms of demonstrating their impact and learning as they worked.

Results

The ‘Impact Framework’ described in this article offers a practical approach to capturing the impact of improvement work at any scale, taking account of unintended outcomes, considering attribution and contribution, and using a narrative approach to uncover the difference made by improvement initiatives in rich detail. In this article, we describe how the Impact Framework has been used with one of NHS England’s national programmes, Time for Care, which was delivered between 2016 and 2020.

Conclusions

The Impact Framework continues to be used, developed and further tested by national improvement programmes being delivered by NHS England and NHS Improvement and is updated regularly. The framework has been developed to be accessible to frontline teams and is supported by a set of resources to help improvement teams and individuals to use by themselves (https://www.england.nhs.uk/sustainableimprovement/impact-framework/).

Implementing evidence-based individual psychotherapies for adults with eating disorders in a real world clinical setting

OBJECTIVE

This study aimed to evaluate the effectiveness of evidenced-based psychological treatments (specifically, Cognitive-Behaviour Therapy for Eating Disorders [CBT-ED] and Maudsley Anorexia Nervosa Treatment for Adults [MANTRA]) for a transdiagnostic eating disorder population in a routine clinical setting. In particular, it aimed to determine the extent to which treatment was provided in line with current clinical guidelines (NICE, 2017) and how effective treatment was in improving eating disorder and general psychopathology.

METHOD

Three hundred and seventy-nine participants meeting criteria for DSM-5 anorexia nervosa, bulimia nervosa, binge-eating disorder or other specified feeding or eating disorder completed pre- and posttreatment measures of eating disorder pathology and general distress. Clinicians recorded weight and episodes of bingeing and purging.

RESULTS

Ninety seven percent of participants received treatment in line with evidence-based psychotherapies. Treatment was completed by 59.9% of the whole sample. Using stringent criteria and ITT analysis 21.4% met criteria for remission at end of treatment. In the underweight sample, there was a significant increase in BMI, averaging 1.38 kg/m² over treatment, with similar outcomes for MANTRA and CBT-ED.

DISCUSSION

These findings, in a large transdiagnostic population, add to emerging literature on the translation of evidence-based psychotherapies to real-world clinical settings. Our results converge well with prior similar studies. Findings highlight the need for routine data collection in services and for the ongoing improvement of treatments for the eating disorders.

Internalizing symptoms, well-being, and correlates in adolescence: A multiverse exploration via cross-lagged panel network models

Internalizing symptoms are the most prevalent mental health problem in adolescents, with sharp increases seen, particularly for girls, and evidence that young people today report more problems than previous generations. It is therefore critical to measure and monitor these states on a large scale and consider correlates. We used novel panel network methodology to explore relationships between internalizing symptoms, well-being, and inter/intrapersonal indicators. A multiverse design was used with 32 conditions to consider the stability of results across arbitrary researcher decisions in a large community sample over three years (N = 15,843, aged 11-12 at Time 1). Networks were consistently similar for girls and boys. Stable trait-like effects within anxiety, attentional, and social indicators were found. Within-person networks were densely connected and suggested mental health and inter/intrapersonal correlates related to one another in similar complex ways. The multiverse design suggested the particular operationalization of items can substantially influence conclusions. Nevertheless, indicators such as thinking clearly, unhappiness, dealing with stress, and worry showed more consistent centrality, suggesting these indicators may play particularly important roles in the development of mental health in adolescence.

A mixed methods evaluation of the large-scale implementation of a school- and community-based parenting program to reduce violence against children in Tanzania: a study protocol

BACKGROUND

Despite the rapid dissemination of parenting programs aiming to reduce and prevent violence against children (VAC) worldwide, there is limited knowledge about and evidence of the implementation of these programs at scale. This study addresses this gap by assessing the quality of delivery and impact of an evidence-based parenting program for parents/caregivers and their adolescent girls aged 9 to 14-Parenting for Lifelong Health Teens (PLH-Teens), known locally as Furaha Teens-on reducing VAC at scale in Tanzania. The study will explore participating family and staff perspectives on program implementation and examine factors associated with implementation and how implementation quality is associated with intervention outcomes when the program is delivered to approximately 50,000 parent-child dyads (N = 100,000) in schools and community centers across eight districts of Tanzania.

METHODS

This mixed-methods study will answer the following research questions: (1) what is the implementation quality and fidelity of PLH-Teens at scale in Tanzania; (2) what factors are associated with the quality of delivery and implementation fidelity of PLH-Teens; (3) how are implementation quality and fidelity associated with intervention outcomes; (4) what are participant and implementing staff perspectives on the acceptability, appropriateness, feasibility, benefits, and challenges of delivering PLH-Teens in their schools and communities; (5) what is the impact of PLH-Teens on VAC and participant well-being; and (6) how much does it cost to deliver PLH-Teens at scale? Qualitative and quantitative data will be collected directly from implementers, parents/caregivers, and adolescents using pre-post questionnaires, observational assessments, cost surveys, focus groups, and interviews. Qualitative data will be analyzed thematically with the aid of NVIVO software. Quantitative data will be cleaned and analyzed using methods such as correlation, regression, and structural equation models using Stata and R. COREQ and TREND guidelines will be used, where appropriate.

DISCUSSION

Findings will provide vital insights into some of the factors related to quality implementation at scale. Lessons learned regarding the implementation of PLH-Teens at scale will be applied in Tanzania, and also in the delivery of PLH parenting programs globally.

Assessing the usability and user engagement of Thought Spot - A digital mental health help-seeking solution for transition-aged youth

OBJECTIVE

To evaluate the perceived usability of and user engagement with a digital platform (Thought Spot) designed to enhance mental health and wellness help-seeking among transition-aged youth (TAY; 17-29-years old).

MATERIALS AND METHODS

Survey responses and usage patterns were collected as part of a randomized controlled trial evaluating the efficacy of Thought Spot. Participants given Thought Spot completed an adapted Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire to measure perceived usability of the platform. User engagement patterns on Thought Spot were examined using analytics data collected throughout the study (March 2018-June 2019).

RESULTS

A total of 131 transition-aged participants completed the USE questionnaire and logged on to Thought Spot at least once. Ease of learning scored higher than ease of use, usefulness and satisfaction. Participants identified numerous strengths and challenges related to usability, visual appeal, functionality and usefulness of the content. In terms of user engagement, most participants stopped using the platform after 3 weeks. Participants searched and were interested in a variety of resources, including mental health, counselling and social services.

DISCUSSION

Participants reported mixed experiences while using Thought Spot and exhibited low levels of long-term user engagement. User satisfaction, the willingness to recommend Thought Spot to others, and the willingness for future use appeared to be influenced by content relevance, ease of learning, available features, and other contextual factors. Analysis of the types of resources viewed and searches conducted by TAY end-users provided insight into their behaviour and needs.

CONCLUSION

Users had mixed perceptions about the usability of Thought Spot, which may have contributed to the high attrition rate. User satisfaction and engagement appears to be influenced by content relevance, ease of learning, and the types of features available. Further investigation to understand the contextual factors that affect TAYs' adoption and engagement with digital mental health tools is required.

Ethiopian Pediatric Society Quality Improvement Initiative: a pragmatic approach to facility-based quality improvement in low-resource settings

Objectives

To describe critical features of the Ethiopian Pediatric Society (EPS) Quality Improvement (QI) Initiative and to present formative research on mentor models.

Setting

General and referral hospitals in the Addis Ababa area of Ethiopia.

Participants

Eighteen hospitals selected for proximity to the EPS headquarters, prior participation in a recent newborn care training cascade and minimal experience with QI.

Interventions

Education in QI in a 2-hour workshop setting followed by implementation of a facility-based QI project with the support of virtual mentorship or in-person mentorship.

Primary and secondary outcome measures

Primary outcome—QI progress, measured using an adapted Institute for Healthcare Improvement Scale; secondary outcome—contextual factors affecting QI success as measured by the Model for Understanding Success in Quality.

Results

The dose and nature of mentoring encounters differed based on a virtual versus in-person mentoring approach. All QI teams conducted at least one large-scale change. Education of staff was the most common change implemented in both groups. We did not identify contextual factors that predicted greater QI progress.

Conclusions

The EPS QI Initiative demonstrates that education in QI paired with external mentorship can support implementation of QI in low-resource settings. This pragmatic approach to facility-based QI may be a scalable strategy for improving newborn care and outcomes. Further research is needed on the most appropriate instruments for measuring contextual factors in low/middle-income country settings.

Evaluating the dissemination and scale-up of two evidence-based parenting interventions to reduce violence against children: study protocol

Background

Eliminating violence against children is a prominent policy goal, codified in the Sustainable Development Goals, and parenting programs are one approach to preventing and reducing violence. However, we know relatively little about dissemination and scale-up of parenting programs, particularly in low- and middle-income countries (LMICs). The scale-up of two parenting programs, Parenting for Lifelong Health (PLH) for Young Children and PLH for Parents and Teens, developed under Creative Commons licensing and tested in randomized trials, provides a unique opportunity to study their dissemination in 25 LMICs.

Methods

The Scale-Up of Parenting Evaluation Research (SUPER) study uses a range of methods to study the dissemination of these two programs. The study will examine (1) process and extent of dissemination and scale-up, (2) how the programs are implemented and factors associated with variation in implementation, (3) violence against children and family outcomes before and after program implementation, (4) barriers and facilitators to sustained program delivery, and (5) costs and resources needed for implementation.Primary data collection, focused on three case study projects, will include interviews and focus groups with program facilitators, coordinators, funders, and other stakeholders, and a summary of key organizational characteristics. Program reports and budgets will be reviewed as part of relevant contextual information. Secondary data analysis of routine data collected within ongoing implementation and existing research studies will explore family enrolment and attendance, as well as family reports of parenting practices, violence against children, child behavior, and child and caregiver wellbeing before and after program participation. We will also examine data on staff sociodemographic and professional background, and their competent adherence to the program, collected as part of staff training and certification.

Discussion

This project will be the first study of its kind to draw on multiple data sources and methods to examine the dissemination and scale-up of a parenting program across multiple LMIC contexts. While this study reports on the implementation of two specific parenting programs, we anticipate that our findings will be of relevance across the field of parenting, as well as other violence prevention and social programs.

Impact of the initial response to COVID-19 on long-term care for people with intellectual disability: an interrupted time series analysis of incident reports

BACKGROUND

The lockdown-measures in response to COVID-19 taken by long-term care organisations might have impacted problem behaviour and behavioural functioning of people with intellectual disability. This study tested changes in reported incidents, in particular regarding aggression, unexplained absence and, for contrast, medication errors.

METHODS

Metadata on weekly incident and near-incident reports from 2016 to June 2020 involving over 14 000 clients with mild to serious intellectual disability of 's Heeren Loo, a long-term care organisation for people with intellectual disability, were subjected to interrupted time series analysis, comparing the COVID-19 with the pre-COVID-19 period.

RESULTS

The imposition of lockdown-measures coincided with a significant drop in incidents (total, P < .001; aggression, P = .008; unexplained absences, P = .008; and medication errors, P < .001). Incidents in total (P = .001) and with aggression (P < .001) then climbed from this initial low level, while medication errors remained stably low (P = .94).

CONCLUSION

The rise in incidents involving aggression, against the background of generally lowered reporting, underlines the need for pandemic control measures that are suitable for people with intellectual disability in long-term care.

Evaluating Large-Scale Integrated Care Projects: The Development of a Protocol for a Mixed Methods Realist Evaluation Study in Belgium

BACKGROUND

The twelve Integrated Care Program pilot projects (ICPs) created by the government plan 'Integrated Care for Better Health' aim to achieve four outcome types (the Quadruple Aim) for people with chronic diseases in Belgium: improved population health, improved patient and provider experiences and improved cost efficiency. The aim of this article is to present the development of a mixed methods realist evaluation of this large-scale, whole system change programme.

METHODS

A scientific team was commissioned to co-design and implement an evaluation protocol in close collaboration with the government, the ICPs and several other involved stakeholders.

RESULTS

A protocol for a mixed methods realist evaluation was developed to gain insights into the mechanisms that foster successful results in ICPs. The qualitative evaluation proposed will be based on the document analysis of yearly ICP progress reports, selected case studies and focus group interviews with stakeholders. Processes and outcomes of all the projects will be monitored using indicators based on administrative data on population health and the quality and costs of care. A yearly survey will be organized to collect data on patient-reported outcomes and experiences and on provider-reported measures of inter-professional collaboration and proper wellbeing. Using both quantitative and qualitative data, we will develop theories about the mechanisms and the associated contextual factors that lead to integrated care and the Quadruple Aim outcomes.

DISCUSSION

The objective of this study is to deliver policy recommendations on strategies and best practices to improve care integration in Belgium and to implement a sustainable monitoring system that serves both policy makers and the stakeholders within the ICPs. Some challenges due to the large scale of the project and the multiple stakeholders involved may impede the successful implementation of this proposal.

Moving beyond randomized controlled trials in the evaluation of compulsory community treatment

Compulsory community treatment for people with severe mental illness remains controversial due to conflicting research evidence. Recently, there have been challenges to the conventional view that trial-based evidence should take precedence. This paper adds to these challenges in three ways. First, it emphasizes the need for critiques of trials to engage with conceptual and not just technical issues. Second, it develops a critique of trials centred on both how we can have knowledge and what it is we can have knowledge of. Third, it uses this critique to develop a research strategy that capitalizes on the information in large-scale datasets.

Eating disorders

Eating disorders are disabling, deadly, and costly mental disorders that considerably impair physical health and disrupt psychosocial functioning. Disturbed attitudes towards weight, body shape, and eating play a key role in the origin and maintenance of eating disorders. Eating disorders have been increasing over the past 50 years and changes in the food environment have been implicated. All health-care providers should routinely enquire about eating habits as a component of overall health assessment. Six main feeding and eating disorders are now recognised in diagnostic systems: anorexia nervosa, bulimia nervosa, binge eating disorder, avoidant-restrictive food intake disorder, pica, and rumination disorder. The presentation form of eating disorders might vary for men versus women, for example. As eating disorders are under-researched, there is a great deal of uncertainty as to their pathophysiology, treatment, and management. Future challenges, emerging treatments, and outstanding research questions are addressed.

Implementing Narrative Exposure Therapy for unaccompanied asylum-seeking minors with post-traumatic stress disorder: A pilot feasibility report

There are high rates of post-traumatic stress disorder (PTSD) in unaccompanied asylum-seeking minors (UAM) and there is a requirement for feasible, acceptable and evidence-based treatments. Narrative Exposure Therapy (NET) is a short-term treatment for PTSD following multiple traumatic events. This article aims to examine the applicability of NET for UAM in routine clinical practice and to provide preliminary feasibility, acceptability and effectiveness data. The participants were four UAM receiving NET within a dedicated child and adolescent mental health service for refugee children. Semi-structured interviews were conducted to understand the acceptability of this approach and standardised measures of PTSD were used to provide preliminary data regarding the effectiveness of NET for these clients. The clients attended NET consistently with few missed appointments. At post-treatment, two clients' symptom scores were below the clinical cut-off for PTSD and all three clients who completed NET met reliable improvement criteria. The clients reported improvements in functional outcomes and mentioned that they would encourage other young people with similar difficulties to engage in NET. This study was limited by the small sample size and naturalistic time limitations in clinicians' contracts. This article highlights that it is possible to implement NET within routine clinical practice and observed improvements in PTSD symptoms and functional outcomes for UAM.

The Service of Research Analytics to Optimize Digital Health Evidence Generation: Multilevel Case Study

Background

The widespread adoption of digital health interventions for chronic disease self-management has catalyzed a paradigm shift in the selection of methodologies used to evidence them. Recently, the application of digital health research analytics has emerged as an efficient approach to evaluate these data-rich interventions. However, there is a growing mismatch between the promising evidence base emerging from analytics mediated trials and the complexity of introducing these novel research methods into evaluative practice.

Objective

This study aimed to generate transferable insights into the process of implementing research analytics to evaluate digital health interventions. We sought to answer the following two research questions: (1) how should the service of research analytics be designed to optimize digital health evidence generation? and (2) what are the challenges and opportunities to scale, spread, and sustain this service in evaluative practice?

Methods

We conducted a qualitative multilevel embedded single case study of implementing research analytics in evaluative practice that comprised a review of the policy and regulatory climate in Ontario (macro level), a field study of introducing a digital health analytics platform into evaluative practice (meso level), and interviews with digital health innovators on their perceptions of analytics and evaluation (microlevel).

Results

The practice of research analytics is an efficient and effective means of supporting digital health evidence generation. The introduction of a research analytics platform to evaluate effective engagement with digital health interventions into a busy research lab was ultimately accepted by research staff, became routinized in their evaluative practice, and optimized their existing mechanisms of log data analysis and interpretation. The capacity for research analytics to optimize digital health evaluations is highest when there is (1) a collaborative working relationship between research client and analytics service provider, (2) a data-driven research agenda, (3) a robust data infrastructure with clear documentation of analytic tags, (4) in-house software development expertise, and (5) a collective tolerance for methodological change.

Conclusions

Scientific methods and practices that can facilitate the agile trials needed to iterate and improve digital health interventions warrant continued implementation. The service of research analytics may help to accelerate the pace of digital health evidence generation and build a data-rich research infrastructure that enables continuous learning and evaluation.

Predicting admission to long-term care and mortality among community-based, dependent older people in Ireland

OBJECTIVE

To identify factors that predict admission to long-term care (LTC) and mortality among community-based, dependent older people in Ireland, who were in receipt of formal home support.

METHODS

An audit was conducted of all community-dwelling older adults receiving government funded home support during 2017 in the Dublin North Central, Health Service Executive administrative area. Data were extracted from the Common Summary Assessment Report (CSAR), a mandatory form used in the provision of home support. Multiple logistic regression analysis was used to examine the factors associated with admission to LTC and mortality, with the results presented as odds ratios (OR) and 95% confidence intervals.

RESULTS

The audit comprised 1597 community-dwelling older adults with a mean age of 83.3 (SD: 7.2) years. The prevalence of transition to LTC and mortality was 8% and 9%, respectively, during the 12-month period. Factors significantly associated with admission to LTC were "cognitive dysfunction" [OR 2.10 (1.41-3.14), P < .001] and the intensity of home support [OR 1.05 (1.01-1.06), P < .003], as measured by weekly formal care hours. Physical dependency and advanced age (aged 95 years +) were significantly associated with mortality in this population (P < .001).

CONCLUSION

"Cognitive dysfunction" and intensity of formal home support were associated with transition to LTC, while physical dependency and advanced age were associated with mortality. Investment in personalised, cognitive-specific, services and supports are necessary to keep people with dementia and related cognitive impairments living at home for longer.

Working with unaccompanied asylum-seeking young people: cultural considerations and acceptability of a cognitive behavioural group approach

This paper outlines a psychological skills group for unaccompanied asylum-seeking young people with a focus on cultural adaptations in the context of a UK mental health service. Unaccompanied asylum-seeking young people have typically experienced multiple losses, traumatic experiences, significant disruption and psychosocial stressors. These experiences occur during a key developmental period and outside of the context of a supportive family environment. Mental health difficulties are estimated to be present in 41–69% of this population. Prevalence rates are higher than among children seeking asylum with their families or children who are not from refugee or asylum-seeking backgrounds. Cognitive behavioural approaches were considered to be applicable and useful when working with this client group. Group approaches may offer unique benefits for this population through peer support and normalization. The group described was planned around three key themes: physical health needs, emotional wellbeing and resilience-building. A number of adaptations were made to meet the needs of this population which included engagement, considering physical health needs, sleep, language needs, issues related to power, race and status, and thinking about the needs of the group as young people. Attendance ratings, session rating scale outcomes, preliminary effectiveness data and qualitative feedback from young people identified that this is an acceptable approach for these young people. Unaccompanied asylum-seeking young people require a broad package of care; however, making adaptations to routine practice allowed access to evidence-based interventions to support mental health and wellbeing.

Evaluation of reliable improvement rates in depression and anxiety at the end of treatment in adolescents

BACKGROUND

Literature has focused on effect sizes rather than individual-level improvement rates to determine how effectively services address burgeoning numbers of adolescents with anxiety and depression.AimsTo consider how many adolescents report reliable improvement in anxiety, depression and comorbid depression and anxiety by end of treatment.

METHOD

The primary outcome was reliable improvement (i.e. change greater than likely the result of measurement error) in self-reported anxiety and depression for N = 4464 adolescents (mean age 14.5 years, s.d. = 1.9; 75% female; 61% White) seen in specialist mental health services in England.

RESULTS

In total, 53% of those with anxiety, 44% with depression, and 35% with comorbid depression and anxiety showed reliable improvement.

CONCLUSIONS

Improvement rates were higher than previously reported, but lower than generally used in advice to the public. There may be a need to set more realistic expectations, including with young people who seek help.Declaration of interestAll authors were involved in the programme of service transformation that this report draws on. M.W. led the outcomes and evaluation group that agreed the approach to measurement used in the initiative.

Studying complexity in health services research: desperately seeking an overdue paradigm shift

Complexity is much talked about but sub-optimally studied in health services research. Although the significance of the complex system as an analytic lens is increasingly recognised, many researchers are still using methods that assume a closed system in which predictive studies in general, and controlled experiments in particular, are possible and preferred. We argue that in open systems characterised by dynamically changing inter-relationships and tensions, conventional research designs predicated on linearity and predictability must be augmented by the study of how we can best deal with uncertainty, unpredictability and emergent causality. Accordingly, the study of complexity in health services and systems requires new standards of research quality, namely (for example) rich theorising, generative learning, and pragmatic adaptation to changing contexts. This framing of complexity-informed health services research provides a backdrop for a new collection of empirical studies. Each of the initial five papers in this collection illustrates, in different ways, the value of theoretically grounded, methodologically pluralistic, flexible and adaptive study designs. We propose an agenda for future research and invite researchers to contribute to this on-going series.