Online screening and feedback to increase help-seeking for mental health problems: population-based randomised controlled trial

The Trajectories of Online Mental Health Information Seeking: Modeling Search Behavior Before and After Completion of Self-report Screens

There is an appreciable mental health treatment gap in the United States. Efforts to bridge this gap and improve resource accessibility have led to the provision of online, clinically-validated tools for mental health self-assessment. In theory, these screens serve as an invaluable component of information-seeking, representing the preparative and action-oriented stages of this process while altering or reinforcing the search content and language of individuals as they engage with information online. Accordingly, this work investigated the association of screen completion with mental health-related search behaviors. Three-year internet search histories from N=7,572 Microsoft Bing users were paired with their respective depression, anxiety, bipolar disorder, or psychosis online screen completion and sociodemographic data available through Mental Health America. Data was transformed into network representations to model queries as discrete steps with probabilities and times-to-transition from one search type to another. Search data subsequent to screen completion was also modeled using Markov chains to simulate likelihood trajectories of different search types through time. Differences in querying dynamics relative to screen completion were observed, with searches involving treatment, diagnosis, suicidal ideation, and suicidal intent commonly emerging as the highest probability behavioral information seeking endpoints. Moreover, results pointed to the association of low risk states of psychopathology with transitions to extreme clinical outcomes (i.e., active suicidal intent). Future research is required to draw definitive conclusions regarding causal relationships between screens and search behavior.

The efficacy of automated feedback after internet-based depression screening (DISCOVER): an observer-masked, three-armed, randomised controlled trial in Germany

BACKGROUND

Despite the availability of effective treatments, most depressive disorders remain undetected and untreated. Internet-based depression screening combined with automated feedback of screening results could reach people with depression and lead to evidence-based care. We aimed to test the efficacy of two versions of automated feedback after internet-based screening on depression severity compared with no feedback.

METHODS

DISCOVER was an observer-masked, three-armed, randomised controlled trial in Germany. We recruited individuals (aged ≥18 years) who were undiagnosed with depression and screened positive for depression on an internet-based self-report depression rating scale (Patient Health Questionnaire-9 [PHQ-9] ≥10 points). Participants were randomly assigned 1:1:1 to automatically receive no feedback, non-tailored feedback, or tailored feedback on the depression screening result. Randomisation was stratified by depression severity (moderate: PHQ-9 score 10-14 points; severe: PHQ-9 score ≥15 points). Participants could not be masked but were kept unaware of trial hypotheses to minimise expectancy bias. The non-tailored feedback included the depression screening result, a recommendation to seek professional diagnostic advice, and brief general information about depression and its treatment. The tailored feedback included the same basic information but individually framed according to the participants' symptom profiles, treatment preferences, causal symptom attributions, health insurance, and local residence. Research staff were masked to group allocation and outcome assessment as these were done using online questionnaires. The primary outcome was change in depression severity, defined as change in PHQ-9 score 6 months after random assignment. Analyses were conducted following the intention-to-treat principle for participants with at least one follow-up visit. This trial was registered at ClinicalTrials.gov, NCT04633096.

FINDINGS

Between Jan 12, 2021, and Jan 31, 2022, 4878 individuals completed the internet-based screening. Of these, 1178 (24%) screened positive for depression (mean age 37·1 [SD 14·2] years; 824 [70%] woman, 344 [29%] men, and 10 [1%] other gender identity). 6 months after random assignment, depression severity decreased by 3·4 PHQ-9 points in the no feedback group (95% CI 2·9-4·0; within-group d 0·67; 325 participants), by 3·5 points in the non-tailored feedback group (3·0-4·0; within-group d 0·74; 319 participants), and by 3·7 points in the tailored feedback group (3·2-4·3; within-group d 0·71; 321 participants), with no significant differences among the three groups (p=0·72). The number of participants seeking help for depression or initiating psychotherapy or antidepressant treatment did not differ among study groups. The results remained consistent when adjusted for fulfilling the DSM-5-based criteria for major depressive disorder or subjective belief of having a depressive disorder. Negative effects were reported by less than 1% of the total sample 6 months after random assignment.

INTERPRETATION

Automated feedback following internet-based depression screening did not reduce depression severity or prompt sufficient depression care in individuals previously undiagnosed with but affected by depression.

FUNDING

German Research Foundation.

A Brief Workplace Training Program to Support Help-Seeking for Mental Ill-Health: Protocol for the Helipad Cluster Randomized Controlled Trial

BACKGROUND

Most people with mental health problems do not seek help, with delays of even decades in seeking professional help. Lack of engagement with professional mental health services can lead to poor outcomes and functional impairment. However, few effective interventions have been identified to improve help-seeking in adults, and those that exist are not widely implemented to deliver public health impact. Co-designing interventions with people with lived experience of mental ill-health and other relevant stakeholders is critical to increase the likelihood of uptake and engagement with these programs.

OBJECTIVE

This study aims to (1) test the effectiveness of a co-designed help-seeking program on increasing professional help-seeking intentions in employees in a workplace setting; (2) determine whether the program reduces mental illness stigma and improves help-seeking intentions and behavior, mental health literacy, mental health symptoms, and work and activity functioning relative to the control condition; (3) explore factors that facilitate broader implementation of the co-designed program; and (4) explore the cost-effectiveness of the co-designed program compared to the control condition over 6 months.

METHODS

A 2-arm cluster randomized controlled trial will be conducted (target sample: N=900 from 30 to 36 workplaces, with n=25 to 35 participants per workplace). The trial will compare the relative effectiveness of an enhanced interactive program (intervention condition) with a standard psychoeducation-alone program (active control condition) on the primary outcome of professional help-seeking intentions as measured by the General Help-Seeking Questionnaire. Secondary outcomes include the impact on mental illness stigma; mental health literacy; help-seeking attitudes and behavior; work and activity functioning; quality of life; and symptoms of mental ill-health including depression, anxiety, and general psychological distress.

RESULTS

Facilitators of and risks to the trial are identified and addressed in this protocol. Recruitment of workplaces is scheduled to commence in the first quarter of 2024.

CONCLUSIONS

If effective, the program has the potential to be ready for rapid dissemination throughout Australia, with the potential to increase appropriate and efficient service use across the spectrum of evidence-based services.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry (ANZCTR) ACTRN12623000270617p; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=385376.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/55529.

How adults with suspected depressive disorder experience online depression screening: A qualitative interview study

Background

While evidence on the effects and mechanisms of online depression screening is inconclusive, publicly available 'online depression tests' are already frequently used. To further a comprehensive understanding of online depression screening and evince the perspectives of those affected, we aimed to qualitatively explore how adults with undiagnosed but suspected depressive disorder experience the screening process.

Methods

This study is a qualitative follow-up of a German-wide, 3-arm, randomised controlled trial on feedback after online depression screening conducted between Jan 2021 and Sep 2022. A subsample of 26 participants with undiagnosed but suspected depressive disorder (Patient Health Questionnaire-9 ≥ 10; no depression diagnosis/treatment within the last year) were purposefully selected based on maximum variation in gender, age, and study arm. In-depth semi-structured telephone interviews (mean = 37 min) were conducted approximately six months after screening. Data were analysed within a contextualist theoretical framework using inductive reflexive thematic analysis.

Results

Participants were balanced in terms of gender (female/male, n = 15/11), age (range = 22 to 61 years), and study arm (no feedback/standard feedback/tailored feedback, n = 7/11/8). Reported experiences of online depression screening can be described as a two-step process: Step 1 is the initial reaction to the screening procedure and comprises the theme recognition of depressive symptoms: from denial to awareness. Step 2 describes a subsequent self-explorative process encompassing the themes cognitive positioning: rejection vs. acceptance, emotional reaction: between overload and empowerment, and personal activation: from reflection to action.

Conclusions

Findings indicate that online depression screening with and without feedback of results is experienced as a two-step process promoting symptom recognition and subsequent self-exploration. While few participants reported negative effects, the majority described the screening process as insightful, empowering, and activating. Future research should determine to what extent online depression screening may pose a standalone form of low-threshold support for individuals with undiagnosed depressive disorder, while focusing as well on potential negative effects.

Preliminary Clinical Outcomes of the Hello Sunday Morning Alcohol and Wellbeing Self-Assessment: Feasibility and Acceptability Study

BACKGROUND

Alcohol-related injuries and diseases are a leading cause of morbidity and mortality worldwide. Early intervention is essential given the chronic, relapsing nature of alcohol use disorders. There is significant potential for widely accessible web-based screening tools to help individuals determine where they stand in terms of alcohol use and provide support recommendations. Screening and brief interventions (SBIs) provide individuals with a stigma-free opportunity to learn and think about the potential risks of drinking and prompt help-seeking behavior by incorporating behavior change techniques. Furthermore, as excessive alcohol use and mental health problems often occur concurrently, SBIs for both conditions simultaneously can potentially address a critical gap in alcohol and mental health treatment.

OBJECTIVE

We investigated the feasibility, acceptability, and clinical outcomes of participants completing the Alcohol and Wellbeing Self-assessment (A&WS), a web-based SBI.

METHODS

The A&WS is freely available on the Hello Sunday Morning website as part of an uncontrolled observational prospective study. Feasibility was assessed based on the number of respondents who commenced and subsequently completed the A&WS. Acceptability was measured via participant feedback to determine overall satisfaction, perceived helpfulness, and likelihood of recommending the A&WS to others. Clinical outcomes were measured in two ways: (1) self-reported changes in alcohol consumption (Alcohol Use Disorders Identification Test score) or psychological distress (Kessler Psychological Distress Scale score) over time and (2) help seeking-both self-reported and immediate web-based help seeking. Preliminary baseline data collected for the first 9 months (March 2022 to December 2022) of the study were reported, including the 3-month follow-up outcomes.

RESULTS

A total of 17,628 participants commenced the A&WS, and of these, 14,419 (81.8%) completed it. Of those 14,419 who completed the A&WS, 1323 (9.18%) agreed to participate in the follow-up research. Acceptability was high, with 78.46% (1038/1323) reporting high satisfaction levels overall; 95.62% (1265/1323) found the A&WS easy to use and would recommend the tool to others. The 1-, 2-, and 3-month follow-ups were completed by 28.57% (378/1323), 21.09% (279/1323), and 17.61% (233/1323) of the participants, respectively. Significant reductions in the Alcohol Use Disorders Identification Test Consumption subscale (P<.001) and Kessler Psychological Distress Scale scores (P<.001) were observed over the 3-month follow-up period.

CONCLUSIONS

Our results suggest that the A&WS is a highly feasible and acceptable digital SBI that may support individuals in making changes to their alcohol consumption and improve their psychological well-being. In the absence of a control group, positive clinical outcomes cannot be attributed to the A&WS, which should now be subjected to a randomized controlled trial. This scalable, freely available tool has the potential to reach a large number of adults who might not otherwise access help while complementing the alcohol and mental health treatment ecosystem.

Factors Influencing Community Participation in Internet Interventions Compared With Research Trials: Observational Study in a Nationally Representative Adult Cohort

BACKGROUND

Digital mental health (DMH) programs can be effective in treating and preventing mental health problems. However, community engagement with these programs can be poor. Understanding the barriers and enablers of DMH program use may assist in identifying ways to increase the uptake of these programs, which have the potential to provide broad-scale prevention and treatment in the community.

OBJECTIVE

In this study, we aimed to identify and compare factors that may influence participation in DMH programs in practice and research trials, identify any respondent characteristics that are associated with these factors, and assess the relationship between intentions to use DMH programs and actual uptake.

METHODS

Australian adults aged ≥18 years were recruited from market research panels to participate in the study. The sample was representative of the Australian adult population based on age, gender, and location. Participants completed a cross-sectional web-based survey assessing demographic characteristics, mental health symptom measures, attitudes and use of DMH programs in practice and in research studies, and the factors influencing their use in both settings.

RESULTS

Across both research and practice, trust in the organization delivering the service or trial was the top-ranked factor influencing participation, followed by anonymity or privacy and adequate information. There was little variation in rankings across demographic groups, including intentions to use DMH programs or mental health status. Intentions to use DMH programs were a strong predictor of both current (odds ratio 2.50, 99% CI 1.41-4.43; P<.001) and past (odds ratio 2.98, 99% CI 1.71-5.19; P<.001) use behaviors.

CONCLUSIONS

Efforts to increase the uptake of DMH programs or participation in research trials should focus on clearly communicating the following to users: the legitimacy of the organization delivering the program, security and use of participant data, and effectiveness of DMH programs.

Digital Interventions to Enhance Readiness for Psychological Therapy: Scoping Review

BACKGROUND

Psychological therapy is an effective treatment method for mental illness; however, many people with mental illness do not seek treatment or drop out of treatment early. Increasing client uptake and engagement in therapy is key to addressing the escalating global problem of mental illness. Attitudinal barriers, such as a lack of motivation, are a leading cause of low engagement in therapy. Digital interventions to increase motivation and readiness for change hold promise as accessible and scalable solutions; however, little is known about the range of interventions being used and their feasibility as a means to increase engagement with therapy.

OBJECTIVE

This review aimed to define the emerging field of digital interventions to enhance readiness for psychological therapy and detect gaps in the literature.

METHODS

A literature search was conducted in PubMed, PsycINFO, PsycARTICLES, Scopus, Embase, ACM Guide to Computing Literature, and IEEE Xplore Digital Library from January 1, 2006, to November 30, 2021. The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) methodology was applied. Publications were included when they concerned a digitally delivered intervention, a specific target of which was enhancing engagement with further psychological treatment, and when this intervention occurred before the target psychological treatment.

RESULTS

A total of 45 publications met the inclusion criteria. The conditions included depression, unspecified general mental health, comorbid anxiety and depression, smoking, eating disorders, suicide, social anxiety, substance use, gambling, and psychosis. Almost half of the interventions (22/48, 46%) were web-based programs; the other formats included screening tools, videos, apps, and websites. The components of the interventions included psychoeducation, symptom assessment and feedback, information on treatment options and referrals, client testimonials, expectation management, and pro-con lists. Regarding feasibility, of the 16 controlled studies, 7 (44%) measuring actual behavior or action showed evidence of intervention effectiveness compared with controls, 7 (44%) found no differences, and 2 (12%) indicated worse behavioral outcomes. In general, the outcomes were mixed and inconclusive owing to variations in trial designs, control types, and outcome measures.

CONCLUSIONS

Digital interventions to enhance readiness for psychological therapy are broad and varied. Although these easily accessible digital approaches show potential as a means of preparing people for therapy, they are not without risks. The complex nature of stigma, motivation, and individual emotional responses toward engaging in treatment for mental health difficulties suggests that a careful approach is needed when developing and evaluating digital readiness interventions. Further qualitative, naturalistic, and longitudinal research is needed to deepen our knowledge in this area.

Technology-Based Interventions to Improve Help-Seeking for Mental Health Concerns: A Systematic Review

Background

Despite the high prevalence of mental health disorders worldwide, a significant proportion of distressed individuals do not seek professional help. Digital technology can be a potential bridge to reduce the treatment gap for mental disorders. A systematic review was undertaken to examine the technology-based interventions aimed at improving help-seeking attitude, intention, or behavior for mental health concerns.

Methods

The literature search was conducted in January-February 2020 through various e-databases using relevant keywords that targeted help-seeking interventions for mental health disorders via different technology modes.

Results

21 studies (15 randomized controlled trials and six non-randomized studies) were reviewed. The included studies were published between April 2006 to February 2020. Majority of the interventions led to an increase in the help-seeking variables. The crucial role of online delivery, participant involvement, and embedded links to professional services in encouraging help-seeking is highlighted. The review emphasizes the need for understanding utility of multicomponent interventions with personalized elements targeting help-seeking behavior, particularly in low-middle-income countries, and studies involving longer duration follow-ups.

Conclusion

This systematic review is the first of its kind to examine technology-based interventions to improve help-seeking for mental health and suggests that such interventions play a crucial role in positively impacting help-seeking. The complex interplay between the relevant variables such as mental health literacy, stigma, help-seeking attitude, intention and behavior, and the intervention components that may have a differential bearing on these variables are issues that merit urgent attention in further research.

"I Wanted to See How Bad it Was": Online Self-screening as a Critical Transition Point Among Young Adults with Common Mental Health Conditions

Young adults have high rates of mental health conditions, yet they are the age group least likely to seek traditional treatment. They do, however, seek information about their mental health online, including by filling out online mental health screeners. To better understand online self-screening, and its role in help-seeking, we conducted focus groups with 50 young adults who voluntarily completed a mental health screener hosted on an advocacy website. We explored (1) catalysts for taking the screener, (2) anticipated outcomes, (3) reactions to the results, and (4) desired next steps. For many participants, the screener results validated their lived experiences of symptoms, but they were nevertheless unsure how to use the information to improve their mental health moving forward. Our findings suggest that online screeners can serve as a transition point in young people's mental health journeys. We discuss design implications for online screeners, post-screener feedback, and digital interventions broadly.

Group-tailored feedback on online mental health screening for university students: using cluster analysis

BACKGROUND

The method by which mental health screening result reports are given affects the user's health behavior. Lists with the distribution of scores in various mental health areas is difficult for users to understand, and if the results are negative, they may feel more embarrassed than necessary. Therefore, we propose using group-tailored feedback, grouping people of similar mental health types by cluster analysis for comprehensive explanations of multidimensional mental health.

METHODS

This cross-sectional, observational study was conducted using a qualitative approach based on cluster analysis. Data were collected via a developed mental screening website, with depression, anxiety, sleep problems, perfectionism, procrastination, and attention assessed for 2 weeks in January 2020 in Korea. Participants were randomly recruited, and sample size was 174. Total was divided into 25 with severe depression/anxiety (SDA+) and 149 without severe depression/anxiety (SDA-) according to the PHQ-9 and GAD-7 criteria. Cluster analysis was conducted in each group, and an ANOVA was performed to find significant clusters. Thereafter, structured discussion was performed with mental health professionals to define the features of the clusters and construct the feedback content initially. Thirteen expert counselors were interviewed to reconstruct the content and validate the effectiveness of the developed feedback.

RESULTS

SDA- was divided into 3 using the k-means algorithm, which showed the best performance (silhouette score = 0.32, CH score = 91.67) among the clustering methods. Perfectionism and procrastination were significant factors in discretizing the groups. SDA+ subgroups were integrated because only 25 people belonged to this group, and they need professional help rather than self-care. Mental status and treatment recommendations were determined for each group, and group names were assigned to represent their features. The developed feedback was assessed to improve mental health literacy (MHL) through integrative and understandable explanations of multidimensional mental health. Moreover, it appeared that a sense of belonging was induced to reduce reluctance to face the feedback.

CONCLUSIONS

This study suggests group-tailored feedback using cluster analysis, which identifies groups of university students by integrating multidimensions of mental health. These methods can help students increase their interest in mental health and improve MHL to enable timely help.

Impact of online mental health screening tools on help-seeking, care receipt, and suicidal ideation and suicidal intent: Evidence from internet search behavior in a large U.S. cohort

INTRODUCTION

Most people with psychiatric illnesses do not receive treatment for almost a decade after disorder onset. Online mental health screens reflect one mechanism designed to shorten this lag in help-seeking, yet there has been limited research on the effectiveness of screening tools in naturalistic settings.

MATERIAL AND METHODS

We examined a cohort of persons directed to a mental health screening tool via the Bing search engine (n = 126,060). We evaluated the impact of tool content on later searches for mental health self-references, self-diagnosis, care seeking, psychoactive medications, suicidal ideation, and suicidal intent. Website characteristics were evaluated by pairs of independent raters to ascertain screen type and content. These included the presence/absence of a suggestive diagnosis, a message on interpretability, as well as referrals to digital treatments, in-person treatments, and crisis services.

RESULTS

Using machine learning models, the results suggested that screen content predicted later searches with mental health self-references (AUC = 0·73), mental health self-diagnosis (AUC = 0·69), mental health care seeking (AUC = 0·61), psychoactive medications (AUC = 0·55), suicidal ideation (AUC = 0·58), and suicidal intent (AUC = 0·60). Cox-proportional hazards models suggested individuals utilizing tools with in-person care referral were significantly more likely to subsequently search for methods to actively end their life (HR = 1·727, p = 0·007).

DISCUSSION

Online screens may influence help-seeking behavior, suicidal ideation, and suicidal intent. Websites with referrals to in-person treatments could put persons at greater risk of active suicidal intent. Further evaluation using large-scale randomized controlled trials is needed.

Clinical effectiveness of patient-oriented depression feedback in primary care: The empirical method of the GET.FEEDBACK.GP multicenter randomized controlled trial

GET.FEEDBACK.GP₁ is a multicenter randomized controlled trial testing the efficacy of patient-oriented depression feedback in primary care. This paper describes the complex methods and procedures of the trial. The primary outcome is depression severity six months after feedback, and we vary who is the target of the feedback as follows: no one receives feedback, only general practitioners receive feedback, and both patients and general practitioners receive feedback. The procedure includes a baseline assessment in primary care practices and three telephone follow-up interviews after one, six, and twelve months. The patients completed a baseline assessment, which determined their depression severity. Those with at least a moderate depression severity (PHQ-9₅ ≥ 10) were randomly allocated to three groups stratified by depression severity. A standardized mean difference of d = 0.25 with power 1 - β = 0.80 required a total sample size of N = 699. The patients provided responses regarding the primary and secondary outcomes at follow-up. The extensive planning for GET.FEEDBACK.GP involved experts from diverse medical specialties and external corporations. Of particular importance were (a) blinding in the study inclusion and random assignment with data capture software, (b) representative and unbiased patient selection in practice waiting rooms, (c) a data management and safety plan supplied by a specialized trial center, and (d) the use of participant pseudonyms supplied by a specialized service (Mainzelliste). The data collection started in July 2019 and will continue until June 2022. Five university study centers in Germany are participating in the trial.

The impact of communicating personal mental ill-health risk: A randomized controlled non-inferiority trial

AIM

Risk algorithms predicting personal mental ill-health will form an important component of digital and personalized preventive interventions, yet it is unknown whether informing people of personal risk may cause unintended harm. This trial evaluated the comparative effect of communicating personal mental ill-health risk profiles on psychological distress.

METHODS

Australian participants using a mood-monitoring app were randomly allocated to receiving their current personal mental ill-health risk profile (n = 119), their achievable personal risk profile (n = 118) or to a control group (n = 118) in which no risk information was communicated, in a non-inferiority trial design. The primary outcome was psychological distress at four-weeks as assessed on the Kessler Psychological Distress Scale.

RESULTS

There was high attrition in the trial with 64% of data missing at follow up. Per-protocol (completer) analysis found that the lower bounds of the confidence intervals of the estimated mean change of the current risk (m = 0.19, 95% CI: -2.59- 2.98) and achievable risk (m = -0.09, 95% CI: -2.84 to 2.66) groups were within the non-inferiority margin of the control group's mean at follow up. Supplementary intention-to-treat analysis using Multivariate Imputation by Chained Equations (MICE) found that 98/100 imputed datasets of the current risk profile group, and all imputed datasets of the achievable risk profile group showed non-inferiority to the control group.

CONCLUSIONS

This study provides preliminary support that providing personal mental health risk profiles does not lead to unacceptable worsening of distress compared to no risk feedback, although this needs to be replicated in a fully powered RCT.

A cluster randomised controlled trial of a web-based youth mental health service in Australian schools

BACKGROUND

Secondary schools have attempted to address gaps in help-seeking for mental health problems with little success. This trial evaluated the effectiveness of a universal web-based service (Smooth Sailing) for improving help-seeking intentions for mental health problems and other related outcomes among students.

METHODS

A cluster randomised controlled trial was conducted to evaluate the 12-week outcomes of the Smooth Sailing service among 1841 students from 22 secondary schools in New South Wales, Australia. Assignment was conducted at the school level. The control condition received school-as-usual. The primary outcome was help-seeking intentions for general mental health problems at 12-weeks post-baseline. Secondary outcomes included help-seeking behaviour, anxiety and depressive symptoms, psychological distress, psychological barriers to help-seeking, and mental health literacy. Data were analysed using mixed linear models. This trial was registered with the Australian and New Zealand Clinical Trials Registry (ACTRN12618001539224).

FINDINGS

At 12-weeks post-baseline, there was a marginal statistical difference in the relative means of help-seeking intentions (effect size=0•10, 95%CI: -0•02-0•21) that favoured the intervention condition. Help-seeking from adults declined in both conditions. There was a greater reduction in the number of students who "needed support for their mental health but were not seeking help" in the intervention condition (OR: 2•08, 95%CI: 1•72-2.27, P<•0001). No other universal effects were found. Participants found the service easy to use and understand; However, low motivation, time, forgetfulness, and lack of perceived need were barriers to use.

INTERPRETATION

Smooth Sailing led to small improvements in help-seeking intentions. Refinements are needed to improve its effectiveness on other mental health outcomes and to increase student uptake and engagement.

FUNDING

HSBC and Graf Foundation.

Impact of a Web-Based Psychiatric Assessment on the Mental Health and Well-Being of Individuals Presenting With Depressive Symptoms: Longitudinal Observational Study

BACKGROUND

Web-based assessments of mental health concerns hold great potential for earlier, more cost-effective, and more accurate diagnoses of psychiatric conditions than that achieved with traditional interview-based methods.

OBJECTIVE

The aim of this study was to assess the impact of a comprehensive web-based mental health assessment on the mental health and well-being of over 2000 individuals presenting with symptoms of depression.

METHODS

Individuals presenting with depressive symptoms completed a web-based assessment that screened for mood and other psychiatric conditions. After completing the assessment, the study participants received a report containing their assessment results along with personalized psychoeducation. After 6 and 12 months, participants were asked to rate the usefulness of the web-based assessment on different mental health-related outcomes and to self-report on their recent help-seeking behavior, diagnoses, medication, and lifestyle changes. In addition, general mental well-being was assessed at baseline and both follow-ups using the Warwick-Edinburgh Mental Well-being Scale (WEMWBS).

RESULTS

Data from all participants who completed either the 6-month or the 12-month follow-up (N=2064) were analyzed. The majority of study participants rated the study as useful for their subjective mental well-being. This included talking more openly (1314/1939, 67.77%) and understanding one's mental health problems better (1083/1939, 55.85%). Although most participants (1477/1939, 76.17%) found their assessment results useful, only a small proportion (302/2064, 14.63%) subsequently discussed them with a mental health professional, leading to only a small number of study participants receiving a new diagnosis (110/2064, 5.33%). Among those who were reviewed, new mood disorder diagnoses were predicted by the digital algorithm with high sensitivity (above 70%), and nearly half of the participants with new diagnoses also had a corresponding change in medication. Furthermore, participants' subjective well-being significantly improved over 12 months (baseline WEMWBS score: mean 35.24, SD 8.11; 12-month WEMWBS score: mean 41.19, SD 10.59). Significant positive predictors of follow-up subjective well-being included talking more openly, exercising more, and having been reviewed by a psychiatrist.

CONCLUSIONS

Our results suggest that completing a web-based mental health assessment and receiving personalized psychoeducation are associated with subjective mental health improvements, facilitated by increased self-awareness and subsequent use of self-help interventions. Integrating web-based mental health assessments within primary and/or secondary care services could benefit patients further and expedite earlier diagnosis and effective treatment.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/18453.

Interaktive Gesundheitskompetenz, soziodemografische Faktoren und der Zeitraum bis zur Inanspruchnahme psychotherapeutischer oder medikamentöser Behandlung – gibt es Hinweise auf soziale Ungleichheiten in der Therapie der Sozialen Angststörung?

Ziel Ziel ist die Untersuchung des Zusammenhangs zwischen soziodemografischen Merkmalen sowie der interaktiven Gesundheitskompetenz und dem Zeitraum zwischen der Behandlungsentscheidung und der Inanspruchnahme einer Therapie der Sozialen Angststörung.

Methodik Hierzu wurde eine Online-Befragung von N = 311 Personen mit Sozialer Angststörung durchgeführt (Responserate 54,1 %). Neben deskriptiven statistischen Verfahren wurde eine logistische Regressionsanalyse durchgeführt.

Ergebnisse Die Befragten sind durchschnittlich 46 Jahre alt (20–81), 59 % sind Frauen. Höheres Alter (OR 2,579), nicht in Partnerschaft lebend (OR 1,963), Angst vor persönlichen Kontakten (OR 5,716) und geringe (OR 3,585) bzw. mäßige (OR 3,144) interaktive Gesundheitskompetenz sind signifikant mit der Wartezeit auf eine Behandlung assoziiert.

Schlussfolgerung Die Ergebnisse können als Hinweise auf soziale Ungleichheiten in Bezug auf die Inanspruchnahme einer Therapie bei Personen mit sozialer Angststörung gedeutet werden.

Social media recruitment for mental health research: A systematic review

BACKGROUND

Social media holds exciting promise for advancing mental health research recruitment, however, the extent and efficacy to which these platforms are currently in use are underexplored.

OBJECTIVE

A systematic review was conducted to characterize the current use and efficacy of social media in recruiting participants for mental health research.

METHOD

A literature review was performed using MEDLINE, EMBASE, and PsychINFO. Only non-duplicative manuscripts written in the English language and published between 1/1/2004-3/31/2019 were selected for further screening. Data extracted included study type and design, participant inclusion criteria, social media platform, advertising strategy, final recruited sample size, recruitment location, year, monetary incentives, comparison to other recruitment methods if performed, and final cost per participant.

RESULTS

A total of 176 unique studies that used social media for mental health research recruitment were reviewed. The majority of studies were cross-sectional (62.5%) in design and recruited adults. Facebook was overwhelmingly the recruitment platform of choice (92.6%), with the use of paid advertisements being the predominant strategy (60.8%). Of the reviewed studies, substance abuse (43.8%) and mood disorders (15.3%) were the primary subjects of investigation. In 68.3% of studies, social media recruitment performed as well as or better than traditional recruitment methods in the number and cost of final enrolled participants. The majority of studies used Facebook for recruitment at a median cost per final recruited study participant of $19.47. In 55.6% of the studies, social media recruitment was the more cost-effective recruitment method when compared to traditional methods (e.g., referrals, mailing).

CONCLUSION

Social media appears to be an effective and economical recruitment tool for mental health research. The platform raises methodological and privacy concerns not covered in current research regulations that warrant additional consideration.

Prevalence and association of discrimination and bullying and the impact on mental health in an occupational setting

Purpose

Discrimination and bullying contribute to mental ill-health in the workplace. At face value, they would seem linked but are often dealt with by different legislations. Workplace studies generally focus on bullying and population studies on discrimination. The authors aimed to evaluate the prevalence and relationship of discrimination and bullying in a male-dominated workforce, associated factors and relative impact on mental ill-health.

Design/methodology/approach

An online cohort survey was conducted amongst employees of an Australian mining company, measuring discrimination, bullying, demographics and workplace and health factors over two months. Cross-sectional and prospective analyses assessed the prevalence of each, their association and their effects on depression and anxiety.

Findings

A total of 580 employees (82% male) participated. There was no association between workplace bullying (n = 56, 9.7%) and discrimination (n = 160, 27.6%). Discrimination, but not bullying, was associated with higher depression, anxiety and suicidal ideation and lower well-being and resilience. After controlling for demographic, workplace and health and well-being factors, depression had the main effect on discrimination ß = 0.39, p = 0.003. Discrimination predicted an increase in depression scores at follow-up F (1, 129) = 4.88, p = 0.029.

Originality/value

In this male-dominated industry, discrimination was more prevalent than bullying. Discrimination, but not bullying, was associated with poorer mental health both cross sectionally and prospectively. Supporting the need to assess and manage discrimination and bullying in the workplace independently and the need for interventions to reduce a broader range of adverse interpersonal behaviours.

Digital Interventions for Screening and Treating Common Mental Disorders or Symptoms of Common Mental Illness in Adults: Systematic Review and Meta-analysis

BACKGROUND

Digital interventions targeting common mental disorders (CMDs) or symptoms of CMDs are growing rapidly and gaining popularity, probably in response to the increased prevalence of CMDs and better awareness of early help-seeking and self-care. However, no previous systematic reviews that focus on these novel interventions were found.

OBJECTIVE

This systematic review aims to scope entirely web-based interventions that provided screening and signposting for treatment, including self-management strategies, for people with CMDs or subthreshold symptoms. In addition, a meta-analysis was conducted to evaluate the effectiveness of these interventions for mental well-being and mental health outcomes.

METHODS

Ten electronic databases including MEDLINE, PsycINFO, and EMBASE were searched from January 1, 1999, to early April 2020. We included randomized controlled trials (RCTs) that evaluated a digital intervention (1) targeting adults with symptoms of CMDs, (2) providing both screening and signposting to other resources including self-care, and (3) delivered entirely through the internet. Intervention characteristics including target population, platform used, key design features, and outcome measure results were extracted and compared. Trial outcome results were included in a meta-analysis on the effectiveness of users' well-being and mental health outcomes. We also rated the meta-analysis results with the Grading of Recommendations, Assessment, Development, and Evaluations approach to establish the quality of the evidence.

RESULTS

The electronic searches yielded 21 papers describing 16 discrete digital interventions. These interventions were investigated in 19 unique trials including 1 (5%) health economic study. Most studies were conducted in Australia and North America. The targeted populations varied from the general population to allied health professionals. All interventions offered algorithm-driven screening with measures to assess symptom levels and to assign treatment options including automatic web-based psychoeducation, self-care strategies, and signposting to existing services. A meta-analysis of usable trial data showed that digital interventions improved well-being (3 randomized controlled trials [RCTs]; n=1307; standardized mean difference [SMD] 0.40; 95% CI 0.29 to 0.51; I2=28%; fixed effect), symptoms of mental illness (6 RCTs; n=992; SMD -0.29; 95% CI -0.49 to -0.09; I2=51%; random effects), and work and social functioning (3 RCTs; n=795; SMD -0.16; 95% CI -0.30 to -0.02; I2=0%; fixed effect) compared with waitlist or attention control. However, some follow-up data failed to show any sustained effects beyond the post intervention time point. Data on mechanisms of change and cost-effectiveness were also lacking, precluding further analysis.

CONCLUSIONS

Digital mental health interventions to assess and signpost people experiencing symptoms of CMDs appear to be acceptable to a sufficient number of people and appear to have enough evidence for effectiveness to warrant further study. We recommend that future studies incorporate economic analysis and process evaluation to assess the mechanisms of action and cost-effectiveness to aid scaling of the implementation.

Preliminary Analysis of the Factor Structure, Reliability and Validity of an Obsessive-Compulsive Disorder Screening Tool for Use with Adults in Malaysia

This is a preliminary study to examine the factor structure, reliability, and validity of an obsessive-compulsive disorder (OCD) screening tool for use in the Malaysian setting. A total of 199 Malaysian adults were recruited for this study. After cleaning and normalizing the data, 190 samples were left to be analyzed. Principle component analysis using varimax rotation was then performed to examine various factors derived from psychometric tools commonly used to assess OCD patients. The screening tool exhibited three factors that fit the description of obsessions and compulsions from the Diagnostic and Statistical Manual of Mental Disorders-5th Edition (DSM 5), as well as other common symptoms that co-morbid with OCD. The labels given to the three factors were: Severity of Compulsions, Severity of Obsessions, and Symptoms of Depression and Anxiety. Reliability analysis showed high reliability with a Cronbach's alpha of 0.94, whereas convergent validity of the tool with the Yale Brown Obsessive-compulsive Scale-Self Report demonstrated good validity of r = 0.829. The three-factor model explained 68.91% of the total variance. Subsequent studies should focus on OCD factors that are culturally unique in the Malaysian context. Future research may also use online technology, which is cost-efficient and accessible, to further enhance the screening tool.

Increasing intentions to use mental health services among university students. Results of a pilot randomized controlled trial within the World Health Organization's World Mental Health International College Student Initiative

BACKGROUND

The majority of university students with mental health problems are untreated. Only a small empirical literature exists on strategies to increase mental health service use.

AIMS

To investigate the effects and moderators of a brief acceptance-facilitating intervention on intention to use mental health services among university students.

METHOD

Within the German site of the World Health Organization's World Mental Health International College Student (WMH-ICS) initiative, 1,374 university students were randomized to an intervention condition (IC; n = 664) or a control condition (CC; n = 710) that was implemented in the survey itself. Both conditions received the questions assessing mental disorders and suicidality that were included in other WMH-ICS surveys. The IC group then additionally received: Internet-based personalized feedback based on subject symptom severity in the domains of depression, anxiety, substance use, suicidal thoughts and behaviors, and nonsuicidal self-injury; psychoeducation tailored to the personal symptom profile; and information about available university and community mental health services. The primary outcome was reported intention to use psychological interventions in the next semester, which was the last question in the survey. A broad range of potential moderating factors was explored.

RESULTS

There was a significant main effect of the intervention with students randomized to IC, reporting significantly higher intentions to seek help in the next semester than students in the CC condition (d = 0.12, 95% CI: 0.02 to 0.23). Moderator analyses indicated that the intervention was more effective among students that fulfilled the criteria for lifetime (d = 0.34; 95% CI: -0.08 to 0.7) and 12-month panic-disorder (d = 0.32; 95% CI: -0.10 to 0.74) compared with those without lifetime (d = 0.11; 95% CI: 0.00 to 0.22) or 12-month panic disorder (d = 0.11; 95% CI: 0.00 to 0.22), students with lower (d = 0.37; 95% CI: -0.77 to 1.51) than higher (d = -0.01; 95% CI: -0.36 to 0.34) self-reported physical health, and students with nonheterosexual (d = 0.38; 95% CI: 0.08 to 0.67) compared with heterosexual (d = 0.06; 95% CI: -0.06 to 0.17) sexual orientation. The intervention had no effects among students who reported that they recognized that they had an emotional problem and "are already working actively to change it" (Stage 4 "stages of change").

CONCLUSIONS

A simple acceptance-facilitating intervention can increase intention to use mental health services, although effects, are on average, small. Future studies should investigate more personalized approaches with interventions tailored to barriers and clinical characteristics of students. In order to optimize intervention effects, the development and evaluation should be realized in designs that are powered to allow incremental value of different intervention components and tailoring strategies to be evaluated, such as in multiphase optimization designs.

Reductions in quality of life and increased economic burden associated with mental disorders in an Australian adult sample

Objective The aim of this study was to determine the effect of a broad range of common mental disorders and their comorbidity on health-related quality of life and functional disability. Methods In all, 2734 Australians aged ≥18 years, recruited from the general community via Facebook during August–December 2014, completed an online survey assessing demographic characteristics, nine mental disorders, suicidal ideation and attempt. Outcome measures were health-related quality of life (assessed using the Assessment of Quality of Life (AQoL)-4D measure and functional disability (days out of role). Results Overall, 53.1% of the sample met criteria for at least one mental disorder. Participants with each of the 11 mental health problems had significantly lower mean AQoL-4D scores and significantly greater functional disability compared with not having the disorder (P&lt;0.001). A monotonic decrease in quality of life and an increase in functional disability were observed with an increased total number of comorbid disorders (P&lt;0.001). Accounting for disorder prevalence, annual economic burden for each mental disorder was estimated to be in the range of A$870 million–A$17 billion. Conclusions Mental disorders negatively affect health-related quality of life and functional disability, exacerbated by increased comorbidity. The economic burden to participants and employers estimated in this study is of concern, and highlights the importance of evidence-based treatment and prevention approaches. What is known about the topic? Mental disorders are associated with poorer health-related quality of life, increased functional disability and increased economic costs. What does this paper add? This paper furthers our understanding of the associations of nine mental disorders, suicidal ideation and suicide attempts with quality of life in an Australian setting, highlighting the considerable economic implications of these associations. Further, it reveals that comorbidity of mental disorders exacerbates reductions in quality of life and increased functional disability. What are the implications for practitioners? The economic burden associated with lost productivity and quality of life for individuals with mental disorders is considerable. Therefore, prioritising funding to prevention and treatment using evidence-based approaches will have significant effect in terms of economic productivity and personal well-being for individuals.

Impact of Mental Health Screening on Promoting Immediate Online Help-Seeking: Randomized Trial Comparing Normative Versus Humor-Driven Feedback

BACKGROUND

Given the widespread availability of mental health screening apps, providing personalized feedback may encourage people at high risk to seek help to manage their symptoms. While apps typically provide personal score feedback only, feedback types that are user-friendly and increase personal relevance may encourage further help-seeking.

OBJECTIVE

The aim of this study was to compare the effects of providing normative and humor-driven feedback on immediate online help-seeking, defined as clicking on a link to an external resource, and to explore demographic predictors that encourage help-seeking.

METHODS

An online sample of 549 adults were recruited using social media advertisements. Participants downloaded a smartphone app known as "Mindgauge" which allowed them to screen their mental wellbeing by completing standardized measures on Symptoms (Kessler 6-item Scale), Wellbeing (World Health Organization [Five] Wellbeing Index), and Resilience (Brief Resilience Scale). Participants were randomized to receive normative feedback that compared their scores to a reference group or humor-driven feedback that presented their scores in a relaxed manner. Those who scored in the moderate or poor ranges in any measure were encouraged to seek help by clicking on a link to an external online resource.

RESULTS

A total of 318 participants scored poorly on one or more measures and were provided with an external link after being randomized to receive normative or humor-driven feedback. There was no significant difference of feedback type on clicking on the external link across all measures. A larger proportion of participants from the Wellbeing measure (170/274, 62.0%) clicked on the links than the Resilience (47/179, 26.3%) or Symptoms (26/75, 34.7%) measures (χ²=60.35, P<.001). There were no significant demographic factors associated with help-seeking for the Resilience or Wellbeing measures. Participants with a previous episode of poor mental health were less likely than those without such history to click on the external link in the Symptoms measure (P=.003, odds ratio [OR] 0.83, 95% CI 0.02-0.44), and younger adults were less likely to click on the link compared to older adults across all measures (P=.005, OR 0.44, 95% CI 0.25-0.78).

CONCLUSIONS

This pilot study found that there was no difference between normative and humor-driven feedback on promoting immediate clicks to an external resource, suggesting no impact on online help-seeking. Limitations included: lack of personal score control group, limited measures of predictors and potential confounders, and the fact that other forms of professional help-seeking were not assessed. Further investigation into other predictors and factors that impact on help-seeking is needed.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN12616000707460; https://www.anzctr.org.au/ Trial/Registration/TrialReview.aspx?id=370187 (Archived by WebCite at http://www.webcitation.org/6y8m8sVxr).

Adolescents' Perspectives on a Mobile App for Relationships: Cross-Sectional Survey

Background

Adolescence can be a fertile time for relationship issues, with interpersonal conflict being a risk factor for poor mental health. Mobile app interventions may have a significant appeal to young people in assisting with relationship distress. However, currently available apps have not been formally evaluated. Youths’ perspectives on engaging with mobile technology to assist with relationships are also unknown.

Objective

This study aimed to examine adolescents’ attitudes toward the concept of a mobile phone app for relationship help and support, and whether they would be likely to use such an intervention.

Methods

A cross-sectional Web survey consisting of 42 questions, including 13 free responses, was delivered. The proposed app, including character vignettes, was presented, and participants were asked to indicate whether they had experienced the same relationship issues, whether their peers would relate to the issues, and how helpful they found the proposed advice. Participants were also asked to provide their own suggestions for help, which were analyzed using thematic analyses.

Results

A total of 150 adolescents (aged 15 to 18) participated. Overall, 60.7% (91/150) were likely to use an app for relationship problems, and this was not associated with demographics or social support (all P values >.05). Likelihood of app usage was found to be influenced by perceived need for help, personal beliefs about app effectiveness, and whether the app is engaging and easy to use. Overall, adolescents were receptive of the proposed content with an average of 99.3% (149/150), rating the strategies provided as somewhat to very helpful.

Conclusions

Adolescents were likely to use a mobile phone app for relationship support, and use was not influenced by gender, age, social support, or any other background characteristic. Instead, likely use was influenced by need, personal beliefs, usability, and the appropriateness of app content. App developers must address these factors if the app is to have a wide-scale uptake.

Online versus paper-based screening for depression and anxiety in adults with cystic fibrosis in Ireland: a cross-sectional exploratory study

OBJECTIVE

To compare online and paper-based screening for depression and anxiety in adults with cystic fibrosis (CF).

DESIGN AND SETTING

Cross-sectional study in CF clinics in Ireland and through the Cystic Fibrosis Ireland online community.

PARTICIPANTS

160 adult patients aged 18 or above were recruited. Of these, 147 were included in the analysis; 83 online and 64 paper-based. The remaining 13 were excluded because of incomplete data.

MEASURES

Depression and anxiety were measured using the Hospital Anxiety and Depression Scale (HADS). Data on pulmonary function (forced expiratory volume in 1 s %) and body mass index were self-reported based on clinical assessments. Sociodemographic data were collected.

RESULTS

Compared with the paper-based participants, the online participants were more likely to be female (61.7% vs 48.4%), older (mean 32.2 vs 28.2 years) and were more likely to be married (32.5% vs 15.6%), living with their spouse or partner (42.5% vs 22.6%) and working either full time (33.7% vs 15.9%) or part time (30.1%vs 17.5%). The prevalence rates of elevated anxiety and depression were not significantly different (P=0.71 and P=0.56). HADS anxiety and depression scores were not statistically different between online (P=0.83) and paper-based (P=0.92) participants based on Mann-Whitney U test. A significant negative correlation was found between depression and pulmonary function (r=-0.39, P=0.01) and anxiety and pulmonary function (r=-0.36, P=0.02). Based on Cronbach's alpha, there were no statistically significant differences between the online and paper-based participants on the internal consistency of the HADS anxiety (P=0.073) and depression (P=0.378) scales.

CONCLUSIONS

Our findings suggest that online and paper-based screening for depression and anxiety in adult patients with CF yield comparable findings on prevalence rates and scores, associations with health and internal consistency of subscales. This study highlights that online screening offers an alternative method to paper-based screening. Further research with a larger sample and assessment of measurement equivalence between online and paper based screening is needed to confirm our results.

Using New and Emerging Technologies to Identify and Respond to Suicidality Among Help-Seeking Young People: A Cross-Sectional Study

BACKGROUND

Suicidal thoughts are common among young people presenting to face-to-face and online mental health services. The early detection and rapid response to these suicidal thoughts and other suicidal behaviors is a priority for suicide prevention and early intervention efforts internationally. Establishing how best to use new and emerging technologies to facilitate person-centered systematic assessment and early intervention for suicidality is crucial to these efforts.

OBJECTIVE

The aim of this study was to examine the use of a suicidality escalation protocol to respond to suicidality among help-seeking young people.

METHODS

A total of 232 young people in the age range of 16-25 years were recruited from either a primary mental health care service or online in the community. Each young person used the Synergy Online System and completed an initial clinical assessment online before their face-to-face or online clinical appointment. A suicidality escalation protocol was used to identify and respond to current and previous suicidal thoughts and behaviors.

RESULTS

A total of 153 young people (66%, 153/232) reported some degree of suicidality and were provided with a real-time alert online. Further levels of escalation (email or phone contact and clinical review) were initiated for the 35 young people (15%, 35/232) reporting high suicidality. Higher levels of psychological distress (P<.001) and a current alcohol or substance use problem (P=.02) predicted any level of suicidality compared with no suicidality. Furthermore, predictors of high suicidality compared with low suicidality were higher levels of psychological distress (P=.01), psychosis-like symptoms in the last 12 months (P=.01), a previous mental health problem (P=.01), and a history of suicide planning or attempts (P=.001).

CONCLUSIONS

This study demonstrates the use of new and emerging technologies to facilitate the systematic assessment and detection of help-seeking young people presenting with suicidality. This protocol empowered the young person by suggesting pathways to care that were based on their current needs. The protocol also enabled an appropriate and timely response from service providers for young people reporting high suicidality that was associated with additional comorbid issues, including psychosis-like symptoms, and a history of suicide plans and attempts.

Using different Facebook advertisements to recruit men for an online mental health study: Engagement and selection bias

A growing number of researchers are using Facebook to recruit for a range of online health, medical, and psychosocial studies. There is limited research on the representativeness of participants recruited from Facebook, and the content is rarely mentioned in the methods, despite some suggestion that the advertisement content affects recruitment success. This study explores the impact of different Facebook advertisement content for the same study on recruitment rate, engagement, and participant characteristics. Five Facebook advertisement sets ("resilience", "happiness", "strength", "mental fitness", and "mental health") were used to recruit male participants to an online mental health study which allowed them to find out about their mental health and wellbeing through completing six measures. The Facebook advertisements recruited 372 men to the study over a one month period. The cost per participant from the advertisement sets ranged from $0.55 to $3.85 Australian dollars. The "strength" advertisements resulted in the highest recruitment rate, but participants from this group were least engaged in the study website. The "strength" and "happiness" advertisements recruited more younger men. Participants recruited from the "mental health" advertisements had worse outcomes on the clinical measures of distress, wellbeing, strength, and stress. This study confirmed that different Facebook advertisement content leads to different recruitment rates and engagement with a study. Different advertisement also leads to selection bias in terms of demographic and mental health characteristics. Researchers should carefully consider the content of social media advertisements to be in accordance with their target population and consider reporting this to enable better assessment of generalisability.