Population-based outreach versus care as usual to prevent suicide attempt: study protocol for a randomized controlled trial

Recognizing 30 Years of Accomplishments and Envisioning an Innovative Future - The 2024 Annual Conference of the Health Care Systems Research Network

The Health Care Systems Research Network (HCSRN) kicked off the 2024 Annual Conference on April 9, 2024, in Milwaukee at the Hyatt Regency with nearly 275 participants from 19 HCSRN member institutions. This year, HCSRN attendees joined their colleagues to reconnect and network during the three-day conference featuring the theme, "Advancing High-Quality, Equitable Research in the Age of New Health Care Technologies."

Development and preliminary evaluation of a suicidal risk assessment protocol in a randomised controlled trial using the Patient Health Questionnaire (PHQ-9)

BACKGROUND

Participants in research trials often disclose severe depression symptoms, including thoughts of self-harm and suicidal ideation, in validated self-administered questionnaires such as the Patient Health Questionnaire (PHQ-9). However, there is no standard protocol for responding to such disclosure, and the opportunity to support people at risk is potentially missed. We developed and evaluated a risk assessment protocol for the IBD-BOOST randomised controlled trial (ISRCTN71618461 09/09/2019).

METHODS

Participants completed the PHQ-9 at baseline and 6-month and 12-month follow-ups. The trial database automatically alerted the research team to risk assess participants. Trial researchers, trained in the protocol, contacted participants by telephone, completed the risk assessment, and signposted participants to appropriate professional services.

RESULTS

Seven hundred eighty participants were randomised in the trial; 41 required risk assessment. One participant declined assessment, so 40 risk assessments were completed. Twenty-four participants were assessed as low-risk and 16 participants as medium-risk, with 12 declaring previous suicide attempts. None were rated as high-risk. Trial participants expressed appreciation for being contacted, and all except two wished to receive information about professional support services. Trial risk assessors reported positive experiences of conducting the risk assessment with suggestions for improvement, which resulted in minor modifications to the protocol.

DISCUSSION

Our evaluation demonstrated that it was viable for a research trial team to successfully conduct a risk-assessment protocol for trial participants reporting thoughts of self-harm, with training and support from senior colleagues. Resources are required for training and delivery, but it is not unduly onerous. Trial participants appeared to find completing the assessment acceptable.

Information and communication technology-based interventions for suicide prevention implemented in clinical settings: a scoping review

BACKGROUND

A large number of information and communication technology (ICT) based interventions exist for suicide prevention. However, not much is known about which of these ICTs are implemented in clinical settings and their implementation characteristics. In response, this scoping review aimed to systematically explore the breadth of evidence on ICT-based interventions for suicide prevention implemented in clinical settings and then to identify and characterize implementation barriers and facilitators, as well as evaluation outcomes, and measures.

METHODS

We conducted this review following the Joanna Briggs Institute methodology for scoping reviews. A search strategy was applied to the following six databases between August 17-20, 2021: MEDLINE, Embase, CINAHL, PsycINFO, Web of Science, and Library, Information Science and Technology Abstracts. We also supplemented our search with Google searches and hand-searching reference lists of relevant reviews. To be included in this review, studies must include ICT-based interventions for any spectrum of suicide-related thoughts and behaviours including non-suicidal self-injury. Additionally, these ICTs must be implemented in clinical settings, such as emergency department and in-patient units. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews (PRISMA-ScR) checklist to prepare this full report.

RESULTS

This review included a total of 75 citations, describing 70 studies and 66 ICT-based interventions for suicide prevention implemented in clinical settings. The majority of ICTs were computerized interventions and/or applications (n = 55). These ICTs were commonly used as indicated strategies (n = 49) targeting patients who were actively presenting with suicide risk. The three most common suicide prevention intervention categories identified were post-discharge follow-up (n = 27), screening and/or assessment (n = 22), and safety planning (n = 20). A paucity of reported information was identified related to implementation strategies, barriers and facilitators. The most reported implementation strategies included training, education, and collaborative initiatives. Barriers and facilitators of implementation included the need for resource supports, knowledge, skills, motivation as well as engagement with clinicians with research teams. Studies included outcomes at patient, clinician, and health system levels, and implementation outcomes included acceptability, feasibility, fidelity, and penetration.

CONCLUSION

This review presents several trends of the ICT-based interventions for suicide prevention implemented in clinical settings and identifies a need for future research to strengthen the evidence base for improving implementation. More effort is required to better understand and support the implementation and sustainability of ICTs in clinical settings. The findings can also serve as a future resource for researchers seeking to evaluate the impact and implementation of ICTs.

Randomized controlled trial of an internet-based self-guided hand exercise program to improve hand function in people with systemic sclerosis: the Scleroderma Patient-centered Intervention Network Hand Exercise Program (SPIN-HAND) trial

BACKGROUND

Systemic sclerosis (scleroderma; SSc) is a rare autoimmune connective tissue disease. Functional impairment of hands is common. The Scleroderma Patient-centered Intervention Network (SPIN)-HAND trial compared effects of offering access to an online self-guided hand exercise program to usual care on hand function (primary) and functional health outcomes (secondary) in people with SSc with at least mild hand function limitations.

METHODS

The pragmatic, two-arm, parallel-group cohort multiple randomized controlled trial was embedded in the SPIN Cohort. Cohort participants with Cochin Hand Function Scale (CHFS) scores ≥ 3 and who indicated interest in using the SPIN-HAND Program were randomized (3:2 ratio) to an offer of program access or to usual care (targeted N = 586). The SPIN-HAND program consists of 4 modules that address (1) thumb flexibility and strength; (2) finger bending; (3) finger extension; and (4) wrist flexibility and strength. The primary outcome analysis compared CHFS scores 3 months post-randomization between participants offered versus not offered the program. Secondary outcomes were CHFS scores 6 months post-randomization and functional health outcomes (Patient-Reported Outcomes Measurement Information System profile version 2.0 domain scores) 3 and 6 months post-randomization.

RESULTS

In total, 466 participants were randomized to intervention offer (N = 280) or usual care (N = 186). Of 280 participants offered the intervention, 170 (61%) consented to access the program. Of these, 117 (69%) viewed at least one hand exercise instruction video and 77 (45%) logged into the program website at least 3 times. In intent-to-treat analyses, CHFS scores were 1.2 points lower (95% CI - 2.8 to 0.3) for intervention compared to usual care 3 months post-randomization and 0.1 points lower (95% CI - 1.8 to 1.6 points) 6 months post-randomization. There were no statistically significant differences in other outcomes.

CONCLUSION

The offer to use the SPIN-HAND Program did not improve hand function. Low offer uptake, program access, and minimal usage among those who accessed the program limited our ability to determine if using the program would improve function. To improve engagement, the program could be tested in a group format or as a resource to support care provided by a physical or occupational therapist.

TRIAL REGISTRATION

NCT03419208 . Registered on February 1, 2018.

Accuracy of ICD-10-CM encounter diagnoses from health records for identifying self-harm events

OBJECTIVE

Assess the accuracy of ICD-10-CM coding of self-harm injuries and poisonings to identify self-harm events.

MATERIALS AND METHODS

In 7 integrated health systems, records data identified patients reporting frequent suicidal ideation. Records then identified subsequent ICD-10-CM injury and poisoning codes indicating self-harm as well as selected codes in 3 categories where uncoded self-harm events might be found: injuries and poisonings coded as undetermined intent, those coded accidental, and injuries with no coding of intent. For injury and poisoning encounters with diagnoses in those 4 groups, relevant clinical text was extracted from records and assessed by a blinded panel regarding documentation of self-harm intent.

RESULTS

Diagnostic codes selected for review include all codes for self-harm, 43 codes for undetermined intent, 26 codes for accidental intent, and 46 codes for injuries without coding of intent. Clinical text was available for review for 285 events originally coded as self-harm, 85 coded as undetermined intent, 302 coded as accidents, and 438 injury events with no coding of intent. Blinded review of full-text clinical records found documentation of self-harm intent in 254 (89.1%) of those originally coded as self-harm, 24 (28.2%) of those coded as undetermined, 24 (7.9%) of those coded as accidental, and 48 (11.0%) of those without coding of intent.

CONCLUSIONS

Among patients at high risk, nearly 90% of injuries and poisonings with ICD-10-CM coding of self-harm have documentation of self-harm intent. Reliance on ICD-10-CM coding of intent to identify self-harm would fail to include a small proportion of true self-harm events.

Leveraging behavioral economics and implementation science to engage patients at risk for suicide in mental health treatment: a pilot study protocol

BACKGROUND

Primary care is an ideal setting to connect individuals at risk for suicide to follow-up care; however, only half of the patients referred from the primary care attend an initial mental health visit. We aim to develop acceptable, feasible, low-cost, and effective new strategies to increase treatment initiation among at-risk individuals identified in primary care.

METHODS

We will conduct a multi-phase, mixed-methods study. First, we will conduct a chart review study by using administrative data, including medical records, to identify characteristics of primary care patients at risk for suicide who do or do not attend an initial mental health visit following a referral. Second, we will conduct a mixed methods study by using direct observations and qualitative interviews with key stakeholders (N = 65) to understand barriers and facilitators to mental health service initiation among at-risk individuals. Stakeholders will include patients with suicidal ideation referred from primary care who do and do not attend a first mental health visit, primary care and behavioral health providers, and individuals involved in the referral process. We also will collect preliminary self-report and behavioral data regarding potential mechanisms of behavior change (i.e., self-regulation and social support) from patients. Third, we will leverage these findings, relevant frameworks, and the extant literature to conduct a multi-arm, non-randomized feasibility trial. During this trial, we will rapidly prototype and test strategies to support attendance at initial mental health visits. Strategies will be developed with subject matter experts (N = 10) and iteratively pilot tested (~5 patients per strategy) and refined. Research will be completed in the Penn Integrated Care Program (PIC), which includes fourteen primary care clinics in Philadelphia that provide infrastructure for electronic referrals, patient communication, and data access.

DISCUSSION

We will leverage frameworks and methods from behavioral economics and implementation science to develop strategies to increase mental health treatment initiation among individuals at risk for suicide identified in primary care. This project will lead to an evaluation of these strategies in a fully powered randomized trial and contribute to improvements in access to and engagement in mental health services for individuals at risk for suicide.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT05021224.

Effect of Offering Care Management or Online Dialectical Behavior Therapy Skills Training vs Usual Care on Self-harm Among Adult Outpatients With Suicidal Ideation: A Randomized Clinical Trial

Importance

People at risk of self-harm or suicidal behavior can be accurately identified, but effective prevention will require effective scalable interventions.

Objective

To compare 2 low-intensity outreach programs with usual care for prevention of suicidal behavior among outpatients who report recent frequent suicidal thoughts.

Design, Setting, and Participants

Pragmatic randomized clinical trial including outpatients reporting frequent suicidal thoughts identified using routine Patient Health Questionnaire depression screening at 4 US integrated health systems. A total of 18 882 patients were randomized between March 2015 and September 2018, and ascertainment of outcomes continued through March 2020.

Interventions

Patients were randomized to a care management intervention (n = 6230) that included systematic outreach and care, a skills training intervention (n = 6227) that introduced 4 dialectical behavior therapy skills (mindfulness, mindfulness of current emotion, opposite action, and paced breathing), or usual care (n = 6187). Interventions, lasting up to 12 months, were delivered primarily through electronic health record online messaging and were intended to supplement ongoing mental health care.

Main Outcomes and Measures

The primary outcome was time to first nonfatal or fatal self-harm. Nonfatal self-harm was ascertained from health system records, and fatal self-harm was ascertained from state mortality data. Secondary outcomes included more severe self-harm (leading to death or hospitalization) and a broader definition of self-harm (selected injuries and poisonings not originally coded as self-harm).

Results

A total of 18 644 patients (9009 [48%] aged 45 years or older; 12 543 [67%] female; 9222 [50%] from mental health specialty clinics and the remainder from primary care) contributed at least 1 day of follow-up data and were included in analyses. Thirty-one percent of participants offered care management and 39% offered skills training actively engaged in intervention programs. A total of 540 participants had a self-harm event (including 45 deaths attributed to self-harm and 495 nonfatal self-harm events) over 18 months following randomization: 172 (3.27%) in care management, 206 (3.92%) in skills training, and 162 (3.27%) in usual care. Risk of fatal or nonfatal self-harm over 18 months did not differ significantly between the care management and usual care groups (hazard ratio [HR], 1.07; 97.5% CI, 0.84-1.37) but was significantly higher in the skills training group than in usual care (HR, 1.29; 97.5% CI, 1.02-1.64). For severe self-harm, care management vs usual care had an HR of 1.03 (97.5% CI, 0.71-1.51); skills training vs usual care had an HR of 1.34 (97.5% CI, 0.94-1.91). For the broader self-harm definition, care management vs usual care had an HR of 1.10 (97.5% CI, 0.92-1.33); skills training vs usual care had an HR of 1.17 (97.5% CI, 0.97-1.41).

Conclusions and Relevance

Among adult outpatients with frequent suicidal ideation, offering care management did not significantly reduce risk of self-harm, and offering brief dialectical behavior therapy skills training significantly increased risk of self-harm, compared with usual care. These findings do not support implementation of the programs tested in this study.

Trial Registration

ClinicalTrials.gov Identifier: NCT02326883.

Connecting Research and Practice: Implementation of Suicide Prevention Strategies in Learning Health Care Systems

The health care systems affiliated with the Mental Health Research Network strive to be learning health care systems that identify and address evidence gaps of importance to clinicians, patients, and funders. This column describes how research guides clinical care and clinical care guides research in the area of suicide prevention as well as some of the challenges of conducting embedded research.

Use of pre-enrollment randomization and delayed consent to maximize participation in a clinical trial of opt-in versus opt-out tobacco treatment

Background: Enrollment in smoking cessation trials remain sub-optimal. The aim of this analysis was to determine the effectiveness of a modified Zelen's design in engaging hospitalized patients who smoke in a pragmatic OPT-IN versus OPT-OUT tobacco treatment trial. Methods: At bedside, clinical staff screened smokers for eligibility, randomized eligible into study arms, and delivered the appropriate treatment approach. Study staff called randomized patients at one-month post-discharge, debriefed patients on the study design, and collected consent to participate. We used frequencies and percentages for categorical variables and means and standard deviations for quantitative variables to describe the characteristics of those who consented and were enrolled versus those who did not enroll. We also compared the characteristics of participants who consented and those who were reached and explicitly refused consent at one-month follow-up. We used the Cohen's d measure of effect size to evaluate differences. Results: Of the 1,000 randomized, 741 (74.1%) consented to continue in the study at one-month follow-up. One hundred and twenty-seven (12.7%) refused consent and 132 (13.2%) were unreachable. Cohen's d effect size differences between those who consented/enrolled (n = 741) and those who were not enrolled (n = 259) were negligible (<0.2) for age, gender, race/ethnicity, and most forms of insurance. The effect size was small for Medicaid (0.36), and other public insurance (0.48). After excluding those unreached at 1 month (12.7%), there were medium Cohen's d effect size differences between those who consented to participate (n = 741) and those who explicitly refused (n = 127) with respect to age (0.55) and self-pay or no insurance (0.51). There were small to negligible effect size differences with respect to sex, race/ethnicity, and other forms of health insurance. Conclusions: The modified Zelen's design resulted in successful enrollment of most participants who were initially randomized into the trial, including those not motivated to quit.

A Pragmatic Clinical Trial Approach to Assessing and Monitoring Suicidal Ideation: Results from A National US Trauma Care System Study

OBJECTIVE

Few investigations have comprehensively described methods for assessing and monitoring suicidal ideation in pragmatic clinical trials of mental health services interventions. This investigation's goal was to assess a collaborative care intervention's effectiveness in reducing suicidal ideation and describe suicide monitoring implementation in a nationwide protocol.

METHOD

The investigation was a secondary analysis of a stepped wedge cluster randomized trial at 25-Level I trauma centers. Injury survivors (N = 635) were randomized to control (n = 370) and intervention (n = 265) conditions and assessed at baseline hospitalization and follow-up at 3-, 6- and 12-months post-injury. The Patient Health Questionnaire (PHQ-9) item-9 was used to evaluate patients for suicidal ideation. Mixed model regression was used to assess intervention versus control group changes in PHQ-9 item-9 scores over time and associations between baseline characteristics and development of suicidal ideation longitudinally. As part of the study implementation process assessment, suicide outreach call logs were also reviewed.

RESULTS

Over 50% of patients endorsed suicidal ideation at ≥1 assessment. Intervention patients relative to control patients demonstrated reductions in endorsements of suicidal ideation that did not achieve statistical significance (F[3,1461] = 0.74, P = .53). The study team completed outreach phone calls, texts or voice messages to 268 patients with PHQ-9 item-9 scores ≥1 (n = 161 control, n = 107 intervention).

CONCLUSIONS

Suicide assessment and monitoring can be feasibly implemented in large-scale pragmatic clinical trials. Intervention patients demonstrated less suicidal ideation over time; however, these comparisons did not achieve statistical significance. Intensive pragmatic trial monitoring may mask treatment effects by providing control patients a supportive intervention.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02655354.

Risk Assessment of Self-Injurious Behavior and Suicide Presentation in the Emergency Department: An Integrative Review

INTRODUCTION

Globally, there is a lack of clarity regarding the best practice to distinguish patients at the highest risk of suicide. This review explores the use of risk assessment tools in emergency departments to identify patients at high risk of repeat self-harm, suicide attempts, or death by suicide.

METHODS

The review question ("Does the use of risk assessment tools in emergency departments identify patients at high risk of repeat self-harm, suicide attempts, or death by suicide?") focused on exposure and outcome. Studies of any design were included. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines were used. Study characteristics and concepts were extracted, compared, and verified. An integrative approach was used for reporting through narrative synthesis.

RESULTS

Nine studies were identified for inclusion. Two risk assessment tools were found to have good predictive ability for suicide ideation and self-harm. Three had modest prediction of patient disposition, but in one study, the clinical impression of nurses had higher predictive ability. One tool showed modest predictive ability for patients requiring admission.

DISCUSSION

This review found no strong evidence to indicate that any particular risk tool has a superior predictive ability to identify repeat self-harm, suicide attempts, or death by suicide. Best practice lacks clarity to determine patients at highest risk of suicide, but the use of risk assessment tools has been recommended. Nevertheless, such tools should not be used in isolation from clinical judgment and experience to evaluate patients at risk. Education and training to augment risk assessment within the emergency department are recommended.

Using the Delphi Method for Determining Key Performance Elements for Delivery of Optimal Suicide-Specific Interventions in Emergency Departments

OBJECTIVE

Evidence-based suicide prevention interventions directed to those seeking psychiatric crisis services for suicidality in the emergency department (ED) can reduce death by suicide and related suffering. Best practice guidelines for the care of suicidal patients in the ED exist but are not accompanied by fidelity tools for use in determining whether the interventions were applied, particularly when more than one intervention is delivered concurrently. We sought to develop a universal, treatment-agnostic Suicide Care Fidelity Checklist comprised of Key Performance Elements (KPE) across the recommended suicide-specific ED interventions.

METHOD

A comprehensive review of published care standards was first conducted to determine suicide-specific ED best practice treatment domains and KPEs. Subject matter experts (SMEs) were identified for each domain. Using the Delphi Consensus method, SMEs iteratively revised and refined the KPEs within their domain until achieving KPE item consensus.

RESULTS

A total of three iterations was required to obtain consensus in five of six domains: comprehensive suicide assessment, lethal means counseling, suicide crisis planning, behavioral skills training, and psychoeducation about suicidality. Consensus was not fully attained for the domain involving engagement with people with lived experience.

CONCLUSIONS

We successfully identified six intervention domains and 74 KPEs across domains (60 deemed essential, and 14 deemed optional), with full consensus reached for 70 KPEs. While replication of the initial findings is required, the Suicide Care Fidelity Checklist can be used as a fidelity checklist to verify delivery of suicide-specific ED interventions.HIGHLIGHTSApplied Delphi Consensus method with suicide-specific subject matter experts.Generated a treatment-agnostic, universal set of suicide prevention KPEs for EDs.Expert-derived KPEs help real-world settings to assess suicide care fidelity.

Enhancing the use of EHR systems for pragmatic embedded research: lessons from the NIH Health Care Systems Research Collaboratory

OBJECTIVE

We identified challenges and solutions to using electronic health record (EHR) systems for the design and conduct of pragmatic research.

MATERIALS AND METHODS

Since 2012, the Health Care Systems Research Collaboratory has served as the resource coordinating center for 21 pragmatic clinical trial demonstration projects. The EHR Core working group invited these demonstration projects to complete a written semistructured survey and used an inductive approach to review responses and identify EHR-related challenges and suggested EHR enhancements.

RESULTS

We received survey responses from 20 projects and identified 21 challenges that fell into 6 broad themes: (1) inadequate collection of patient-reported outcome data, (2) lack of structured data collection, (3) data standardization, (4) resources to support customization of EHRs, (5) difficulties aggregating data across sites, and (6) accessing EHR data.

DISCUSSION

Based on these findings, we formulated 6 prerequisites for PCTs that would enable the conduct of pragmatic research: (1) integrate the collection of patient-centered data into EHR systems, (2) facilitate structured research data collection by leveraging standard EHR functions, usable interfaces, and standard workflows, (3) support the creation of high-quality research data by using standards, (4) ensure adequate IT staff to support embedded research, (5) create aggregate, multidata type resources for multisite trials, and (6) create re-usable and automated queries.

CONCLUSION

We are hopeful our collection of specific EHR challenges and research needs will drive health system leaders, policymakers, and EHR designers to support these suggestions to improve our national capacity for generating real-world evidence.

Zelen design clinical trials: why, when, and how

BACKGROUND

In 1979, Marvin Zelen proposed a new design for randomized clinical trials intended to facilitate clinicians' and patients' participation. The defining innovation of Zelen's proposal was random assignment of treatment prior to patient or participant consent. Following randomization, a participant would receive information and asked to consent to the assigned treatment.

METHODS

This narrative review examined recent examples of Zelen design trials evaluating clinical and public health interventions.

RESULTS

Zelen designs have often been applied to questions regarding real-world treatment or intervention effects under conditions of incomplete adherence. Examples include evaluating outreach or engagement interventions (especially for stigmatized conditions), evaluating treatments for which benefit may vary according to participant motivation, and situations when assignment to a control or usual care condition might prompt a disappointment effect. Specific practical considerations determine whether a Zelen design is scientifically appropriate or practicable. Zelen design trials usually depend on identifying participants automatically from existing records rather than by advertising, referral, or active recruitment. Assessments of baseline or prognostic characteristics usually depend on available records data rather than research-specific assessments. Because investigators must consider how exposure to treatments or interventions might bias ascertainment of outcomes, assessment of outcomes from routinely created records is often necessary. A Zelen design requires a waiver of the usual requirement for informed consent prior to random assignment of treatment. The Revised Common Rule includes specific criteria for such a waiver, and those criteria are most often met for evaluation of a low-risk and potentially beneficial intervention added to usual care. Investigators and Institutional Review Boards must also consider whether the scientific or public health benefit of a Zelen design trial outweighs the autonomy interests of potential participants. Analysis of Zelen trials compares outcomes according to original assignment, regardless of any refusal to accept or participate in the assigned treatment.

CONCLUSIONS

A Zelen design trial assesses the real-world consequences of a specific strategy to prompt or promote uptake of a specific treatment. While such trials are poorly suited to address explanatory or efficacy questions, they are often preferred for addressing pragmatic or policy questions.

Accounting for quality improvement during the conduct of embedded pragmatic clinical trials within healthcare systems: NIH Collaboratory case studies

Embedded pragmatic clinical trials (ePCTs) and quality improvement (QI) activities often occur simultaneously within healthcare systems (HCSs). Embedded PCTs within HCSs are conducted to test interventions and provide evidence that may impact public health, health system operations, and quality of care. They are larger and more broadly generalizable than QI initiatives, and may generate what is considered high-quality evidence for potential use in care and clinical practice guidelines. QI initiatives often co-occur with ePCTs and address the same high-impact health questions, and this co-occurrence may dilute or confound the ability to detect change as a result of the ePCT intervention.

During the design, pilot, and conduct phases of the large-scale NIH Collaboratory Demonstration ePCTs, many QI initiatives occurred at the same time within the HCSs. Although the challenges varied across the projects, some common, generalizable strategies and solutions emerged, and we share these as case studies.

Accuracy Requirements for Cost-effective Suicide Risk Prediction Among Primary Care Patients in the US

Importance

Several statistical models for predicting suicide risk have been developed, but how accurate such models must be to warrant implementation in clinical practice is not known.

Objective

To identify threshold values of sensitivity, specificity, and positive predictive value that a suicide risk prediction method must attain to cost-effectively target a suicide risk reduction intervention to high-risk individuals.

Design, Setting, and Participants

This economic evaluation incorporated published data on suicide epidemiology, the health care and societal costs of suicide, and the costs and efficacy of suicide risk reduction interventions into a novel decision analytic model. The model projected suicide-related health economic outcomes over a lifetime horizon among a population of US adults with a primary care physician. Data analysis was performed from September 19, 2019, to July 5, 2020.

Interventions

Two possible interventions were delivered to individuals at high predicted risk: active contact and follow-up (ACF; relative risk of suicide attempt, 0.83; annual health care cost, $96) and cognitive behavioral therapy (CBT; relative risk of suicide attempt, 0.47; annual health care cost, $1088).

Main Outcomes and Measures

Fatal and nonfatal suicide attempts, quality-adjusted life-years (QALYs), health care sector costs and societal costs (in 2016 US dollars), and incremental cost-effectiveness ratios (ICERs) (with ICERs ≤$150 000 per QALY designated cost-effective).

Results

With a specificity of 95% and a sensitivity of 25%, primary care-based suicide risk prediction could reduce suicide death rates by 0.5 per 100 000 person-years (if used to target ACF) or 1.6 per 100 000 person-years (if used to target CBT) from a baseline of 15.3 per 100 000 person-years. To be cost-effective from a health care sector perspective at a specificity of 95%, a risk prediction method would need to have a sensitivity of 17.0% or greater (95% CI, 7.4%-37.3%) if used to target ACF and 35.7% or greater (95% CI, 23.1%-60.3%) if used to target CBT. To achieve cost-effectiveness, ACF required positive predictive values of 0.8% for predicting suicide attempt and 0.07% for predicting suicide death; CBT required values of 1.7% for suicide attempt and 0.2% for suicide death.

Conclusions and Relevance

These findings suggest that with sufficient accuracy, statistical suicide risk prediction models can provide good health economic value in the US. Several existing suicide risk prediction models exceed the accuracy thresholds identified in this analysis and thus may warrant pilot implementation in US health care systems.

When Can We Trust Real-World Data To Evaluate New Medical Treatments?

Concerns regarding both the limited generalizability and the slow pace of traditional randomized trials have led to calls for greater use of real‐world evidence (RWE) in the evaluation of new treatments or products. RWE studies often rely on real‐world data (RWD), including data extracted from healthcare records or data captured by mobile phones or other consumer devices. Global assessments of RWD sources are not helpful in assessing whether any specific RWD element is fit for any specific purpose. Instead, evidence generators and evidence consumers should clearly identify the specific health state or clinical phenomenon of interest and then consider each step between that clinical phenomenon and its representation in a research database. We propose specific questions regarding potential error or bias affecting each of those steps: Would a person experiencing this clinical phenomenon present for care in this setting or interact with this recording device? Would this clinical phenomenon be accurately recognized or assessed? How might the recording environment or tools affect accurate and consistent recording of this clinical phenomenon? Can data elements from different sources be harmonized, both technically (same format) and semantically (same meaning)? Can the original data elements be consistently reduced to a useful clinical phenotype? Addressing these questions requires a range of clinical, organizational, and technical expertise. Transparency regarding each step in the creation of RWD is essential if evidence consumers are to rely on RWE studies.

Psychosocial interventions for self-harm in adults

BACKGROUND

Self-harm (SH; intentional self-poisoning or self-injury regardless of degree of suicidal intent or other types of motivation) is a growing problem in most counties, often repeated, and associated with suicide. There has been a substantial increase in both the number of trials and therapeutic approaches of psychosocial interventions for SH in adults. This review therefore updates a previous Cochrane Review (last published in 2016) on the role of psychosocial interventions in the treatment of SH in adults.

OBJECTIVES

To assess the effects of psychosocial interventions for self-harm (SH) compared to comparison types of care (e.g. treatment-as-usual, routine psychiatric care, enhanced usual care, active comparator) for adults (aged 18 years or older) who engage in SH.

SEARCH METHODS

We searched the Cochrane Common Mental Disorders Specialised Register, the Cochrane Library (Central Register of Controlled Trials [CENTRAL] and Cochrane Database of Systematic reviews [CDSR]), together with MEDLINE, Ovid Embase, and PsycINFO (to 4 July 2020).

SELECTION CRITERIA

We included all randomised controlled trials (RCTs) comparing interventions of specific psychosocial treatments versus treatment-as-usual (TAU), routine psychiatric care, enhanced usual care (EUC), active comparator, or a combination of these, in the treatment of adults with a recent (within six months of trial entry) episode of SH resulting in presentation to hospital or clinical services. The primary outcome was the occurrence of a repeated episode of SH over a maximum follow-up period of two years. Secondary outcomes included treatment adherence, depression, hopelessness, general functioning, social functioning, suicidal ideation, and suicide.

DATA COLLECTION AND ANALYSIS

We independently selected trials, extracted data, and appraised trial quality. For binary outcomes, we calculated odds ratio (ORs) and their 95% confidence intervals (CIs). For continuous outcomes, we calculated mean differences (MDs) or standardised mean differences (SMDs) and 95% CIs. The overall quality of evidence for the primary outcome (i.e. repetition of SH at post-intervention) was appraised for each intervention using the GRADE approach.

MAIN RESULTS

We included data from 76 trials with a total of 21,414 participants. Participants in these trials were predominately female (61.9%) with a mean age of 31.8 years (standard deviation [SD] 11.7 years). On the basis of data from four trials, individual cognitive behavioural therapy (CBT)-based psychotherapy may reduce repetition of SH as compared to TAU or another comparator by the end of the intervention (OR 0.35, 95% CI 0.12 to 1.02; N = 238; k = 4; GRADE: low certainty evidence), although there was imprecision in the effect estimate. At longer follow-up time points (e.g., 6- and 12-months) there was some evidence that individual CBT-based psychotherapy may reduce SH repetition. Whilst there may be a slightly lower rate of SH repetition for dialectical behaviour therapy (DBT) (66.0%) as compared to TAU or alternative psychotherapy (68.2%), the evidence remains uncertain as to whether DBT reduces absolute repetition of SH by the post-intervention assessment. On the basis of data from a single trial, mentalisation-based therapy (MBT) reduces repetition of SH and frequency of SH by the post-intervention assessment (OR 0.35, 95% CI 0.17 to 0.73; N = 134; k = 1; GRADE: high-certainty evidence). A group-based emotion-regulation psychotherapy may also reduce repetition of SH by the post-intervention assessment based on evidence from two trials by the same author group (OR 0.34, 95% CI 0.13 to 0.88; N = 83; k = 2; moderate-certainty evidence). There is probably little to no effect for different variants of DBT on absolute repetition of SH, including DBT group-based skills training, DBT individual skills training, or an experimental form of DBT in which participants were given significantly longer cognitive exposure to stressful events. The evidence remains uncertain as to whether provision of information and support, based on the Suicide Trends in At-Risk Territories (START) and the SUicide-PREvention Multisite Intervention Study on Suicidal behaviors (SUPRE-MISS) models, have any effect on repetition of SH by the post-intervention assessment. There was no evidence of a difference for psychodynamic psychotherapy, case management, general practitioner (GP) management, remote contact interventions, and other multimodal interventions, or a variety of brief emergency department-based interventions.

AUTHORS' CONCLUSIONS

Overall, there were significant methodological limitations across the trials included in this review. Given the moderate or very low quality of the available evidence, there is only uncertain evidence regarding a number of psychosocial interventions for adults who engage in SH. Psychosocial therapy based on CBT approaches may result in fewer individuals repeating SH at longer follow-up time points, although no such effect was found at the post-intervention assessment and the quality of evidence, according to the GRADE criteria, was low. Given findings in single trials, or trials by the same author group, both MBT and group-based emotion regulation therapy should be further developed and evaluated in adults. DBT may also lead to a reduction in frequency of SH. Other interventions were mostly evaluated in single trials of moderate to very low quality such that the evidence relating to the use of these interventions is inconclusive at present.

Brief Interventions via Electronic Health Record Messaging for Population-Based Suicide Prevention: Mixed Methods Pilot Study

Background

New opportunities to create and evaluate population-based selective prevention programs for suicidal behavior are emerging in health care settings. Standard depression severity measures recorded in electronic medical records (EMRs) can be used to identify patients at risk for suicide and suicide attempt, and promising interventions for reducing the risk of suicide attempt in at-risk populations can be adapted for web-based delivery in health care.

Objective

This study aims to evaluate a pilot of a psychoeducational program, focused on developing emotion regulation techniques via a web-based dialectical behavior therapy (DBT) skills site, including four DBT skills, and supported by secure message coaching, including elements of caring messages.

Methods

Patients were eligible based on the EMR-documented responses to the Patient Health Questionnaire indicating suicidal thoughts. We measured feasibility via the proportion of invitees who opened program invitations, visited the web-based consent form page, and consented; acceptability via qualitative feedback from participants about the DBT program; and engagement via the proportion of invitees who began DBT skills as well as the number of website visits for DBT skills and the degree of site engagement.

Results

A total of 60 patients were invited to participate. Overall, 93% (56/60) of the patients opened the invitation and 43% (26/60) consented to participate. DBT skills website users visited the home page on an average of 5.3 times (SD 6.0). Procedures resulted in no complaints and some participant feedback emphasizing the usefulness of DBT skills.

Conclusions

This study supports the potential of using responses to patient health questionnaires in EMRs to identify a high-risk population and offer key elements of caring messages and DBT adapted for a low-intensity intervention. A randomized trial evaluating the effectiveness of this program is now underway (ClinicalTrials.gov: NCT02326883).

Safer use of antipsychotics in youth (SUAY) pragmatic trial protocol

BACKGROUND

Programs such as the Pediatric Access Line in Washington state have shown decreases in antipsychotic medication use by youth with non-psychotic disorders. Program outcomes have been studied with observational designs. This manuscript describes the protocol for Targeted and Safer Use of Antipsychotics in Youth (SUAY), a randomized controlled trial of psychiatrist review of prescriptions and facilitated access to psychosocial care. The aim of the intervention is to reduce the number of person-days of antipsychotic use among participants.

METHODS

Recruitment occurs at 4 health systems. Targeted enrollment is 800 youth aged 3-17 years. Clinicians are block randomized to intervention versus usual care prior to the study. Youth are nested within the arm of the prescribing clinician. Clinicians in the intervention group receive an EHR-based best practice alert with options to expedite access to psychosocial care and all medication orders are reviewed by a child and adolescent psychiatrist with feedback provided to the prescriber. The primary outcome is person-days of antipsychotic medication use in the 6 months following the initial order. All randomized individuals contribute data regardless of their level of participation (including declining all services).

DISCUSSION

The trial has been approved by the institutional review boards at each of the 4 sites. The intervention has 4 novel design features including automated recruitment using a best practice alert, psychiatrist medication order review and consultation, telephone navigation to psychosocial care, and telemental health visits. Recruitment began in March of 2018 and will be completed in June 2020. Follow-up will be completed December 31, 2020.

TRIAL REGISTRATION

Clinicaltrials.gov, NCT03448575.

Health benefits and economic advantages associated with increased utilization of a smoking cessation program

Rationale, aim & objective: The goal of this study was to examine the health and economic impacts related to increased utilization of the Duke Smoking Cessation Program resulting from the addition of two relatively new referral methods - Best Practice Advisory and Population Outreach. Materials & methods: In a companion paper 'Comparison of Referral Methods into a Smoking Cessation Program', we report results from a retrospective, observational, comparative effectiveness study comparing the impact of three referral methods - Traditional Referral, Best Practice Advisory and Population Outreach on utilization of the Duke Smoking Cessation Program. In this paper we take the next step in this comparative assessment by developing a Markov model to estimate the improvement in health and economic outcomes when two referral methods - Best Practice Advisory and Population Outreach - are added to Traditional Referral. Data used in this analysis were collected from Duke Primary Care and Disadvantaged Care clinics over a 1-year period (1 October 2017-30 September 2018). Results: The addition of two new referral methods - Best Practice Advisory and Population Outreach - to Traditional Referral increased the utilization of the Duke Smoking Cessation Program in Primary Care clinics from 129 to 329 smokers and in Disadvantaged Care clinics from 206 to 401 smokers. The addition of these referral methods was estimated to result in 967 life-years gained, 408 discounted quality-adjusted life-years saved and total discounted lifetime direct healthcare cost savings of US$46,376,285. Conclusion: Health systems may achieve increased patient health and decreased healthcare costs by adding Best Practice Advisory and Population Outreach strategies to refer patients to smoking cessation services.

How Do Patients with Mental Health Diagnoses Use Online Patient Portals? An Observational Analysis from the Veterans Health Administration

Online patient portals may be effective for engaging patients with mental health conditions in their own health care. This retrospective database analysis reports patient portal use among Veterans with mental health diagnoses. Unadjusted and adjusted odds of portal feature use was calculated using logistic regressions. Having experienced military sexual trauma or having an anxiety disorder, post-traumatic stress disorder, or depression were associated with increased odds of portal use; bipolar, substance use, psychotic and adjustment disorders were associated with decreased odds. Future research should examine factors that influence portal use to understand diagnosis-level differences and improve engagement with such tools.

Internet-Based Cognitive Behavioral Therapy to Reduce Suicidal Ideation: A Systematic Review and Meta-analysis

IMPORTANCE

Suicidal ideation is a widespread phenomenon. However, many individuals at risk for suicide do not seek treatment, which might be addressed by providing low-threshold, internet-based self-help interventions.

OBJECTIVE

To investigate whether internet-based self-help interventions directly targeting suicidal ideation or behavior are associated with reductions in suicidal ideation.

DATA SOURCES

A systematic search of PsycINFO, MEDLINE, Cochrane Central Register of Controlled Trials (CENTRAL), and the Centre for Research Excellence of Suicide Prevention (CRESP) databases for trials from inception to April 6, 2019, was performed, supplemented by reference searches. Search strings consisted of various search terms related to the concepts of internet, suicide, and randomized clinical trials.

STUDY SELECTION

Two independent researchers reviewed titles, abstracts, and full texts. Randomized clinical trials evaluating the effectiveness of internet-based self-help interventions to reduce suicidal ideation were included. Interventions were eligible if they were based on psychotherapeutic elements. Trials had to report a quantitative measure of a suicide-specific outcome. Mobile-based and gatekeeper interventions were excluded; no further restrictions were placed on participant characteristics or date of publication.

DATA EXTRACTION AND SYNTHESIS

This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guidelines. Risk of bias was evaluated using the Cochrane Risk of Bias Tool. Standardized mean differences were calculated using a random-effects model.

MAIN OUTCOMES AND MEASURES

Suicidal ideation was the a priori primary outcome.

RESULTS

Six unique eligible trials (1567 unique participants; 1046 [66.8%] female; pooled mean [SD] age, 36.2 [12.5] years) were included in the systematic review and meta-analysis. All identified interventions were internet-based cognitive behavioral therapy (iCBT). Participants assigned to the iCBT condition experienced a significantly reduced suicidal ideation compared with controls following intervention in all 6 trials (standardized mean difference, -0.29; 95% CI, -0.40 to -0.19; P < .001). Heterogeneity was low (I2 = 0%). The effect appeared to be maintained at follow-up in 4 trials (standardized mean difference, -0.18; 95% CI, -0.34 to -0.02; P = .03; I2 = 36%). Studies did not report sufficient data on completed suicides and suicide attempts to assess potential associations.

CONCLUSIONS AND RELEVANCE

These results show that iCBT interventions are associated with significant reductions in suicidal ideation compared with control conditions. Considering their high scalability, iCBT interventions have the potential to reduce suicide mortality. Future research should assess the effect of these digital health interventions on suicidal behavior and identify moderators and mediators to advance understanding of the mechanisms of effectiveness of these interventions.

Using predictive analytics to improve pragmatic trial design

Clinical trials embedded in health systems can randomize large populations using automated data sources to determine trial eligibility and assess outcomes. The suicide prevention outreach trial used real-world data for trial design and randomized 18,868 individuals in four health systems using patient-reported thoughts of death or self-harm (Patient Health Questionnaire item 9). This took 3.5 years. We consider if using predictive analytics, that is, suicide risk estimates based on prediction models, could improve trial "efficiency." We used data on mental health outpatient visits between 1 January 2009 and 30 September 2017 in seven health systems (HealthPartners; Henry Ford Health System; and Colorado, Hawaii, Northwest, Southern California, and Washington Kaiser Permanente regions). We used a suicide risk prediction model developed in these same systems. We compared five trial designs with different eligibility criteria: a response of a 2 or 3 on Patient Health Questionnaire item 9, a response of a 3, suicide risk score above 90th, 95th, or 99th percentile. We compared the sample that met each criterion, 90-day suicide attempt rate following first eligible visit, and necessary sample sizes to detect a 15%, 25%, and 35% relative reduction in the suicide attempt rate, assuming 90% power, for each eligibility criterion. Our sample included 24,355,599 outpatient visits. Despite wide-spread use of Patient Health Questionnaire, 21,026,985 (86.3%) visits did not have a recorded Patient Health Questionnaire. Of the 2,928,927 individuals in our sample, 109,861 had a recorded Patient Health Questionnaire item 9 response of a 2 or 3 over the study years with a 1.40% 90-day suicide attempt rate and 50,047 had a response of a 3 (suicide attempt rate 1.98%). More patients met criteria requiring a certain risk score or higher: 331,273 had a 90th percentile risk score or higher (suicide attempt rate: 1.36%); 182,316 a 95th percentile or higher (suicide attempt rate 2.16%), and 78,655 a 99th percentile or higher (suicide attempt rate: 3.95%). Eligibility criterion of a Patient Health Questionnaire item 9 response of a 2 or 3 would require randomizing 44,081 individuals (40.2% of eligible population in our sample); eligibility criterion of a 3 would require 31,024 individuals (62.0% of eligible population). Eligibility criterion of a suicide risk score of 90th percentile or higher would require 45,675 individuals (13.8% of eligible population), 95th percentile 28,699 individuals (15.7% of eligible population), and 99th percentile 15,509 (19.7% of eligible population). A suicide risk prediction calculator could improve trial "efficiency"; identifying more individuals at increased suicide risk than relying on patient-report. It is an open scientific question if individuals identified using predictive analytics would respond differently to interventions than those identified by more traditional means.

A Pragmatic Approach to Psychometric Comparisons between the DSM-IV and DSM-5 Posttraumatic Stress Disorder (PTSD) Checklists in Acutely Injured Trauma Patients

Objective: Prior investigations suggest the relative equivalence of the DSM-IV and DSM-5 versions of the Posttraumatic Stress Disorders (PTSD) Checklist, yet no investigations have assessed the psychometric properties of the scales in pragmatic trials of acutely injured trauma survivors. Method: DSM-IV and DSM-5 versions of the PTSD Checklist were included in follow-up interviews of physically injured patients enrolled longitudinally in a pragmatic clinical trial; pragmatic trials aim to efficiently implement research procedures to inform healthcare system policy changes. Psychometric comparisons of the DSM-IV, DSM-5, and a 20-item blended version of the scale included evaluations of internal consistency, correlational assessments, evaluation of item level agreements, and estimation of DSM-5 cutoffs that optimize electronic health record screening protocols. Results: 128 patients were included in the pragmatic psychometric study. Cronbach's alphas for the 3 versions of the PTSD Checklist ranged from 0.93 to 0.95. Correlations between the 3 scales ranged from 0.79 to 0.99. All 3 measures demonstrated excellent convergent and discriminant properties. Item level agreement ranged from 70-80%. For the DSM-5 and blended versions of the scale, a score of 30 and 24, respectively, best approximated the DSM-IV cutoff of ≥35 that had previously optimized PTSD detection in conjunction with EHR screening. Conclusions: Among injured trauma survivors, the psychometric performance of the DSM-IV PTSD Checklist with the addition of the 4 new DSM-5 PTSD Checklist items is nearly equivalent to the DSM-5 PTSD Checklist. The investigation also suggests that pragmatic psychometric methods can catalyze the rapid translation of research findings into real-world practice settings.

Principles and procedures for data and safety monitoring in pragmatic clinical trials

Background

All clinical trial investigators have ethical and regulatory obligations to monitor participant safety and trial integrity. Specific procedures for meeting these obligations, however, may differ substantially between pragmatic trials and traditional explanatory clinical trials.

Methods/Results

Appropriate monitoring of clinical trials typically includes assessing rate of recruitment or enrollment; monitoring safe and effective delivery of study treatments; assuring that study staff act to minimize risks; monitoring quality and timeliness of study data; and considering interim analyses for early detection of benefit, harm, or futility. Each of these responsibilities applies to pragmatic clinical trials. Just as design of pragmatic trials typically involves specific and necessary departures from methods of explanatory clinical trials, appropriate monitoring of pragmatic trials typically requires specific departures from monitoring procedures used in explanatory clinical trials. We discuss how specific aspects of pragmatic trial design and operations influence selection of monitoring procedures and illustrate those choices using examples from three ongoing pragmatic trials conducted by the Mental Health Research Network.

Conclusions

Pragmatic trial investigators should not routinely adopt monitoring procedures used in explanatory clinical trials. Instead, investigators should consider core principles of trial monitoring and design monitoring procedures appropriate for each pragmatic trial.

Current Practices in Electronic Capture of Patient-Reported Outcomes for Measurement-Based Care and the Use of Patient Portals to Support Behavioral Health

Electronic health records combined with tethered patient portals now support a range of functions including electronic data capture of patient-reported outcomes, trend reporting on clinical targets, secure messaging, and patient-mediated health information exchange. The applications of these features require special consideration in psychiatric and behavioral health settings. Nonetheless, their potential to engage patients suffering from disorders in which passivity and withdrawal are endemic to their mental health condition, is great. This article presents the growing research base on these topics, including discussion of key issues and recommendations for optimal implementation of patient portals in behavioral health settings.

Development and Evaluation of a Web-Based Resource for Suicidal Thoughts: NowMattersNow.org

Background

Nearly half of people who die by suicide see a health care provider in the month before their death. With the release of new care guidelines, detection of suicidal patients will likely increase. Providers need access to suicide-specific resources that can be used as part of immediate, brief interventions with a suicidal patient. Web-based suicide prevention resources have the potential to address this need.

Objective

This study aimed to describe the development of the NowMattersNow.org website as a resource for individuals with suicidal thoughts and to evaluate the utility of the site via user experience surveys.

Methods

NowMattersNow.org is an online video-based free public resource that provides evidence-based teachings, examples, and resources for managing suicidal thoughts and intense emotions focused largely around skills from dialectical behavior therapy. Developed with assistance from mental health consumers, it is intended to address gaps in access to services for suicidal patients in health care systems. Visitors stay an average of a minute and a half on the website. From March 2015 to December 2017, a user experience survey measured self-reported changes on a 1 (not at all) to 5 (completely overwhelming) scale regarding intensity of suicidal thoughts and negative emotions while on the website. Longitudinal regression analyses using generalized estimating equations evaluated the magnitude and statistical significance of user-reported changes in suicidal ideation and negative emotion. In secondary analyses, user-reported changes specific to subgroups, including men aged 36 to 64 years, mental health care providers, and other health care providers were evaluated.

Results

During the period of analysis, there were 138,386 unique website visitors. We analyzed surveys (N=3670) collected during that time. Subsamples included men aged 36 to 64 years (n=512), mental health providers (n=460), and other health care providers (n=308). A total of 28% (1028/3670) of survey completers rated their suicidal thoughts as a 5 or “completely overwhelming” when they entered the website. We observed significant reductions in self-reported intensity of suicidal thoughts (–0.21, P<.001) and negative emotions (–0.32, P<.001), including decreases for users with the most severe suicidal thoughts (–6.4%, P<.001), most severe negative emotions (–10.9%, P<.001), and for middle-aged men (–0.13, P<001). Results remained significant after controlling for length of visit to website (before the survey) and technology type (mobile, desktop, and tablet).

Conclusions

Survey respondents reported measurable reductions in intensity of suicidal thoughts and emotions, including those rating their suicidal thoughts as completely or almost completely overwhelming and among middle-aged men. Although results from this user-experience survey administered at one point in time to a convenience sample of users must be interpreted with caution, results provide preliminary support for the potential effectiveness of the NowMattersNow.org website as a tool for short-term management of suicidal thoughts and negative emotions.

Improving pragmatic clinical trial design using real-world data

BACKGROUND

Pragmatic clinical trials often use automated data sources such as electronic health records, claims, or registries to identify eligible individuals and collect outcome information. A specific advantage that this automated data collection often yields is having data on potential participants when design decisions are being made. We outline how this data can be used to inform trial design.

METHODS

Our work is motivated by a pragmatic clinical trial evaluating the impact of suicide-prevention outreach interventions on fatal and non-fatal suicide attempts in the 18 months after randomization. We illustrate our recommended approaches for designing pragmatic clinical trials using historical data from the health systems participating in this study. Specifically, we illustrate how electronic health record data can be used to inform the selection of trial eligibility requirements, to estimate the distribution of participant characteristics over the course of the trial, and to conduct power and sample size calculations.

RESULTS

Data from 122,873 people with patient health questionnaire (PHQ) responses, recorded in their electronic health records between 1 July 2010 and 31 March 2012, were used to show that the suicide attempt rate in the 18 months following completion of the questionnaire varies by response to item nine of the PHQ. We estimated that the proportion of individuals with a prior recorded elevated PHQ (i.e. history of suicidal ideation) would decrease from approximately 50% at the beginning of a trial to about 5%, 50 weeks later. Using electronic health record data, we conducted simulations to estimate the power to detect a 25% reduction in suicide attempts. Simulation-based power calculations estimated that randomizing 8000 participants per randomization arm would allow 90% power to detect a 25% reduction in the suicide attempt rate in the intervention arm compared to usual care at an alpha rate of 0.05.

CONCLUSIONS

Historical data can be used to inform the design of pragmatic clinical trials, a strength of trials that use automated data collection for randomizing participants and assessing outcomes. In particular, realistic sample size calculations can be conducted using real-world data from the health systems in which the trial will be conducted. Data-informed trial design should yield more realistic estimates of statistical power and maximize efficiency of trial recruitment.

Pragmatic clinical trials offer unique opportunities for disseminating, implementing, and sustaining evidence-based practices into clinical care: Proceedings of a workshop

The National Institutes of Health (NIH) Health Care Systems (HCS) Research Collaboratory hosted a workshop to explore challenges and strategies for the dissemination, implementation, and sustainability of findings from pragmatic clinical trials (PCTs) embedded in HCS. PCTs are designed to assess the impact of interventions delivered in usual or real-world conditions and leverage existing infrastructure to answer important clinical questions. The goal of the workshop was to discuss strategies for conducting impactful future PCTs that bridge the gap between evidence, practice, and policy. This paper summarizes presentations about how to design and conduct PCTs embedded in HCS and use dissemination and implementation strategies during the planning and conduct of projects, emphasizing the ever-changing world of care delivery and the need for pragmatic trial operations to adapt at various levels of operation.

Ethical Acceptability of Postrandomization Consent in Pragmatic Clinical Trials

IMPORTANCE

Pragmatic clinical trials that seek informed consent after randomization (ie, postrandomization consent) are increasingly used, but debate on ethics persists because control arm patients are not specifically informed about the trials and randomization occurs before consent for the trials. The public's attitude toward postrandomization consent trials is unknown, but the way the trials are described could bias people's views.

OBJECTIVES

To assess the attitudes of the US general public toward postrandomization informed consent for pragmatic trials and to measure potential framing and other factors associated with those attitudes.

DESIGN, SETTING, AND PARTICIPANTS

An online, 2 × 2 experimental survey (fielded between February 23 and April 3, 2018) portraying 4 scenarios of postrandomization informed consent (with prior broad consent for medical record use) was conducted. These scenarios included traditional randomized clinical trial language framing vs alternative framing in a high-stakes trial (ie, survival in leukemia) or low-stakes trial (ie, blood glucose level in diabetes). A total of 3793 individuals invited to participate were part of an existing panel representative of the US general public (GfK KnowledgePanel).

MAIN OUTCOMES AND MEASURES

The proportion of participants who would recommend that an ethics review board approve a postrandomization consent pragmatic trial.

RESULTS

A total of 2042 of 3739 invitees (54.6%) responded; after exclusion of 38 incomplete surveys, 2004 participants were included in the analysis. Of these, 997 (49.8%) were women, 1440 (71.9%) were white non-Hispanic, 199 (9.9%) were black non-Hispanic, and 233 (11.6%) were Hispanic. Mean (SD) age was 47.5 (17.4) years. Across scenarios, weighted data showed that 75.4% of the participants would recommend approval of the postrandomization consent pragmatic trial, 20.4% would probably not recommend approval, and 4.2% would definitely not recommend approval. Approval was not sensitive to framing language (traditional vs new framing in high-stakes scenario, 74.3% vs 76.8%, P = .40; in low-stakes scenario, 77.7% vs 72.9%, P = .10) or to the stakes (low vs high stakes in traditional framing, 77.7% vs 74.3%, P = .25; in new framing, 72.9% vs 76.8%, P = .18). Better understanding of the postrandomization consent design was associated with higher rate of approval (78.1% vs 65.0%, P = .002 for high-stakes scenario; 77.2% vs 64.9%, P = .004 for low-stakes scenario), especially among those with less education. However, opinions about personal involvement in the control arm were more cautious (range depending on scenario, 45.6%-59.7%) and sensitive to stakes but not to framing.

CONCLUSIONS AND RELEVANCE

The public's generally high rate of approval of the ethics of postrandomization informed consent for pragmatic trial designs does not appear to be affected by whether postrandomization consent design is framed using traditional randomized clinical trial terminology, regardless of the stakes of the trial. Promoting better understanding of the design may increase its acceptance by the public.

Overcoming Challenges to Evidence-Based Policy Development in a Large, Integrated Delivery System

OBJECTIVE

To describe a new Veterans Health Administration (VHA) program to foster the learning health system paradigm by rigorously evaluating health care initiatives and to report key lessons learned in designing those evaluations.

PRINCIPAL FINDINGS

The VHA's Quality Enhancement Research Initiative and its Health Services Research and Development Service are cooperating on several large, randomized program evaluations aimed at improving the care veterans receive and the efficiency with which it is delivered. The evaluations we describe involve collaborative design, outcomes assessment, and implementation science through partnerships between VHA operations and researchers. We review key factors to assess before committing to an evaluation. In addition to traditional design issues (such as ensuring adequate power and availability of data), these include others that are easily overlooked: the stability of intervention financing, means of controlling and commitment to adhering to randomized roll-out, degree of buy-in from key implementation staff, and feasibility of managing multiple veto points for interventions that span several programs, among others.

CONCLUSIONS

Successful program implementation and rigorous evaluation require resources, specialized expertise, and careful planning. If the learning health system model is to be sustained, organizations will need dedicated programs to prioritize resources and continuously adapt evaluation designs.

Ethics and practice of Trials within Cohorts: An emerging pragmatic trial design

BACKGROUND

With increasing emphasis on pragmatic trials, new randomized clinical trial designs are being proposed to enhance the "real world" nature of the data generated. We describe one such design, appropriate for unmasked pragmatic clinical trials in which the control arm receives usual care, called "Trials within Cohorts" that is increasingly used in various countries because of its efficiency in recruitment, advantages in reducing subject burden, and ability to better mimic real-world consent processes.

METHODS

Descriptive, ethical, and US regulatory analysis of the Trials within Cohorts design.

RESULTS

Trials within Cohorts design involves, after recruitment into a cohort, randomization of eligible subjects, followed by an asymmetric treatment of the two arms: those selected for the experimental arm provide informed consent for the intervention trial, while the data from the control arm are used based on prior broad permission. Thus, unlike the traditional Zelen post-randomization consent design, the cohort participants are informed about future research within the cohort; however, the extent of this disclosure currently varies among studies. Thus, ethical analysis is provided for two types of situations: when the pre-randomization disclosure and consent regarding the embedded trials are fairly explicit and detailed versus when they consist of only general statements about future data use. These differing ethical situations could have implications for how ethics review committees apply US research rules regarding waivers and alterations of informed consent.

CONCLUSION

Trials within Cohorts is a promising new pragmatic randomized controlled trial design that is being increasingly used in various countries. Although the asymmetric consent procedures for the experimental versus control arm subjects can initially raise ethical concerns, it is ethically superior to previous post-randomization consent designs and can have important advantages over traditional trial designs.

Effectiveness of online and mobile telephone applications ('apps') for the self-management of suicidal ideation and self-harm: a systematic review and meta-analysis

BACKGROUND

Online and mobile telephone applications ('apps') have the potential to improve the scalability of effective interventions for suicidal ideation and self-harm. The aim of this review was therefore to investigate the effectiveness of digital interventions for the self-management of suicidal ideation or self-harm.

METHODS

Seven databases (Applied Science & Technology; CENTRAL; CRESP; Embase; Global Health; PsycARTICLES; PsycINFO; Medline) were searched to 31 March, 2017. Studies that examined the effectiveness of digital interventions for suicidal ideation and/or self-harm, or which reported outcome data for suicidal ideation and/or self-harm, within a randomised controlled trial (RCT), pseudo-RCT, or observational pre-test/post-test design were included in the review.

RESULTS

Fourteen non-overlapping studies were included, reporting data from a total of 3,356 participants. Overall, digital interventions were associated with reductions for suicidal ideation scores at post-intervention. There was no evidence of a treatment effect for self-harm or attempted suicide.

CONCLUSIONS

Most studies were biased in relation to at least one aspect of study design, and particularly the domains of participant, clinical personnel, and outcome assessor blinding. Performance and detection bias therefore cannot be ruled out. Digital interventions for suicidal ideation and self-harm may be more effective than waitlist control. It is unclear whether these reductions would be clinically meaningful at present. Further evidence, particularly with regards to the potential mechanisms of action of these interventions, as well as safety, is required before these interventions could recommended.

Suicidal ideation reported on the PHQ9 and risk of suicidal behavior across age groups

OBJECTIVE

The Joint Commission recommends all patients be screened for suicide. However, differences in suicide attempt and death rates may affect how well tools predict risk across age groups. Our objective was to determine whether item 9 of the Patient Health Questionnaire (PHQ9) predicts risk for suicide attempts and deaths across age groups.

METHODS

PHQ9s completed by adult outpatients treated for mental health conditions in 2010-2012 at four Mental Health Research Network-affiliated healthcare systems were used to measure depression severity and suicidal ideation. Suicide attempts were identified via ICD-9 codes and suicide deaths via ICD-10 codes and state death certificates.

RESULTS

In all, 939,268PHQ9s were completed by 297,290 outpatients. Compared to those without, those with nearly daily suicidal ideation were 5-to-8 times more likely to attempt suicide and 3-to-11times more likely to die by suicide within 30 days, and 2-to-4times more likely to attempt suicide and 2-to-6 times more likely to die by suicide within 365 days. The increased risk of suicide death for those with any level of suicidal ideation persisted over two years. The relationships between suicide thoughts and attempts and deaths were similar across age groups.

LIMITATIONS

Our sample was limited to outpatients completing a PHQ9 and relied on pre-existing clinical and administrative data.

CONCLUSIONS

Suicidal ideation reported on the PHQ9 was a robust predictor of suicide attempts and deaths regardless of age, and this increased risk persisted for two years. Healthcare systems should address both the immediate and sustained risk for suicide for patients of all ages.