Two brief valid measures of therapeutic alliance in counseling for tobacco dependence

Conversational Chatbot for Cigarette Smoking Cessation: Results From the 11-Step User-Centered Design Development Process and Randomized Controlled Trial

BACKGROUND

Conversational chatbots are an emerging digital intervention for smoking cessation. No studies have reported on the entire development process of a cessation chatbot.

OBJECTIVE

We aim to report results of the user-centered design development process and randomized controlled trial for a novel and comprehensive quit smoking conversational chatbot called QuitBot.

METHODS

The 4 years of formative research for developing QuitBot followed an 11-step process: (1) specifying a conceptual model; (2) conducting content analysis of existing interventions (63 hours of intervention transcripts); (3) assessing user needs; (4) developing the chat's persona ("personality"); (5) prototyping content and persona; (6) developing full functionality; (7) programming the QuitBot; (8) conducting a diary study; (9) conducting a pilot randomized controlled trial (RCT); (10) reviewing results of the RCT; and (11) adding a free-form question and answer (QnA) function, based on user feedback from pilot RCT results. The process of adding a QnA function itself involved a three-step process: (1) generating QnA pairs, (2) fine-tuning large language models (LLMs) on QnA pairs, and (3) evaluating the LLM outputs.

RESULTS

We developed a quit smoking program spanning 42 days of 2- to 3-minute conversations covering topics ranging from motivations to quit, setting a quit date, choosing Food and Drug Administration-approved cessation medications, coping with triggers, and recovering from lapses and relapses. In a pilot RCT with 96% three-month outcome data retention, QuitBot demonstrated high user engagement and promising cessation rates compared to the National Cancer Institute's SmokefreeTXT text messaging program, particularly among those who viewed all 42 days of program content: 30-day, complete-case, point prevalence abstinence rates at 3-month follow-up were 63% (39/62) for QuitBot versus 38.5% (45/117) for SmokefreeTXT (odds ratio 2.58, 95% CI 1.34-4.99; P=.005). However, Facebook Messenger intermittently blocked participants' access to QuitBot, so we transitioned from Facebook Messenger to a stand-alone smartphone app as the communication channel. Participants' frustration with QuitBot's inability to answer their open-ended questions led to us develop a core conversational feature, enabling users to ask open-ended questions about quitting cigarette smoking and for the QuitBot to respond with accurate and professional answers. To support this functionality, we developed a library of 11,000 QnA pairs on topics associated with quitting cigarette smoking. Model testing results showed that Microsoft's Azure-based QnA maker effectively handled questions that matched our library of 11,000 QnA pairs. A fine-tuned, contextualized GPT-3.5 (OpenAI) responds to questions that are not within our library of QnA pairs.

CONCLUSIONS

The development process yielded the first LLM-based quit smoking program delivered as a conversational chatbot. Iterative testing led to significant enhancements, including improvements to the delivery channel. A pivotal addition was the inclusion of a core LLM-supported conversational feature allowing users to ask open-ended questions.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03585231; https://clinicaltrials.gov/study/NCT03585231.

'Quitlink': Outcomes of a randomised controlled trial of peer researcher facilitated referral to a tailored quitline tobacco treatment for people receiving mental health services

OBJECTIVE

The aim of this study was to test the effectiveness of a tailored quitline tobacco treatment ('Quitlink') among people receiving support for mental health conditions.

METHODS

We employed a prospective, cluster-randomised, open, blinded endpoint design to compare a control condition to our 'Quitlink' intervention. Both conditions received a brief intervention delivered by a peer researcher. Control participants received no further intervention. Quitlink participants were referred to a tailored 8-week quitline intervention delivered by dedicated Quitline counsellors plus combination nicotine replacement therapy. The primary outcome was self-reported 6 months continuous abstinence from end of treatment (8 months from baseline). Secondary outcomes included additional smoking outcomes, mental health symptoms, substance use and quality of life. A within-trial economic evaluation was conducted.

RESULTS

In total, 110 participants were recruited over 26 months and 91 had confirmed outcomes at 8 months post baseline. There was a difference in self-reported prolonged abstinence at 8-month follow-up between Quitlink (16%, n = 6) and control (2%, n = 1) conditions, which was not statistically significant (OR = 8.33 [0.52, 132.09] p = 0.131 available case). There was a significant difference in favour of the Quitlink condition on 7-day point prevalence at 2 months (OR = 8.06 [1.27, 51.00] p = 0.027 available case). Quitlink costs AU$9231 per additional quit achieved.

CONCLUSION

The Quitlink intervention did not result in significantly higher rates of prolonged abstinence at 8 months post baseline. However, engagement rates and satisfaction with the 'Quitlink' intervention were high. While underpowered, the Quitlink intervention shows promise. A powered trial to determine its effectiveness for improving long-term cessation is warranted.

The Bucket Approach: Developing and Implementing an On-line Training Program in Tobacco Dependence Interventions Tailored for Behavioral Health Clinicians

People coping with a mental illness and/or addictive disorders have a very high prevalence of smoking cigarettes. The Bucket Approach, a free online training, tailors evidence-based tobacco dependence interventions for behavioral health clinicians to increase the likelihood that they will also address the tobacco use of their patients. From October 2019 through August 2021, 999 people enrolled in and 447 people completed the training. Individuals who completed the training evaluated it highly with an overall mean score of 8.4 (scale = 1 for very poor to 10 for very good). 3- and 6-month follow-up surveys documented continued impact. The training resulted in substantial changes in beliefs about treating tobacco dependence. For example, before training, 18.3% of trainees strongly agreed with the statement, "The skills currently possessed by behavioral health clinicians can be easily applied to the treatment of tobacco dependence." This increased to 40.7% at the end of training.

The Effects of Opt-out vs Opt-in Tobacco Treatment on Engagement, Cessation, and Costs: A Randomized Clinical Trial

Importance

Tobacco use causes 7 million deaths per year; most national guidelines require people who use tobacco to opt in to care by affirming they are willing to quit. Use of medications and counseling is low even in advanced economy countries.

Objective

To evaluate the efficacy of opt-out care vs opt-in care for people who use tobacco.

Design, Setting, and Participants

In Changing the Default (CTD), a Bayesian adaptive population-based randomization trial, eligible patients were randomized into study groups, treated according to group assignment, and debriefed and consented for participation at 1-month follow-up. A total of 1000 adult patients were treated at a tertiary care hospital in Kansas City. Patients were randomized from September 2016 to September 2020; final follow-up was in March 2021.

Interventions

At bedside, counselors screened for eligibility, conducted baseline assessment, randomized patients to study group, and provided opt-out care or opt-in care. Counselors and medical staff provided opt-out patients with inpatient nicotine replacement therapy, prescriptions for postdischarge medications, a 2-week medication starter kit, treatment planning, and 4 outpatient counseling calls. Patients could opt out of any or all elements of care. Opt-in patients willing to quit were offered each element of treatment described previously. Opt-in patients who were unwilling to quit received motivational counseling.

Main Outcomes and Measures

The main outcomes were biochemically verified abstinence and treatment uptake at 1 month after randomization.

Results

Of a total of 1000 eligible adult patients who were randomized, most consented and enrolled (270 [78%] of opt-in patients; 469 [73%] of opt-out patients). Adaptive randomization assigned 345 (64%) to the opt-out group and 645 (36%) to the opt-in group. The mean (SD) age at enrollment was 51.70 (14.56) for opt-out patients and 51.21 (14.80) for opt-out patients. Of 270 opt-in patients, 123 (45.56%) were female, and of 469 opt-out patients, 226 (48.19%) were female. Verified quit rates for the opt-out group vs the opt-in group were 22% vs 16% at month 1 and 19% vs 18% at 6 months. The Bayesian posterior probability that opt-out care was better than opt-in care was 0.97 at 1 month and 0.59 at 6 months. Treatment use for the opt-out group vs the opt-in group was 60% vs 34% for postdischarge cessation medication (bayesian posterior probability of 1.0), and 89% vs 37% for completing at least 1 postdischarge counseling call (bayesian posterior probability of 1.0). The incremental cost-effectiveness ratio was $678.60, representing the cost of each additional quit in the opt-out group.

Conclusions and Relevance

In this randomized clinical trial, opt-out care doubled treatment engagement and increased quit attempts, while enhancing patients' sense of agency and alliance with practitioners. Stronger and longer treatment could increase cessation.

Trial Registration

ClinicalTrials.gov Identifier: NCT02721082.

The Working Alliance Inventory's Measurement Properties: A Systematic Review

Measurement properties of the Working Alliance Inventory (WAI) and its various translations and adaptations for specific target groups have been investigated for over 30 years. No systematic review analyzing studies on measurement properties of the WAI has been conducted to date. COnsensus-based Standards for the selection of health Measurements INstruments (COSMIN) were developed for conducting high-quality systematic reviews on measurement properties in a transparent and standardized way. Aim of this study was to systematically review studies on measurement properties of the WAI, and its adapted versions, within psychotherapy, and other healthcare contexts using COSMIN criteria. PsycINFO, Medline, and EMBASE were searched (1989–2021). In all phases of the review procedure, study selection, data extraction, risk of bias assessment, rating of the quality of measurement properties, and rating of the quality of evidence for measurement properties, disagreement between reviewers was resolved by discussion. Results on validity, internal structure, reliability, construct validity, and responsiveness were analyzed. In total 66 studies were included. In most studies, evidence for measurement properties was according to COSMIN criteria, insufficient, lacking, or conflicting. Content validity was rated insufficient because neither patients nor healthcare professionals were involved in the development and validation process. Hence evidence for content validity of the WAI is unknown. Conflicting evidence was found for structural validity. Evidence for internal consistency could not be established. Limited evidence was found for inter-rater reliability and convergent validity. Conflicting evidence was also found for test-retest reliability and divergent validity. COSMIN criteria exposed persistent problems in validation studies of the WAI. These findings may indicate that measurement properties of the WAI are not up to current standards, or that COSMIN criteria may be less appropriate for assessing measurement properties of the WAI, or it could indicate both. The results of this systematic review suggest that WAI outcomes should be interpreted with caution and further research is needed regarding the content validity and hypotheses development. For the future, the theoretical framework underlying the measurement of the working alliance needs to be studied in psychotherapy and other health contexts, and tested in methodologically sound studies.

Measuring Therapeutic Alliance for Tobacco Cessation Counseling for Behavioral Health Clinicians

INTRODUCTION

Those coping with significant mental illness smoke at a high prevalence rate. Increasingly, behavioral health clinicians (BHCs) are being asked to provide tobacco-dependence interventions. In this context, it is important to measure their success at doing so. While the Working Alliance Inventory (WAI) is a well-established measurement of the effectiveness of therapeutic alliance, it is not specific to tobacco-dependence interventions. The Working Alliance Inventory for Tobacco (WAIT-3) has been found valid for tobacco cessation counselors (health providers who address tobacco), but its validity has not been established when BHCs address tobacco cessation as part of addressing all other needs of their patients. The purpose of this study was to examine the validity of the WAIT-3 in the context of behavioral health clinicians.

METHODS

Wisconsin Community Support Programs and Comprehensive Community Services programs distributed an anonymous, brief (14 items) survey to 1,930 of their clients. Measured variables included smoking status, behavioral intentions regarding quitting, and perception of help received from their clinic. Respondents could enter a chance to win a gift card as a thank you.

RESULTS

WAIT-3 scores were correlated with quitting-related variables. Compared to those with lower WAIT-3 scores, those with higher scores reported more attempts to quit, were more motivated to quit, were more likely to have a smoking cessation/reduction goal in their general treatment plan, had more conversations about quitting with their BHC, and wanted more help from their BHC to quit.

CONCLUSIONS

The WAIT-3 may be a valid way to measure the effectiveness of BHCs to address the tobacco use of their patients. Next steps include establishing its predictive validity.

Decídetexto: Mobile cessation support for Latino smokers. Study protocol for a randomized clinical trial

INTRODUCTION

Latinos, the largest minority group in the U.S., experience tobacco-related disparities, including limited access to cessation resources. Evidence supports the efficacy of mobile interventions for smoking cessation, which may be greater among Latinos, the highest users of text messaging.

OBJECTIVES

To describe the methodology of a randomized clinical trial to evaluate the impact of Decídetexto, a culturally appropriate mobile smoking cessation intervention versus standard care on smoking abstinence (cotinine-verified 7-day point prevalence abstinence) at Month 6 among Latino smokers.

METHODS

Latino smokers (N = 618) will be randomized to one of two conditions: 1) Decídetexto or 2) standard of care. Decídetexto is a mobile smoking cessation intervention (available in English and Spanish) that incorporates three integrated components: 1) a tablet-based software that collects smoking-related information to develop an individualized quit plan, 2) a 24-week text messaging counseling program with interactive capabilities, and 3) pharmacotherapy support. Decídetexto follows the Social Cognitive Theory as theoretical framework. Standard of care consists of printed smoking cessation materials along with referral to telephone quitline. Participants in both groups are given access to free pharmacotherapy (nicotine patches or gum) by calling study phone number. Promotores de Salud will rely on community-based approaches and clinical settings to recruit smokers into the study. All participants will complete follow-up assessments at Week 12 and Month 6.

DISCUSSION

If successful, Decídetexto will be ready to be implemented in different community- and clinic-based settings to reduce tobacco-related disparities.

Feasibility and Acceptability of a Culturally- and Linguistically-Adapted Smoking Cessation Text Messaging Intervention for Latino Smokers

Objective: Assess the feasibility and acceptability of a culturally- and linguistically-adapted smoking cessation text messaging intervention for Latino smokers.

Methods: Using a community-based strategy, 50 Latino smokers were recruited to participate in a smoking cessation pilot study. Participants received a 12-week text messaging intervention and were offered Nicotine Replacement Therapy (NRT) at no cost. We assessed biochemically verified abstinence at 12 weeks, text messaging interactivity with the program, NRT utilization, self-efficacy, therapeutic alliance, and satisfaction.

Results: Participants were 44.8 years old on average (SD 9.80), and they were primarily male (66%) and had no health insurance (78%). Most of the participants were born in Mexico (82%) and were light smokers (1–10 CPD) (68%). All participants requested the first order of NRT, and 66% requested a refill. Participants sent an average of 39.7 text messages during the 12-week intervention (SD 82.70). At 12 weeks, 30% of participants were biochemically verified abstinent (88% follow-up rate) and working alliance mean value was 79.2 (SD 9.04). Self-efficacy mean score increased from 33.98 (SD 10.36) at baseline to 40.05 (SD 17.65) at follow-up (p = 0.04). The majority of participants (90.9%, 40/44) reported being very or extremely satisfied with the program.

Conclusion: A culturally- and linguistically-adapted smoking cessation text messaging intervention for Latinos offers a promising strategy to increase the use of NRT, generated high satisfaction and frequent interactivity, significantly increased self-efficacy, produced high therapeutic alliance, and resulted in noteworthy cessation rates at the end of treatment. Additional testing as a formal randomized clinical trial is warranted.

"Quitlink"-A Randomized Controlled Trial of Peer Worker Facilitated Quitline Support for Smokers Receiving Mental Health Services: Study Protocol

Introduction: Although smokers with severe mental illnesses (SSMI) make quit attempts at comparable levels to other smokers, fewer are successful in achieving smoking cessation. Specialized smoking cessation treatments targeting their needs can be effective but have not been widely disseminated. Telephone delivered interventions, including by quitlines, show promise. However, few SSMI contact quitlines and few are referred to them by health professionals. Mental health peer workers can potentially play an important role in supporting smoking cessation. This study will apply a pragmatic model using peer workers to engage SSMI with a customized quitline service, forming the "Quitlink" intervention. Methods: A multi-center prospective, cluster-randomized, open, blinded endpoint (PROBE) trial. Over 3 years, 382 smokers will be recruited from mental health services in Victoria, Australia. Following completion of baseline assessment, a brief intervention will be delivered by a peer worker. Participants will then be randomly allocated either to no further intervention, or to be referred and contacted by the Victorian Quitline and offered a targeted 8-week cognitive behavioral intervention along with nicotine replacement therapy (NRT). Follow-up measures will be administered at 2-, 5-, and 8-months post-baseline. The primary outcome is 6 months continuous abstinence from end of treatment with biochemical verification. Secondary outcomes include 7-day point prevalence abstinence from smoking, increased quit attempts, and reductions in cigarettes per day, cravings and withdrawal, mental health symptoms, and other substance use, and improvements in quality of life. We will use a generalized linear mixed model (linear regression for continuous outcomes and logistic regression for dichotomous outcomes) to handle clustering and the repeated measures at baseline, 2-, 5-, and 8-months; individuals will be modeled as random effects, cluster as a random effect, and group assignment as a fixed effect. Discussion: This is the first rigorously designed RCT to evaluate a specialized quitline intervention accompanied by NRT among SSMI. The study will apply a pragmatic model to link SSMI to the Quitline, using peer workers, with the potential for wide dissemination. Clinical Trial Registration: Trial Registry: The trial is registered with ANZCTR (www.anzctr.org.au): ACTRN12619000244101 prior to the accrual of the first participant and updated regularly as per registry guidelines. Trial Sponsor: University of Newcastle, NSW, Australia.