Increasing follow-up questionnaire response rates in a randomized controlled trial of telehealth for depression: three embedded controlled studies

Feasibility of In-Home Virtual Reality for Chronic Pain in Sickle Cell Disease

PURPOSE

This study assessed the feasibility of an in-home virtual reality intervention for chronic pain in adults with sickle cell disease.

DESIGN

Two-group, parallel, randomized, multiple methods design with surveys, and interviews.

METHODS

Participants were randomized to virtual reality or audio control, with 2-16-minute daily modules for 8 weeks, a daily pain diary survey, and a post study interview. Chronic pain and pain correlates were evaluated at baseline and every 4 weeks for 3 months. Feasibility outcomes were participant enrollment (set at > 50%), questionnaire response (> 50%), intervention use, and cybersickness (< 20%).

RESULTS

Of the individuals approached, 67.8% (n = 19) were enrolled. Questionnaire response rates were 100% at baseline, 57.8% at week 4, and < 50% at weeks 8 and 12. The intervention was used for a median of 781 minutes and 210 minutes in the virtual reality and audio groups, respectively. Participants reported slight symptoms of cybersickness with no reports of severe symptoms, and the intervention was acceptable.

CONCLUSIONS

Home-based virtual reality can be used in future sickle cell disease research. To further strengthen evaluations of virtual reality in adults with sickle cell who experience chronic pain, future trials should address sample size limitations and incorporate recommended strategies to address cybersickness and questionnaire response.

CLINICAL IMPLICATIONS

The first known application of in-home virtual reality for chronic pain in adults with sickle cell disease was successful. Findings can inform future in-home investigations of virtual reality in this underserved population.

Methods to increase response to postal and electronic questionnaires

BACKGROUND

Self-administered questionnaires are widely used to collect data in epidemiological research, but non-response reduces the effective sample size and can introduce bias. Finding ways to increase response to postal and electronic questionnaires would improve the quality of epidemiological research.

OBJECTIVES

To identify effective strategies to increase response to postal and electronic questionnaires.

SEARCH METHODS

We searched 14 electronic databases up to December 2021 and manually searched the reference lists of relevant trials and reviews. We contacted the authors of all trials or reviews to ask about unpublished trials; where necessary, we also contacted authors to confirm the methods of allocation used and to clarify results presented.

SELECTION CRITERIA

Randomised trials of methods to increase response to postal or electronic questionnaires. We assessed the eligibility of each trial using pre-defined criteria.

DATA COLLECTION AND ANALYSIS

We extracted data on the trial participants, the intervention, the number randomised to intervention and comparison groups and allocation concealment. For each strategy, we estimated pooled odds ratios (OR) and 95% confidence intervals (CI) in a random-effects model. We assessed evidence for selection bias using Egger's weighted regression method and Begg's rank correlation test and funnel plot. We assessed heterogeneity amongst trial odds ratios using a Chi2 test and quantified the degree of inconsistency between trial results using the I2 statistic.

MAIN RESULTS

Postal We found 670 eligible trials that evaluated over 100 different strategies of increasing response to postal questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response almost doubled when: using monetary incentives (odds ratio (OR) 1.86; 95% confidence interval (CI) 1.73 to 1.99; heterogeneity I2 = 85%); using a telephone reminder (OR 1.96; 95% CI 1.03 to 3.74); and when clinical outcome questions were placed last (OR 2.05; 95% CI 1.00 to 4.24). The odds of response increased by about half when: using a shorter questionnaire (OR 1.58; 95% CI 1.40 to 1.78); contacting participants before sending questionnaires (OR 1.36; 95% CI 1.23 to 1.51; I2 = 87%); incentives were given with questionnaires (i.e. unconditional) rather than when given only after participants had returned their questionnaire (i.e. conditional on response) (OR 1.53; 95% CI 1.35 to 1.74); using personalised SMS reminders (OR 1.53; 95% CI 0.97 to 2.42); using a special (recorded) delivery service (OR 1.68; 95% CI 1.36 to 2.08; I2 = 87%); using electronic reminders (OR 1.60; 95% CI 1.10 to 2.33); using intensive follow-up (OR 1.69; 95% CI 0.93 to 3.06); using a more interesting/salient questionnaire (OR 1.73; 95% CI 1.12 to 2.66); and when mentioning an obligation to respond (OR 1.61; 95% CI 1.16 to 2.22). The odds of response also increased with: non-monetary incentives (OR 1.16; 95% CI 1.11 to 1.21; I2 = 80%); a larger monetary incentive (OR 1.24; 95% CI 1.15 to 1.33); a larger non-monetary incentive (OR 1.15; 95% CI 1.00 to 1.33); when a pen was included (OR 1.44; 95% CI 1.38 to 1.50); using personalised materials (OR 1.15; 95% CI 1.09 to 1.21; I2 = 57%); using a single-sided rather than a double-sided questionnaire (OR 1.13; 95% CI 1.02 to 1.25); using stamped return envelopes rather than franked return envelopes (OR 1.23; 95% CI 1.13 to 1.33; I2 = 69%), assuring confidentiality (OR 1.33; 95% CI 1.24 to 1.42); using first-class outward mailing (OR 1.11; 95% CI 1.02 to 1.21); and when questionnaires originated from a university (OR 1.32; 95% CI 1.13 to 1.54). The odds of response were reduced when the questionnaire included questions of a sensitive nature (OR 0.94; 95% CI 0.88 to 1.00). Electronic We found 88 eligible trials that evaluated over 30 different ways of increasing response to electronic questionnaires. We found substantial heterogeneity amongst trial results in half of the strategies. The odds of response tripled when: using a brief letter rather than a detailed letter (OR 3.26; 95% CI 1.79 to 5.94); and when a picture was included in an email (OR 3.05; 95% CI 1.84 to 5.06; I2 = 19%). The odds of response almost doubled when: using monetary incentives (OR 1.88; 95% CI 1.31 to 2.71; I2 = 79%); and using a more interesting topic (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by half when: using non-monetary incentives (OR 1.60; 95% CI 1.25 to 2.05); using shorter e-questionnaires (OR 1.51; 95% CI 1.06 to 2.16; I2 = 94%); and using a more interesting e-questionnaire (OR 1.85; 95% CI 1.52 to 2.26). The odds of response increased by a third when: offering survey results as an incentive (OR 1.36; 95% CI 1.16 to 1.59); using a white background (OR 1.31; 95% CI 1.10 to 1.56); and when stressing the benefits to society of response (OR 1.38; 95% CI 1.07 to 1.78; I2 = 41%). The odds of response also increased with: personalised e-questionnaires (OR 1.24; 95% CI 1.17 to 1.32; I2 = 41%); using a simple header (OR 1.23; 95% CI 1.03 to 1.48); giving a deadline (OR 1.18; 95% CI 1.03 to 1.34); and by giving a longer time estimate for completion (OR 1.25; 95% CI 0.96 to 1.64). The odds of response were reduced when: "Survey" was mentioned in the e-mail subject (OR 0.81; 95% CI 0.67 to 0.97); when the email or the e-questionnaire was from a male investigator, or it included a male signature (OR 0.55; 95% CI 0.38 to 0.80); and by using university sponsorship (OR 0.84; 95%CI 0.69 to 1.01). The odds of response using a postal questionnaire were over twice those using an e-questionnaire (OR 2.33; 95% CI 2.25 to 2.42; I2 = 98%). Response also increased when: providing a choice of response mode (electronic or postal) rather than electronic only (OR 1.76 95% CI 1.67 to 1.85; I2 = 97%); and when administering the e-questionnaire by computer rather than by smartphone (OR 1.62 95% CI 1.36 to 1.94).

AUTHORS' CONCLUSIONS

Researchers using postal and electronic questionnaires can increase response using the strategies shown to be effective in this Cochrane review.

The risks of attrition bias in longitudinal surveys of the impact of COVID-19

As the COVID-19 pandemic continues, interest in mental health impacts is shifting from short-term to long-term outcomes. As part of a longitudinal online survey study examining mental health impacts of the pandemic, we assessed the risk of attrition bias related to a history of depression-a condition research shows can increase challenges of recruitment and retention. Among 5023 participants who completed the baseline survey, significantly more reporting a history of depression were lost to follow-up: baseline to 3 months: 497/760 (65.4%) vs 2228/4263 (52.3%), P < 0.001; 3 to 6 months: 179/263 (68.1%) vs 1183/2035 (58.1%), P = 0.002. Participants reporting a history of depression also had greater adjusted odds of a Patient Health Questionnaire-8 score ≥10 (odds ratio [OR] = 3.97, 95% confidence interval [CI] 3.27, 4.84), Generalized Anxiety Disorder-7 score ≥10 (OR = 3.77, 95% CI 3.07, 4.62), and Posttraumatic Diagnostic Scale for DSM V score ≥ 28 (OR = 7.17, 95% CI 4.67, 11.00) at baseline, indicating a need to account for attrition bias when examining these outcomes. Similar considerations likely apply to other longitudinal survey studies and are important to address to ensure accurate evidence is available to support policy decisions regarding resource allocation and funding.

Does advance contact with research participants increase response to questionnaires: an updated systematic review and meta-analysis

Background

Questionnaires remain one of the most common forms of data collection in epidemiology, psychology and other human-sciences. However, results can be badly affected by non-response. One way to potentially reduce non-response is by sending potential study participants advance communication. The last systematic review to examine the effect of questionnaire pre-notification on response is 10 years old, and lacked a risk of bias assessment.

Objectives

Update the section of the Cochrane systematic review, Edwards et al. (2009), on pre-notification to include 1) recently published studies, 2) an assessment of risk of bias, 3) Explore if heterogeneity is reduced by: delay between pre-contact and questionnaire delivery, the method of pre-contact, if pre-contact and questionnaire delivery differ, if the pre-contact includes a foot-in-the-door manipulation, and study’s the risk of bias.

Methods

Inclusion criteria: population: any population, intervention: comparison of some type of pre-notification, comparison group: no pre-notification, outcome: response rates. Study design: randomised controlled trails. Exclusion criteria: NA. Data sources: Studies which cited or were included in Edwards et al. (2009); We additionally searched: CINAHL, Web of Science, PsycInfo, MEDLINE, EconLit, EMBASE, Cochrane Central, Cochrane CMR, ERIC, and Sociological Abstracts. The searches were implemented in June 2018 and May 2021. Study screening: a single reviewer screened studies, with a random 10% sample independently screened to ascertain accuracy. Data extraction: data was extracted by a single reviewer twice, with a week between each extraction. Risk of Bias: within studies bias was assessed using the Cochrane Risk of Bias tool (ROB1) by a single unblinded reviewer, across studies bias was assessed using funnel plots. Synthesis Method: study results were meta-analysed with a random effects model using the final response rate as the outcome. Evaluation of Uncertainty: Uncertainty was evaluated using the GRADE approach.

Results

One hundred seven trials were included with 211,802 participants. Over-all pre-notification increased response, OR = 1.33 (95% CI: 1.20–1.47). However, there was a large amount of heterogeneity (I² = 97.1%), which was not explained by the subgroup analyses. In addition, when studies at high or unclear risk of bias were excluded the effect was to reduced OR = 1.09 (95% CI: 0.99–1.20). Because of the large amount of heterogeneity, even after restricting to low risk of bias studies, there is still moderate uncertainty in these results.

Conclusions

Using the GRADE evaluation, this review finds moderate evidence that pre-notification may not have an effect on response rates.

Funding

Economic and Social Research Council.

Preregistration

None.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01435-2.

Mental health specialist video consultations versus treatment as usual in patients with depression or anxiety disorders in primary care: study protocol for an individually randomised superiority trial (the PROVIDE-C trial)

BACKGROUND

Most people with mental disorders, including those with severe and chronic disorders, are treated solely by their general practitioner (GP). Nevertheless, specialised mental health care may be required for specific patients. Notably, the accessibility of mental health specialist care is mainly complicated by (a) long waiting times for an appointment with specialists, (b) long travel distances to specialists, particularly in rural and remote areas, and (c) patients' reservations about mental health specialist care (including fear of being stigmatised by seeking such care). To mitigate those barriers, technology-based integrated care models have been proposed. The purpose of this study is to examine the effectiveness and cost-effectiveness of a mental health specialist video consultations model versus treatment as usual in patients with depression or anxiety disorders in primary care.

METHODS

In an individually randomised, prospective, two-arm superiority trial with parallel group design, N = 320 patients with anxiety and/or depressive disorder will be recruited in general practices in Germany. The intervention includes a newly developed treatment model based on video consultations with focus on diagnostics, treatment planning, and short-term intervention by mental health specialists. We will systematically compare the effectiveness, cost-effectiveness, and adverse effects of this new model with usual care by the GP: the primary outcome is the absolute change in the mean depressive and anxiety symptom severity measured on the Patient Health Questionnaire Anxiety and Depression Scale (PHQ-ADS) from baseline to 6 months after baseline assessment. Follow-up in both groups will be conducted by blinded outcome assessors at 6 months and 12 months after baseline. The main analysis will be based on the intention-to-treat principle. We will optimise the likelihood of treatment effectiveness by strict inclusion criteria for patients, enhanced intervention integrity, and conducting a process evaluation.

DISCUSSION

To the best of our knowledge, this is the first confirmatory study on a video-based, integrated care model for the treatment of anxiety and depressive disorders in GP patients in Germany.

TRIAL REGISTRATION

ClinicalTrials.gov, United States National Institutes of Health NCT04316572 . Prospectively registered on 20 March 2020.

Strategies to improve retention in randomised trials

BACKGROUND

Poor retention of participants in randomised trials can lead to missing outcome data which can introduce bias and reduce study power, affecting the generalisability, validity and reliability of results. Many strategies are used to improve retention but few have been formally evaluated.

OBJECTIVES

To quantify the effect of strategies to improve retention of participants in randomised trials and to investigate if the effect varied by trial setting.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Scopus, PsycINFO, CINAHL, Web of Science Core Collection (SCI-expanded, SSCI, CPSI-S, CPCI-SSH and ESCI) either directly with a specified search strategy or indirectly through the ORRCA database. We also searched the SWAT repository to identify ongoing or recently completed retention trials. We did our most recent searches in January 2020.

SELECTION CRITERIA

We included eligible randomised or quasi-randomised trials of evaluations of strategies to increase retention that were embedded in 'host' randomised trials from all disease areas and healthcare settings. We excluded studies aiming to increase treatment compliance.

DATA COLLECTION AND ANALYSIS

We extracted data on: the retention strategy being evaluated; location of study; host trial setting; method of randomisation; numbers and proportions in each intervention and comparator group. We used a risk difference (RD) and 95% confidence interval (CI) to estimate the effectiveness of the strategies to improve retention. We assessed heterogeneity between trials. We applied GRADE to determine the certainty of the evidence within each comparison.

MAIN RESULTS

We identified 70 eligible papers that reported data from 81 retention trials. We included 69 studies with more than 100,000 participants in the final meta-analyses, of which 67 studies evaluated interventions aimed at trial participants and two evaluated interventions aimed at trial staff involved in retention. All studies were in health care and most aimed to improve postal questionnaire response. Interventions were categorised into broad comparison groups: Data collection; Participants; Sites and site staff; Central study management; and Study design. These intervention groups consisted of 52 comparisons, none of which were supported by high-certainty evidence as determined by GRADE assessment. There were four comparisons presenting moderate-certainty evidence, three supporting retention (self-sampling kits, monetary reward together with reminder or prenotification and giving a pen at recruitment) and one reducing retention (inclusion of a diary with usual follow-up compared to usual follow-up alone). Of the remaining studies, 20 presented GRADE low-certainty evidence and 28 presented very low-certainty evidence. Our findings do provide a priority list for future replication studies, especially with regard to comparisons that currently rely on a single study.

AUTHORS' CONCLUSIONS

Most of the interventions we identified aimed to improve retention in the form of postal questionnaire response. There were few evaluations of ways to improve participants returning to trial sites for trial follow-up. None of the comparisons are supported by high-certainty evidence. Comparisons in the review where the evidence certainty could be improved with the addition of well-done studies should be the focus for future evaluations.

Digital tools for the recruitment and retention of participants in randomised controlled trials: a systematic map

BACKGROUND

Recruiting and retaining participants in randomised controlled trials (RCTs) is challenging. Digital tools, such as social media, data mining, email or text-messaging, could improve recruitment or retention, but an overview of this research area is lacking. We aimed to systematically map the characteristics of digital recruitment and retention tools for RCTs, and the features of the comparative studies that have evaluated the effectiveness of these tools during the past 10 years.

METHODS

We searched Medline, Embase, other databases, the Internet, and relevant web sites in July 2018 to identify comparative studies of digital tools for recruiting and/or retaining participants in health RCTs. Two reviewers independently screened references against protocol-specified eligibility criteria. Included studies were coded by one reviewer with 20% checked by a second reviewer, using pre-defined keywords to describe characteristics of the studies, populations and digital tools evaluated.

RESULTS

We identified 9163 potentially relevant references, of which 104 articles reporting 105 comparative studies were included in the systematic map. The number of published studies on digital tools has doubled in the past decade, but most studies evaluated digital tools for recruitment rather than retention. The key health areas investigated were health promotion, cancers, circulatory system diseases and mental health. Few studies focussed on minority or under-served populations, and most studies were observational. The most frequently-studied digital tools were social media, Internet sites, email and tv/radio for recruitment; and email and text-messaging for retention. One quarter of the studies measured efficiency (cost per recruited or retained participant) but few studies have evaluated people's attitudes towards the use of digital tools.

CONCLUSIONS

This systematic map highlights a number of evidence gaps and may help stakeholders to identify and prioritise further research needs. In particular, there is a need for rigorous research on the efficiency of the digital tools and their impact on RCT participants and investigators, perhaps as studies-within-a-trial (SWAT) research. There is also a need for research into how digital tools may improve participant retention in RCTs which is currently underrepresented relative to recruitment research.

REGISTRATION

Not registered; based on a pre-specified protocol, peer-reviewed by the project's Advisory Board.

Design of a comparative outcome analysis of open, laparoscopic, or robotic-assisted incisional or inguinal hernia repair utilizing surgeon experience and a novel follow-up model

BACKGROUND

In a recent publication, the International Guidelines for Groin Hernia Management by the European Hernia Society (EHS) recognized the need to individualize and tailor the surgical approach for hernia repair. There may be different opportunities for optimization of the surgical technique for surgeons performing open, laparoscopic, or robotic-assisted hernia repair. Robotic-assisted hernia repair is a relatively new minimally invasive surgical approach compared to laparoscopic and open repair. Currently, there is a lack of comparative prospective studies designed to evaluate long-term outcomes of patients undergoing robotic-assisted, laparoscopic, or open hernia repair.

MATERIALS & METHODS

This manuscript presents an innovative study design with two study cohorts (incisional and inguinal hernia repair) that contain three arms (robotic-assisted, laparoscopic, and open). The trial objective is to collect short-term and long-term outcomes for patients undergoing robotic-assisted, laparoscopic, or open hernia repair. The present publication will discuss the trial design, methods used to ensure consistency in surgeon expertise, and provides strategies to obtain long-term (> 3 months) follow-up data for enrolled patients.

RESULTS

One hundred subjects underwent incisional and one hundred underwent inguinal hernia repair at the time of this manuscript. Surgeon experience was analyzed across the three surgical techniques and follow-up compliance was assessed through 1 year. The follow-up completion rates for both study cohorts were >80% for all visits.

CONCLUSIONS

The innovative trial design helped to improve the quality and quantity of long-term follow-up. More innovative options to improve patient retention may be tested in future trials of similar design.

Electronic Patient-Reported Outcomes: Semi-Automated Data Collection in the Interventional Radiology Clinic

INTRODUCTION

Patient-reported outcomes are important for clinical research and will likely be used in the near future as a metric for physician reimbursement. This study aims to evaluate the implementation of an electronic data collection system for deep vein thrombosis and lymphedema quality-of-life (QOL) questionnaires in a tertiary care interventional radiology practice.

METHODS

A single provider's clinic patients were automatically e-mailed validated questionnaires 1 week before their appointments. If not completed via e-mail, the questionnaire was administered on an electronic tablet in clinic by a research coordinator. Patients were also sent postprocedure questionnaires.

RESULTS

In all, 106 patients visited the clinic for a pre-intervention venous consultation. Of them, 96% (n = 102 of 106) completed the pre-intervention questionnaire: 48% (n = 47 of 98) via e-mail and 52% (n = 51 of 98) via tablet. Of the patients who had procedures and were sent questionnaires, 49% (n = 26 of 53) were seen in person. Of the postprocedure in-person clinic patients, 76% (n = 20 of 26) completed the questionnaire via e-mail, and the remainder with the tablet in clinic. Twenty-seven of the 53 (51%) patients did not return for follow-up and instead were sent an electronic questionnaire as their only source of follow-up, of which 74% (n = 20 of 27) complied.

CONCLUSION

After an initial introduction to electronic QOL reporting, patients were more likely to complete the questionnaires remotely for their follow-up appointment. A semi-automated electronic QOL system allows physicians to collect patient outcome data even in the absence of a clinic visit.

An evidence-based approach to the use of telehealth in long-term health conditions: development of an intervention and evaluation through pragmatic randomised controlled trials in patients with depression or raised cardiovascular risk

Background

Health services internationally are exploring the potential of telehealth to support the management of the growing number of people with long-term conditions (LTCs).

Aim

To develop, implement and evaluate new care programmes for patients with LTCs, focusing on two common LTCs as exemplars: depression or high cardiovascular disease (CVD) risk.

Methods

Development

We synthesised quantitative and qualitative evidence on the effectiveness of telehealth for LTCs, conducted a qualitative study based on interviews with patients and staff and undertook a postal survey to explore which patients are interested in different forms of telehealth. Based on these studies we developed a conceptual model [TElehealth in CHronic disease (TECH) model] as a framework for the development and evaluation of the Healthlines Service for patients with LTCs.

Implementation

The Healthlines Service consisted of regular telephone calls to participants from health information advisors, supporting them to make behaviour change and to use tailored online resources. Advisors sought to optimise participants’ medication and to improve adherence.

Evaluation

The Healthlines Service was evaluated with linked pragmatic randomised controlled trials comparing the Healthlines Service plus usual care with usual care alone, with nested process and economic evaluations. Participants were adults with depression or raised CVD risk recruited from 43 general practices in three areas of England. The primary outcome was response to treatment and the secondary outcomes included anxiety (depression trial), individual risk factors (CVD risk trial), self-management skills, medication adherence, perceptions of support, access to health care and satisfaction with treatment.

Trial results

Depression trial

In total, 609 participants were randomised and the retention rate was 86%. Response to treatment [Patient Health Questionnaire 9-items (PHQ-9) reduction of ≥ 5 points and score of < 10 after 4 months] was higher in the intervention group (27%, 68/255) than in the control group (19%, 50/270) [odds ratio 1.7, 95% confidence interval (CI) 1.1 to 2.5;p = 0.02]. Anxiety also improved. Intervention participants reported better access to health support, greater satisfaction with treatment and small improvements in self-management, but not improved medication adherence.

CVD risk trial

In total, 641 participants were randomised and the retention rate was 91%. Response to treatment (maintenance of/reduction in QRISK®2 score after 12 months) was higher in the intervention group (50%, 148/295) than in the control group (43%, 124/291), which does not exclude a null effect (odds ratio 1.3, 95% CI 1.0 to 1.9;p = 0.08). The intervention was associated with small improvements in blood pressure and weight, but not smoking or cholesterol. Intervention participants were more likely to adhere to medication, reported better access to health support and greater satisfaction with treatment, but few improvements in self-management.

The Healthlines Service was likely to be cost-effective for CVD risk, particularly if the benefits are sustained, but not for depression. The intervention was implemented largely as planned, although initial delays and later disruption to delivery because of the closure of NHS Direct may have adversely affected participant engagement.

Conclusion

The Healthlines Service, designed using an evidence-based conceptual model, provided modest health benefits and participants valued the better access to care and extra support provided. This service was cost-effective for CVD risk but not depression. These findings of small benefits at extra cost are consistent with previous pragmatic research on the implementation of comprehensive telehealth programmes for LTCs.

Trial registration

Current Controlled Trials ISRCTN14172341 (depression trial) and ISRCTN27508731 (CVD risk trial).

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.