Comparison between electronic and paper versions of patient-reported outcome measures in subjects with chronic obstructive pulmonary disease: an observational study with a cross-over administration

Development and validation of an electronic version of Sydney Swallow Questionnaire

BACKGROUND AND AIMS

Accurate assessment of patient-reported oropharyngeal dysphagia (OPD) is essential to guide appropriate management and evaluate response. The Sydney Swallow Questionnaire (SSQ) is a paper-based 17-item inventory developed and validated to objectively detect risk of OPD. An easy-to-use electronic version with digital output has significant potential in streamlining patient assessment. This study aims to develop and validate an electronic version of the SSQ (eSSQ) against the original paper version.

METHOD

The English-based paper SSQ was adapted on the online REDcap (Research Electronic Data Capture) platform to be accessible on computer and mobile devices. Patients with OPD and asymptomatic controls completed both electronic and paper versions in randomized order. Patients with stable symptoms then repeated the eSSQ after ≥14 days for test-retest reliability. Paper-based and eSSQs were also collected from an independent cohort for external validation. Agreement of total scores between both versions and eSSQ test-retest reliability were calculated using two-way mixed-effects intra-class correlation coefficient (ICC).

RESULTS

47 dysphagic patients, 32 controls, and 31 patients from an external validation cohort were recruited. The most common underlying etiology was head and neck cancer. Mean eSSQ total score was 789 in dysphagic patients, and 68 in controls. eSSQ had excellent agreement with paper SSQ in total scores among all participants, with ICC 0.97 (95% CI [0.93, 0.98]) in controls, 0.97 (95% CI [0.94, 0.98]) in dysphagic patients and 0.96 (95% CI [0.92, 0.98]) in validation cohort. Test-retest reliability was also excellent (ICC 0.96, 95% CI [0.90, 0.98]).

CONCLUSION

The newly developed eSSQ shows excellent agreement with the paper version and test-retest reliability. Future applications of its use may allow for more efficient and accessible patient assessment.

Health Status Progression Measured Using Weekly Telemonitoring of COPD Assessment Test Scores Over 1 Year and Its Association With COPD Exacerbations

Background

A previous longitudinal study of chronic obstructive pulmonary disease (COPD) Assessment Test (CAT) score changes suggested patients fall into 3 patterns: stable, improving, and worsening. This study assessed the evolution of CAT scores over time and its relationship to exacerbations.

Methods

In total, 84 participants used a telemedicine platform to complete CAT weekly for 52 weeks. Completion rates, annualized change in CAT scores, and learning effects were measured, as well as CAT changes of >4 units during look-back periods of 4 and 8 weeks. In a subgroup of participants with at least a 25% completion rate (adherent group, n=68 [81%]), the relationship between change in CAT score and exacerbations at any time during the study was examined post hoc.

Results

Linear regression showed that 50%, 22%, and 28% of the adherent subgroup had CAT scores indicating worsening, stable, and improving health status, respectively. In the adherent subgroup, 70% (n=7/10) of participants who had an exacerbation during the study had worsening CAT scores, versus 47% (n=27/58) without an exacerbation. The hazard ratio association between CAT score increase and moderate exacerbation was 1.13 (95% confidence interval: 1.03-1.24). Most participants experienced at least one CAT score change of >4 units, and 7% showed an initial learning effect with a median of 2 weeks.

Conclusion

Measuring trends in CAT scores may allow future studies to group patients into 3 defined categories of change over time and quantify CAT change trajectories to assess treatment response and potentially predict medium-term outcomes within individual patients.

Best Practices for the Electronic Implementation and Migration of Patient-Reported Outcome Measures

While the use of electronic methods to collect patient-reported outcome data in clinical trials continues to increase, it remains the case that many patient-reported outcome measures (PROMs) have originally been developed and validated on paper. Careful consideration during the move from paper PROMs to electronic format is required to preserve the integrity of the measure and ensure a "faithful migration." Relevant literature has long called out the importance of following migration best practices during this process; nevertheless, such best practices are distributed across multiple documents. This article consolidates and builds upon existing electronic PROM implementation best practice recommendations to provide a comprehensive, up-to-date, single point of reference. It reflects the current consensus based on the significant advances in technology capabilities and knowledge gleaned from the growing evidence base on electronic migration and implementation, to balance the need for maintaining the integrity of the measure while optimizing respondent usability. It also specifies whether the practice is rooted in evidence or expert consensus, to enable those using these best practices to make informed and considered decisions when conducting migration.

ISO/TS 21564:2019- based Evaluation of a Semantic Map between Variables in the ISARIC Freestanding Follow Up Survey and ORCHESTRA Studies

The COVID-19 pandemic has led to tremendous investment in clinical studies to generate much-needed knowledge on the prevention, diagnosis, treatment and long-term effects of the disease. Case report forms, comprised of questions and answers (variables), are commonly used to collect data in clinical trials. Maximizing the value of study data depends on data quality and on the ability to easily pool and share data from several sources. ISARIC, in collaboration with the WHO, has created a case report form that is available for use by the scientific community to collect COVID-19 trial data. One of such research initiatives collecting and analyzing multi-country and multi-cohort COVID-19 study data is the Horizon 2020 project ORCHESTRA. Following the ISO/TS 21564:2019 standard, a mapping between five ORCHESTRA studies' variables and the ISARIC Freestanding Follow-Up Survey elements was created. Measures of correspondence of shared semantic domain of 0 (perfect match), 1 (fully inclusive match), 2 (partial match), 4 (transformation required) or 4* (not present in ORCHESTRA) as compared to the target code system, ORCHESTRA study variables, were assigned to each of the elements in the ISARIC FUP case report form (CRF) which was considered the source code system. Of the ISARIC FUP CRF's variables, around 34% were found to show an exact match with corresponding variables in ORCHESTRA studies and about 33% showed a non-inclusive overlap. Matching variables provided information on patient demographics, COVID-19 testing, hospital admission and symptoms. More in-depth details are covered in ORCHESTRA variables with regards to treatment and comorbidities. ORCHESTRA's Long-Term Sequelae and Fragile population studies' CRFs include 32 and 27 variables respectively which were evaluated as a perfect match to variables in the ISARIC FUP CRF. Our study serves as an example of the kind of maps between case report form variables from different research projects needed to link ongoing COVID-19 research efforts and facilitate collaboration and data sharing. To enable data aggregation across two data systems, the information they contain needs to be connected through a map to determine compatibility and transformation needs. Combining data from various clinical studies can increase the power of analytical insights.

Learning from COVID-19 related trial adaptations to inform efficient trial design-a sequential mixed methods study

BACKGROUND

Many clinical trial procedures were often undertaken in-person prior to the COVID-19 pandemic, which has resulted in adaptations to these procedures to enable trials to continue. The aim of this study was to understand whether the adaptations made to clinical trials by UK Clinical Trials Units (CTUs) during the pandemic have the potential to improve the efficiency of trials post-pandemic.

METHODS

This was a mixed methods study, initially involving an online survey administered to all registered UK CTUs to identify studies that had made adaptations due to the pandemic. Representatives from selected studies were qualitatively interviewed to explore the adaptations made and their potential to improve the efficiency of future trials. A literature review was undertaken to locate published evidence concerning the investigated adaptations. The findings from the interviews were reviewed by a group of CTU and patient representatives within a workshop, where discussions focused on the potential of the adaptations to improve the efficiency of future trials.

RESULTS

Forty studies were identified by the survey. Fourteen studies were selected and fifteen CTU staff were interviewed about the adaptations. The workshop included 15 CTU and 3 patient representatives. Adaptations were not seen as leading to direct efficiency savings for CTUs. However, three adaptations may have the potential to directly improve efficiencies for trial sites and participants beyond the pandemic: a split remote-first eligibility assessment, recruitment outside the NHS via a charity, and remote consent. There was a lack of published evidence to support the former two adaptations, however, remote consent is widely supported in the literature. Other identified adaptations may benefit by improving flexibility for the participant. Barriers to using these adaptations include the impact on scientific validity, limitations in the role of the CTU, and participant's access to technology.

CONCLUSIONS

Three adaptations (a split remote-first eligibility assessment, recruitment outside the NHS via a charity, and remote consent) have the potential to improve clinical trials but only one (remote consent) is supported by evidence. These adaptations could be tested in future co-ordinated 'studies within a trial' (SWAT).