Can a computerized format replace a paper form in PRO and HRQL evaluation? Psychometric testing of the computer-assisted LCSS instrument (eLCSS-QL)

Standardized Usage of Electronic Patient-Reported Outcome Measurements is Time-Efficient and Feasible

(1) Background: Digitization is of the utmost importance in improving the transfer of medical data. In order to emphasize the need for the greater implementation of digital solutions, we compared analog PROMs (aPROMs) to electronic PROMs (ePROMs) to emphasize the time benefits for clinical everyday life. (2) Methods: This prospective, observational study compared the evaluation of SF-36 in patients between 18 and 80 years old with musculoskeletal pathologies. We performed an age-independent and age-dependent analysis. (3) Results: After the import of aPROMs data, ePROMs took significantly less time (11.97 ± 3.00 min vs. 9.41 ± 3.12 min, p = 0.002, d = 0.797). There were no significant differences associated with age for aPROMs (7.23 ± 2.57 min vs. 8.38 ± 2.71 min, p = 0.061, d = -0.607) or ePROMs (8.72 ± 2.19 min vs. 10.09 ± 3.80 min, p = 0.130, d = -0.436), respectively. (4) Conclusions: This study indicates that ePROMs are a time-feasible method for collecting data to guide patient-personalized treatment approaches.

Patient-reported outcomes in lung cancer surgery: A narrative review

Lung cancer is a leading cause of cancer-related mortality worldwide, profoundly affecting patients' quality of life. Patient-reported outcomes (PROs) provide essential insights from the patients' perspective, a crucial aspect often overlooked by traditional clinical outcomes. This review synthesizes research on the role of PROs in lung cancer surgery to enhance patient care and outcomes. We conducted a comprehensive literature search across PubMed, Scopus, and Web of Science up to March 2024, using terms such as "lung cancer," "Patient Reported Outcome," "lobectomy," "segmentectomy," and "lung surgery." The criteria included original studies on lung cancer patients who underwent surgical treatment and reported on PROs. After screening and removing duplicates, reviews, non-English articles, and irrelevant studies, 36 research articles were selected, supported by an additional 53 publications, totaling 89 references. The findings highlight the utility of PROs in assessing post-surgical outcomes, informing clinical decisions, and facilitating patient-centered care. However, challenges in standardization, patient burden, and integration into clinical workflows remain, underscoring the need for further research and methodological refinement. PROs are indispensable for understanding the quality-of-life post-surgery and enhancing communication and decision-making in clinical practice. Their integration into routine care is vital for a holistic approach to lung cancer treatment, promising significant improvements in patient outcomes and quality of care.

Validation of smartphone app-based digital patient reported outcomes in full-endoscopic spine surgery

PURPOSE

Patient reported outcome measures (PROMs) are important tools to assess patient function, pain, disability, and quality of life. We aim to study the efficiency and validity of digital PROMs collection using a smartphone app compared to traditional paper PROMs.

METHODS

Patients undergoing evaluation for full-endoscopic spine surgery were recruited from the outpatient clinic at Harborview Medical Center. Visual analogue scale (VAS), Oswestry disability index (ODI), and EQ5-5D PROMs were administered on paper and through a smartphone app called SpineHealthie. Compliance rates were collected, and PROM results were assessed for correlation between paper and digital methods.

RESULTS

123 patients were enrolled. 57.7% of patients completed paper PROMs, 82.9% completed digital PROMs, and 48.8% completed both. Of the patients that completed both, Spearman's correlation was greatest for VAS leg, ODI, and EQ5 index scores. Correlation was weaker for VAS back pain, neck pain, and upper extremity pain. Patients tended to report lower disability and higher quality of life on the digital PROM compared to the paper PROM.

CONCLUSION

The SpineHealthie app effectively and accurately collects PROMs digitally, showing strong concordance with traditional paper PROMs. We conclude that digital PROMs constitute a promising strategy for monitoring patients after spine surgery over time.

Updated Recommendations on Evidence Needed to Support Measurement Comparability Among Modes of Data Collection for Patient-Reported Outcome Measures: A Good Practices Report of an ISPOR Task Force

The ISPOR Task Force on measurement comparability between modes of data collection for patient-reported outcome measures (PROMs) has updated the good practice recommendations from the 2009 ISPOR electronic patient-reported outcome and 2014 patient-reported outcome mixed modes Good Research Practices Task Force reports in light of accumulated evidence of measurement comparability among different modes of PROM data collection. Furthermore, with the increasing use of electronic formats of clinical outcome assessments in clinical trials and the US Food and Drug Administration's encouragement of electronic data collection, this new task force report provides stakeholders with best practice recommendations reflecting the current body of evidence and enables them to respond to future developments in research and technology. This task force recommends an evidence-based approach to determine whether new research is needed to evaluate measurement comparability for a given questionnaire or technology. The suitability of existing evidence depends upon whether it satisfactorily demonstrates that the change in data collection mode has not affected the PROM's measurement properties. In cases where sufficient evidence of measurement comparability exists and best practices for faithful migration are followed, this task force concludes that further testing of measurement comparability among the data collection modes is unnecessary, including cases of "mixing modes" within clinical trials such as bring your own device designs.

Validation of the Mobile App Version of the EQ-5D-5L Quality of Life Questionnaire Against the Gold Standard Paper-Based Version: Randomized Crossover Study

Background

Study participants and patients often perceive (long) questionnaires as burdensome. In addition, paper-based questionnaires are prone to errors such as (unintentionally) skipping questions or filling in a wrong type of answer. Such errors can be prevented with the emergence of mobile questionnaire apps.

Objective

This study aimed to validate an innovative way to measure the quality of life using a mobile app based on the EQ-5D-5L questionnaire. This validation study compared the EQ-5D-5L questionnaire requested by a mobile app with the gold standard paper-based version of the EQ-5D-5L.

Methods

This was a randomized, crossover, and open study. The main criteria for participation were participants should be aged ≥18 years, healthy at their own discretion, in possession of a smartphone with at least Android version 4.1 or higher or iOS version 9 or higher, digitally skilled in downloading the mobile app, and able to read and answer questionnaires in Dutch. Participants were recruited by a market research company that divided them into 2 groups balanced for age, gender, and education. Each participant received a digital version of the EQ-5D-5L questionnaire via a mobile app and the EQ-5D-5L paper-based questionnaire by postal mail. In the mobile app, participants received, for 5 consecutive days, 1 question in the morning and 1 question in the afternoon; as such, all questions were asked twice (at time point 1 [App T1] and time point 2 [App T2]). The primary outcomes were the correlations between the answers (scores) of each EQ-5D-5L question answered via the mobile app compared with the paper-based questionnaire to assess convergent validity.

Results

A total of 255 participants (healthy at their own discretion), 117 (45.9%) men and 138 (54.1%) women in the age range of 18 to 64 years, completed the study. To ensure randomization, the measured demographics were checked and compared between groups. To compare the results of the electronic and paper-based questionnaires, polychoric correlation analysis was performed. All questions showed a high correlation (0.64-0.92; P<.001) between the paper-based and the mobile app–based questions at App T1 and App T2. The scores and their variance remained similar over the questionnaires, indicating no clear difference in the answer tendency. In addition, the correlation between the 2 app-based questionnaires was high (>0.73; P<.001), illustrating a high test-retest reliability, indicating it to be a reliable replacement for the paper-based questionnaire.

Conclusions

This study indicates that the mobile app is a valid tool for measuring the quality of life and is as reliable as the paper-based version of the EQ-5D-5L, while reducing the response burden.

Measurement Comparability of Electronic and Paper Administration of Visual Analogue Scales: A Review of Published Studies

BACKGROUND

Visual analogue scales (VASs) are used in a variety of patient-, observer- and clinician-reported outcome measures. While typically included in measures originally developed for pen-and-paper completion, a greater number of clinical trials currently use electronic approaches to their collection. This leads researchers to question whether the measurement properties of the scale have been conserved during the migration to an electronic format, particularly because electronic formats often use a different scale length than the 100 mm paper standard.

METHODS

We performed a review of published studies investigating the measurement comparability of paper and electronic formats of the VAS.

RESULTS

Our literature search yielded 26 studies published between 1997 and 2018 that reported comparison of paper and electronic formats using the VAS. After excluding 2 publications, 23 of the remaining 24 studies included in this review reported electronic formats of the VAS (eVAS) and paper formats (pVAS) to be equivalent. A further study concluded that eVAS and pVAS were both acceptable but should not be interchanged. eVAS length varied from 21 to 200 mm, indicating that 100 mm length is not a requirement.

CONCLUSIONS

The literature supports the hypothesis that eVAS and pVAS provide comparable results regardless of the VAS length. When implementing a VAS on a screen-based electronic mode, we recommend following industry best practices for faithful migration to minimise the likelihood of non-comparability with pVAS.

Do Patients Regret Having Received Systemic Treatment for Advanced Non-Small Cell Lung Cancer: A Prospective Evaluation

BACKGROUND

Thousands of patients annually receive treatment for advanced non-small cell lung cancer (NSCLC), but little is known about their views on the decision to receive that treatment, or regret. This trial prospectively evaluated the incidence of regret and whether baseline characteristics, patient decision-making parameters, or clinical progress early in the treatment course predicts regret.

MATERIALS AND METHODS

Patients receiving systemic treatment for advanced NSCLC completed every 3-week patient reported outcome (PRO) assessment using the electronic Lung Cancer Symptom Scale (eLCSS-QL), including the 3-Item Global Index (3-IGI; assessing overall distress, activities, and quality of life [QL]). A prespecified secondary aim was to determine the frequency of regret evaluated at 3 months after starting treatment. Patients were randomized to usual care or enhanced care (which included use of the DecisionKEYS decision aid).

RESULTS

Of 164 patients entered, 160 received treatment and 142 were evaluable for regret. In total, 11.5% of patients and 9% of their supporters expressed regret. Baseline characteristics did not predict regret; regret was rarely expressed by those who had a less than 20% decline or improvement in the 3-IGI PRO score after two treatment cycles. In contrast, when asked if they would make the same decision again, only 1% not having a 20% 3-IGI decline expressed regret, versus 14% with a 3-IGI decline (p = .01).

CONCLUSION

The majority of patients having regret were identified early using the PRO 3-IGI of the eLCSS-QL measure. Identifying patients at risk for regret allows for interventions, including frank discussions of progress and goals early in the treatment course, which could address regret in patients and their supporters.

IMPLICATIONS FOR PRACTICE

This report documents prospectively, for the first time, the incidence of treatment-related regret in patients with advanced lung cancer and outlines that risk of regret is associated with patient-determined worsening health status early in the course of treatment. Identifying patients at risk for regret early in treatment (before the third cycle of treatment) appears to be crucial. Counseling at that time should include a discussion of consideration of treatment change and the reason for this change.

A randomized trial of the electronic Lung Cancer Symptom Scale for quality-of-life assessment in patients with advanced non-small-cell lung cancer

Introduction

Improving health-related quality of life (hrqol) is a key goal of systemic therapy in advanced lung cancer, although routine assessment remains challenging. We analyzed the impact of a real-time electronic hrqol tool, the electronic Lung Cancer Symptom Scale (elcss-ql), on palliative care (pc) referral rates, patterns of chemotherapy treatment, and use of other supportive interventions in patients with advanced non-small-cell lung cancer (nsclc) receiving first-line chemotherapy.

Methods

Patients with advanced nsclc starting first-line chemotherapy were randomized to their oncologist receiving or not receiving their elcss-ql data before each clinic visit. Patients completed the elcss-ql at baseline, before each chemotherapy cycle, and at subsequent follow-up visits until disease progression. Prospective data about the pc referral rate, hrqol, and use of other supportive interventions were collected.

Results

For the 95 patients with advanced nsclc who participated, oncologists received real-time elcss-ql data for 44 (elcss-ql arm) and standard clinical assessment alone for 51 (standard arm). The primary endpoint, the pc referral rate, was numerically higher, but statistically similar, for patients in the elcss-ql and standard arms. The hrqol scores over time were not significantly different between the two study arms.

Conclusions

The elcss-ql is feasible as a tool for use in routine clinical practice, although no statistically significant effect of its use was demonstrated in our study. Improving access to supportive care through the collection of patient-reported outcomes and hrqol should be an important component of care for patients with advanced lung cancer.

Benefits and Disadvantages of Electronic Patient-reported Outcome Measures: Systematic Review

Background

Patient-reported outcome measures (PROMs) are important in clinical practice and research. The growth of electronic health technologies provides unprecedented opportunities to systematically collect information via PROMs.

Objective

The aim of this study was to provide an objective and comprehensive overview of the benefits, barriers, and disadvantages of the digital collection of qualitative electronic patient-reported outcome measures (ePROMs).

Methods

We performed a systematic review of articles retrieved from PubMED and Web of Science. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed during all stages. The search strategy yielded a total of 2333 records, from which 32 met the predefined inclusion and exclusion criteria. The relevant ePROM-related information was extracted from each study.

Results

Results were clustered as benefits and disadvantages. Reported benefits of ePROMs were greater patient preference and acceptability, lower costs, similar or faster completion time, higher data quality and response rates, and facilitated symptom management and patient-clinician communication. Tablets were the most used ePROM modality (14/32, 44%), and, as a platform, Web-based systems were used the most (26/32, 81%). Potential disadvantages of ePROMs include privacy protection, a possible large initial financial investment, and exclusion of certain populations or the “digital divide.”

Conclusions

In conclusion, ePROMs offer many advantages over paper-based collection of patient-reported outcomes. Overall, ePROMs are preferred over paper-based methods, improve data quality, result in similar or faster completion time, decrease costs, and facilitate clinical decision making and symptom management. Disadvantages regarding ePROMs have been outlined, and suggestions are provided to overcome the barriers. We provide a path forward for researchers and clinicians interested in implementing ePROMs.

Trial Registration

PROSPERO CRD42018094795; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=94795

Feasibility and Utility of Incorporating Patient-Reported Outcomes into Surveillance Strategies for Advanced Lung Cancer

PURPOSE

To identify and to describe patient-reported outcomes (PROs) in lung cancer patients and to evaluate the feasibility and utility of PROs into surveillance strategies, a review was carried out.

PATIENTS AND METHODS

A systematic search in bibliographic databases evaluating the instruments used in PROs of non-small-Cell lung cancer (NSCLC) patients was done.

RESULTS

From August 2014 to August 2019, 33 studies were included in this review and 16,491 patients were evaluated. PROs were divided into 6 different categories: 1) PROs as a guide in therapeutic choice, 2) PROs as indicator of disease progression, 3) agreement between PROs and the evaluated parameters, 4) PROs to evaluate the effects of immunotherapy, 5) need to deepen the knowledge of PROs, and 6) use of new electronic PROs.

CONCLUSION

The most frequently used instruments are EORTC QLQ-30 (16, 50%) and EORTC LC-13 (14, 43.75%) and in some studies (37.5%) they are used together. For different reasons (disease progression, adverse event, death, incomplete participation, etc.), the completion of these instruments decreased over time from baseline to subsequent measurements. This review demonstrates that PROs can play an important role as part of health care, and that routine use implementation could improve patient management in addition to the traditionally collected outcome.

Training on the Use of Technology to Collect Patient-Reported Outcome Data Electronically in Clinical Trials: Best Practice Recommendations from the ePRO Consortium

Electronic capture of patient-reported outcome (PRO) data has many advantages over paper-based data collection. Regulatory agencies have consistently supported the use of electronic PRO (ePRO) data capture and recommended participant and site staff training on the correct use of electronic data capture systems. The objective of this paper is to outline best practice recommendations for training end users, including site staff and study participants, on the use of ePRO technology in clinical trials to enable consistent, accurate, and complete data collection. Site personnel should be trained on study-specific as well as technology-specific topics and be given instructions on whom to contact to obtain technical support. Optimal training takes place over time using multiple modalities, including hands-on, face-to-face training at an investigator meeting or directly in the clinical site; remote training via webinar or teleconference; interactive on-demand self-paced-training via e-learning modalities; and supplemented by proxy training performed by study clinical research associates. Like site personnel training, study participants should be provided with individual, hands-on training by site staff at the initiation of the trial and in conjunction with interactive electronic training modules that can be accessed on-demand throughout the duration of the trial. The recommendations put forth in this paper provide a structured framework for the training that site personnel and study participants need to optimize the advantages trials can gain from using ePRO data collection systems.

Enhancing evaluation of sarcopenia in patients with non-small cell lung cancer (NSCLC) by assessing skeletal muscle index (SMI) at the first lumbar (L1) level on routine chest computed tomography (CT)

PURPOSE

Ongoing cancer cachexia trials evaluate sarcopenia by skeletal muscle index (SMI) at the L3 vertebrae level, commonly used as a standard. Routine chest CT institutional protocols widely differ in including L3. We investigated whether SMI at L1 assessment, rather than L3, would be reliable and more practicable for non-small cell lung cancer (NSCLC).

METHODS

NSCLC patients with routine CT chest had SMI measurements performed at L1 using Slice-O-Matic software. Accuracy of including L1 level, imaging quality, and ability to detect sarcopenia was collected and correlation of L1 SMI with body mass index (BMI) was performed.

RESULTS

Thirty-seven patients with NSCLC (73 CT assessments) were enlisted at three institutions. Characteristics: 47% female; medians: age 59, KPS 80%; BMI 25.49, weight 72.97 kg, SMI 59.24. Sarcopenia was detected in 14.7% of patients; 20% had sarcopenic obesity. Of the 73 CTs, 94.5% included L1 (95% CI 86.6-98.5%). Three images (4%) were difficult to evaluate. Inclusion of L1 was similar among the three participating institutions (90.4 to 96.7% inclusion). BMI correlation with SMI was weak (r = 0.329).

CONCLUSIONS

SMI assessment at L1 is achievable in patients with NSCLC receiving routine chest CT, with 96% having acceptable quality evaluations. Similar to results previously reported at L3, BMI showed poor correlation and low sensitivity to detect muscle mass loss. The use of CT at L1 is reliable and presents the opportunity for easier patient evaluation of sarcopenia in patients with lung cancer without the need for additional testing or radiation exposure.

Feasibility and Pilot Studies in Palliative Care Research: A Systematic Review

CONTEXT

Feasibility and pilot study designs are common in palliative care research. Finding standard guidelines on the structure and reporting of these study types is difficult.

OBJECTIVES

In feasibility and pilot studies in palliative care research, to determine 1) how commonly a priori feasibility are criteria reported and whether results are subsequently reported against these criteria? and 2) how commonly are participants' views on acceptability of burden of the study protocol assessed?

METHODS

Four databases (OVID Medline, EMBASE, CINAHL, and PubMed via caresearch.com.au.) were searched. Search terms included palliative care, terminal care, advance care planning, hospice, pilot, feasibility, with a publication date between January 1, 2012 and December 31, 2013. Articles were selected and appraised by two independent reviewers.

RESULTS

Fifty-six feasibility and/or pilot studies were included in this review. Only three studies had clear a priori criteria to measure success. Sixteen studies reported participant acceptability or burden with measures. Forty-eight studies concluded feasibility.

CONCLUSION

The terms "feasibility" and "pilot" are used synonymously in palliative care research when describing studies that test for feasibility. Few studies in palliative care research outline clear criteria for success. The assessment of participant acceptability and burden is uncommon. A gold standard for feasibility study design in palliative care research that includes both clear criteria for success and testing of the study protocol for participant acceptability and burden is needed. Such a standard would assist with consistency in the design, conduct and reporting of feasibility and pilot studies.

Key issues affecting quality of life and patient-reported outcomes in prostate cancer: an analysis conducted in 2128 patients with initial psychometric assessment of the prostate cancer symptom scale (PCSS)

OBJECTIVE

Evidence-based quality of life (QL) questionnaires require the identification of issues of importance to patients. The primary aim of this study was to inform providers on patient-expressed issues while enhancing the content validity of instruments assessing QL and patient-reported outcomes (PROs) in prostate cancer. The study provided additional psychometric properties for the new PRO and QL instrument, the Prostate Cancer Symptom Scale (PCSS).

METHODS

An anonymous web-based survey of 2128 patients with prostate cancer was conducted with patients rating 18 QL items on a five-point scale.

RESULTS

Most respondents (74%) were aged 55-74 years, had early stage disease at diagnosis (81%) and were diagnosed within 2 years of the survey (81%). The top five-rated issues were: overall QL, ability to perform normal activities, maintaining independence, ability to sleep and not being a burden. These items were ranked as either 'very important' or 'important' by at least 88% of patients. None of the most highly ranked issues were symptoms. Instead, the highest ranked items were global issues reflecting the impact of symptoms on patients. In addition to the enhanced content validity findings, good reliability results and initial support for construct validity are reported for the PCSS.

CONCLUSIONS

This is the largest survey providing patient-expressed background for content validity for QL and PRO measures. The findings of this study should aid development of newer practical questionnaires, such as the PCSS, which can be adapted to electronic platforms enhancing rapid and accurate PRO and QL evaluation.

Symptoms, Quality of Life, and Daily Activities in People With Newly Diagnosed Solid Tumors Presenting to a Medical Oncologist

INTRODUCTION

Symptom and Quality of Life (QOL) data are important patient reported outcomes. Early identification of these is critical for appropriate interventions. Data collection may be helped by modern information technology.

AIM

This study examined symptoms and QOL in people with solid tumors at their first visit to a medical oncologist. We also evaluated the clinical utility of tablet computers (TC) to collect this data.

METHODS

This was a prospective study of 105 consecutive patients in the cancer outpatient clinic of a tertiary level academic medical center. Symptom and QOL data was collected by TC with wireless database upload.

RESULTS

One-third participants had moderate to severe pain; almost half clinically significant pain that interfered with daily activities. Tiredness, anxiety, and drowsiness were common (prevalence - 79%, 63% and 50% respectively). One-third of those who had items identified from the Edmonton System Assessment System also volunteered other symptoms, mostly gastrointestinal problems. Many of those affected also reported impaired Global Wellbeing and low Overall QOL. There was a 98% completion rate, which took on average ten minutes. Direct observation and informal feedback from patients and physicians regarding the acceptability of TC in this setting was uniformly positive.

CONCLUSIONS

Amongst people with newly diagnosed solid tumors clinically important psychological and physical symptoms, QOL problems and difficulties with daily activities were commonly present in the 24-hour period and in the week before a first Medical Oncology visit. Symptom and QOL data collection by TC in busy outpatient clinics showed good clinical utility.

Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013

OBJECTIVE

To conduct a systematic review and meta-analysis of the equivalence between electronic and paper administration of patient reported outcome measures (PROMs) in studies conducted subsequent to those included in Gwaltney et al's 2008 review.

METHODS

A systematic literature review of PROM equivalence studies conducted between 2007 and 2013 identified 1,997 records from which 72 studies met pre-defined inclusion/exclusion criteria. PRO data from each study were extracted, in terms of both correlation coefficients (ICCs, Spearman and Pearson correlations, Kappa statistics) and mean differences (standardized by the standard deviation, SD, and the response scale range). Pooled estimates of correlation and mean difference were estimated. The modifying effects of mode of administration, year of publication, study design, time interval between administrations, mean age of participants and publication type were examined.

RESULTS

Four hundred thirty-five individual correlations were extracted, these correlations being highly variable (I2 = 93.8) but showing generally good equivalence, with ICCs ranging from 0.65 to 0.99 and the pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Standardised mean differences for 307 studies were small and less variable (I2 = 33.5) with a pooled standardised mean difference of 0.037 (95% CI 0.031 to 0.042). Average administration mode/platform-specific correlations from 56 studies (61 estimates) had a pooled estimate of 0.88 (95% CI 0.86 to 0.90) and were still highly variable (I2 = 92.1). Similarly, average platform-specific ICCs from 39 studies (42 estimates) had a pooled estimate of 0.90 (95% CI 0.88 to 0.92) with an I2 of 91.5. After excluding 20 studies with outlying correlation coefficients (≥3SD from the mean), the I2 was 54.4, with the equivalence still high, the overall pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Agreement was found to be greater in more recent studies (p < 0.001), in randomized studies compared with non-randomised studies (p < 0.001), in studies with a shorter interval (<1 day) (p < 0.001), and in respondents of mean age 28 to 55 compared with those either younger or older (p < 0.001). In terms of mode/platform, paper vs Interactive Voice Response System (IVRS) comparisons had the lowest pooled agreement and paper vs tablet/touch screen the highest (p < 0.001).

CONCLUSION

The present study supports the conclusion of Gwaltney's previous meta-analysis showing that PROMs administered on paper are quantitatively comparable with measures administered on an electronic device. It also confirms the ISPOR Taskforce´s conclusion that quantitative equivalence studies are not required for migrations with minor change only. This finding should be reassuring to investigators, regulators and sponsors using questionnaires on electronic devicesafter migration using best practices. Although there is data indicating that migrations with moderate changes produce equivalent instrument versions, hence do not require quantitative equivalence studies, additional work is necessary to establish this. Furthermore, there is the need to standardize migration practices and reporting practices (i.e. include copies of tested instrument versions and screenshots) so that clear recommendations regarding equivalence testing can be made in the future.raising questions about the necessity of conducting equivalence testing moving forward.

Equivalence of electronic and paper administration of patient-reported outcome measures: a systematic review and meta-analysis of studies conducted between 2007 and 2013

OBJECTIVE

To conduct a systematic review and meta-analysis of the equivalence between electronic and paper administration of patient reported outcome measures (PROMs) in studies conducted subsequent to those included in Gwaltney et al's 2008 review.

METHODS

A systematic literature review of PROM equivalence studies conducted between 2007 and 2013 identified 1,997 records from which 72 studies met pre-defined inclusion/exclusion criteria. PRO data from each study were extracted, in terms of both correlation coefficients (ICCs, Spearman and Pearson correlations, Kappa statistics) and mean differences (standardized by the standard deviation, SD, and the response scale range). Pooled estimates of correlation and mean difference were estimated. The modifying effects of mode of administration, year of publication, study design, time interval between administrations, mean age of participants and publication type were examined.

RESULTS

Four hundred thirty-five individual correlations were extracted, these correlations being highly variable (I2 = 93.8) but showing generally good equivalence, with ICCs ranging from 0.65 to 0.99 and the pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Standardised mean differences for 307 studies were small and less variable (I2 = 33.5) with a pooled standardised mean difference of 0.037 (95% CI 0.031 to 0.042). Average administration mode/platform-specific correlations from 56 studies (61 estimates) had a pooled estimate of 0.88 (95% CI 0.86 to 0.90) and were still highly variable (I2 = 92.1). Similarly, average platform-specific ICCs from 39 studies (42 estimates) had a pooled estimate of 0.90 (95% CI 0.88 to 0.92) with an I2 of 91.5. After excluding 20 studies with outlying correlation coefficients (≥3SD from the mean), the I2 was 54.4, with the equivalence still high, the overall pooled correlation coefficient being 0.88 (95% CI 0.87 to 0.88). Agreement was found to be greater in more recent studies (p < 0.001), in randomized studies compared with non-randomised studies (p < 0.001), in studies with a shorter interval (<1 day) (p < 0.001), and in respondents of mean age 28 to 55 compared with those either younger or older (p < 0.001). In terms of mode/platform, paper vs Interactive Voice Response System (IVRS) comparisons had the lowest pooled agreement and paper vs tablet/touch screen the highest (p < 0.001).

CONCLUSION

The present study supports the conclusion of Gwaltney's previous meta-analysis showing that PROMs administered on paper are quantitatively comparable with measures administered on an electronic device. It also confirms the ISPOR Taskforce´s conclusion that quantitative equivalence studies are not required for migrations with minor change only. This finding should be reassuring to investigators, regulators and sponsors using questionnaires on electronic devicesafter migration using best practices. Although there is data indicating that migrations with moderate changes produce equivalent instrument versions, hence do not require quantitative equivalence studies, additional work is necessary to establish this. Furthermore, there is the need to standardize migration practices and reporting practices (i.e. include copies of tested instrument versions and screenshots) so that clear recommendations regarding equivalence testing can be made in the future.raising questions about the necessity of conducting equivalence testing moving forward.

Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods

BACKGROUND

Self-administered survey questionnaires are an important data collection tool in clinical practice, public health research and epidemiology. They are ideal for achieving a wide geographic coverage of the target population, dealing with sensitive topics and are less resource-intensive than other data collection methods. These survey questionnaires can be delivered electronically, which can maximise the scalability and speed of data collection while reducing cost. In recent years, the use of apps running on consumer smart devices (i.e., smartphones and tablets) for this purpose has received considerable attention. However, variation in the mode of delivering a survey questionnaire could affect the quality of the responses collected.

OBJECTIVES

To assess the impact that smartphone and tablet apps as a delivery mode have on the quality of survey questionnaire responses compared to any other alternative delivery mode: paper, laptop computer, tablet computer (manufactured before 2007), short message service (SMS) and plastic objects.

SEARCH METHODS

We searched MEDLINE, EMBASE, PsycINFO, IEEEXplore, Web of Science, CABI: CAB Abstracts, Current Contents Connect, ACM Digital, ERIC, Sociological Abstracts, Health Management Information Consortium, the Campbell Library and CENTRAL. We also searched registers of current and ongoing clinical trials such as ClinicalTrials.gov and the World Health Organization (WHO) International Clinical Trials Registry Platform. We also searched the grey literature in OpenGrey, Mobile Active and ProQuest Dissertation & Theses. Lastly, we searched Google Scholar and the reference lists of included studies and relevant systematic reviews. We performed all searches up to 12 and 13 April 2015.

SELECTION CRITERIA

We included parallel randomised controlled trials (RCTs), crossover trials and paired repeated measures studies that compared the electronic delivery of self-administered survey questionnaires via a smartphone or tablet app with any other delivery mode. We included data obtained from participants completing health-related self-administered survey questionnaire, both validated and non-validated. We also included data offered by both healthy volunteers and by those with any clinical diagnosis. We included studies that reported any of the following outcomes: data equivalence; data accuracy; data completeness; response rates; differences in the time taken to complete a survey questionnaire; differences in respondent's adherence to the original sampling protocol; and acceptability to respondents of the delivery mode. We included studies that were published in 2007 or after, as devices that became available during this time are compatible with the mobile operating system (OS) framework that focuses on apps.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data from the included studies using a standardised form created for this systematic review in REDCap. They then compared their forms to reach consensus. Through an initial systematic mapping on the included studies, we identified two settings in which survey completion took place: controlled and uncontrolled. These settings differed in terms of (i) the location where surveys were completed, (ii) the frequency and intensity of sampling protocols, and (iii) the level of control over potential confounders (e.g., type of technology, level of help offered to respondents). We conducted a narrative synthesis of the evidence because a meta-analysis was not appropriate due to high levels of clinical and methodological diversity. We reported our findings for each outcome according to the setting in which the studies were conducted.

MAIN RESULTS

We included 14 studies (15 records) with a total of 2275 participants; although we included only 2272 participants in the final analyses as there were missing data for three participants from one included study.Regarding data equivalence, in both controlled and uncontrolled settings, the included studies found no significant differences in the mean overall scores between apps and other delivery modes, and that all correlation coefficients exceeded the recommended thresholds for data equivalence. Concerning the time taken to complete a survey questionnaire in a controlled setting, one study found that an app was faster than paper, whereas the other study did not find a significant difference between the two delivery modes. In an uncontrolled setting, one study found that an app was faster than SMS. Data completeness and adherence to sampling protocols were only reported in uncontrolled settings. Regarding the former, an app was found to result in more complete records than paper, and in significantly more data entries than an SMS-based survey questionnaire. Regarding adherence to the sampling protocol, apps may be better than paper but no different from SMS. We identified multiple definitions of acceptability to respondents, with inconclusive results: preference; ease of use; willingness to use a delivery mode; satisfaction; effectiveness of the system informativeness; perceived time taken to complete the survey questionnaire; perceived benefit of a delivery mode; perceived usefulness of a delivery mode; perceived ability to complete a survey questionnaire; maximum length of time that participants would be willing to use a delivery mode; and reactivity to the delivery mode and its successful integration into respondents' daily routine. Finally, regardless of the study setting, none of the included studies reported data accuracy or response rates.

AUTHORS' CONCLUSIONS

Our results, based on a narrative synthesis of the evidence, suggest that apps might not affect data equivalence as long as the intended clinical application of the survey questionnaire, its intended frequency of administration and the setting in which it was validated remain unchanged. There were no data on data accuracy or response rates, and findings on the time taken to complete a self-administered survey questionnaire were contradictory. Furthermore, although apps might improve data completeness, there is not enough evidence to assess their impact on adherence to sampling protocols. None of the included studies assessed how elements of user interaction design, survey questionnaire design and intervention design might influence mode effects. Those conducting research in public health and epidemiology should not assume that mode effects relevant to other delivery modes apply to apps running on consumer smart devices. Those conducting methodological research might wish to explore the issues highlighted by this systematic review.

Presurgical assessment of bariatric patients with the Patient Health Questionnaire (PHQ)--a screening of the prevalence of psychosocial comorbidity

BACKGROUND

Bariatric surgery has gained increasing relevance due to the dramatic rise in morbid obesity prevalence. A sound body of scientific literature demonstrates positive long-term outcome of bariatric surgery in decreasing mental and physical health morbidity. Still, there is a need for a manageable presurgical screening to assess major mental disorders. The aim of this study was to assess the frequency of common psychiatric syndromes in bariatric surgery candidates using a computerized version of the Patient Health Questionnaire (PHQ).

METHODS

In a prospective cohort study from August 2009 to July 2011 morbidly obese individuals seeking bariatric treatment were evaluated for mental health disorders using the PHQ (computerized German version).

RESULTS

A total of 159 patients were included in this study. The median age of participants was 42 years, the median BMI was 49 kg/m(2). The PHQ revealed a prevalence of 84 % for mental health disorders, 50 % of the participants had three or more mental health disorders. A high somatic symptom burden (46 %), depressive syndromes (62 %) and anxiety disorders (29 %) were the most frequent psychiatric syndromes. The median number of psychiatric syndromes was 3 for women and 1 for men (p = 0.007). No correlation between BMI and a single syndrome or the sum of syndromes was observed.

CONCLUSION

84 % of the patients seeking bariatric treatment were screened positive for at least one mental health disorder. The computerized PHQ with automated reporting appears to be a useful instrument for presurgical assessment of bariatric patients in routine medical settings.

Equivalence of electronic and paper-based patient-reported outcome measures

AIM

Electronic formats (ePROs) of paper-based patient-reported outcomes (PROs) should be validated before they can be reliably used. This review aimed to examine studies investigating measurement equivalence between ePROs and their paper originals to identify methodologies used and to determine the extent of such validation.

METHODS

Three databases (OvidSP, Web of Science and PubMed) were searched using a set of keywords. Results were examined for compliance with inclusion criteria. Articles or abstracts that directly compared screen-based electronic versions of PROs with their validated paper-based originals, with regard to their measurement equivalence, were included. Publications were excluded if the only instruments reported were stand-alone visual analogue scales or interactive voice response formats. Papers published before 2007 were excluded, as a previous meta-analysis examined papers published before this time.

RESULTS

Fifty-five studies investigating 79 instruments met the inclusion criteria. 53 % of the 79 instruments studied were condition specific. Several instruments, such as the SF-36, were reported in more than one publication. The most frequently reported formats for ePROs were Web-based versions. In 78 % of the publications, there was evidence of equivalence or comparability between the two formats as judged by study authors. Of the 30 publications that provided preference data, 87 % found that overall participants preferred the electronic format.

CONCLUSIONS

When examining equivalence between paper and electronic versions of PROs, formats are usually judged by authors to be equivalent. Participants prefer electronic formats. This literature review gives encouragement to the further widespread development and use of ePROs.

Reliability, validity and sensitivity of a computerized visual analog scale measuring state anxiety

BACKGROUND AND OBJECTIVES

Assessment of state anxiety is frequently required in clinical and research settings, but its measurement using standard multi-item inventories entails practical challenges. Such inventories are increasingly complemented by paper-and-pencil, single-item visual analog scales measuring state anxiety (VAS-A), which allow rapid assessment of current anxiety states. Computerized versions of VAS-A offer additional advantages, including facilitated and accurate data collection and analysis, and applicability to computer-based protocols. Here, we establish the psychometric properties of a computerized VAS-A.

METHODS

Experiment 1 assessed the reliability, convergent validity, and discriminant validity of the computerized VAS-A in a non-selected sample. Experiment 2 assessed its sensitivity to increase in state anxiety following social stress induction, in participants with high levels of social anxiety.

RESULTS

Experiment 1 demonstrated the computerized VAS-A's test-retest reliability (r = .44, p < .001); convergent validity with the State-Trait Anxiety Inventory's state subscale (STAI-State; r = .60, p < .001); and discriminant validity as indicated by significantly lower correlations between VAS-A and different psychological measures relative to the correlation between VAS-A and STAI-State. Experiment 2 demonstrated the VAS-A's sensitivity to changes in state anxiety via a significant pre- to during-stressor rise in VAS-A scores (F(1,48) = 25.13, p < .001).

LIMITATIONS

Set-order administration of measures, absence of clinically-anxious population, and gender-unbalanced samples.

CONCLUSIONS

The adequate psychometric characteristics, combined with simple and rapid administration, make the computerized VAS-A a valuable self-rating tool for state anxiety. It may prove particularly useful for clinical and research settings where multi-item inventories are less applicable, including computer-based treatment and assessment protocols. The VAS-A is freely available: http://people.socsci.tau.ac.il/mu/anxietytrauma/visual-analog-scale/.

An investigation of symptom burden and quality of life in Chinese chemo-naïve advanced lung cancer patients by using the Instrument-Cloud QOL System

OBJECTIVES

This study was designed to assess the disease-related symptom burden and quality of life (QOL) in Chinese chemo-naïve advanced lung cancer patients.

MATERIALS AND METHODS

Chemo-naïve patients with stage III/IV lung cancer were enrolled. 43 centers from 16 provinces of China participated in the study. Functional Assessment of Cancer Therapy-Lung (FACT-L) scale and Cloud QOL System were applied in the study.

RESULTS

376 eligible patients were analyzed. The three most common and severe symptoms were appetite loss (84.3%, scored 2.46), breathing difficulty (79.0%, scored 2.56), and cough (75.5%, scored 2.81). Significant correlation was found between QOL and symptoms. Regression analysis of QOL indicated that almost every symptom item (except shortness of breath) was the negative indicator of QOL. Moreover, pulmonary diseases, pleural metastases and brain metastases had significant negative impact on both symptoms and QOL. Relatively poor performance status affected the QOL only, while cardiovascular diseases merely affected the symptoms. And patients with EGFR mutations had less symptom burden than those with wild-type EGFR.

CONCLUSION

QOL evaluation by using the Cloud QOL System was feasible. Appetite loss, breathing difficulty and cough were the three most common and severe symptoms seen in Chinese chemo-naïve advanced lung cancer patients. Almost all symptoms had negative impact on QOL. And some clinical characteristics could predict the symptoms and QOL.

The Computer-based Health Evaluation Software (CHES): a software for electronic patient-reported outcome monitoring

BACKGROUND

Patient-reported Outcomes (PROs) capturing e.g., quality of life, fatigue, depression, medication side-effects or disease symptoms, have become important outcome parameters in medical research and daily clinical practice. Electronic PRO data capture (ePRO) with software packages to administer questionnaires, storing data, and presenting results has facilitated PRO assessment in hospital settings. Compared to conventional paper-pencil versions of PRO instruments, ePRO is more economical with regard to staff resources and time, and allows immediate presentation of results to the medical staff.The objective of our project was to develop software (CHES - Computer-based Health Evaluation System) for ePRO in hospital settings and at home with a special focus on the presentation of individual patient's results.

METHODS

Following the Extreme Programming development approach architecture was not fixed up-front, but was done in close, continuous collaboration with software end users (medical staff, researchers and patients) to meet their specific demands. Developed features include sophisticated, longitudinal charts linking patients' PRO data to clinical characteristics and to PRO scores from reference populations, a web-interface for questionnaire administration, and a tool for convenient creating and editing of questionnaires.

RESULTS

By 2012 CHES has been implemented at various institutions in Austria, Germany, Switzerland, and the UK and about 5000 patients participated in ePRO (with around 15000 assessments in total). Data entry is done by the patients themselves via tablet PCs with a study nurse or an intern approaching patients and supervising questionnaire completion.

DISCUSSION

During the last decade several software packages for ePRO have emerged for different purposes. Whereas commercial products are available primarily for ePRO in clinical trials, academic projects have focused on data collection and presentation in daily clinical practice and on extending cancer registries with PRO data. CHES includes several features facilitating the use of PRO data for individualized medical decision making. With its web-interface it allows ePRO also when patients are home. Thus, it provides complete monitoring of patients'physical and psychosocial symptom burden.