Electronic versus paper questionnaires: a further comparison in persons with asthma

ISO/TS 21564:2019- based Evaluation of a Semantic Map between Variables in the ISARIC Freestanding Follow Up Survey and ORCHESTRA Studies

The COVID-19 pandemic has led to tremendous investment in clinical studies to generate much-needed knowledge on the prevention, diagnosis, treatment and long-term effects of the disease. Case report forms, comprised of questions and answers (variables), are commonly used to collect data in clinical trials. Maximizing the value of study data depends on data quality and on the ability to easily pool and share data from several sources. ISARIC, in collaboration with the WHO, has created a case report form that is available for use by the scientific community to collect COVID-19 trial data. One of such research initiatives collecting and analyzing multi-country and multi-cohort COVID-19 study data is the Horizon 2020 project ORCHESTRA. Following the ISO/TS 21564:2019 standard, a mapping between five ORCHESTRA studies' variables and the ISARIC Freestanding Follow-Up Survey elements was created. Measures of correspondence of shared semantic domain of 0 (perfect match), 1 (fully inclusive match), 2 (partial match), 4 (transformation required) or 4* (not present in ORCHESTRA) as compared to the target code system, ORCHESTRA study variables, were assigned to each of the elements in the ISARIC FUP case report form (CRF) which was considered the source code system. Of the ISARIC FUP CRF's variables, around 34% were found to show an exact match with corresponding variables in ORCHESTRA studies and about 33% showed a non-inclusive overlap. Matching variables provided information on patient demographics, COVID-19 testing, hospital admission and symptoms. More in-depth details are covered in ORCHESTRA variables with regards to treatment and comorbidities. ORCHESTRA's Long-Term Sequelae and Fragile population studies' CRFs include 32 and 27 variables respectively which were evaluated as a perfect match to variables in the ISARIC FUP CRF. Our study serves as an example of the kind of maps between case report form variables from different research projects needed to link ongoing COVID-19 research efforts and facilitate collaboration and data sharing. To enable data aggregation across two data systems, the information they contain needs to be connected through a map to determine compatibility and transformation needs. Combining data from various clinical studies can increase the power of analytical insights.

Updated Recommendations on Evidence Needed to Support Measurement Comparability Among Modes of Data Collection for Patient-Reported Outcome Measures: A Good Practices Report of an ISPOR Task Force

The ISPOR Task Force on measurement comparability between modes of data collection for patient-reported outcome measures (PROMs) has updated the good practice recommendations from the 2009 ISPOR electronic patient-reported outcome and 2014 patient-reported outcome mixed modes Good Research Practices Task Force reports in light of accumulated evidence of measurement comparability among different modes of PROM data collection. Furthermore, with the increasing use of electronic formats of clinical outcome assessments in clinical trials and the US Food and Drug Administration's encouragement of electronic data collection, this new task force report provides stakeholders with best practice recommendations reflecting the current body of evidence and enables them to respond to future developments in research and technology. This task force recommends an evidence-based approach to determine whether new research is needed to evaluate measurement comparability for a given questionnaire or technology. The suitability of existing evidence depends upon whether it satisfactorily demonstrates that the change in data collection mode has not affected the PROM's measurement properties. In cases where sufficient evidence of measurement comparability exists and best practices for faithful migration are followed, this task force concludes that further testing of measurement comparability among the data collection modes is unnecessary, including cases of "mixing modes" within clinical trials such as bring your own device designs.

Differences in Center for Epidemiologic Studies Depression Scale, Generalized Anxiety Disorder-7 and Kessler Screening Scale for Psychological Distress Scores between Smartphone Version versus Paper Version Administration: Evidence of Equivalence

The use of electronic patient-reported outcomes has increased recently, and smartphones offer distinct advantages over other devices. However, previous systematic reviews have not investigated the reliability of the Center for Epidemiologic Studies Depression Scale (CES-D), Generalized Anxiety Disorder-7 (GAD-7), and Kessler Screening Scale for Psychological Distress (K6) when used with smartphones, and this has not been fully explored. This study aimed to evaluate the equivalence of the paper and smartphone versions of the CES-D, GAD-7, and K6, which were compared following a randomized crossover design method in 100 adults in Gunma, Japan. Participants responded to the paper and smartphone versions at 1-week intervals. The equivalence of paper and smartphone versions was evaluated using the intraclass correlation coefficient (ICC_agreement). The mean participant age was 19.86 years (SD = 1.08, 23% male). The ICC_agreements for the paper and smartphone versions of the CES-D, GAD-7, and K6 were 0.76 (95% confidence interval [CI] 0.66-0.83), 0.68 (95% CI 0.59-0.77), and 0.83 (95% CI 0.75-0.88), respectively. Thus, the CES-D and K6 scales are appropriate for use in a smartphone version, which could be applied to clinical and research settings in which the paper or smartphone versions could be used as needed.

Equivalence of Paper and Electronic-Based Patient Reported Outcome Measures for Children: A Systematic Review

Patient-reported outcome measures (PROMs) exist for a variety of chronic gastrointestinal disorders in children. The availability of electronic (e-)formats of PROMs enhance the accessibility of these tools. The International Society for Pharmacoeconomic and Outcomes Research (ISPOR) defines measurement equivalence (ME) as "comparability of the psychometric properties of data" obtained from the administration of original and adapted versions of PROMs. Consideration of proxy PROM versions is unique to pediatrics and must be included in ME evaluations. We conducted a systematic review (SR) of the literature evaluating ME of e-versions adapted from pediatric paper-based PROMs. A literature search was conducted through Medline, Embase, APA PsychInfo, and the Cochrane Library. Titles, abstracts, and manuscripts were reviewed by 2 independent reviewers. The search yielded 19 studies meeting pre-defined criteria. Just over half (52.6%) of 19 PROMs were disease-specific ones. ME between paper- and e-PROM versions was reported as present in all 19 studies evaluating 5653 participants under the age of 18 years. However, only 6 (31.6%) studies evaluated ME in proxy reported e-versions. Despite the use of PROMs for children with a variety of chronic gastrointestinal disorders, only 1 study evaluated a PROM in this population (IMPACT III for inflammatory bowel disease). Findings from this SR highlight strategic opportunities for the pediatric gastroenterologist to broaden the clinical and research armamentarium to include e-PROMs.

The effective delivery of digital CBT: a service evaluation exploring the outcomes of young people who completed video conferencing therapy in 2020

Despite its impressive evidence base, there is a widening access gap to receiving cognitive behavioural therapy (CBT). Video conferencing therapy (VCT) offers an effective solution for logistical barriers to treatment, which has been salient throughout the Coronavirus pandemic. However, research concerning the delivery of CBT via VCT for children and young people (CYP) is in its infancy, and clinical outcome data are limited. The aim of this service evaluation was to explore the effectiveness of a VCT CBT intervention for CYP referred from Child and Adolescent Mental Health Services (CAMHS) in the UK. A total of 989 records of CYP who had completed CBT via VCT in 2020 with Healios, a digital mental health company commissioned by the National Health Service (NHS), were examined to determine changes in anxiety, depression and progress towards personalised goals. Routine outcome measures (ROMs) were completed at baseline and endpoint, as well as session by session. Feedback was collected from CYP and their families at the end of treatment. There was a significant reduction in symptoms of anxiety and depression and significant progress towards goals, with pre- to post-effect sizes (Cohen’s d) demonstrating medium to large effects (d=.45 to d=−1.39). Reliable improvement ranged from 31 to 80%, clinical improvement ranged from 33 to 50%, and 25% clinically and reliably improved on at least one measure; 92% reported that they would recommend Healios. This service evaluation demonstrates that Healios’ CBT delivered via VCT is effective for CYP receiving it as part of routine mental health care.

Key learning aims

(1)

Smartphone assisted oral health data recording - an android based software application development

Background

Smartphone compared to the traditional pen-paper method could enhance oral health data recording procedure by reducing the cost of data collection, risk of data loss, early detection of errors and reducing data entry time. The present research developed a mobile/tablet-based software application to capture oral health data and test its adaptability and operations in oral health surveys.

Methods

A comparative cross-sectional study was conducted among the general population of Sanwer town, Indore district. The initial testing of the application was done on 120 individuals. A random sampling (lottery method) followed by a systematic sampling strategy was employed to select 120 households. A "one per household" design was implemented for the survey. The initial oral health data collection was done using mobile-assisted software application followed by a second examination scheduled after 15 days on the same participants using the conventional Pen-paper method to collect oral health data.

Results

Six Investigator Recorder (IR) teams conducted the oral health data collection. Data collection through Smartphone-based application displayed less meantime (3.57 minutes) in comparison to pen-paper method (4.87 minutes) (p≤0.001). Survey team response showed the majority of investigators having strong agreement on user satisfaction and speed of data entry using software application.

Conclusion

The initial testing of mobile-assisted recording system (MARS) efficiently captured oral health data among the general population with wide variations in oral disease level. The application facilitated minimal or no wastage of paper and had a high level of user-satisfaction, accuracy, speed of entry and low potential for any data loss.

Comparison between the health-related quality of life of children/adolescents with asthma and that of their caregivers: a systematic review and meta-analysis

Objective:

To evaluate the health-related quality of life (HRQoL) of children/adolescents with asthma and that of their caregivers, comparing the two.

Methods:

This was a systematic review and meta-analysis based on the criteria of Preferred Reporting Items for Systematic Reviews and Meta-Analyses, with a strategy of searching five health-related databases (MEDLINE/PubMed, EMBASE, ScienceDirect, SciELO, and LILACS). We included studies that evaluated the HRQoL of children/adolescents with asthma and that of their caregivers with the Pediatric Asthma Quality of Life Questionnaire and the Pediatric Asthma Caregiver’s Quality of Life Questionnaire, respectively, using the total scores and the scores on the domains activity limitation, symptoms (children/adolescents only), and emotional function.

Results:

We identified 291 articles, and we evaluated 133 of those. A total of 33 articles, collectively including 4,101 subjects, were included in the meta-analysis. An analysis stratified by study design showed no differences between the HRQoL of the caregivers and that of the children/adolescents in the activity limitation domain and in the total score. However, the mean emotional function domain scores were significantly higher (better) among children/adolescents with asthma than among their caregivers in longitudinal studies-Δ = 0.82 (0.21-1.44)-and randomized clinical trials-Δ = 0.52 (0.29-0.79)-although not in cross-sectional studies-Δ = −0.20 (−0.03 to 0.43).

Conclusions:

The total HRQoL scores proved to be similar between children/adolescents with asthma and their caregivers. However, the two groups differed in their perception of their emotional function, the caregivers scoring significantly lower than the children/adolescents in that domain.

Comparison between electronic and paper versions of patient-reported outcome measures in subjects with chronic obstructive pulmonary disease: an observational study with a cross-over administration

OBJECTIVES

A wide range of electronic devices can be used for data collection of patient-reported outcome (PRO) measures in subjects with chronic obstructive pulmonary disease (COPD). Although comparisons between electronic and paper-based PRO measures have been undertaken in asthmatics, it is currently uncertain whether electronic questionnaires work equally as well as paper versions in elderly subjects with COPD. The aim of this study was to compare the responses to paper and electronic versions of the Evaluating Respiratory Symptoms in COPD (E-RS) and the COPD Assessment Test (CAT).

DESIGN

A randomised cross-over design was used to compare the responses to paper and electronic versions of the two tools. The interval between the two administrations was 1 week.

SETTING

Electronic versions were self-administered under supervision using a tablet computer at our outpatient clinic (secondary care hospital in Japan) while paper questionnaires completed at home were requested to be returned by mail. It was intended that half of the patients completed the electronic versions of both questionnaires first, followed by the paper versions while the other half completed the paper versions first.

PARTICIPANTS

Eighty-one subjects with stable COPD were included.

RESULTS

The E-RS total scores (possible range 0-40) were 6.8±7.4 and 5.0±6.6 in the paper-based and electronic versions, respectively, and the CAT scores (possible range 0-40) were 10.0±7.4 and 8.6±7.8. In both questionnaires, higher scores indicate worse status. The relationship between electronic and paper versions showed significant reliability for both the E-RS total score and CAT score (intraclass correlation coefficient=0.82 and 0.89, respectively; both p<0.001). However, both the E-RS total and CAT scores were significantly higher in the paper versions (p<0.05).

CONCLUSIONS

In both cases, the two versions of the same questionnaire cannot be used interchangeably even though they have both been validated.

Electronic and paper mode of data capture when assessing patient-reported outcomes in the National Dental Practice-Based Research Network

AIM

Our objectives were to describe the approach used in the National Dental Practice-Based Research Network to capture patient-reported outcomes and to compare electronic and paper modes of data capture in a specific network study.

METHODS

This was a prospective, multicenter cohort study of 1862 patients with dentin hypersensitivity. Patient-reported outcomes were assessed based on patients' perception of pain using Visual Analog Scales and Labeled Magnitude scales at baseline and at 1, 4 and 8 weeks post-baseline.

RESULTS

Eighty-five percent of study patients chose to complete follow-up assessments via an electronic mode; 15% completed them via a paper mode. There was not a significant difference in the proportions of patients who completed the 8-week assessment when comparing the electronic mode to the paper mode (92% vs. 90.8%, P = 0.31, Rao-Scott clustered χ² -test).

CONCLUSION

The electronic mode of data capture was as operational as the traditional paper mode, while also providing the advantage of eliminating data entry errors, not involving site research coordinators in measuring the patient-reported outcomes, and not incurring cost and potential delays due to mailing study forms. Electronic data capture of patient reported outcomes could be successfully implemented in the community dental practice setting.

Orthokeratology and Contact Lens Quality of Life Questionnaire (OCL-QoL)

OBJECTIVES

To develop a quality of life (QoL) instrument for patients fitted with Orthokeratology (OrthoK) and other modalities of contact lenses.

METHODS

A 45-item pilot questionnaire was developed from an extensive literature review, focus groups, and interviews. Subsequently, 234 participants fitted with contact lenses (OrthoK, soft lenses, and gas permeable lenses) or spectacles completed the pilot questionnaire. Rasch analysis was used to assess the psychometric properties in terms of person separation, item fit statistics, category threshold ordering, differential item functioning (DIF), and targeting. A subset of participants completed the final questionnaire twice (1 week apart) and the repeatability assessed (within-subject standard deviation). The Spearman rank-order correlation coefficient between the questionnaire score and binocular visual acuity was calculated to assess construct validity.

RESULTS

Inspection of category probability curves suggested that one category was underused and the response options were collapsed. Furthermore, Rasch analyses indicated 22 items misfitted the model and were removed. The final questionnaire contained 23 items with a person separation of 2.18, item fit statistics within the range 0.83 to 1.15, targeting 1.05 logits, and no significant DIF (<0.8 logits). The repeatability (Sr) was 4.21 and there was no correlation between questionnaire scores and binocular visual acuity (rs=0.03, P=0.65).

CONCLUSIONS

The Orthokeratology and Contact Lens Quality of Life Questionnaire demonstrated excellent psychometric properties and is capable of measuring vision-related QoL in patients fitted with various forms of contact lenses including OrthoK lenses. The tool is scored on a 0 to 100 interval level scale with higher scores indicating better QoL.

Mobile electronic versus paper case report forms in clinical trials: a randomized controlled trial

BACKGROUND

Regulations, study design complexity and amounts of collected and shared data in clinical trials render efficient data handling procedures inevitable. Recent research suggests that electronic data capture can be key in this context but evidence is insufficient. This randomized controlled parallel group study tested the hypothesis that time efficiency is superior when electronic (eCRF) instead of paper case report forms (pCRF) are used for data collection. We additionally investigated predictors of time saving effects and data integrity.

METHODS

This study was conducted on top of a clinical weight loss trial performed at a clinical research facility over six months. All study nurses and patients participating in the clinical trial were eligible to participate and randomly allocated to enter cross-sectional data obtained during routine visits either through pCRF or eCRF. A balanced randomization list was generated before enrolment commenced. 90 and 30 records were gathered for the time that 27 patients and 2 study nurses required to report 2025 and 2037 field values, respectively. The primary hypothesis, that eCRF use is faster than pCRF use, was tested by a two-tailed t-test. Analysis of variance and covariance were used to evaluate predictors of entry performance. Data integrity was evaluated by descriptive statistics.

RESULTS

All randomized patients were included in the study (eCRF group n = 13, pCRF group n = 14). eCRF, as compared to pCRF, data collection was associated with significant time savings  across all conditions (8.29 ± 5.15 min vs. 10.54 ± 6.98 min, p = .047). This effect was not defined by participant type, i.e. patients or study nurses (F_(1,112) = .15, p = .699), CRF length (F_(2,112) = .49, p = .609) or patient age (Beta = .09, p = .534). Additional 5.16 ± 2.83 min per CRF were saved with eCRFs due to data transcription redundancy when patients answered questionnaires directly in eCRFs. Data integrity was superior in the eCRF condition (0 versus 3 data entry errors).

CONCLUSIONS

This is the first study to prove in direct comparison that using eCRFs instead of pCRFs increases time efficiency of data collection in clinical trials, irrespective of item quantity or patient age, and improves data quality.

TRIAL REGISTRATION

Clinical Trials NCT02649907 .

Electronic case report forms and electronic data capture within clinical trials and pharmacoepidemiology

AIMS

Researchers in clinical and pharmacoepidemiology fields have adopted information technology (IT) and electronic data capture, but these remain underused despite the benefits. This review discusses electronic case report forms and electronic data capture, specifically within pharmacoepidemiology and clinical research.

METHODS

The review used PubMed and the Institute of Electrical and Electronic Engineers library. Search terms used were agreed by the authors and documented. PubMed is medical and health based, whereas Institute of Electrical and Electronic Engineers is technology based. The review focuses on electronic case report forms and electronic data capture, but briefly considers other relevant topics; consent, ethics and security.

RESULTS

There were 1126 papers found using the search terms. Manual filtering and reviewing of abstracts further condensed this number to 136 relevant manuscripts. The papers were further categorized: 17 contained study data; 40 observational data; 27 anecdotal data; 47 covering methodology or design of systems; one case study; one literature review; two feasibility studies; and one cost analysis.

CONCLUSION

Electronic case report forms, electronic data capture and IT in general are viewed with enthusiasm and are seen as a cost-effective means of improving research efficiency, educating participants and improving trial recruitment, provided concerns about how data will be protected from misuse can be addressed. Clear operational guidelines and best practises are key for healthcare providers, and researchers adopting IT, and further work is needed on improving integration of new technologies with current systems. A robust method of evaluation for technical innovation is required.

Validation of the Oral Survey-B System for Electronic Data Capture in National Oral Health Surveys

The aims of the present study were to incorporate and to validate the electronic capture of participant-related outcomes into the Oral Survey-B System, which was originally developed for the electronic capture of clinical data. The validation process compared the performances of electronic and handwritten data captures. The hypothesis of noninferiority would be established if participants performed electronic data capture of the questionnaire survey with an effectiveness of at least 95% of that of handwritten data capture. In this multicenter, randomized, one-period crossover study design, participants (n = 261) were allocated to start with either electronic or handwritten data capture. The incorporation of the electronic self-completed questionnaire into the Oral Survey-B System was successful. The validation of the electronic questionnaire was performed by participants aged from 18 to 75 years. The interrater reliability of participants performing electronic and handwritten data capture of nonclinical assessments per questionnaire and per entry showed a kappa value of 0.72 (95% CI: 0.53-0.94). The noninferiority of electronic data capture in relation to that of the handwritten data capture and transfer was shown (p < 0.0001; 95% CI: 1.47-2.99). In conclusion, the electronic capture of participant-related outcomes with the Oral Survey-B System, originally designed for capture of clinical data, was validated. The electronic data capture was accurate and limited the number of errors. The participants were able to perform electronic data capture effectively, supporting its implementation in further National Oral Health Surveys. With the consideration of participant preference and time savings, this could lead to the implementation of electronic data capture worldwide in National Oral Health Surveys.

High Level of Agreement between Electronic and Paper Mode of Administration of a Thyroid-Specific Patient-Reported Outcome, ThyPRO

INTRODUCTION AND PURPOSE

Use of electronic questionnaires to collect health-related quality-of-life data has evolved as an alternative to paper questionnaires. For the electronic questionnaire to be used interchangeably with the validated paper questionnaire, measurement properties similar to the original must be demonstrated. The aim of the present study was to assess the equivalence between the paper version and the electronic version of the thyroid-related quality-of-life questionnaire ThyPRO.

METHODS

Patients with Graves' hyperthyroidism or autoimmune hypothyroidism in a clinically stable phase were included. The patients were recruited from two endocrine outpatient centers. All patients completed both versions in a randomized test-retest set-up. Scores were compared using intraclass correlation coefficients (ICCs), paired t tests and Bland-Altman plots. Limits of agreement were compared with data from a previous paper-paper test-retest study.

RESULTS

104 patients were included. ICCs were generally high for the 13 scales, ranging from 0.76 to 0.95. There was a small but significant difference in the scale score between paper and electronic administration for the Cosmetic complaints scale, but no differences were found for any other scale. Bland-Altman plots showed similar limits of agreement compared to the earlier test-retest study of the paper version of ThyPRO.

CONCLUSION

Based on our analyses using ICCs, paired t tests and Bland-Altman plots, we found adequate agreement between the paper and electronic questionnaires. The statistically significant difference in score found in the Cosmetic complaints scale is small and probably clinically insignificant.

Assessing the burden of childhood asthma: validation of electronic versions of the Mini Pediatric and Pediatric Asthma Caregiver's Quality of Life Questionnaires

PURPOSE

To validate electronic versions of the Mini Pediatric and Pediatric Asthma Caregiver's Quality of Life Questionnaires (MiniPAQLQ and PACQLQ, respectively), determine completion times and correlate QOL of children and caregivers.

METHODS

A total of 63 children and 64 caregivers completed the paper and electronic MiniPAQLQ or PACQLQ. Agreement between versions of each questionnaire was summarized by intraclass correlation coefficients (ICC). The correlation between MiniPAQLQ and PACQLQ scores from child-caregiver pairs was assessed using Pearson's correlation coefficient.

RESULTS

There was no significant difference (mean difference = 0.1, 95% CI -0.1, 0.2) in MiniPAQLQ Overall Scores between paper (5.9 ± 1.0, mean ± SD) and electronic (5.8 ± 1.0) versions, or any of the domains. ICCs ranged from 0.89 (Overall) to 0.86 (Emotional Function). Overall PACQLQ scores for both versions were comparable (5.9 ± 0.9 and 5.8 ± 1.0; mean difference = 0.0; 95% CI -0.1, 0.2). ICCs ranged from 0.81 (Activity Limitation) to 0.88 (Emotional Function). The electronic PACQLQ took 26 s longer (95% CI 11, 41; p < 0.001). Few participants (3-11%) preferred the paper format. MiniPAQLQ and PACQLQ scores were significantly correlated (all p < 0.05) for Overall (r paper = 0.33, r electronic = 0.27) and Emotional Function domains (r paper = 0.34, r electronic = 0.29).

CONCLUSIONS

These electronic QOL questionnaires are valid, and asthma-related QOL of children and caregivers is related.

Assessing the impact of non-severe hypoglycemic events and treatment in adults: development of the Treatment-Related Impact Measure-Non-severe Hypoglycemic Events (TRIM-HYPO)

PURPOSE

Non-severe hypoglycemic events (NSHEs) are commonly experienced by diabetes patients, particularly among insulin users, and can have serious impacts on daily functioning, emotional well-being, sleep, work productivity, and treatment adherence. Currently, no PRO measures are available to assess the impacts of non-severe hypoglycemia. To address this gap, the Treatment-Related Impact Measure-Non-severe Hypoglycemic Events (TRIM-HYPO) was developed. This paper describes the TRIM-HYPO development and validation.

METHODS

The creation of the TRIM-HYPO followed FDA's guideline for PRO development. Concept elicitation data were gathered from literature review, clinical expert interviews, and focus groups of patients with Type 1 or 2 diabetes in four countries. Based on the qualitative analysis, draft items were generated and cognitively debriefed. Psychometric validation included factor analysis, item response theory analysis, and assessment of psychometric characteristics for the TRIM-HYPO.

RESULTS

Eight clinical experts and 167 patients participated in concept elicitation. The validation study included 407 patients. Thirteen of the 46 items from the preliminary measure were dropped due to ceiling/floor effects and high correlations between conceptually similar items. Factor analysis confirmed five domains in the TRIM-HYPO: daily function, emotional well-being, diabetes management, sleep disruption, and work productivity. All scores were internally consistent (0.86-0.95) and reproducible with a test-retest range of 0.75-0.98. All but one a priori hypothesized associations for validity were confirmed.

CONCLUSIONS

Study findings demonstrate that the final, 33-item TRIM-HYPO is reliable and valid and may be useful for assessing impacts related to NSHEs in research and clinical practice.

Inter-method reliability of paper surveys and computer assisted telephone interviews in a randomized controlled trial of yoga for low back pain

Background

Little is known about the reliability of different methods of survey administration in low back pain trials. This analysis was designed to determine the reliability of responses to self-administered paper surveys compared to computer assisted telephone interviews (CATI) for the primary outcomes of pain intensity and back-related function, and secondary outcomes of patient satisfaction, SF-36, and global improvement among participants enrolled in a study of yoga for chronic low back pain.

Results

Pain intensity, back-related function, and both physical and mental health components of the SF-36 showed excellent reliability at all three time points; ICC scores ranged from 0.82 to 0.98. Pain medication use showed good reliability; kappa statistics ranged from 0.68 to 0.78. Patient satisfaction had moderate to excellent reliability; ICC scores ranged from 0.40 to 0.86. Global improvement showed poor reliability at 6 weeks (ICC = 0.24) and 12 weeks (ICC = 0.10).

Conclusion

CATI shows excellent reliability for primary outcomes and at least some secondary outcomes when compared to self-administered paper surveys in a low back pain yoga trial. Having two reliable options for data collection may be helpful to increase response rates for core outcomes in back pain trials.

Trial registration

ClinicalTrials.gov: NCT01761617. Date of trial registration: December 4, 2012.

Comprehensive Evaluation of Hemostasis in Normal Women

Women with mild bleeding disorders (MBDs) pose a diagnostic challenge and menorrhagia, the most common presenting symptom that remains underreported. We tested the hypothesis that screening apparently normal females using general and gynecological bleeding assessment tools and a global hemostatic assay together with platelet aggregation can help predict MBDs. We assessed 47 women using electronic bleeding assessment tools; e-bleeding questionnaire; and e-Pictorial Bleeding Assessment Chart (e-PBAC) based on previously validated methods, thrombelastography (TEG), and platelet aggregation together with basic coagulation testing. Three women had elevated bleeding score with von Willebrand disease diagnosis confirmed in one case and eleven cases had elevated e-PBAC. We report normal ranges for TEG and platelet aggregation in women during the first half of the menstrual cycle and show 23.4% of apparently normal women may have general or heavy menstrual bleeding. This is a prelude to a larger study to determine the validity of bleeding assessment tools in screening for MBDs in women.

Comparação das formas de aplicação papel e caneta e eletrônica do índice de estresse parental versão reduzida (PSI-SF)

Objetivo avaliar se a aplicação eletrônica do Índice de Estresse Parental – versão reduzida (PSI-SF) é comparável à aplicação em formato papel e caneta. Verificar o estresse em pais de crianças com desenvolvimento normal. Métodos quarenta adultos, pais de crianças entre seis meses e 10 anos, foram divididos em quatro grupos, sendo pareados por idade, sexo, escolaridade e idade da criança. Cada participante completou o questionário em duas ocasiões, com intervalo de sete a dez dias, nas versões: papel-caneta/papel-caneta (PP), papel-caneta/eletrônico (PE), eletrônico/eletrônico (EE), eletrônico/papel-caneta (EP). O PSI-SF apresenta 36 afirmações, divididas em três subescalas: Sofrimento Parental (SP); Interações Disfuncionais entre Pai e Criança (IDPC) e Criança Difícil (CD). Para análise dos dados foram realizadas as correlações (Pearson) e comparação da pontuação do PSI-SF intra e inter-grupos. Resultados foram obtidas correlações positivas fortes e médias e significantes entre a pontuação total e das subescalas do PSI-SF na primeira e segunda aplicação, para todos os grupos. Diferenças significantes foram observadas entre as médias das pontuações para o grupo PP (subescala “Criança Difícil”;) e grupo EE (subescala “Criança Difícil”; e pontuação total). No entanto, tais diferenças de pontuação não alteraram a interpretação do resultado do questionário. Não houve diferença significante entre os grupos para as quatro subescalas analisadas, confirmando equivalência da variância entre os grupos. O estresse dos participantes, em todas as subescalas, recaiu dentro da normalidade. Conclusão a aplicação no formato eletrônico do questionário PSI-SF apresenta resultados semelhantes à aplicação papel e caneta. Os níveis de estresse observados foram considerados normais.

Measurement equivalence and feasibility of the EORTC QLQ-PR25: paper-and-pencil versus touch-screen administration

OBJECTIVE

We assessed the measurement equivalence and feasibility of the paper-and-pencil and touch-screen modes of administration of the Taiwan Chinese version of the EORTC QLQ-PR25, a commonly used questionnaire to evaluate the health-related quality of life (HRQOL) in patients with prostate cancer in Taiwan.

METHODS

A cross-over design study was conducted in 99 prostate cancer patients at an urology outpatient clinic. Descriptive exact and global agreement percentages, intraclass correlation, and equivalence test based on minimal clinically important difference (MCID) approach were used to examine the equity of HRQOL scores between these two modes of administration. We also evaluated the feasibility of computerized assessment based on patients' acceptability and preference. Additionally, we used Rasch rating scale model to assess differential item functioning (DIF) between the two modes of administration.

RESULTS

The percentages of global agreement in all domains were greater than 85% in the EORTC QLQ-PR25. All results from equivalence tests were significant, except for Sexual functioning, indicating good equivalence. Only one item exhibited DIF between the two modes. Although nearly 80% of the study patients had no prior computer-use experience, the overall proportion of acceptance and preference for the touch-screen mode were quite high and there was no significant difference across age groups or between computer-use experience groups.

CONCLUSIONS

The study results showed that the data obtained from the modes of administration were equivalent. The touch-screen mode of administration can be a feasible and suitable alternative to the paper-and-pencil mode for assessment of patient-reported outcomes in patients with prostate cancer.

Psychometric properties of the Japanese version of the Fear-Avoidance Beliefs Questionnaire (FABQ)

BACKGROUND

The Fear-Avoidance Beliefs Questionnaire (FABQ) is useful for measuring fear-avoidance beliefs in patients with low back pain (LBP); however, no psychometrically validated Japanese version is available. The objective of this study was to evaluate reliability and validity of the Japanese version of the FABQ for use with Japanese workers with LBP.

METHODS

This was conducted as a web-based survey. Both confirmatory and exploratory factor analysis were performed to examine domain structure of the Japanese version of the FABQ. For reliability, internal consistency was assessed with Cronbach's alpha coefficient. For concurrent validity, correlation coefficients between the FABQ and the Pain Catastrophizing Scale (PCS) were calculated. For known-group validity, the relationship between FABQ score and clinical variables such as pain and depression was examined.

RESULTS

Analyses were based on responses of 1,786 adult Japanese workers with LBP. Factor analysis using the principal factor method with promax rotation revealed two factors, work and physical activity, in accordance with the domain structure of the original version of the scale. For reliability, acceptable internal consistency was demonstrated with Cronbach's alpha coefficient of 0.882 and 0.783 for each subscale. For concurrent validity, significantly moderate correlations were demonstrated between FABQ subscales and PCS subscales (r = 0.30-0.39). For known-group validity, as hypothesized, significantly higher FABQ subscale scores were observed in workers who had stronger pain, who experienced routine work disability with sick leave, who experienced recurrence of LBP, and who had depressed mood.

CONCLUSIONS

This analysis showed that the Japanese version of the FABQ is psychometrically reliable and valid to detect fear-avoidance beliefs in Japanese workers with LBP.

Technology-based dietary assessment: development of the Synchronised Nutrition and Activity Program (SNAP)

BACKGROUND

Accurate, reliable and feasible methods of dietary intake and physical activity assessment are required to improve our understanding of the associations between energy balance-related behaviours and health.

METHODS

The Synchronised Nutrition and Activity Program (SNAP) was developed to enhance recall in children by integrating new and established methods of dietary intake and physical activity recall. A list of commonly consumed foods (n = 40), drinks (n = 9) and physical activities (n = 29) was used in SNAP. All foods and drinks were analysed by count (i.e., the number of times a particular food was selected), as a proxy indicator of dietary behaviours. All reported physical activities were assigned an intensity code [in metabolic equivalents (METs)] to determine minutes of moderate-vigorous activity (MVPA; ≥3 METs).

RESULTS

Most participants completed a whole day's recall (both dietary intake and physical activities) in less than 25 min. SNAP was compared against 24-h multiple pass questionnaire and accelerometry in 121 children (aged 7-15 years old). For dietary variables, the accuracy of SNAP(™) (mean difference) was within ±1 count for the majority of food groups. The proportion of the sample with a between-method agreement within ±1 count ranged from 0.40 to 0.99. For MVPA, there was no substantial fixed or proportional bias, with a mean difference between methods (SNAP) - accelerometry) of -9 min of MVPA. Qualitatively, participants have indicated that they find SNAP easy and fun to use.

CONCLUSIONS

SNAP was developed to be a simple, quick and engaging method of assessing energy balance-related behaviours at a group or population level and succeeded because it can collect a whole day's recall (dietary intake and physical activities) in less than 25 min to a reasonable and acceptable degree of accuracy.

Asthma outcomes: quality of life

BACKGROUND

"Asthma-related quality of life" (QOL) refers to the perceived impact that asthma has on the patient's QOL.

OBJECTIVE

National Institutes of Health institutes and other federal agencies convened an expert group to recommend standardized measures of the impact of asthma on QOL for use in future asthma clinical research.

METHODS

We reviewed published documentation regarding the development and psychometric evaluation; clinical research use since 2000; and extent to which the content of each existing QOL instrument provides a unique, reliable, and valid assessment of the intended construct. We classified instruments as core (required in future studies), supplemental (used according to the study's aims and standardized), or emerging (requiring validation and standardization). This work was discussed at an National Institutes of Health-organized workshop convened in March 2010 and finalized in September 2011.

RESULTS

Eleven instruments for adults and 6 for children were identified for review. None qualified as core instruments because they predominantly measured indicators of asthma control (symptoms and/or functional status); failed to provide a distinct, reliable score measuring all key dimensions of the intended construct; and/or lacked adequate psychometric data.

CONCLUSIONS

In the absence of existing instruments that meet the stated criteria, currently available instruments are classified as either supplemental or emerging. Research is strongly recommended to develop and evaluate instruments that provide a distinct, reliable measure of the patient's perception of the impact of asthma on all of the key dimensions of QOL, an important outcome that is not captured in other outcome measures.

Post-discharge follow-up of hospital-associated infections in paediatric patients with conventional questionnaires and electronic surveillance

BACKGROUND

Viral infections are common causes of hospital-associated infections (HAIs) in paediatric patients, with most of these infections only becoming evident after discharge.

AIM

To analyse the benefits and costs of conventional and electronic surveillance methods for conducting HAI follow-ups.

METHODS

A comparison was made between response rate, time required per patient and costs to the healthcare system of conventional and electronic surveillance methods (sms, e-mail, telephone call).

FINDINGS

Altogether 1927 patients participated in the conventional followed up in 2001-2003, of whom 1175 (61%) returned the questionnaire; during the electronic surveillance period in 2005-2007, 2309 patients were followed-up in hospital, and 1940 of them (84%) returned the post-discharge information to us. The time needed by HCWs was 33 min per patient in the conventional follow-up and 13 min in the electronic follow-up, the total costs per patient being €15.07 and €13.61 respectively. A decrease of 17.1% in annual expenses was achieved with the electronic follow-up. The incidence of HAI was 8.4% in the conventional period and 12.2% in the electronic surveillance period, most cases becoming symptomatic after hospitalization.

CONCLUSION

Electronic data collection was a convenient way of implementing a continuous HAI follow-up, achieving both a higher participation rate and lower costs.

Data quality assurance: an analysis of patient non-response

PURPOSE

Patient satisfaction is paramount to maintaining high clinical quality assurance. This study seeks to compare response rates, response bias, and the completeness of data between paper and electronic collection modes of a chiropractic patient satisfaction survey.

DESIGN/METHODOLOGY/APPROACH

A convenience sample of 206 patients presenting to a chiropractic college clinic were surveyed concerning satisfaction with their chiropractic care. Paper (in-clinic and postal) and electronic modes of survey administration were compared for response rates and non-response bias.

FINDINGS

The online data collection mode resulted in fewer non-responses and a higher response rate, and did not evince response bias when compared to paper modes. The postal paper mode predicted non-response rates over the in-clinic paper and online modalities and exhibited a gender bias.

RESEARCH LIMITATIONS/IMPLICATIONS

This current study was a single clinic study; future studies should consider multi-clinic data collections. Busy clinic operations and available staff resources restricted the ability to conduct a random sampling of patients or to invite all eligible patients, therefore limiting the generalizability of collected survey data.

PRACTICAL IMPLICATIONS

Results of this study will provide data to aid development of survey protocols that efficiently, account for available human resources, and are convenient for patients while allowing for the most complete and accurate data collection possible in an educational clinic setting.

ORIGINALITY/VALUE

Understanding patient responses across survey modes is critical for the cultivation of quality business intelligence within college teaching clinic settings. This study bridges measurement evidence from three popular data collection modalities and offers support for higher levels of quality for web-based data collection.

Electronic pain assessment in clinical practice

SUMMARY The assessment and treatment of chronic pain remains an international challenge for healthcare providers among aging patients and rising healthcare costs. This article provides a brief overview of studies on the use of electronic pain diaries and innovative software programs for pain assessment and monitoring among providers and persons with chronic pain. The advent of software pain monitoring programs on smart phones, personal digital assistants and internet-based tracking allow for the collection of large datasets of momentary data to better assist in the management of chronic pain. Electronic tracking in the home and clinic can improve data quality and reduce the biases of recalled information compared with paper diaries and questionnaires. Furthermore, 3D pain mapping programs can enable patients to represent the location and intensity of their pain with greater accuracy. However, despite the benefits, there are a number of barriers to incorporating electronic pain assessment into daily clinical practice. Additional control trials and clinical investigations are needed to demonstrate the efficacy and benefit of electronic pain assessment over and above standard practices.

Emotional competence and quality of life of children with asthma: The mediating effect of coping strategies

This study explored the mediating effect of coping strategies on the relationship between emotional competence (EC) and quality of life (QOL) among children with asthma. Participants were 87 children (M age = 11.72, SD = 2.58) with controlled and partially controlled asthma, undergoing everyday treatment. They filled in questionnaires assessing EC, coping strategies and QOL. Results showed that the association between some ECs and the QOL of children with asthma was fully mediated by two maladaptive cognitive coping strategies. Among children with asthma, a greater ability to differentiate their emotions, a reduced attention to bodily signals of emotions and a reduced analysis of their current emotional state were related to decreased engagement in two coping strategies ('Ignoring Asthma' and 'Worrying about Asthma'), which in turn increased their QOL. These findings show that EC has an indirect effect on QOL through very specific coping strategies. They also emphasise the importance of screening EC in children with asthma and the importance of developing and using multidisciplinary interventions for them.

Can the Toronto Extremity Salvage Score produce reliable results when used online?

BACKGROUND

Web-based questionnaires have become popular, however, access to the Internet can be biased regarding age, gender, and education, among other factors. Therefore, it is unknown whether this is a reasonable avenue to administer a questionnaire to patients or whether Web-based can be a reliable alternative to paper-based.

QUESTIONS/PURPOSES

We determined whether the Internet version of the Toronto Extremity Salvage Score is reproducible compared with the paper-based version and the compliance and completion rates.

PATIENTS AND METHODS

The study population consisted of 81 adults who had had surgery for a musculoskeletal tumor of the lower extremity more than 12 months earlier. The Toronto Extremity Salvage Score was administered by paper at a baseline interview and then readministered via Internet 7 to 14 days later to those with access.

RESULTS

Sixty of the 81 patients (74%) were able to use the Internet. Increasing age and lower education levels were correlated with a lower likelihood of using the Internet. Questionnaires were done online and on paper by 56 patients but 10 were excluded because of self-reported change in circumstances. The mean TESS was 85.7 (range, 41.1-100; SD, 17.26) for the paper-based questionnaire and 85.2 (range, 42.5-100; SD, 17.47) for the Internet-based questionnaire. The intraclass correlation coefficient was 0.97.

CONCLUSIONS

The questionnaire can be transferred successfully to the Internet and can be used reliably instead of a paper-based instrument. Recruitment to use an Internet-based questionnaire is limited only by the percentage of patients able to access and use the Internet.

In-clinic use of electronic pain diaries: barriers of implementation among pain physicians

OBJECTIVES

The aim of this study was to examine barriers to the use of electronic diaries within the clinic setting and determine outcome differences between patients who used electronic diaries to monitor their progress with summary data feedback and patients who monitored their progress with paper diaries without summary data feedback.

METHODS

One hundred thirty-four (n=134) chronic pain patients were asked to monitor their pain, mood, activity interference, medication use, and pain location on either a paper or electronic diary immediately before each monthly clinic visit for 10 months. Patients and their treating physicians in the electronic diary group (n=67) were able to observe changes in their ratings whereas patients using the paper diaries (n=67) had no feedback about their data entry.

RESULTS

Most participants believed that completing pain diaries was beneficial; yet, only 23% of patients in the experimental condition felt that the data from the electronic diaries improved their care and less than 15% believed that their doctor made a change in their treatment based on the summary diary information.

CONCLUSION

In general, treating physicians were positive about the use of electronic diaries, although they admitted that they did not regularly incorporate the summary data in their treatment decision making because either they forgot or they were too busy. Future studies in understanding barriers to physicians' and patients' use of diary data to impact treatment outcome are needed to improve care for persons with chronic pain.

On-site electronic observational assessment tool for discomfort and pain

Over the recent years pen-paper observational assessment scales have proven to be useful to monitor behaviour and responses of humans and animals. Observational assessment tools are typically applied for subjects who are not able to communicate directly. For on-site observational assessment however it is hard to record and evaluate timing patterns of observed events using pen-paper scales. Although timing information is in many cases assumed highly valuable, only (videotaped) laboratory scales are able to benefit from this knowledge. In the work described in this paper we digitize pen-paper assessment scales resulting in new functionalities capable to improve assessment scores. A study of on-site pain and discomfort assessment of severely demented elderly is presented. The resulting system is a mobile electronic device with a graphical user interface (GUI) on a touch screen. Moreover digital information is stored in a database improving administration, providing immediate feedback and allowing applications like: visualisation, statistical analysis and scientific research like data mining. The device allows easily registering and automatically interpreting complex timing patterns of behaviours and responses, on-site. This feature could be employed in the development of new more accurate observational assessment instruments.

Validation of an electronic version of the Mini Asthma Quality of Life Questionnaire

BACKGROUND

The Mini Asthma Quality of Life Questionnaire (MiniAQLQ) is a validated disease-specific quality of life (QOL) paper (p) questionnaire. Electronic (e) versions enable inclusion of asthma QOL in electronic medical records and research databases.

PURPOSE

To validate an e-version of the MiniAQLQ, compare time required for completion of e- and p-versions, and determine which version participants prefer.

METHODS

Adults with stable asthma were randomized to complete either the e- or p-MiniAQLQ, followed by a 2-h rest period before completing the other version. Agreement between versions was measured using the intraclass correlation coefficient (ICC) and Bland-Altman analysis.

RESULTS

Two participants with incomplete p-MiniAQLQ responses were excluded. Forty participants (85% female; age 47.7 +/- 14.9 years; asthma duration 22.6 +/- 16.1 years; FEV(1) 87.1 +/- 21.6% predicted) with both AQLQ scores <6.0 completed the study. Agreement between e- and p-versions for the overall score was acceptable (ICC=0.95) with no bias (difference (Delta) p-e=0.1; P=0.21). ICCs for the symptom, activity limitation, emotional function and environmental stimuli domains were 0.94, 0.89, 0.90, and 0.91 respectively. A small but significant bias (Delta=0.3; P=0.004) was noted in the activity limitation domain. Completion time was significantly longer for the e-version (3.8 +/- 1.9min versus 2.7 +/- 1.1min; P<0.0001). The majority of patients (57.5%) preferred the e-MiniAQLQ; 35% had no preference.

CONCLUSION

This e-version of the MiniAQLQ is valid and was preferred by most participants despite taking slightly longer to complete. Generalizabilty may be limited in younger (12-17) and older (>65) adults.

Development and validation of the Treatment Related Impact Measure of Weight (TRIM-Weight)

Background

The use of prescription anti-obesity medication (AOM) is becoming increasingly common as treatment options grow and become more accessible. However, AOM may not be without a wide range of potentially negative impacts on patient functioning and well being. The Treatment Related Impact Measure (TRIM-Weight) is an obesity treatment-specific patient reported outcomes (PRO) measure designed to assess the key impacts of prescription anti-obesity medication. This paper will present the validation findings for the TRIM-Weight.

Methods

The online validation battery survey was administered in four countries (the U.S., U.K., Australia, and Canada). Eligible subjects were over age eighteen, currently taking a prescription AOM and were currently or had been obese during their life. Validation analyses were conducted according to an a priori statistical analysis plan. Item level psychometric and conceptual criteria were used to refine and reduce the preliminary item pool and factor analysis to identify structural domains was performed. Reliability and validity testing was then performed and the minimally importance difference (MID) explored.

Results

Two hundred and eight subjects completed the survey. Twenty-one of the 43 items were dropped and a five-factor structure was achieved: Daily Life, Weight Management, Treatment Burden, Experience of Side Effects, and Psychological Health. A-priori criteria for internal consistency and test-retest coefficients for the total score and all five subscales were met. All pre-specified hypotheses for convergent and known group validity were also met with the exception of the domain of Daily Life (proven in an ad hoc analysis) as well as the 1/2 standard deviation threshold for the MID.

Conclusion

The development and validation of the TRIM-Weight has been conducted according to well-defined principles for the creation of a PRO measure. Based on the evidence to date, the TRIM-Weight can be considered a brief, conceptually sound, valid and reliable PRO measure.

Cross-national comparability of the WHOQOL-BREF: a measurement invariance approach

PURPOSE

To evaluate whether the WHOQOL-BREF measures the QOL construct in the same way across nations.

METHODS

Students from Flanders, Belgium and Iran completed the WHOQOL-BREF as part of a larger Quality of Life questionnaire. Their responses were compared using a multi-group confirmatory factor analysis.

RESULTS

In general, the QOL construct appears rather similar in both cultures; however, participants from both countries seem to respond differently to particular items of the WHOQOL-BREF. Especially for the physical and psychological domain, this is problematic, because none of their indicators works in the same way across samples.

CONCLUSIONS

Notwithstanding some limitations of this study, it must be concluded that the WHOQOL-BREF should only be used with great caution in cross-national comparisons.

Understanding and assessing the impact of treatment in diabetes: the Treatment-Related Impact Measures for Diabetes and Devices (TRIM-Diabetes and TRIM-Diabetes Device)

Purpose

Diabetes is a debilitating illness requiring lifelong management. Treatments can be varied in terms of mode of administration as well as type of agent. Unfortunately, most patient reported outcome measures currently available to assess the impact of treatment are specific to diabetes type, treatment modality or delivery systems and are designed to be either a HRQoL or treatment satisfaction measure. To address these gaps, the Treatment Related Impact Measure-Diabetes and Device measures were developed. This paper presents the item development and validation of the TRIM Diabetes/Device.

Methods

Patient interviews were conducted to collect the patient perspective and ensure high content validity. Interviews were hand coded and qualitatively analyzed to identify common themes. A conceptual model of the impact of diabetes medication was developed and preliminary items for the TRIM-Diabetes/Device were generated and cognitively debriefed. Validation data was collected via an on-line survey and analyzed according to an a priori statistical analysis plan to validate the overall score as well as each domain. Item level criteria were used to reduce the preliminary item pool. Next, factor analysis to identify structural domains was performed. Reliability and validity testing was then performed.

Results

One hundred and five patients were interviewed in focus groups, individual interviews and for cognitive debriefing. Five hundred seven patients participated in the validation study. Factor analysis identified seven domains: Treatment Burden, Daily Life; Diabetes Management; Psychological Health; Compliance and Device Function and Bother. Internal consistency reliability coefficients of the TRIM-Diabetes/Device ranged from 0.80 and 0.94. Test-retest reliability of the TRIM-Diabetes/Device ranged from 0.71 to 0.89. All convergent and known groups validity hypotheses were met for the TRIM-Diabetes/Device total scores and sub-scales.

Conclusion

Validation is an ongoing and iterative process. These findings are the first step in that process and have shown that both the TRIM-Diabetes and the TRIM-Diabetes Device have acceptable psychometric properties. Future research is needed to continue the validation process and examine responsiveness and the validity of the TRIM-Diabetes/Device in a clinical trial population.

Feasibility and acceptance of electronic quality of life assessment in general practice: an implementation study

Background

Patients' health related quality of life (HRQoL) has rarely been systematically monitored in general practice. Electronic tools and practice training might facilitate the routine application of HRQoL questionnaires. Thorough piloting of innovative procedures is strongly recommended before the conduction of large-scale studies. Therefore, we aimed to assess i) the feasibility and acceptance of HRQoL assessment using tablet computers in general practice, ii) the perceived practical utility of HRQoL results and iii) to identify possible barriers hindering wider application of this approach.

Methods

Two HRQoL questionnaires (St. George's Respiratory Questionnaire SGRQ and EORTC QLQ-C30) were electronically presented on portable tablet computers. Wireless network (WLAN) integration into practice computer systems of 14 German general practices with varying infrastructure allowed automatic data exchange and the generation of a printout or a PDF file. General practitioners (GPs) and practice assistants were trained in a 1-hour course, after which they could invite patients with chronic diseases to fill in the electronic questionnaire during their waiting time. We surveyed patients, practice assistants and GPs regarding their acceptance of this tool in semi-structured telephone interviews. The number of assessments, HRQoL results and interview responses were analysed using quantitative and qualitative methods.

Results

Over the course of 1 year, 523 patients filled in the electronic questionnaires (1–5 times; 664 total assessments). On average, results showed specific HRQoL impairments, e.g. with respect to fatigue, pain and sleep disturbances. The number of electronic assessments varied substantially between practices. A total of 280 patients, 27 practice assistants and 17 GPs participated in the telephone interviews. Almost all GPs (16/17 = 94%; 95% CI = 73–99%), most practice assistants (19/27 = 70%; 95% CI = 50–86%) and the majority of patients (240/280 = 86%; 95% CI = 82–91%) indicated that they would welcome the use of electronic HRQoL questionnaires in the future. GPs mentioned availability of local health services (e.g. supportive, physiotherapy) (mean: 9.4 ± 1.0 SD; scale: 1 – 10), sufficient extra time (8.9 ± 1.5) and easy interpretation of HRQoL results (8.6 ± 1.6) as the most important prerequisites for their use. They believed HRQoL assessment facilitated both communication and follow up of patients' conditions. Practice assistants emphasised that this process demonstrated an extra commitment to patient centred care; patients viewed it as a tool, which contributed to the physicians' understanding of their personal condition and circumstances.

Conclusion

This pilot study indicates that electronic HRQoL assessment is technically feasible in general practices. It can provide clinically significant information, which can either be used in the consultation for routine care, or for research purposes. While GPs, practice assistants and patients were generally positive about the electronic procedure, several barriers (e.g. practices' lack of time and routine in HRQoL assessment) need to be overcome to enable broader application of electronic questionnaires in every day medical practice.

Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report

BACKGROUND

Patient-reported outcomes (PROs) are the consequences of disease and/or its treatment as reported by the patient. The importance of PRO measures in clinical trials for new drugs, biological agents, and devices was underscored by the release of the US Food and Drug Administration's draft guidance for industry titled "Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims." The intent of the guidance was to describe how the FDA will evaluate the appropriateness and adequacy of PRO measures used as effectiveness end points in clinical trials. In response to the expressed need of ISPOR members for further clarification of several aspects of the draft guidance, ISPOR's Health Science Policy Council created three task forces, one of which was charged with addressing the implications of the draft guidance for the collection of PRO data using electronic data capture modes of administration (ePRO). The objective of this report is to present recommendations from ISPOR's ePRO Good Research Practices Task Force regarding the evidence necessary to support the comparability, or measurement equivalence, of ePROs to the paper-based PRO measures from which they were adapted.

METHODS

The task force was composed of the leadership team of ISPOR's ePRO Working Group and members of another group (i.e., ePRO Consensus Development Working Group) that had already begun to develop recommendations regarding ePRO good research practices. The resulting task force membership reflected a broad array of backgrounds, perspectives, and expertise that enriched the development of this report. The prior work became the starting point for the Task Force report. A subset of the task force members became the writing team that prepared subsequent iterations of the report that were distributed to the full task force for review and feedback. In addition, review beyond the task force was sought and obtained. Along with a presentation and discussion period at an ISPOR meeting, a draft version of the full report was distributed to roughly 220 members of a reviewer group. The reviewer group comprised individuals who had responded to an emailed invitation to the full membership of ISPOR. This Task Force report reflects the extensive internal and external input received during the 16-month good research practices development process. RESULTS/RECOMMENDATIONS: An ePRO questionnaire that has been adapted from a paper-based questionnaire ought to produce data that are equivalent or superior (e.g., higher reliability) to the data produced from the original paper version. Measurement equivalence is a function of the comparability of the psychometric properties of the data obtained via the original and adapted administration mode. This comparability is driven by the amount of modification to the content and format of the original paper PRO questionnaire required during the migration process. The magnitude of a particular modification is defined with reference to its potential effect on the content, meaning, or interpretation of the measure's items and/or scales. Based on the magnitude of the modification, evidence for measurement equivalence can be generated through combinations of the following: cognitive debriefing/testing, usability testing, equivalence testing, or, if substantial modifications have been made, full psychometric testing. As long as only minor modifications were made to the measure during the migration process, a substantial body of existing evidence suggests that the psychometric properties of the original measure will still hold for the ePRO version. Hence, an evaluation limited to cognitive debriefing and usability testing only may be sufficient. However, where more substantive changes in the migration process has occurred, confirming that the adaptation to the ePRO format did not introduce significant response bias and that the two modes of administration produce essentially equivalent results is necessary. Recommendations regarding the study designs and statistical approaches for assessing measurement equivalence are provided.

CONCLUSIONS

The electronic administration of PRO measures offers many advantages over paper administration. We provide a general framework for decisions regarding the level of evidence needed to support modifications that are made to PRO measures when they are migrated from paper to ePRO devices. The key issues include: 1) the determination of the extent of modification required to administer the PRO on the ePRO device and 2) the selection and implementation of an effective strategy for testing the measurement equivalence of the two modes of administration. We hope that these good research practice recommendations provide a path forward for researchers interested in migrating PRO measures to electronic data collection platforms.

Paper versus web-based administration of the Pelvic Floor Distress Inventory 20 and Pelvic Floor Impact Questionnaire 7

Web-based questionnaires are increasingly employed for clinical research. To investigate whether web-based and paper versions of the Pelvic Floor Distress Inventory 20 (PFDI-20) and Pelvic Floor Impact Questionnaire 7 (PFIQ-7) yield similar results, we compared results obtained with these two modes of administration. Women with pelvic floor disorders completed both versions of these questionnaires. Scores between modes of administration were compared using the paired t test and the intraclass correlation coefficient (ICC). Among the 52 participants, there were no significant differences in scores or scale scores between the web-based and paper questionnaires. The ICC was 0.91 for the PFDI-20 score and 0.81 for the PFIQ-7 score (p < 0.001 for each). The web-based format was preferred by 22 participants (53%), ten (24%) preferred the paper format, and nine (21%) had no preference. The acceptability and score equivalence recommend these web-based questionnaires as an alternative to paper questionnaires for clinical research.

Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review

OBJECTIVES

Patient-reported outcomes (PROs; self-report assessments) are increasingly important in evaluating medical care and treatment efficacy. Electronic administration of PROs via computer is becoming widespread. This article reviews the literature addressing whether computer-administered tests are equivalent to their paper-and-pencil forms.

METHODS

Meta-analysis was used to synthesize 65 studies that directly assessed the equivalence of computer versus paper versions of PROs used in clinical trials. A total of 46 unique studies, evaluating 278 scales, provided sufficient detail to allow quantitative analysis.

RESULTS

Among 233 direct comparisons, the average mean difference between modes averaged 0.2% of the scale range (e.g., 0.02 points on a 10-point scale), and 93% were within +/-5% of the scale range. Among 207 correlation coefficients between paper and computer instruments (typically intraclass correlation coefficients), the average weighted correlation was 0.90; 94% of correlations were at least 0.75. Because the cross-mode correlation (paper vs. computer) is also a test-retest correlation, with potential variation because of retest, we compared it to the within-mode (paper vs. paper) test-retest correlation. In four comparisons that evaluated both, the average cross-mode paper-to-computer correlation was almost identical to the within-mode correlation for readministration of a paper measure (0.88 vs. 0.91).

CONCLUSIONS

Extensive evidence indicates that paper- and computer-administered PROs are equivalent.

Electronic Diaries as a Tool to Improve Pain Management: Is There Any Evidence?

OBJECTIVE

Chronic pain is a common and costly syndrome which affects approximately one in three US adults. Factors such as shortened length of the medical visit, increased availability of technological approaches to care, and a more informed patient all suggest that a new paradigm is required for chronic pain management. Although much has been written about the use of electronic diaries in clinical trials, little has been presented about the use of these diaries in clinic practice and their potential for changing pain behavior. The intent of this preliminary study is to measure accessibility and usability of a software program designed for use on a personal digital assistant and to discuss how the software program may impact clinic practice and patient behavior.

METHODS

We present the results of a preliminary, randomized, comparison, crossover trial of 36 chronic pain patients who were asked to monitor their pain, mood, activity interference, medication use, and pain location on either a paper or electronic diary for 2 weeks. Patients in the electronic diary condition were able to observe changes in their ratings over time and view them on a secure web site.

RESULTS

No differences were found between paper and electronic tracking on pain descriptors, pain interference, mood, or helpfulness of medication. Similar to past findings, patients found the electronic diary easier to use (P < 0.0001) and would continue using it (P < 0.05) over paper if given the choice. Importantly, patients using the electronic diary reported more frequently that a provider suggested medication change (P < 0.05) based on feedback from the electronic diary. One trend requiring further investigation is that electronic diary users reported that the diary enabled them and their doctor to make care adjustments according to changes in pain status.

CONCLUSION

This study goes beyond previous research on preference and data quality to investigate how the information provided may affect patient and physician perspectives toward pain management. Although not the initial intent of this study, findings indicate that electronic tracking may provide information which can affect management decisions. A follow-up study is ongoing to investigate these initial results. If found to be true, electronic monitoring may have broad implications for health care, policy, and improvement in quality of care for chronic pain sufferers in the future.

Computerised paediatric asthma quality of life questionnaires in routine care

BACKGROUND

Asthma quality of life questionnaires are not readily incorporated into clinical care. We therefore computerised the Paediatric Asthma Quality of Life Questionnaire (standardised) (PAQLQ(S)) and the Paediatric Asthma Caregivers Quality of Life Questionnaire (PACQLQ), with a colour-coded printed graphical report.

OBJECTIVES

To (a) assess the feasibility of the electronic questionnaires in clinical care and (b) compare the child's PAQLQ scores with the parent's score, physician's clinical score and spirometry.

METHODS

Children with asthma were given a clinical severity score of 1-4 (increasing severity) and then completed the PAQLQ(S) electronically (scores 1-7 for increasing quality of life in emotional, symptoms and activity limitation domains) followed by spirometry and physician review. Parents completed the PACQLQ. Inclusion criteria required fluent Hebrew and reliable performance of spirometry. Children with additional chronic diseases were excluded.

RESULTS

147 children with asthma aged 7-17 years completed PAQLQs and 115 accompanying parents completed PACQLQs, taking 8.3 (4.3-15) and 4.4 (1.5-12.7) min, respectively (mean (range)). Graphical reports enabled physicians to address quality of life during even brief visits. Children's (PAQLQ) and parents' (PACQLQ) total scores correlated (r = 0.61, p<0.001), although the children's median emotional score of 6.3 was higher than their parents' 5.7 (p<0.001), whereas median activity limitation score was lower than their parents': 5.0 and 6.8, respectively (p<0.001). No correlation was found with physician's clinical score or spirometry.

CONCLUSIONS

Electronic PAQLQs are easy to use, providing additional insight to spirometry and physician's assessment, in routine asthma care. Future studies must assess impact on asthma management.