Updated Recommendations on Evidence Needed to Support Measurement Comparability Among Modes of Data Collection for Patient-Reported Outcome Measures: A Good Practices Report of an ISPOR Task Force

The ISPOR Task Force on measurement comparability between modes of data collection for patient-reported outcome measures (PROMs) has updated the good practice recommendations from the 2009 ISPOR electronic patient-reported outcome and 2014 patient-reported outcome mixed modes Good Research Practices Task Force reports in light of accumulated evidence of measurement comparability among different modes of PROM data collection. Furthermore, with the increasing use of electronic formats of clinical outcome assessments in clinical trials and the US Food and Drug Administration's encouragement of electronic data collection, this new task force report provides stakeholders with best practice recommendations reflecting the current body of evidence and enables them to respond to future developments in research and technology. This task force recommends an evidence-based approach to determine whether new research is needed to evaluate measurement comparability for a given questionnaire or technology. The suitability of existing evidence depends upon whether it satisfactorily demonstrates that the change in data collection mode has not affected the PROM's measurement properties. In cases where sufficient evidence of measurement comparability exists and best practices for faithful migration are followed, this task force concludes that further testing of measurement comparability among the data collection modes is unnecessary, including cases of "mixing modes" within clinical trials such as bring your own device designs.

Identification of prescription opioid misuse and abuse behaviors and risk factors in chronic pain patients using the Prescription Opioid Misuse and Abuse Questionnaire (POMAQ)

OBJECTIVES

To identify patient risk factors associated with prescription opioid misuse and abuse as well as groupings of misuse and abuse behaviors as measured by the Prescription Opioid Misuse and Abuse Questionnaire (POMAQ).

METHODS

Adults with chronic pain requiring long-term treatment with opioids completed the POMAQ and other study questionnaires. Latent class analysis (LCA) was used to examine underlying subgroups exhibiting particular risk profiles. Patient demographic and clinical characteristics were examined as covariates and the concordance between the identified latent classes at-risk classifications and the POMAQ clinical scoring algorithm was assessed.

RESULTS

Analysis of data from 809 patients revealed four classes: "chronic pain, low risk" (n = 473, low to no prevalence of POMAQ behaviors), "chronic pain, comorbid condition" (n = 152, high prevalence of anti-anxiety, sleeping pill, and antihistamine use), "at risk" (n = 154, taking more opioids than prescribed and drinking alcohol with opioids more frequently than other groups), and "high risk" (n = 30, highest prevalence of each behavior). The "high risk" group was associated with being younger, less educated, and unemployed compared to other groups. When examining the LCA classes by groups defined by the original POMAQ clinical scoring algorithm, the "high risk" class had the highest proportion of participants identified with abuse behaviors (46.7%), compared to just 4.7% in the "chronic pain, low risk" group.

CONCLUSIONS

Findings suggest there are four distinct subgroups of patients defined by chronic opioid misuse and abuse behaviors and support the use of the POMAQ to identify risk factors associated with prescription opioid misuse and abuse.

Association of Tafamidis With Health Status in Patients With ATTR Cardiac Amyloidosis: A Post Hoc Analysis of the ATTR-ACT Randomized Clinical Trial

Importance

Tafamidis reduced all-cause mortality and cardiovascular-related hospitalizations and minimized patient-reported health status deterioration at 30 months in patients with transthyretin (ATTR) amyloidosis. However, the clinical significance of health status changes remains unclear, particularly in patients with New York Heart Association (NYHA) class III symptoms who experienced more cardiovascular-related hospitalizations than those with NYHA class I-II symptoms.

Objective

To evaluate the health status of patients taking tafamidis with baseline NYHA class III symptoms.

Design, Setting, and Participants

This randomized clinical trial post hoc analysis evaluated data for patients with transthyretin (ATTR) cardiac amyloidosis and NYHA class I-III symptoms at baseline who were enrolled in ATTR-ACT, a placebo-controlled study of tafamidis held at 48 sites in 13 countries.

Interventions

Tafamidis meglumine, 80 mg or 20 mg (pooled cohort), vs placebo.

Main Outcomes and Measures

Established thresholds for clinical benefit on the Kansas City Cardiomyopathy Questionnaire Overall Summary (KCCQ-OS) were used to define response groups (very large decline to very large improvement); the proportion of patients in each group was calculated within each baseline NYHA class.

Results

Among 441 patients (264 tafamidis, 177 placebo), the mean (SD) age was 74.3 (7.0) years; 398 (90%) were male and 43 (10%) were female. Mean (SD) baseline KCCQ-OS scores were 67.3 (21.4) in the tafamidis group and 65.9 (21.7) in the placebo group (range: 0-100, with 100 indicating the best health). There was a significant shift toward better KCCQ-OS scores in patients receiving tafamidis (odds ratio for 10-point improvement 2.4; 95% CI, 1.6-3.4; P < .001). More patients taking tafamidis were alive and not worse at all time points (37% vs 15% at month 30). These findings were similar in patients with NYHA class III symptoms. In patients with NYHA class III symptoms alive at 30 months, improvements in health status were more common (35% vs 10%) and declines were less common (38% vs 57%) with tafamidis vs placebo.

Conclusions and Relevance

In ATTR-ACT, although patients with baseline NYHA class III symptoms had worse overall outcomes, treatment with tafamidis yielded better health status compared with placebo.

Trial Registration

ClinicalTrials.gov Identifier: NCT01994889.

Best Practice Recommendations for Electronic Patient-Reported Outcome Dataset Structure and Standardization to Support Drug Development

The electronic patient-reported outcome (ePRO) Dataset Structure and Standardization Project is a multistakeholder initiative formed by Critical Path Institute's PRO Consortium and Electronic Clinical Outcome Assessment (eCOA) Consortium to address issues related to ePRO dataset structure and standardization and to provide best practice recommendations for clinical trial sponsors and eCOA providers. Given the many benefits of utilizing electronic modes to capture PRO data, clinical trials are increasingly using these methods, yet there are challenges to using data generated by eCOA systems. Clinical Data Interchange Standards Consortium (CDISC) standards are used in clinical trials to ensure consistency in data collection, tabulation, and analysis and to facilitate regulatory submission. Currently, ePRO data are not required to follow a standard model, and the data models used often vary by eCOA provider and sponsor. This lack of consistency creates risks for programming and analysis and difficulties for analytics functions generating the required analysis and submission datasets. There is a disconnect between data standards used for study data submission and those used for data collection via case report forms and ePRO forms, which would be mitigated through the application of CDISC standards for ePRO data capture and transfer. The project was formed to collate and examine the issues arising from the lack of adoption of standardized approaches and this paper details recommendations to address those issues. Recommendations to address issues with ePRO dataset structure and standardization include adopting CDISC standards in the ePRO data platform, timely involvement of key stakeholders, ensuring ePRO controls are implemented, addressing issues of missing data early in development, ensuring quality control and validation of ePRO datasets, and use of read-only datasets.

Clinical scoring algorithm for the prescription opioid misuse and abuse questionnaire (POMAQ)

OBJECTIVE

The Prescription Opioid Misuse and Abuse Questionnaire (POMAQ) was developed to identify prescription opioid misuse and abuse among patients with chronic pain. A clinical scoring algorithm was developed and refined to align with the patient experience.

METHODS

This study utilized data from the POMAQ validation study (3033-4, NCT02660606) conducted on a sample of patients with chronic pain living in the United States. The study was carried out in two phases. Two purposefully enriched patient samples, one for each phase, were created based on patient responses to select POMAQ items and the availability of urine and hair samples. Two clinical experts (SHS, SFB) reviewed patient data to classify prescription opioid use behavior. Classification differences were adjudicated by a third clinical expert (JTF). Comparisons were made between the final clinical classification determined by the experts and the proposed classification based on the POMAQ algorithm.

RESULTS

Sixty patients were included in Phase I (only POMAQ data) and 52 in Phase II (including POMAQ and ancillary sources [e.g. electronic medical records, urine toxicity screen]). Refinements were made to the POMAQ scoring algorithm following discussions with clinical experts to ensure it was clinically relevant. For both phases, classifications were reviewed and discussed to achieve maximal concordance of classifications across experts. The proposed scoring algorithm was then modified to maximize agreement with the expert interpretation of clinically relevant patient experiences.

CONCLUSION

The clinical scoring algorithm for the POMAQ was developed and refined to reflect clinically relevant patient behaviors identified by expert review. Future testing is needed to determine the sensitivity and specificity of this measure.

Construct validity and reproducibility of the Prescription Opioid Misuse And Abuse Questionnaire (POMAQ)

OBJECTIVE

The Prescription Opioid Misuse and Abuse Questionnaire was developed to identify prescription opioid abuse and misuse among patients with chronic pain, however, evidence of construct validity and reproducibility is needed.

METHODS

Chronic pain patients were recruited from five Department of Defense Military Health System clinics across the United States. Construct validity was examined using subjective clinician-reported and patient-reported measures as well as objective information (e.g. hair/urine drug screens and electronic medical records). Test-retest reliability was assessed across 2 timepoints among a subgroup of patients with stable chronic pain.

RESULTS

Of 3,263 screened patients, 938 (28.7%) met eligibility and were enrolled; 809 (86.2%) completed the Prescription Opioid Misuse and Abuse Questionnaire. Construct validity was supported by comparison to other validated questionnaires and hair and urine screens which yielded high agreements with patient reports on the Prescription Opioid Misuse and Abuse Questionnaire. Electronic medical record data supported patients' Prescription Opioid Misuse and Abuse Questionnaire responses regarding physician and emergency room visits and opioid refills. The Prescription Opioid Misuse and Abuse Questionnaire had excellent test-retest reliability; the percentage agreement between the two Prescription Opioid Misuse and Abuse Questionnaire administrations was high (>90%) for most questions.

DISCUSSION

Results suggest that the Prescription Opioid Misuse and Abuse Questionnaire is a valid and reproducible tool that can be used to assess the presence of prescription opioid misuse and abuse among patients with chronic pain.

Establishing the content validity of the Prescription Opioid Misuse and Abuse Questionnaire (POMAQ) among chronic pain patients

OBJECTIVES

Content validation is essential in the development of patient-reported instruments to ensure relevancy and understandability. The aim was to evaluate patient understanding of the Prescription Opioid Misuse and Abuse Questionnaire (POMAQ) using cognitive interviewing among adults with chronic moderate to severe pain.

METHODS

This qualitative study involved a one-time in-clinic visit to conduct one-on-one cognitive interviews among participants with chronic moderate to severe pain from four groups: (1) Known Opioid Abuse; (2) Known Abuse of Other Substances (e.g. alcohol, benzodiazepines); (3) Opioid Non-abuse; and (4) No Chronic Opioid Use. Patients were recruited from 6 US clinical centers. Concept elicitation questions regarding misuse and abuse were asked at interview start; the POMAQ was completed via a web interface followed by a cognitive interview regarding POMAQ items and response options.

RESULTS

56 patients were enrolled. Mean age was 48.7 ± 12.3 years; 57% female; 80% Caucasian; mean duration of chronic pain was 11.2 ± 8.2 years with lower back pain predominating at 75%. Overall, the POMAQ was well-understood and received positive feedback. A few (n = 6, 11%) expressed concerns about completing the POMAQ using a secure internet site as they either indicated they were not computer savvy (n = 3, 5%) or were concerned about internet security (n = 3, 5%). Minor wording modifications were made to the POMAQ to enhance clarity and understanding of the POMAQ.

CONCLUSIONS

The POMAQ demonstrated content validity among patients with moderate to severe chronic pain and is undergoing psychometric evaluation among a larger cohort of patients.

Insight into chronic pain in the United States: descriptive results from the Prescription Opioid Misuse and Abuse Questionnaire (POMAQ) validation study

OBJECTIVE

A chronic pain patient sample living in the United States who participated in a cross-sectional study to evaluate the validity and reproducibility of the Prescription Opioid Misuse and Abuse Questionnaire is characterized.

METHODS

Patients with chronic pain identified through electronic medical records as refilling at least one opioid prescription within the prior 3 months were recruited from five United States Department of Defense Military Health System clinics. Patients completed the Prescription Opioid Misuse and Abuse Questionnaire, Brief Pain Inventory-Short Form, Medical Outcomes Study: 36-item Short Form, and sociodemographic questions online. Clinical characteristics and electronic medical records for 1 year prior to consent were collected.

RESULTS

809 (86.2%) participants completed the Prescription Opioid Misuse and Abuse Questionnaire. Mean (± standard deviation) age was 55.4 ± 12.7 years; the majority female (55.5%) and white (74.8%). Mean duration of chronic pain was 14.7 ± 10.5 years; the most common pain conditions were lower back pain (76.6%), neck or shoulder pain (60.3%), and osteoarthritis (38.7%). The most commonly prescribed opioids were oxycodone (35.7%), tramadol (34.5%), and hydrocodone (26.9%); 54.8% took one opioid, 44.9% took 2 or more opioids.

DISCUSSION

Participants' health status was poor; pain severity and interference were moderate. Electronic medical record data revealed high healthcare resource utilization. This chronic pain population was severely impacted by their pain condition(s).

Tafamidis Treatment for Patients with Transthyretin Amyloid Cardiomyopathy

BACKGROUND

Transthyretin amyloid cardiomyopathy is caused by the deposition of transthyretin amyloid fibrils in the myocardium. The deposition occurs when wild-type or variant transthyretin becomes unstable and misfolds. Tafamidis binds to transthyretin, preventing tetramer dissociation and amyloidogenesis.

METHODS

In a multicenter, international, double-blind, placebo-controlled, phase 3 trial, we randomly assigned 441 patients with transthyretin amyloid cardiomyopathy in a 2:1:2 ratio to receive 80 mg of tafamidis, 20 mg of tafamidis, or placebo for 30 months. In the primary analysis, we hierarchically assessed all-cause mortality, followed by frequency of cardiovascular-related hospitalizations according to the Finkelstein-Schoenfeld method. Key secondary end points were the change from baseline to month 30 for the 6-minute walk test and the score on the Kansas City Cardiomyopathy Questionnaire-Overall Summary (KCCQ-OS), in which higher scores indicate better health status.

RESULTS

In the primary analysis, all-cause mortality and rates of cardiovascular-related hospitalizations were lower among the 264 patients who received tafamidis than among the 177 patients who received placebo (P<0.001). Tafamidis was associated with lower all-cause mortality than placebo (78 of 264 [29.5%] vs. 76 of 177 [42.9%]; hazard ratio, 0.70; 95% confidence interval [CI], 0.51 to 0.96) and a lower rate of cardiovascular-related hospitalizations, with a relative risk ratio of 0.68 (0.48 per year vs. 0.70 per year; 95% CI, 0.56 to 0.81). At month 30, tafamidis was also associated with a lower rate of decline in distance for the 6-minute walk test (P<0.001) and a lower rate of decline in KCCQ-OS score (P<0.001). The incidence and types of adverse events were similar in the two groups.

CONCLUSIONS

In patients with transthyretin amyloid cardiomyopathy, tafamidis was associated with reductions in all-cause mortality and cardiovascular-related hospitalizations and reduced the decline in functional capacity and quality of life as compared with placebo. (Funded by Pfizer; ATTR-ACT ClinicalTrials.gov number, NCT01994889 .).

Cost of opioid medication abuse with and without tampering in the USA

Objective

To provide per-patient estimates of the economic burden for opioid medication abuse with and without tampering.

Patients and methods

Adults in the US who participated in the 2010 and/or 2011 National Health and Wellness Survey were resurveyed to provide information on use and abuse of prescription opioids in the previous 3 months. Participants (N=20,885) were categorized as those who abused and tampered (n=107), abused without tampering (n=118), those who reported using of opioids as prescribed (n=981), and non-opioid controls (n=19,679). Average wages from the Bureau of Labor Statistics and health care unit costs from the Truven MarketScan database were applied to self-reported work impairment (absenteeism, presenteeism, and overall work impairment) and health care resource utilization (health care provider visits, emergency room visits, hospitalizations, and drug rehabilitation) to estimate indirect and direct medical costs, respectively. Estimated mean costs for these groups were compared using analysis of variance, and generalized linear models were used to compare costs adjusted for confounders.

Results

Those who abused and tampered had significantly higher mean indirect (work impairment: $3,614 vs $2,938, p<0.05) and direct (health care use: $23,328 vs $4,514, p<0.001) costs over 3 months than those who abused without tampering. This included higher mean incremental costs for non-opioid-related medical visits ($14,180 vs $2,236, p<0.001), opioid-related medical visits ($8,790 vs $2,223, p<0.001), and drug rehabilitation ($358 vs $55, p<0.001). Increased total direct costs were associated with tampering after adjusting for confounders (p<0.001). Median incremental costs were also higher among those who tampered.

Conclusion

Tampering with prescription opioid medications is associated with significantly increased medical costs compared to abuse without tampering. Reducing tampering may provide net health care savings.

Patient-reported health-related quality of life, work productivity, and activity impairment during treatment with ALO-02 (extended-release oxycodone and sequestered naltrexone) for moderate-to-severe chronic low back pain

Background

The efficacy of ALO-02, an abuse-deterrent formulation containing extended-release oxycodone and sequestered naltrexone, in the treatment of chronic low back pain (CLBP) was studied in a 12-week randomized controlled trial. Primary efficacy endpoint results have been published previously (Rauck et al., 2015). The current paper focuses on patient-reported outcomes for health-related quality of life (HRQL), work productivity, and activity impairment that were assessed during this study.

Methods

This was a double-blind, placebo-controlled, randomized withdrawal study in patients with moderate-to-severe CLBP. After a screening period (≤2 weeks), patients entered an open-label titration period (4–6 weeks). Treatment responders were then randomized to a double-blind placebo-controlled treatment period (12 weeks). HRQL was assessed using changes in the Short Form-36 v2 Health Survey (SF-36v2) and the EuroQol-5 Dimensions Health Questionnaire 3-Level version (EQ-5D-3L). Work productivity and regular activities were evaluated using the Work Productivity and Activity Impairment Questionnaire: Specific Health Problem (WPAI:SHP).

Results

A total of 410 patients received ALO-02 during the open-label titration period, of which 280 (intent-to-treat (ITT) population) were treated during the double-blind placebo-controlled treatment period (placebo, n = 134; ALO-02, n = 146). Significant improvement was observed for all SF-36v2 subscales and component scores (p < 0.005) and the EQ-5D-3L summary index and visual analog scale (p < 0.0001) during the titration period. Improvement was also significant (p < 0.0001) for all WPAI:SHP outcomes except ‘work time missed due to CLBP’ for the titration period. Significant differences favoring ALO-02 compared with placebo were only observed for the SF-36v2 Bodily Pain subscale (p ≤ 0.0232; ITT population) during the double-blind treatment period and the overall study period (screening to the end of the double-blind treatment period). The percentage change in activity impairment due to low back pain subscale of the WPAI:SHP significantly favored ALO-02 compared with placebo for the ITT population when considering the overall study period (p = 0.0040).

Conclusions

HRQL, work productivity, and activity impairment may be improved with ALO-02 treatment.

Trial registration

ClinicalTrials.gov NCT01571362, registered April 3, 2012.

The content validation of the Self-Reported Misuse, Abuse and Diversion of Prescription Opioids (SR-MAD) instrument for use in patients with acute or chronic pain

BACKGROUND

Establishing content validity is an essential component of instrument development.

OBJECTIVE

To assess the content validity and patient interpretation of the Self-Reported Misuse, Abuse and Diversion of Prescription Opioids (SR-MAD) instrument.

METHODS

A cross-sectional, qualitative study was conducted in patients with chronic or acute pain. Patients were recruited from three patient groups (opioid naïve, known opioid abusers, and chronic opioid non-abusers). After patients completed the SR-MAD, they participated in an in-person cognitive interview to assess the patient's understanding of the instrument. Descriptive statistics and content analysis were performed.

RESULTS

Fifty-seven patients (Wave 1: 20; Wave 2: 37) were enrolled and completed the SR-MAD and cognitive interview. Mean age was 54.5 ± 13.7 years (range 25-84) with 12.5 years of living with pain. The most common chronic pain conditions were back pain (68%), neck pain (32%), and osteoarthritis (25%). Overall, most patients understood the meaning of each question and were able to describe each item using their own words. Many patients reported that some questions were not applicable to them but understood the meaning of the questions as well as the need to ask questions about misuse, abuse, and diversion of opioid medications. Minor revisions to the SR-MAD wording, response options, recall period, and the definition of "opioid", were recommended by the patients in both waves.

LIMITATIONS

Given its qualitative design, this study has a small sample size. Additionally, quantitative validation of the SR-MAD is needed.

CONCLUSION

The SR-MAD, developed based on expert consensus and revised with patient input, is a 15-item self-report instrument that can be used to identify and monitor prescription opioid abuse, misuse, and diversion.

Prescription opioid abuse and tampering in the United States: results of a self-report survey

OBJECTIVE

The objective of this study is to estimate the prevalence and impact of prescription opioid abuse and tampering among US adults.

METHODS

Participants from the US National Health and Wellness Survey were invited to complete an online survey assessing use, misuse, and abuse of prescription opioid medications in the preceding 3 months. A total of 25,864 adults were screened for self-reported opioid abuse. Prevalence was calculated using weights based on age, gender, race, and education. Respondents reporting abuse or medical use of prescription opioid medication in the prior 3 months (N = 1,242) completed a questionnaire assessing health care resource use and the Work Productivity and Activity Impairment questionnaire.

RESULTS

The prevalence of prescription opioid abuse in the 3 months prior to the survey was estimated at 1.31% of US adults, with approximately half (0.67%) tampering during that time. Opioid abuse increased with younger age, male sex, minority race, psychiatric illness, alcoholism, cigarette smoking, being employed, and higher household income. Respondents abusing opioid medications reported greater impairment in work and nonwork activities and more health care use than nonusers. Tampering with opioid medication was associated with greater productivity loss and increased use of health care (all P < 0.05).

CONCLUSIONS

Tampering with opioid medications to get high is associated with substantial loss of productivity and health care use. Technologies that reduce users' ability to tamper may reduce the burden of opioid abuse on the health care system.

Moving towards a comprehensive assessment of lower urinary tract symptoms (LUTS)

AIMS

To evaluate the utility of the International Prostate Symptom Score (IPSS) and the LUTS Tool when assessing lower urinary tract symptoms (LUTS). Secondary objectives were to examine associations of LUTS and treatment seeking.

METHODS

Analyses were performed using EpiLUTS data, a population-based, cross-sectional, Internet survey of men and women (aged ≥40) in Sweden, UK, and US with a sample of 30,000 participants. Participants completed the IPSS and the LUTS Tool. Prevalence rates of symptoms captured by the LUTS Tool were compared using IPSS summary scores: 0, 1-7, 8-19, and ≥20. LUTS Tool subscale scores were calculated. Pearson correlations between the LUTS Tool symptoms and subscales and IPSS symptoms were performed. Logistic regressions evaluated the associations of IPSS scores and LUTS Tool subscales with treatment seeking for LUTS.

RESULTS

The IPSS did not assess some symptoms (i.e., incontinence) for which there was a high prevalence among participants. Correlations between the 7 symptoms assessed by the IPSS and LUTS Tool were moderate to high ranging between 0.37 (nocturia women) and 0.77 (weak stream men), indicating concordance. While the LUTS Tool subscales and the total IPSS score were all significantly associated with treatment seeking, the LUTS Tool OAB subscale in men and Voiding subscale in women most strongly predicted treatment seeking.

CONCLUSION

LUTS that are excluded from the IPSS, most notably incontinence, were prevalent even among mildly symptomatic participants. Since storage symptoms appear to drive treatment seeking, identifying, and treating these symptoms is essential when caring for patients with LUTS.

Web-based data management for a phase II clinical trial in ALS

The objective was to report on the creation, features and performance of a web-based data management system for a two-stage phase II randomized clinical trial of Co-Enzyme Q10 in ALS. We created a relatively comprehensive web-based data system that provided electronic data entry; patient management utilities; adverse event reporting, safety monitoring, and invoice generation; and standardized coding for medications and adverse events. In stage 1, clinical sites submitted 7207 forms reporting on 105 patients followed for 10 months. Less than 0.7% of submitted forms contained errors. At the time of the delivery of the analysis data set, only four errors remained unresolved. Data were available quickly, with a median time from event to data posting of two days. The data set was locked and the analysis data set produced nine days after the final patient visit. A survey of trial personnel yielded generally positive feedback, with 75% of respondents wishing to use a similar system in the future. Given sufficient resources, a comprehensive web-based data management system can meet the need for clean, available data in clinical trials in ALS and similar diseases, and can contribute significantly to their efficient execution.

Excellent inter‐rater, intra‐rater, and telephone‐administered reliability of the ALSFRS‐R in a multicenter clinical trial

We wished to determine whether the Amyotrophic Lateral Sclerosis Functional Rating Scale-revised (ALSFRS-R) is reliable when used as primary outcome measure in a multicenter clinical trial. To establish inter-rater reliability, we randomly assigned 19 primary raters and 11 back-up raters to score nine amyotrophic lateral sclerosis (ALS) patients using the ALSFRS-R. To assess intra-rater reliability and reliability of telephone administration, we randomly assigned consecutive participants of the Clinical Trial of High Dose Coenzyme Q10 in ALS (QALS) to have in-person ALSFRS-R interviews at both screening and baseline visits (n=41 patients) or to have the ALSFRS-R interview by telephone at screening and in person at the baseline visit (n=27). An intraclass correlation coefficient (ICC) of reliability was calculated using a one-way random effects analysis of variance model. In the inter-rater reliability assessment, the primary raters performed 54 ratings on nine patients with ICC=0.93 (95% CI 0.84-0.98). For back-up raters, 32 ratings on nine patients resulted in ICC=0.93 (95% CI 0.82-0.98). The intra-rater reliability for in-person interviews was ICC = 0.95 (95% CI 0.92-0.98). The reliability of telephone administration compared to in-person interviews was ICC=0.97 (95% CI 0.93-0.98). We conclude that the ALSFRS-R shows excellent inter- and intra-rater reliability, and reliability of telephone administration when used as primary outcome measure in a multicenter ALS trial.