Linking Scores with Patient-Reported Health Outcome Instruments:A VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS

Cancer-specific utility instrument for health economic evaluations: A synopsis of the EORTC QLU-C10D user manual and current validity evidence

The European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Utility - Core 10 Dimensions (QLU-C10D) adds a preference-based scoring algorithm to the EORTC measurement portfolio. It is built on the most widely used health-related quality of life (HRQoL) measure in oncology, the EORTC Quality of Life Questionnaire - Core 30 (QLQ-C30), allowing for the calculation of both HRQoL profiles and health utilities. This is an important advancement for integrating cancer-specific values into health economic evaluations and decision making, offering greater content validity and statistical power than some generic measures. This article presents an outline of the EORTC QLU-C10D User Manual to inform health-technology bodies, academic researchers, and industry, on the why and how of using this instrument for utility measurement. It covers basic concepts, proper use of the tool, including administration and scoring. Further, we summarise the presently published valuation studies, country-specific value sets, and body of evidence about psychometric properties. Current methodological questions are discussed, including mapping between utility measures and adaptions of generic measures for use in cancer populations. Taking into account empirical studies on psychometric properties, like content and construct validity, and comparisons with generic measures, we argue that the QLU-C10D is a reliable and valid instrument for cancer populations. Without imposing additional patient and administrative burden it has the potential to support health economic decisions in cancer by providing high-quality cancer-specific utility scores, to complement the more detailed HRQoL information from the QLQ-C30.

Developing Linkages Between PROMIS Physical Function CAT and QuickDASH Scores in Hand Surgery: A Crosswalk Study

BACKGROUND

There is no standardization within hand and upper-extremity surgery regarding which patient-reported outcome measures (PROMs) are collected and reported. This limits the ability to compare or combine cohorts that utilize different PROMs. The aim of this study was to develop a linkage model for the QuickDASH (shortened version of the Disabilities of the Arm, Shoulder and Hand) and PROMIS PF CAT (Patient-Reported Outcomes Measurement Information System Physical Function computerized adaptive testing) instruments to allow interconversion between these PROMs in a hand surgery population.

METHODS

A retrospective review was conducted to identify adults (≥18 years old) who had completed the QuickDASH and PROMIS PF CAT instruments at the same clinical encounter. Patients with shoulder pathology were excluded. The linear relationship between scores was evaluated with use of the Pearson correlation coefficient. Linking was performed with use of several common methods, and an optimal linkage model was recommended on the basis of a higher R2, strong intraclass correlation coefficient (ICC), and lower standard error (SE). The recommended model was further evaluated in subgroups based on age (<60 or ≥60 years), sex, etiology for presentation (traumatic versus atraumatic), and treatment type (operative versus nonoperative).

RESULTS

A total of 15,019 patients (mean age, 49 years; 54% female; 86% White) were included. The mean QuickDASH score (and standard deviation) was 37 ± 22, and the mean PROMIS PF CAT score was 45 ± 10. There was a strong negative linear relationship between the QuickDASH and PROMIS PF CAT (r = -0.73). The circle-arc linkage model demonstrated good accuracy and reliability (R2 = 0.55; ICC = 0.71), and crosswalk tables were developed from this model. The subgroup analysis demonstrated age-related bias in the linkage model (root expected mean squared difference, 0.12). To address this, a separate crosswalk table was developed, which was dichotomized by age category.

CONCLUSIONS

The QuickDASH and PROMIS PF CAT scores were successfully linked. Utilization of the developed crosswalks-one specific to patients <60 years old and another specific to patients ≥60 years old-will allow for score interconversion in future meta-analyses and multicenter hand surgery studies.

LEVEL OF EVIDENCE

Prognostic Level III. See Instructions for Authors for a complete description of levels of evidence.

The Reha-Toolbox Project: Linking Item Subsets of 3 Established Rehabilitation PROMs to 9 Domains of the Patient-Reported Outcomes Measurement Information System (PROMIS)

OBJECTIVE

The overarching goal of the patient-reported outcomes measurement information system (PROMIS) is to standardize patient-reported outcomes across settings and health conditions globally. Following this purpose, the Reha-Toolbox study aimed to link item subsets of the World Health Organization Disability Assessment Schedule 2.0 (WHODAS 2.0), the Indicators of Rehabilitation Status (IRES-3), and the Hamburg Modules for the Assessment of Psychosocial Health (HEALTH-49) to the standardized metrics provided by PROMIS.

DESIGN

Cross-sectional, single-group linking study.

SETTING

Online survey.

PARTICIPANTS

Experts (N=5) mapped items from the 3 rehabilitation measures to PROMIS scales. Data were collected online from a general population sample (N=1000). Items from the rehabilitation measures and their corresponding PROMIS short forms were administered to facilitate item linkage.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

WHODAS 2.0, IRES-3, HEALTH-49, and PROMIS scales.

RESULTS

Overall, 96 of 171 outcome items (56%) from the legacy measures and 1 additional IRES-3 item were mapped to 9 PROMIS domains, including pain interference, physical function, dyspnea, fatigue, depression, anxiety, cognitive function, ability to participate in social roles and activities, and satisfaction with participation in social roles and activities. Ninety-five items fulfilled the linking assumptions of construct similarity, unidimensionality, and measurement invariance. The legacy items were successfully calibrated on the corresponding PROMIS metrics using graded-response models. The range and precision of the measures varied, depending on the number of items in each domain. Domains that were assessed with 4 or more items achieved sufficient reliability for group-based analyses. Crosswalk tables were created for each measure and domain. We discussed the reasons for and implications of the fact that the rehabilitation measures were only partially linked to the PROMIS metrics.

CONCLUSIONS

The study achieved robust linking between subsets of WHODAS 2.0, IRES-3, HEALTH-49 items, and 9 PROMIS scales.

The influence of early childhood education and care on the relation between early-life social adversity and children's mental health in the environmental influences for Child Health Outcomes Program

Early adversity increases risk for child mental health difficulties. Stressors in the home environment (e.g., parental mental illness, household socioeconomic challenges) may be particularly impactful. Attending out-of-home childcare may buffer or magnify negative effects of such exposures. Using a longitudinal observational design, we leveraged data from the NIH Environmental influences on Child Health Outcomes Program to test whether number of hours in childcare, defined as 1) any type of nonparental care and 2) center-based care specifically, was associated with child mental health, including via buffering or magnifying associations between early exposure to psychosocial and socioeconomic risks (age 0-3 years) and later internalizing and externalizing symptoms (age 3-5.5 years), in a diverse sample of N = 2,024 parent-child dyads. In linear regression models, childcare participation was not associated with mental health outcomes, nor did we observe an impact of childcare attendance on associations between risk exposures and symptoms. Psychosocial and socioeconomic risks had interactive effects on internalizing and externalizing symptoms. Overall, the findings did not indicate that childcare attendance positively or negatively influenced child mental health and suggested that psychosocial and socioeconomic adversity may need to be considered as separate exposures to understand child mental health risk in early life.

Comparing the contents of patient-reported outcome measures for fatigue: EORTC CAT Core, EORTC QLQ-C30, EORTC QLQ-FA12, FACIT, PRO-CTCAE, PROMIS, Brief Fatigue Inventory, Multidimensional Fatigue Inventory, and Piper Fatigue Scale

BACKGROUND

To assess fatigue in cancer patients, several patient-reported outcome measures (PROMs) are available that differ in content. To support the selection of suitable measures for specific applications and to evaluate possibilities of quantitative linking, the present study provides a content comparison of common fatigue measures, scales, and item banks. We included the EORTC CAT Core, EORTC QLQ-FA12, EORTC QLQ-C30, FACIT-F, PROMIS Fatigue (Cancer item bank v1.0), Brief Fatigue Inventory (BFI), Multidimensional Fatigue Inventory (MFI-20), Piper Fatigue Scale (PFS-12), and PRO-CTCAE.

METHODS

All items of the included measures were linked to the International Classification of Functioning, Disability and Health (ICF). Additionally, they were categorized as assessing general, physical, emotional, or cognitive fatigue. Descriptive statistics were used to display the contents covered in each measure and to allow for a qualitative comparison.

RESULTS

The measures consist of 160 items in total and covered primarily contents of the ICF components 'Body functions', 'Activities and participation', and 'Environmental Factors'. Most ICF codings refer to 'b1300 Energy level' (9-67% of the codings per instrument; 47% of all coded content). Within the broad categorization of types of fatigue, most items were classified as general fatigue (33-100% of the codings per instrument; 49% of the overall item pool). While the EORTC CAT Core focuses exclusively on physical and general fatigue, FACIT and BFI additionally assess emotional fatigue. The EORTC QLQ-FA12, PROMIS, MFI-20, and PFS-12 cover all fatigue components, including cognitive fatigue.

DISCUSSION

The review provides an in-depth content comparison of PROMs assessing cancer-related fatigue. This can inform the selection of suitable measures in different clinical contexts. Furthermore, it will inform quantitative analyses to facilitate comparison of scores obtained with different PROMs.

Using a measurement type-independent metric to compare patterns of determinants between patient-reported versus performance-based physical function in hemodialysis patients

PURPOSE

We applied a previously established common T-score metric for patient-reported and performance-based physical function (PF), offering the unique opportunity to directly compare measurement type-specific patterns of associations with potential laboratory-based, psychosocial, sociodemographic, and health-related determinants in hemodialysis patients.

METHODS

We analyzed baseline data from the CONVINCE trial (N = 1,360), a multinational randomized controlled trial comparing high-flux hemodialysis with high-dose hemodiafiltration. To explore the associations of potential determinants with performance-based versus patient-reported PF, we conducted multiple linear regression (backward elimination with cross-validation and Lasso regression). We used standardized T-scores as estimated from the PROMIS PF short-form 4a (patient-reported PF) and the Physical Performance Test (performance-based PF) as dependent variables.

RESULTS

Performance-based and patient-reported PF were both significantly associated with a laboratory marker-based indicator of muscle mass (simplified creatinine index), although the effects were relatively small (partial f2 = 0.04). Age was negatively associated with PF; the effect size was larger for performance-based (partial f2 = 0.12) than for patient-reported PF (partial f2 = 0.08). Compared to performance-based PF, patient-reported PF showed a stronger association with self-reported health domains, particularly pain interference and fatigue. When using the individual difference between patient-reported and performance-based T-scores as outcome, we found that younger age and more fatigue were associated with lower patient-reported PF compared to performance-based PF (small effect size).

CONCLUSION

Patient-reported and performance-based assessments were similarly associated with an objective marker of physical impairment in hemodialysis patients. Age and fatigue may result in discrepancies when comparing performance-based and patient-reported scores on the common PF scale. Trial Registration CONVINCE is registered in the Dutch Trial Register (Register ID: NL64750.041.18). The registration can be accessed at: https://onderzoekmetmensen.nl/en/trial/52958 .

Towards Standardized Assessment of Outcomes in Back Pain-Validation of Linking Studies Between Disease-Specific and Generic Patient-Reported Outcome Measures

Background: Comparing outcomes across different health measurement tools is essential where various patient-reported outcome measures (PROMs) are used. In spinal surgery, where recent studies show that over 30 different PROMs are applied, this need becomes even more pressing. Although several statistical transformations between the Oswestry Disability Index (ODI) and the PROMIS Profile 29 have been proposed, validation studies on conversion equations and cross-walk tables remain limited. In this study, we examined the agreement between observed ODI scores and those predicted from the PROMIS Profile 29 in a large sample of patients with low back pain, collected from routine clinical care. Methods: We compared the performance of regression and linking models at both the individual and group levels. Using Bland-Altman plots, we assessed the mean difference, 95% limits of agreement, root mean squared error (RMSE), and standardized mean differences (Cohen's d) between predicted and observed ODI scores. Results: While group-level agreement was satisfactory, with negligible effect sizes, individual prediction accuracy was relatively poor. Additionally, regression models showed inconsistent performance across the ODI score range, though incorporating more domains marginally improved predictions. Conclusions: The equipercentile linking approach demonstrated stable agreement across all ODI scores, making it the preferred method. Future regression models should account for nonlinear relationships between PROMs to enhance prediction accuracy.

Developing a European Psychotherapy Consortium (EPoC): Towards Adopting a Single-Item Self-Report Outcome Measure Across European Countries

Background

Complementing the development of evidence-based psychological therapies, practice-based evidence has developed from patient samples collected in routine care, addressing questions relevant to patients and practitioners, and thereby expanding our knowledge of psychological therapies and their impact. Implementation of assessments in routine care allows for timely clinical decision support and the collection of multiple practice-based data sets by addressing the needs of patients and clinicians (e.g., routine outcome monitoring) and the needs of researchers (e.g., identifying the impact of therapist variables on outcomes).

Method

In this article we describe an initiative developed in Europe, through the European Chapter of the Society for Psychotherapy Research, aimed at creating a consortium that has the potential for collecting data on tens of thousands of patients per year.

Results

A survey identified one of the main problems in the development of a common data set to be the heterogeneity of measures used by members (e.g., 87 different pre-post outcomes). We report on the results of the survey and the initial stage of identifying a single-item - the Emotional and Psychological Outcome (EPO-1) - measure and the process of its translation into multiple European languages.

Conclusions

We conclude this first stage of the overall project by discussing the future potential of the Consortium in relation to the development of procedures that allow crosswalks of outcome measures and the creation of a task force that may be consulted when new data sets are collected, aiming for new common measures to be implemented and shared.

Crosswalking 4 Pain Impact Measures in a Nationally Representative Sample of Adults With Back Pain

OBJECTIVE

To generate crosswalk equations and tables for 4 pain impact measures: the Impact Stratification Score (ISS), Oswestry Disability Index (ODI), Roland-Morris Disability Questionnaire (RMDQ), and the Pain, Enjoyment of Life and General Activity Scale (PEG).

DESIGN

Cross-sectional survey assessing demographics and pain impact. Crosswalks were developed using item-response theory (IRT) cocalibrations and linear regressions between the ISS, ODI, RMDQ, and PEG.

SETTING

Online panel.

PARTICIPANTS

Population-based sample of United States adults aged 18 and older. Eligibility criteria were reporting current back pain, not reporting 2 fake health conditions, and having data for 2 or more pain measures (N=1530; 37% of sample). Crosswalks were developed (n=1030) and cross-validated in a subsample of 500 participants (n=125 randomly sampled from each ISS quartile).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

ISS, ODI, RMDQ, and the PEG.

RESULTS

Associations of the ISS with the PEG and ODI met the criteria for IRT cocalibration. Other measure pairs were crosswalked using regression. Associations were strongest between the PEG and the ISS (r=0.87, normalized mean absolute error [NMAE]=0.38) and between the ODI and the ISS (r=0.85, NMAE=0.39). Associations were weakest between the PEG and the RMDQ (r=0.69, R2=0.48, NMAE: 0.55-0.58). Regression equations and IRT accounted for 48%-64% of the variance (NMAE: 0.38-0.58) in corresponding pain measures in the cross-validation sample.

CONCLUSIONS

The crosswalks between the ISS and common legacy pain measures created in this study of a nationally representative sample of United States adults with back pain can be used to estimate 1 pain impact measure from another. Further evaluation in clinical samples is recommended.

Protocol for a randomised trial of a self-directed digital pain management intervention (Empowered Relief) tailored to adults with chronic pain and prescription opioid misuse/disorder: the MOBILE Relief study

INTRODUCTION

Chronic pain increases the risk of prescription opioid misuse or opioid use disorder (OUD). Non-pharmacological treatments are needed to dually address pain and opioid risks. The purpose of the Mobile and Online-Based Interventions to Lessen Pain (MOBILE Relief) study is to compare a one-session, video-based, on-demand digital pain relief skills intervention for chronic pain ('Empowered Relief' (ER); tailored to people at risk for opioid misuse or with opioid misuse/OUD) to a one-session digital health education intervention ('Living Better'; no pain management skills).

METHODS AND ANALYSIS

MOBILE Relief is an international online randomised controlled clinical trial. Study participants are adults with chronic, non-cancer pain (≥6 months) with daily pain intensity ≥3/10, taking ≥10 morphine equivalent daily dose and score ≥6 on the Current Opioid Misuse Measure. Participants are recruited through clinician referrals and clinic advertisements. Study procedures include electronic eligibility screening, informed consent, automated 1:1 randomisation to the treatment group, baseline measures, receipt of assigned digital treatment and six post-treatment surveys spanning 3 months. Study staff will call participants at baseline and 1-month and 3 months post-treatment to verify the opioid prescription. The main statistical analyses will include analysis of covariance and mixed effects model for repeated measurements regression.

MAIN OUTCOMES

Primary outcomes are self-reported pain catastrophising, pain intensity, pain interference, opioid craving and opioid misuse at 1-month and 3 months post-treatment. We will determine the feasibility of ER (≥50% participant engagement, ≥70% treatment appraisal ratings). We hypothesise the ER group will be superior to the Living Better group in the reduction of multiprimary pain outcomes at 1-month post-treatment and opioid outcomes at 1-month and 3 months post-treatment.

ETHICS AND DISSEMINATION

The study protocol was approved by the Stanford University School of Medicine Institutional Review Board (IRB 61643). We will publish results in peer-reviewed journals; National Institute of Drug Abuse (funder) and MOBILE Relief participants will receive result summaries.

TRIAL REGISTRATION NUMBER

NCT05152134.

Linking QuickDASH and PROMIS Upper-Extremity Computer-Adaptive Test Scores in Hand Surgery: A Crosswalk Study

PURPOSE

Assessment of patient-reported outcome measures (PROMs) for hand and upper-extremity surgery patients using measures such as the Quick Disabilities of the Arm, Shoulder, and Hand (qDASH), as well as general measures including the Patient-Reported Outcomes Measurement Information System Upper Extremity Physical Function domain via a Computer-Adaptive Test (PROMIS UE CAT), has become commonplace. The aim of this study was to link, for crosswalking, the qDASH measure to both versions of the PROMIS UE CAT (v1.2 and v2.0).

METHODS

We included 18,944 hand and upper-extremity patients who completed both versions of the PROMIS UE CAT and the qDASH at the same clinical encounter. Shoulder pathology was excluded. Score linkage was performed using the R package equate, and multiple equating models (linear regression, identity, mean, linear, equipercentile, and circle-arc models) were used to establish crosswalk tables.

RESULTS

Mean qDASH and PROMIS UE CAT v1.2 scores were 38.2 (SD = 23.1) and 36.6 (SD = 9.8), respectively. Mean qDASH and PROMIS UE CAT v2.0 scores were 37.3 (SD = 21.8) and 38.3 (SD = 10.4), respectively. Pearson correlations had very strong linear relationships between the qDASH and the PROMIS UE CAT v1.2 and PROMIS UE CAT v2.0 (r = -0.83 [-0.84, -0.92] and r = -0.80 [-0.81, -0.80], respectively). For the equipercentile equating models, the intraclass correlation coefficient (ICC) had very strong positive relationships to linking measures with ICC = 0.85 (0.84, 0.86) for the qDASH-UE CAT v1.2 crosswalk and ICC = 0.83 (0.82, 0.84) for the qDASH-UE CAT v2.0 crosswalk.

CONCLUSIONS

The linkages establish crosswalk tables using equipercentile equating models to convert the PROMIS UE CAT v1.2 and v2.0 scores to the qDASH and vice versa.

CLINICAL RELEVANCE

This study provides crosswalk tables for commonly collected PROMs in hand surgery, increasing the comparability of results between centers using different PROMs to study the same conditions or treatments.

Crosswalk between the PROMIS physical function CAT and PROMIS upper extremity CAT v1.2 in a hand surgery population

BACKGROUND

There is no gold standard patient-reported outcome measure (PROM) in hand surgery. As a result, a diverse array of PROM instruments have been utilized across centers over time. Lack of score interchangeability limits the ability to compare or conglomerate scores when new instruments are introduced. Our aim was to develop a linkage for the PROMIS UE CAT v1.2 and PROMIS PF CAT scores and develop crosswalk tables for interconversion between these PROMs.

METHODS

Retrospective review was conducted to identify adult (≥ 18y) patients seen by orthopaedic hand surgeons at a single academic tertiary care hospital who had completed PROMIS UE CAT v1.2 and PROMIS PF CAT score at the same visit. For those with multiple visits, only one randomly selected visit was included in the analyses. Pearson's correlation was calculated to determine the linear relationship between the scores. Linkage from PF to UE was performed utilizing several commonly utilized equating models (identity, mean, linear, equipercentile and circle-arc methods). The performance of the models was assessed using intraclass correlation (ICC) between observed PROMIS UE CAT v1.2 and estimated PROMIS UE CAT v1.2 scores generated using the model as well as Root Mean Square Error (RMSE). The model chosen as the 'best' was further assessed for population invariance using root expected mean squared difference (REMSD) where < 0.08 were considered good.

RESULTS

Of 10,081 included patients, mean age was 48.3 (SD = 17.0), and 54% were female (5,477/10,081). Mean UE CAT v1.2 and PF CAT scores were 37 (SD = 9.8) and 46 (SD = 10.0), respectively. There was a strong correlation between the scores (Pearson correlation r = 0.70). All methods performed acceptably (ICC ≥ 0.66 and RMSE < = 7.52 for all). The equipercentile method had the highest ICC (ICC = 0.70 (95% CI 0.69-0.71)) while the mean and circle arc methods had the lowest RMSE. The circle arc method is the most reliable with the smallest standard error and has satisfactory population invariance across age group (REMSD 0.065) and sex (REMSD 0.036).

CONCLUSIONS

Crosswalk tables to be used for bidirectional conversion between scores were created.

LEVEL OF EVIDENCE

III.

Equipercentile equating of scores from common patient-reported outcome measures of physical function in patients with cancer

OBJECTIVE

To provide equipercentile equating of physical function (PF) scores from frequently used patient-reported outcome measures (PROMs) in cancer patients to facilitate data pooling and comparisons.

STUDY DESIGN AND SETTING

Adult cancer patients from five European countries completed the European Organization for Research and Treatment of Cancer (EORTC) computer adaptive test (CAT) Core, EORTC Quality of Life Questionnaire Version 3.0 (QLQ-C30), Functional Assessment of Cancer Therapy - General (FACT-G), 36-item Short Form Health Survey (SF-36), and the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function 20a short form. The R package "equate" was used to establish conversion tables of PF scores on those measures with a bivariate rank correlation of at least 0.75.

RESULTS

In total, 953 patients with cancer (mean age 58.9 years, 54.7% men) participated. Bivariate rank correlations between PF scores from the EORTC CAT Core, EORTC QLQ-C30, SF-36, and PROMIS were all above 0.85, but below 0.69 for the FACT-G. Conversion tables were established for all measures but the FACT-G. These tables indicate which score from one PROM best matches the score from another PROM and provide standard errors of converted scores.

CONCLUSION

Our analysis indicates that linking of PF scores from both EORTC measures (CAT and QLQ-C30) with PROMIS and SF-36 is possible, whereas the physical domain of the FACT-G seems to be different. The established conversion tables may be used for comparing results or pooling data from clinical studies using different PROMs.

Establishing the interpretability and utility of the 4-item BriefPCS

To reduce the patient burden associated with completing the 13-item Pain Catastrophizing Scale (PCS), the 4-item "BriefPCS" was developed. To date, no crosswalk has been developed that associates scores on the BriefPCS with PCS scores. Further, no study has compared the use of BriefPCS and PCS scores in a randomized clinical trial (RCT). We aimed to: (1) establish the interpretability of BriefPCS scores in reference to PCS scores, (2) compare the concurrent validity between the BriefPCS and PCS, and (3) asssess the use of BriefPCS in an RCT. First, we conducted equipercentile linking, created a crosswalk that associated scores of BriefPCS with PCS, and calculated differences between PCS and crosswalked PCS scores. Secondly, we compared Bootstrap correlation coefficients between PCS and self-reported measures of other domains. Lastly, we compared results from an RCT using BriefPCS scores versus PCS scores. Findings indicated that the correlation coefficient estimates with the BriefPCS and PCS scores were not significantly different. BriefPCS and PCS scores had similar ability to detect treatment-related changes. The BriefPCS scores validly, reliably, and accurately distinguish levels of pain catastrophizing. Additionally, the BriefPCS scores are sensitive to changes after behavioral interventions, with less respondent burden compared to the PCS scores.

Common measures or common metrics? the value of IRT-based common metrics

There is a clear need to harmonize outcome measurement. Some authors propose to express scores as T scores to facilitate interpretation of PROM results in clinical practice. While this is a step in the right direction, there are important limitations to the acceptance of the T score metric as a common metric when T scores are based on raw sum scores of ordinal items: Such T scores of different instruments are not exactly comparable because they are not interval scaled; T scores of different measures are only on the same scale if exactly the same reference group is used; and the T sore metric cannot be maintained because it is reference population-dependent and needs to be updated regularly. These limitations can be overcome by using an item response theory (IRT)-based metric. Items from different measures can be placed on the same IRT metric to make scores comparable on an interval scale. The PROMIS initiative used IRT to develop item banks for measuring various health outcomes. Other PROMs have been linked to the PROMIS metric. Although PROMIS uses a T-score metric for practical reasons, the underlying PROMIS metric is actually an IRT metric. An IRT approach also enables further development of an item bank while preserving the underlying metric. Therefore, IRT-based metrics should be considered as common metrics for the future.

Establishing a common metric for physical function: Linking SARC-F and PROMIS® physical function

INTRODUCTION

Aligned with the increasing need for standardized assessment of physical function in older individuals with cancer and other conditions, several patient-reported outcome measures (PROMs) have been developed and published. The aim of this study is to link the Strength, Assistance with walking, Rising from a chair, Climbing stairs, and Falls questionnaire (SARC-F), and the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Physical Function Short Form 8c (PROMIS PF 8c), and make their scores convertible, in order to expand the use of both instruments in research and inform clinicians and researchers about the interchangeability of critical cut-off scores.

MATERIALS AND METHODS

The sample included 300 participants recruited from an online panel. Participants were included if they had received a cancer diagnosis from a clinician and reported receiving anti-cancer treatment. We conducted five linking procedures and selected an optimal one to generate the crosswalk table between the two measures.

RESULTS

The linked T scores of all five methods showed acceptably small mean differences from the observed T scores, and the standard deviation (SD), and root-mean-squared deviation (RMSD) of the differences were generally similar across all methods. After comparing across all statistics, the Stocking-Lord approach was considered as the optimal approach to compute the crosswalk table for converting SARC-F raw scores to PROMIS PF 8c scores. The crosswalk table shows that the SARC-F cut-off value of 4 between healthy versus symptomatic with a corresponding score of 37 fell in the range of moderate physical function limitation from 30 to 39 on the PROMI PF 8c T score metric.

DISCUSSION

The linkage in this study has potential for improving clinical and research activities for people with cancer and perhaps others with a similar range of physical function. It facilitates the interpretability in scores of both measures on a common metric anchored on general population for further group-level analysis. Researchers can use this crosswalk to harmonize data collected from either instrument without requiring all cohorts to administer the same instrument for a prospective data collection or retrospective data analysis purpose or for a cross-study effectiveness study.

Overcoming floor and ceiling effects in knee arthroplasty outcome measurement

Aims

To map the Oxford Knee Score (OKS) and High Activity Arthroplasty Score (HAAS) items to a common scale, and to investigate the psychometric properties of this new scale for the measurement of knee health.

Methods

Patient-reported outcome measure (PROM) data measuring knee health were obtained from the NHS PROMs dataset and Total or Partial Knee Arthroplasty Trial (TOPKAT). Assumptions for common scale modelling were tested. A graded response model (fitted to OKS item responses in the NHS PROMs dataset) was used as an anchor to calibrate paired HAAS items from the TOPKAT dataset. Information curves for the combined OKS-HAAS model were plotted. Bland-Altman analysis was used to compare common scale scores derived from OKS and HAAS items. A conversion table was developed to map between HAAS, OKS, and the common scale.

Results

We included 3,329 response sets from 528 patients undergoing knee arthroplasty. These generally met the assumptions of unidimensionality, monotonicity, local independence, and measurement invariance. The HAAS items provided more information than OKS items at high levels of knee health. Combining both instruments resulted in higher test-level information than either instrument alone. The mean error between common scale scores derived from the OKS and HAAS was 0.29 logits.

Conclusion

The common scale allowed more precise measurement of knee health than use of either the OKS or HAAS individually. These techniques for mapping PROM instruments may be useful for the standardization of outcome reporting, and pooling results across studies that use either PROM in individual-patient meta-analysis.

A systematic review of health-related quality of life outcomes in psychosocial intervention trials for adolescent and young adult cancer survivors

BACKGROUND

The National Cancer Institute has catalyzed research in adolescent and young adult (AYA) oncology by identifying the need for supportive care intervention studies and psychometrically robust health-related quality of life (HRQOL) measures. We evaluated progress toward these goals (1) examining changes in the number of registered psychosocial intervention trials being conducted with AYAs over time; (2) determining what domains of HRQOL were assessed across these intervention trials; and (3) identifying the most frequently used measures of HRQOL.

METHODS

We conducted a systematic review of psychosocial intervention trials for AYAs registered on ClinicalTrials.gov from 2007 to 2021. Following identification of relevant trials, we extracted the outcome measures and determined whether they were measures of HRQOL and which HRQOL domains were evaluated. Descriptive statistics were used to summarize trial and outcome characteristics.

RESULTS

We identified 93 studies that met our inclusion criteria and 326 HRQOL outcomes across studies. The average number of clinical trials conducted annually has increased from 2 (SD = 1) during the years of 2007-2014 to 11 (SD = 4) during the years of 2015-2021. 19 trials (20.4%) did not include a measure of HRQOL. HRQOL measures varied widely, and most evaluated psychological and physical domains. Of the 9 measures used 5 + times, none were developed to cover the full AYA age spectrum.

CONCLUSIONS

This review demonstrated that the number of AYA psychosocial intervention trials conducted annually has increased. However, it also revealed several important areas for additional work including: (1) ensuring psychosocial trials include HRQOL measures; (2) increasing the frequency of evaluation of underrepresented domains of HRQOL (e.g., body image, fertility/sexuality and spiritual); and (3) improving the validity and standardization of measures used to evaluate domains of HRQOL across AYA-focused trials to improve the field's ability to compare the impact of different psychosocial interventions on HRQOL outcomes.

Physical performance tasks were linked to the PROMIS physical function metric in patients undergoing hemodialysis

OBJECTIVES

To investigate whether a multi-item performance outcome measure, the physical performance test (PPT), can be calibrated to a common scale with patient-reported outcome measures, using the Patient-Reported Outcomes Measurement Information System (PROMIS) physical function (PF) metric.

STUDY DESIGN AND SETTING

We analyzed baseline data (N = 1,113) from the CONVINCE study, an international trial in end-stage kidney disease patients comparing high-dose hemodiafiltration with high-flux hemodialysis. Assumptions of item response theory (IRT) modelling were investigated for the combined set of the nine-item PPT and a four-item PROMIS PF short form (PROMIS-PF4a). We applied unidimensional IRT linking for calibrating the PPT to the PROMIS PF metric.

RESULTS

Although some evidence for multidimensionality was found, classical test statistics (Cronbach's Alpha = 0.93), Mokken (Loevinger's H = 0.50), and bifactor analysis (explained common variance = 0.65) indicated that PPT and PROMIS-PF4a items can be used to assess a common PF construct. On the group level, the agreement between PROMIS-PF4a and linked PPT scores was stable across several subsamples. On the individual level, scores differed considerably.

CONCLUSION

We found preliminary evidence that the PPT can be linked to the PROMIS PF metric in hemodialysis patients, enabling group comparisons across patient-reported outcome and performance outcome measures. Alternative linking methods should be applied in future studies using a more comprehensive PROMIS PF item set.

Patient-reported outcome measures for physical function in cancer patients: content comparison of the EORTC CAT Core, EORTC QLQ-C30, SF-36, FACT-G, and PROMIS measures using the International Classification of Functioning, Disability and Health

BACKGROUND

Patient-reported physical function (PF) is a key endpoint in cancer clinical trials. Using complex statistical methods, common metrics have been developed to compare scores from different patient-reported outcome (PRO) measures, but such methods do not account for possible differences in questionnaire content. Therefore, the aim of our study was a content comparison of frequently used PRO measures for PF in cancer patients.

METHODS

Relying on the framework of the International Classification of Functioning, Disability and Health (ICF) we categorized the item content of the physical domains of the following measures: EORTC CAT Core, EORTC QLQ-C30, SF-36, PROMIS Cancer Item Bank for Physical Function, PROMIS Short Form for Physical Function 20a, and the FACT-G. Item content was linked to ICF categories by two independent reviewers.

RESULTS

The 118 items investigated were assigned to 3 components ('d - Activities and Participation', 'b - Body Functions', and 'e - Environmental Factors') and 11 first-level ICF categories. All PF items of the EORTC measures but one were assigned to the first-level ICF categories 'd4 - Mobility' and 'd5 - Self-care', all within the component 'd - Activities and Participation'. The SF-36 additionally included item content related to 'd9 - Community, social and civic life' and the PROMIS Short Form for Physical Function 20a also included content related to 'd6 - domestic life'. The PROMIS Cancer Item Bank (v1.1) covered, in addition, two first-level categories within the component 'b - Body Functions'. The FACT-G Physical Well-being scale was found to be the most diverse scale with item content partly not covered by the ICF framework.

DISCUSSION

Our results provide information about conceptual differences between common PRO measures for the assessment of PF in cancer patients. Our results complement quantitative information on psychometric characteristics of these measures and provide a better understanding of the possibilities of establishing common metrics.

Equity-Driven Implementation of Patient-Reported Outcome Measures in Musculoskeletal Care: Advancing Value for All

ABSTRACT

The clinical use of patient-reported outcome measures (PROMs) in musculoskeletal care is expanding, encompassing both individual patient management and population-level applications. However, without thoughtful implementation, we risk introducing or exacerbating disparities in care processes or outcomes. We outline examples of opportunities, challenges, and priorities throughout PROM implementation to equitably advance value-based care at both the patient and population level. Balancing standardization with tailored strategies can enable the large-scale implementation of PROMs while optimizing care processes and outcomes for all patients.

Development of a set of patient reported outcome measures for patients with benign liver tumours and cysts: patient focus groups and systematic review

BACKGROUND

Patient reported outcome measures (PROMs) may be useful for patients with benign liver tumours and cysts (BLTC) to evaluate the impact of treatment and/or guide shared decision making. Yet, a set of PROMs relevant to patients with BLTC is currently unavailable. In this study, we selected a PROMs set for patients with BLTC.

METHODS

Potentially relevant patient reported outcomes (PROs) were selected by psychologist-researchers based on keywords used or suggested by participants of two virtual focus groups meetings consisting of thirteen female BLTC patients with a median age of 50 years. Subsequently, patients were asked to report their most relevant PROs. PROMs identified by systematic literature review and computerized adaptive tests (CATs) in the Patient-Reported Outcomes Measurement Information System (PROMIS) were considered in selecting the final PROMs set to assess relevant outcomes.

RESULTS

The most important PROs were: insecurity/anxiety (11/12 patients), pain (9/12 patients), fatigue (8/12 patients), and limitations in daily life (5/12 patients). The literature review included 23 studies, which used various generic and disease-specific PROMs, often not measuring (all) relevant PROs. The final selected PROMs set included numerical rating scales for pain, two questions on overall health and quality of life and four PROMIS CATs.

CONCLUSIONS

A PROMs set generically and efficiently measuring outcomes relevant for patients with BLTC was developed and may be used in future research and clinical practice.

Can a linking crosswalk table be applied to a different population? An independent validation study for a crosswalk between BSI depression and PROMIS depression scales

A linking procedure establishes a "bridge" between the scores from different patient-reported outcome (PRO) instruments that measure similar constructs. After developing a linking relationship however, it is critical to evaluate whether this relationship can be generalized to different groups. Our study aims to validate a published crosswalk for score conversion between the Brief Symptom Inventory Depression subscale and the Patient-Reported Outcomes Measurement Information System Depression 8a using an independent sample. Data were from a sample of young men who have sex with men (MSM), which differs in terms of participant age, race, and ethnicity from the sample used to develop the existing crosswalk. The validity of the newly derived crosswalk was evaluated in terms of the correlation, mean difference and standard deviation between the observed and the linked scores. The two crosswalks were further compared to evaluate if the difference was within an acceptable range. More than half of the item parameters obtained from the two samples were found to overlap in their confidence intervals. Differences between each pair of scores in the two crosswalks was within three T-score points, well within the range of each crosswalk score's standard error. This study concludes that an existing crosswalk is replicable on a sample that differs from that used for crosswalk development, but future research should continue to examine the generalizability of the linked parameters and evaluate the reproducibility of this crosswalk to other populations.

Head-to-head comparison of somatic symptom scales: The Patient Health Questionnaire (PHQ-15) and the somatization scale of the Four-Dimensional Symptom Questionnaire (4DSQ-S)

OBJECTIVE

The goal of this study was to compare the 15-item Patient Health Questionnaire (PHQ-15) and the somatization subscale of the Four-Dimensional Symptoms Questionnaire (4DSQ-S) with respect to their latent structure and reliability, and to examine whether their scores are affected by age and gender, and whether the scales measure the same construct(s).

METHODS

The study population consisted of individuals with a tendency to experience persistent somatic symptoms, recruited in multiple healthcare settings, who completed the PHQ-15 and 4DSQ-S concurrently. We analyzed the scales' latent factor structure using confirmatory factor analysis (CFA), the scales' reliability, and differential item functioning (DIF) due to age and gender. We performed a head-to-head comparison by fitting structural equation models of the questionnaires' factors.

RESULTS

We included 234 participants. CFA showed that both questionnaires fitted a bifactor model with a general factor and four specific factors, three of which (labeled "musculoskeletal", "gastrointestinal", and "cardiopulmonary") were substantively similar. Both scales were essentially unidimensional. The reliability of the PHQ-15 and 4DSQ-S was equally high (omega 0.933 and 0.942, respectively). DIF-analysis showed minor DIF for age in one item of each questionnaire, with negligible impact on the scale score. Head-to-head comparison showed that the PHQ-15 and 4DSQ-S measured the same constructs. We present PHQ-15 - 4DSQ-S cross-walk tables.

CONCLUSIONS

Both questionnaires mainly measure a single somatic symptom burden dimension of which all symptoms (covered by the questionnaires) are adequate indicators. They do so equally accurately and they behave the same across gender and age categories.

Analyses of Cross-Sectional Data to Link the PEG With the Patient Reported Outcomes Measurement and Information System (PROMIS) Global Physical Health Scale

It is challenging to synthesize findings across studies of pain impact. This study develops a link to estimate the Patient-Reported Outcomes Measurement Information System (PROMIS) global health measure from the 3-item Pain intensity, interference with Enjoyment of life, interference with General activity (PEG) scale. The PROMIS and PEG items were administered to 795 adults (average age = 51; 54% female, 79% White). We estimated correlations among the PEG and PROMIS items and conducted factor analysis to identify the best subset of PROMIS items for linking to the PEG. An item response theory graded response model was estimated to link the PEG with the 4-item PROMIS global physical health scale. A categorical single-factor model and a bifactor model provided support for a single dimension for the PEG and PROMIS global physical health items. The product-moment correlation between estimated PROMIS global physical health scale from the PEG and the actual global physical health score was .74. The mean difference between estimated PROMIS global physical health scale score from the PEG and the observed global physical health score was less than a T-score point. This study makes it possible to estimate the average global physical health for group-level comparisons in research that includes the PEG. PERSPECTIVE: This article describes an empirical link of the PEG to the PROMIS global physical health scale that makes it possible to estimate the average global physical health in studies that include the PEG. This link can facilitate comparisons among studies that have not administered the PEG or the PROMIS global health scale.

Common measures or common metrics? A plea to harmonize measurement results

OBJECTIVE

There is a great variety of measurement instruments to assess similar constructs in clinical research and practice. This complicates the interpretation of test results and hampers the implementation of measurement-based care.

METHOD

For reporting and discussing test results with patients, we suggest converting test results into universally applicable common metrics. Two well-established metrics are reviewed: T scores and percentile ranks. Their calculation is explained, their merits and drawbacks are discussed, and recommendations for the most convenient reference group are provided.

RESULTS

We propose to express test results as T scores with the general population as reference group. To elucidate test results to patients, T scores may be supplemented with percentile ranks, based on data from a clinical sample. The practical benefits are demonstrated using the published data of four frequently used instruments for measuring depression: the CES-D, PHQ-9, BDI-II and the PROMIS depression measure.

DISCUSSION

Recent initiatives have proposed to mandate a limited set of outcome measures to harmonize clinical measurement. However, the selected instruments are not without flaws and, potentially, this directive may hamper future instrument development. We recommend using common metrics as an alternative approach to harmonize test results in clinical practice, as this will facilitate the integration of measures in day-to-day practice.

A Patient Reported Outcome Ontology: Conceptual Issues and Challenges Addressed by the Patient-Reported Outcomes Measurement Information System® (PROMIS®)

We briefly review the history of measuring perceptions of health and quality of life, followed by an examination of conceptual issues related to terminology that have led to potentially conflicting ontologies. Then, we discuss challenges posed by the lack of consensus on common meaning and the proliferation of measures. Next, we suggest a solution grounded in an ontology adopted by the National Institutes of Health (NIH) funded Patient-Reported Outcomes Measurement Information System (PROMIS) project. We conclude by discussing issues associated with mapping the PROMIS domain framework onto other familiar ontologies and recommend a way forward for PROMIS to provide a sustainable ontological structure to enable coherent common measurement.

Validity and reliability of Patient-Reported Outcomes Measurement Information System (PROMIS®) using Computerized Adaptive Testing (CAT) in patients with advanced chronic kidney disease

BACKGROUND

The Patient-Reported Outcomes Measurement Information System (PROMIS®) has been recommended for computerized adaptive testing (CAT) of health-related quality of life (HRQOL). This study compared the content, validity and reliability of seven PROMIS CATs to the 12-item Short-Form Health Survey (SF-12) in patients with advanced chronic kidney disease (CKD).

METHODS

Adult CKD patients with an eGFR < 30 ml/min.1.73m2 not receiving dialysis treatment completed seven PROMIS CATs (assessing physical function, pain interference, fatigue, sleep disturbance, anxiety, depression, and ability to participate in social roles and activities), the SF-12 and, additionally, the PROMIS Pain Intensity single item and Dialysis Symptom Index (DSI) at inclusion and 2-weeks. A content comparison was performed between PROMIS CATs and SF-12. Construct validity of PROMIS CATs was assessed using Pearson's correlations. Test-retest reliability of all patient-reported outcome measures (PROMs) was assessed by calculating the intra-class correlation coefficient (ICC) and minimal detectable change (MDC).

RESULTS

In total, 207 patients participated in the study. A median of 45 items (10 minutes) was completed for PROMIS CATs. All PROMIS CATs showed evidence for sufficient construct validity. PROMIS CATs, most SF-12 domains and summary scores, and DSI showed sufficient test-retest reliability (ICC ≥ 0.70). PROMIS CATs had a lower MDC compared to the SF-12 (5.7-7.4 compared to 11.2-21.7 across domains, respectively).

CONCLUSION

PROMIS CATs showed sufficient construct validity and test-retest reliability in patients with advanced CKD. PROMIS CATs required more items but showed better reliability than the SF-12. Future research is needed to investigate the feasibility of PROMIS CATs for routine nephrology care.

Linking the KOOS-PS to PROMIS Physical Function in Knee Patients Evaluated for Surgery

INTRODUCTION

The Knee Injury and Osteoarthritis Outcome Score-Physical Function Short-form and the Patient-Reported Outcomes Measurement Information System Physical Function are widely used patient-reported outcome measures in orthopaedic practice and research. It would be helpful for clinicians and researchers to compare scores obtained on one instrument with those collected on another. To achieve this goal, this study conducted a linking analysis and computed a crosswalk table between these two scales.

DATA

The data of this study were collected as part of the clinical care of total knee arthroplasty patients in a large urban and suburban health system. The sample was a mix of responses from nonsurgical (no surgery performed), preoperative (before surgical intervention), and postoperative (after surgical intervention) groups.

METHODS

This study applied five linking methods: the item response theory (IRT)-based linking methods including fixed-parameter calibration, separate-parameter calibration with Stocking-Lord constants, and calibrated projection; and the equipercentile methods with log-linear smoothing and nonsmoothing approaches. Before conducting the linking analysis, we checked the linking assumptions including the similar content of the two scales, the unidimensionality of the combined scales, and the population invariance. The results of the five linking methods were evaluated by mean difference, SD, root-mean-squared deviation, intraclass correlation coefficient of the observed T scores and the crosswalk-derived T scores.

RESULTS

The linking assumptions were all met. T scores generated from the Stocking-Lord crosswalk had the smallest mean difference (= -0.03) and relatively small SD (= 4.91) and root-mean-squared deviation (= 4.91) among the five linking methods. We validated this crosswalk in a larger sample with the nonsurgical, preoperative, and postoperative groups and in an external sample.

DISCUSSION

This study provides clinicians and researchers a practical tool (ie, a crosswalk table) to link scores from two popular physical function measures. Given the diversity of patient-reported outcome measures in use for knee conditions, these crosswalk tables would accelerate clinical and research interpretation of aggregating functional outcomes among the patients evaluated for knee surgery each year.

Multidimensional Computerized Adaptive Testing: A Potential Path Toward the Efficient and Precise Assessment of Applied Cognition, Daily Activity, and Mobility for Hospitalized Patients

OBJECTIVE

To develop and evaluate an efficient and precise variable-length functional assessment of applied cognition, daily activity, and mobility to inform mobility preservation and rehabilitation service delivery among hospitalized patients.

DESIGN

A multidimensional item bank tapping into these dimensions was developed, with all items calibrated using a multidimensional graded response model. The items were adaptively selected from the item banks to maximize the test information, and the test ended when a joint stopping rule was satisfied. A simulation study was conducted based on the completed instrument, the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT), to compare its measurement precision and efficiency capabilities relative to conventional unidimensional computerized adaptive testing. Precision was measured by the bias and root mean squared error between the estimated and true (ie, simulated) θ estimates, whereas efficiency was measured by average test length. Data were collected by an interviewer reading questions from a tablet computer and entering patients' responses.

SETTING

A large Midwestern hospital.

PARTICIPANTS

A total of 4143 patients hospitalized with medical diagnosis and/or surgical complications, with 2060 in the calibration sample and 2083 in the validation cohort.

INTERVENTION

Not applicable.

RESULTS

Among the 2083 patients in the validation sample, FAMCAT administration required an average of 6 (SD=3.11) minutes. Ninety-six percent had their tests terminated by the standard error rule after responding to an average of 22.05 (SD=7.98) items, whereas 15 were terminated by the change in θ rule, with an average test length of 45.27 (SD=11.49). The remaining 76 responded until reaching the maximum test length of 60 items.

CONCLUSIONS

The FAMCAT has the potential to satisfy the need for structured, frequent, and precise assessment of functional domains among hospitalized patients with medical diagnosis and/or surgical complications. The results are promising and may be informative for others who wish to develop similar instruments when concurrent assessment of correlated domains is required.

Health-Related Quality of Life Measurement in Public Health

Patient-reported outcomes are recognized as essential for the evaluation of medical and public health interventions. Over the last 50 years, health-related quality of life (HRQoL) research has grown exponentially from 0 to more than 17,000 papers published annually. We provide an overview of generic HRQoL measures used widely in epidemiological studies, health services research, population studies, and randomized clinical trials [e.g., Medical Outcomes Study SF-36 and the Patient-Reported Outcomes Measurement Information System (PROMIS^®)-29]. In addition, we review methods used for economic analysis and calculation of the quality-adjusted life year (QALY). These include the EQ-5D, the Health Utilities Index (HUI), the self-administered Quality of Well-being Scale (QWB-SA), and the Health and Activities Limitation Index (HALex). Furthermore, we consider hybrid measures such as the SF-6D and the PROMIS-Preference (PROPr). The plethora of HRQoL measures has impeded cumulative science because incomparable measures have been used in different studies. Linking among different measures and consensus on standard HRQoL measurement should now be prioritized. In addition, enabling widespread access to common measures is necessary to accelerate future progress. Expected final online publication date for the Annual Review of Public Health, Volume 43 is April 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Solving the Tower of Babel Problem for Patient-Reported Outcome Measures : Comments on: Linking Scores with Patient-Reported Health Outcome Instruments: A Validation Study and Comparison of Three Linking Methods

The PROsetta Stone Project, summarized in this issue by Schalet et al. (Psychometrika 86, 2021), is a major step forward in enabling comparability between different patient-reported outcomes measures. Schalet et al. clearly describe the psychometric methods used in the PROsetta Stone project and other projects from the Patient-Reported Outcomes Measurement Information System (PROMIS): linking based on unidimensional item response theory (IRT), equipercentile linking, and calibrated projection based on multidimensional IRT. Analyses in a validation data set and simulation studies provide strong support that the linking methods are robust when basic assumptions are fulfilled. The links already established will be of great value to the field, and the methodology described by Schalet et al. will hopefully inspire the next series of linking studies. Among potential improvements that should be considered by new studies are: (1) a thorough evaluation of the content of the measures to be linked to better guide the evaluation of measurement assumptions, (2) improvements in the design of linking studies such as selection of the optimal sample to provide data in the score ranges where linking precision is most critical and using counterbalanced designs to control for order effects. Finally, it may be useful to consider how the linking algorithms are used in subsequent data analyses. Analytic strategies based on plausible values or latent regression IRT models may be preferable to the simple transformation of scores from one patient at the time.