Evaluation of an item bank for a computerized adaptive test of activity in children with cerebral palsy

Psychometric Assessment of an Item Bank for Adaptive Testing on Patient-Reported Experience of Care Environment for Severe Mental Illness: Validation Study

BACKGROUND

The care environment significantly influences the experiences of patients with severe mental illness and the quality of their care. While a welcoming and stimulating environment enhances patient satisfaction and health outcomes, psychiatric facilities often prioritize staff workflow over patient needs. Addressing these challenges is crucial to improving patient experiences and outcomes in mental health care.

OBJECTIVE

This study is part of the Patient-Reported Experience Measure for Improving Quality of Care in Mental Health (PREMIUM) project and aims to establish an item bank (PREMIUM-CE) and to develop computerized adaptive tests (CATs) to measure the experience of the care environment of adult patients with schizophrenia, bipolar disorder, or major depressive disorder.

METHODS

We performed psychometric analyses including assessments of item response theory (IRT) model assumptions, IRT model fit, differential item functioning (DIF), item bank validity, and CAT simulations.

RESULTS

In this multicenter cross-sectional study, 498 patients were recruited from outpatient and inpatient settings. The final PREMIUM-CE 13-item bank was sufficiently unidimensional (root mean square error of approximation=0.082, 95% CI 0.067-0.097; comparative fit index=0.974; Tucker-Lewis index=0.968) and showed an adequate fit to the IRT model (infit mean square statistic ranging between 0.7 and 1.0). DIF analysis revealed no item biases according to gender, health care settings, diagnosis, or mode of study participation. PREMIUM-CE scores correlated strongly with satisfaction measures (r=0.69-0.78; P<.001) and weakly with quality-of-life measures (r=0.11-0.21; P<.001). CAT simulations showed a strong correlation (r=0.98) between CAT scores and those of the full item bank, and around 79.5% (396/498) of the participants obtained a reliable score with the administration of an average of 7 items.

CONCLUSIONS

The PREMIUM-CE item bank and its CAT version have shown excellent psychometric properties, making them reliable measures for evaluating the patient experience of the care environment among adults with severe mental illness in both outpatient and inpatient settings. These measures are a valuable addition to the existing landscape of patient experience assessment, capturing what truly matters to patients and enhancing the understanding of their care experiences.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02491866; https://clinicaltrials.gov/study/NCT02491866.

Properties of a Transport Instrument for Measuring Psychological Impacts of Delay on Commuters, Mokken Scale Analysis

This study assessed the validity of instrument including various negative psychological and physical behaviors of commuters due to the public transport delay. Instruments have been mostly evaluated by parametric method of item response theory (IRT). However, the IRT has been characterized by some restrictive assumptions about the data, focusing on detailed model fit evaluation. The Mokken scale analysis (MSA), as a scaling procedure is a non-parametric method, which does not require adherence to any distribution. The results of the study show that in most regards, our instrument meets the minimum requirements highlighted by the MSA. However, the instrument did not adhere to the minimum requirements of the "scalability" for two variables including "stomach pain" and "increased heart rate". So, modifications were proposed to address the violations. Although MSA technique has been used frequently in other fields, this is one of the earliest studies to implement the technique in the context of transport psychology.

A View of the Development of Patient Reported Outcomes Measures, their Clinical Integration, Electronification, and Potential Impact on Rehabilitation Service Delivery

Recognition of the importance of a patient's perception of their status and experience has become central to medical care and its evaluation. This recognition has led to a growing reliance on the use of patient reported outcome measures (PROMs). Nevertheless, while awareness of PROMs and acceptance of their utility has increased markedly, few of us have a good insight into their development; their utility relative to clinician-rated and performance measures such as the FIM and 6-Minute Walk Test or how their "electronification" and incorporation into electronic health records (EHRs) may improve the individualization, value, and quality of medical care. In all, the goal of this commentary is to provide some insight into historical factors and technology developments that we believe have shaped modern clinical PROMs as they relate to medicine in general and to rehabilitation in particular. In addition, we speculate that while the growth of PROM use may have been triggered by an increased emphasis on the centrality of the patient in their care, future uptake will be shaped by their embedding in EHRs and use to improve clinical decision support though their integration with other sources of clinical and sociodemographic data.

Measurement Precision and Efficiency of Computerized Adaptive Testing for the Activities-specific Balance Confidence Scale in People With Stroke

OBJECTIVE

Administrative burden often prevents clinical assessment of balance confidence in people with stroke. A computerized adaptive test (CAT) version of the Activities-specific Balance Confidence Scale (ABC CAT) can dramatically reduce this burden. The objective of this study was to test balance confidence measurement precision and efficiency in people with stroke with an ABC CAT.

METHODS

We conducted a retrospective, cross-sectional, simulation study with data from 406 adults approximately 2 months post-stroke in the Locomotor-Experience Applied Post-Stroke trial. Item parameters for CAT calibration were estimated with the Rasch model using a random sample of participants (n = 203). Computer simulation was used with response data from the remaining 203 participants to evaluate the ABC CAT algorithm under varying stopping criteria. We compared estimated levels of balance confidence from each simulation to actual levels predicted from the Rasch model (Pearson correlations and mean standard error [SE]).

RESULTS

Results from simulations with number of items as a stopping criterion strongly correlated with actual ABC scores (full item, r = 1, 12-item, r = 0.994; 8-item, r = 0.98; 4-item, r = 0.929). Mean SE increased with decreasing number of items administered (full item, SE = 0.31; 12 item, SE = 0.33; 8 item, SE = 0.38; 4 item, SE = 0.49). A precision-based stopping rule (mean SE = 0.5) also strongly correlated with actual ABC scores (r = 0.941) and optimized the relationship between number of items administrated with precision (mean number of items 4.37, range [4-9]).

CONCLUSION

An ABC CAT can determine accurate and precise measures of balance confidence in people with stroke with as few as 4 items. Individuals with lower balance confidence may require a greater number of items (up to 9) which could be attributed to the Locomotor-Experience Applied Post-Stroke trial excluding more functionally impaired persons.

IMPACT

Computerized adaptive testing can drastically reduce the ABC test's administration time while maintaining accuracy and precision. This should greatly enhance clinical utility, facilitating adoption of clinical practice guidelines in stroke rehabilitation.

LAY SUMMARY

If you have had a stroke, your physical therapist will likely test your balance confidence. A CAT version of the ABC scale can accurately identify balance with as few as 4 questions, which takes much less time.

Psychometric Properties and Efficiency of the Computerized Adaptive Testing System for Measuring Self-Care Performance in Taiwanese Children With Developmental Disabilities

OBJECTIVE

To investigate the psychometric properties and efficiency of the computerized adaptive testing system for measuring self-care performance (CAT-SC). The psychometric properties included intrarater and interrater reliabilities, concurrent validity, minimal detectable change, minimal important difference, and responsiveness.

DESIGN

Criterion standard study.

SETTING

A teaching hospital.

PARTICIPANTS

A convenience sample of 60 caregivers of children with developmental disabilities was recruited at the initial assessment, and 95% of the children (N=57) were followed-up at 3 and 6 months.

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

The CAT-SC and the Pediatric Evaluation of Disability Inventory Chinese version were used at the initial assessment. We assessed the CAT-SC and asked caregivers to rate children's changes in self-care performance with a separate question rated on a 15-point Likert-type scale at the 3- and 6-month follow-ups.

RESULTS

The CAT-SC had excellent intrarater and interrater reliabilities (intraclass coefficient=0.99 and 0.92, respectively), high concurrent validity with the Pediatric Evaluation of Disability Inventory (r=0.91-0.92), and trivial to small responsiveness at 3- and 6-month follow-ups (effect size=0.02 and 0.12, standardized response mean=0.08 and 0.33). The minimal detectable change of intrarater reliabilities and the minimal important difference at the first follow-up were 0.22 and 0.17 logits, respectively. The administration time of the CAT-SC was about 5 minutes.

CONCLUSIONS

The results of our study validated the sound psychometric properties and good efficiency of the CAT-SC. Moreover, the values of the minimal detectable change and minimal smallest change can be used as a reference for clinicians and caregivers to interpret children's progress.

Development and calibration of a novel positive mindset item bank to measure health-related quality of life (HRQoL) in Singapore

BACKGROUND

Positive mindset (PM) is an important domain of health-related quality of life in Singapore, a multi-ethnic urban city state in Southeast Asia. We therefore developed and calibrated a novel item bank to measure and improve PM.

METHODS

We developed an initial candidate pool of 48 items from focus groups, in-depth interviews and existing instruments locally developed and validated for use in Singapore. We administered all items in English to a multi-stage sample stratified for age and gender, of subjects with and without medical conditions recruited from the community and a hospital, and calibrated their responses using Samejima's Graded Response Model. We evaluated a final 36-item bank with respect to Item Response Theory (IRT) model assumptions, model fit, differential item functioning (DIF), concurrent and known-groups validity.

RESULTS

Among 493 participants (49.3% male, 41.6% above 50 years old, 33% Chinese, Malay and Indian), bifactor model analyses supported unidimensionality: explained common variance of the general factor was 0.86 and omega hierarchical was 0.97. Local independence was deemed acceptable: the average absolute residual correlations were <0.06 and 3.3% of the total item-pair residuals were flagged for local dependence. The overall model fit was adequate and provided good coverage of the PM construct (theta range: -3.6 to +2.4). Five items exhibited DIF with respect to ethnicity and gender, but were retained without modification of scores because they measured important aspects of PM. Scores correlated in the hypothesized direction with a self-reported measure of global health (Spearman's rho = -0.28, p<0.001) and discriminated between groups of participants with and without a self-reported diagnosis of a mood disorder (p = 0.007) adjusting for age, gender, ethnicity, education and marital status.

CONCLUSION

The 36-item PM item bank demonstrated satisfactory psychometric properties for the English-speaking Singaporean population. IRT model assumptions were sufficiently met and scores showed concurrent and known-groups validity. Future studies to evaluate the validity of PM scores when items are administered adaptively are needed.

Development and calibration of a novel social relationship item bank to measure health-related quality of life (HRQoL) in Singapore

BACKGROUND

Social relationships (SR) is an important domain of health-related quality of life. We developed and calibrated a novel item bank to measure SR in Singapore, a multi-ethnic city in Southeast Asia.

METHODS

We developed an initial candidate pool of 51 items from focus groups, individual in-depth interviews and existing instruments that had been developed and/or validated for use in Singapore. We administered all items in English to a multi-stage sample of subjects, stratified for age and gender, with and without medical conditions, recruited from community and hospital settings. We calibrated their responses using Samejima's Graded Response Model (SGRM). We evaluated a final 30-item bank with respect to Item Response Theory (IRT) model assumptions, model fit, differential item functioning (DIF), and concurrent and known-groups validity.

RESULTS

Among 503 participants (47.7% male, 41.4% above 50 years old, 34.0% Chinese, 33.6% Malay and 32.4% Indian), bi-factor model analyses supported essential unidimensionality: explained common variance of the general factor was 0.805 and omega hierarchical was 0.98. Local independence was deemed acceptable: the average absolute residual correlations were < 0.06 and 1.8% of the total item-pair residuals were flagged for local dependence. The overall SGRM model fit was adequate (p = 0.146). Five items exhibited DIF with respect to age, ethnicity and education, but were retained without modification of scores because they measured important aspects of SR. The SR scores correlated in the hypothesized direction with a self-reported measure of global health (Spearman's rho = - 0.28, p < 0.001).

CONCLUSION

The 30-item SR item bank has shown acceptable psychometric properties. Future studies to evaluate the validity of SR scores when items are administered adaptively are needed.

The Cerebral Palsy Profile of Health and Function: Upper-Extremity Domain's Sensitivity to Change Following Musculoskeletal Surgery

PURPOSE

The Cerebral Palsy Profile of Health and Function (CP-PRO) Computerized Adaptive Tests (CAT) are quality of life measures developed specifically for use in children with cerebral palsy. This study examined the ability of the upper-extremity (UE) CP-PRO CAT to detect change in function after UE surgery compared with the Pediatric Outcomes Data Collection Instrument (PODCI), ABILHAND-Kids, and Box and Blocks test.

METHODS

From 2009 to 2013, children with cerebral palsy who had UE musculoskeletal surgery completed the UE CP-PRO CAT, PODCI-UE, ABILHAND-Kids, and Box and Blocks tests before surgery (97 children) and at 3 postoperative intervals: 6 months (80 children), 12 months (73 children), and 24 months (52 children). Mean, SD, effect size (ES), and standardized response mean (SRM) values for each measure at each time interval and each level of the Manual Ability Classification System were calculated and compared. Finally, the minimal detectable change at the 90% confidence level was determined.

RESULTS

Values for the ES (0.40) and SRM (0.53) for the UE CP-PRO CAT at baseline to 6 months were moderate and significantly greater than the PODCI-UE (ES, 0.18; SRM, 0.25). The ES and SRM for the PODCI-UE, ABILHAND-Kids, and Box and Blocks tests were not significantly greater than for the UE CP-PRO CAT at any period. From baseline to 6 months, the UE CP-PRO CAT detected a large and significant improvement for Manual Ability Classification System level II (SRM, 0.70; ES, 0.70). The minimal detectable change for the UE CP-PRO CAT was 5.20.

CONCLUSIONS

The UE CP-PRO CAT is significantly better in detecting change in UE function in the first 6 months after surgery and is comparable to other measures at 12 and 24 months.

TYPE OF STUDY/LEVEL OF EVIDENCE

Diagnostic II.

Functional Gains in Children With Spastic Hemiplegia Following a Tendon Achilles Lengthening Using Computerized Adaptive Testing-A Pilot Study

Purpose:

This pilot study evaluated the outcomes of tendon Achilles lengthening in 12 children (mean age: 11.2 years) with spastic hemiplegia.

Methods:

Cerebral Palsy Computer Adaptive Tests, the timed up-and-go, the Gross Motor Function Measure, the Gillette Functional Assessment Questionnaire, and the Pediatric Outcomes Data Collection Instrument were administered at baseline and at 6, 12, and 24 months postsurgery.

Results:

Significant improvement at the latest follow-up (12-24 months following surgery) was seen in all domains of the Cerebral Palsy Computer Adaptive Test: activity (P = .017), lower extremity (P = .005), global (P = .005), pain (P = .005), and fatigue (P = .028), as well as in the Gross Motor Function Measure-standing domain (P = .02) and the mobility domain of the Pediatric Outcomes Data Collection Instrument (P = .04).

Conclusion:

These findings indicate that the tendon Achilles lengthening improved functional outcome in these children as measured by tests of physical function, walking speed, and activity performance.

Examination of psychometric properties of PROMIS®: Pediatric upper limb measures in youth with cerebral palsy

Introduction

This study examines the validity and distribution characteristics of the PROMIS® pediatric upper limb measures in a sample of young people with cerebral palsy.

Method

Data are a cross-sectional subset of a larger prospective study of the responsiveness to change of PROMIS® pediatric measures following surgery to improve functioning in young people with cerebral palsy. Ninety-three participants between the ages of eight and 21 years completed the PROMIS® pediatric mobility and upper limb computer adaptive tests and short forms in conjunction with a set of static “legacy” measures of physical functioning, including a parent-report of upper limb function.

Results

The PROMIS® Pediatric upper limb short form demonstrated acceptable internal consistency (Cronbach’s alpha = 0.85). PROMIS® pediatric upper limb computer adaptive tests and short form mean values (42.1(11) and 43(10.4), respectively) were nearly 1 SD below normal, which is appropriate when a generic measure is used in a sample of young persons with cerebral palsy. The PROMIS® pediatric upper limb computer adaptive tests had a higher frequency of ceiling effects (29.50%) compared to the short form (18.30%).

Conclusion

Results of this study suggest that the PROMIS® pediatric upper limb computer adaptive tests and the short form are valid indicators of upper limb function in young people with cerebral palsy. The item bank can be replenished to address ceiling effects.

Development of a Computerized Adaptive Test of Children's Gross Motor Skills

OBJECTIVES

To (1) develop a computerized adaptive test for gross motor skills (GM-CAT) as a diagnostic test and an outcome measure, using the gross motor skills subscale of the Comprehensive Developmental Inventory for Infants and Toddlers (CDIIT-GM) as the candidate item bank; and (2) examine the psychometric properties and the efficiency of the GM-CAT.

DESIGN

Retrospective study.

SETTING

A developmental center of a medical center.

PARTICIPANTS

Children with and without developmental delay (N=1738).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURES

The CDIIT-GM contains 56 universal items on gross motor skills assessing children's antigravity control, locomotion, and body movement coordination.

RESULTS

The item bank of the GM-CAT had 44 items that met the dichotomous Rasch model's assumptions. High Rasch person reliabilities were found for each estimated gross motor skill for the GM-CAT (Rasch person reliabilities =.940-.995, SE=.68-2.43). For children aged 6 to 71 months, the GM-CAT had good concurrent validity (r values =.97-.98), adequate to excellent diagnostic accuracy (area under receiver operating characteristics curve =.80-.98), and moderate to large responsiveness (effect size =.65-5.82). The averages of items administered for the GM-CAT were 7 to 11, depending on the age group.

CONCLUSIONS

The results of this study support the use of the GM-CAT as a diagnostic and outcome measure to estimate children's gross motor skills in both research and clinical settings.

Improving the utility of the fine motor skills subscale of the comprehensive developmental inventory for infants and toddlers: a computerized adaptive test

AIM

This study aimed at improving the utility of the fine motor subscale of the comprehensive developmental inventory for infants and toddlers (CDIIT) by developing a computerized adaptive test of fine motor skills.

METHODS

We built an item bank for the computerized adaptive test of fine motor skills using the fine motor subscale of the CDIIT items fitting the Rasch model. We also examined the psychometric properties and efficiency of the computerized adaptive test of fine motor skills with simulated computerized adaptive tests.

RESULTS AND CONCLUSIONS

Data from 1742 children with suspected developmental delays were retrieved. The mean scores of the fine motor subscale of the CDIIT increased along with age groups (mean scores = 1.36-36.97). The computerized adaptive test of fine motor skills contains 31 items meeting the Rasch model's assumptions (infit mean square = 0.57-1.21, outfit mean square = 0.11-1.17). For children of 6-71 months, the computerized adaptive test of fine motor skills had high Rasch person reliability (average reliability >0.90), high concurrent validity (rs = 0.67-0.99), adequate to excellent diagnostic accuracy (area under receiver operating characteristic = 0.71-1.00), and large responsiveness (effect size = 1.05-3.93). The computerized adaptive test of fine motor skills used 48-84% fewer items than the fine motor subscale of the CDIIT. The computerized adaptive test of fine motor skills used fewer items for assessment but was as reliable and valid as the fine motor subscale of the CDIIT. Implications for Rehabilitation We developed a computerized adaptive test based on the comprehensive developmental inventory for infants and toddlers (CDIIT) for assessing fine motor skills. The computerized adaptive test has been shown to be efficient because it uses fewer items than the original measure and automatically presents the results right after the test is completed. The computerized adaptive test is as reliable and valid as the CDIIT.

Instrumentos que avaliam a independência funcional em crianças com paralisia cerebral: uma revisão sistemática de estudos observacionais

RESUMO Este artigo teve como objetivo fazer uma revisão sistemática de instrumentos que avaliam a independência funcional de crianças com paralisia cerebral. As bases eletrônicas da MEDLINE/PubMed, Scopus e Web of Science foram usadas para as buscas. Estudos observacionais dos últimos cinco anos, com texto completo disponível e sem restrição de idioma foram incluídos nesta revisão. Foram encontrados 222 artigos, dos quais, 63 foram analisados e 24 foram incluídos no estudo. Os principais instrumentos encontrados foram: PEDI, WeeFIM, ASK, PODCI, VABS-II, LIFE-H e CAPE/PAC.

Framework and guidance for implementing patient-reported outcomes in clinical practice: evidence, challenges and opportunities

Patient-reported outcomes (PROs) are reports of the status of a patient's health condition that come directly from the patient. While PRO measures are a well-developed technology with robust standards in research, their use for informing healthcare decisions is still poorly understood. We review relevant examples of their application in the provision of healthcare and examine the challenges associated with implementing PROs in clinical settings. We evaluate evidence for their use and examine barriers to their uptake, and present an evidence-based framework for the successful implementation of PROs in clinical practice. We discuss current and future developments for the use of PROs in clinical practice, such as individualized measurement and computer-adaptive testing.

Intervention for an Adolescent With Cerebral Palsy During Period of Accelerated Growth

PURPOSE

The purpose of this case report was to describe changes in body functions and structures, activities, and participation after a biweekly 10-week program of home physical therapy and hippotherapy using a weighted compressor belt.

PARTICIPANT

A 13-year-old boy with spastic diplegic cerebral palsy, Gross Motor Function Classification System level II, was referred because of accelerated growth and functional impairments that limited daily activities.

OUTCOME MEASURES

The Modified Ashworth Scale, passive range of motion, 1-Minute Walk Test, Timed Up and Down Stairs, Pediatric Balance Scale, Pediatric Evaluation of Disability Inventory Computer Adaptive Test, and Dimensions of Mastery Questionnaire 17 were examined at baseline, 5, and 10 weeks.

OUTCOMES

Data at 5 and 10 weeks demonstrated positive changes in passive range of motion, balance, strength, functional activities, and motivation, with additional improvements in endurance and speed after 10 weeks.

CLINICAL IMPLICATIONS

This report reveals enhanced body functions and structures and activities and improved participation and motivation.

A new computerized adaptive test advancing the measurement of health-related quality of life (HRQoL) in children: the Kids-CAT

PURPOSE

Assessing health-related quality of life (HRQoL) via Computerized Adaptive Tests (CAT) provides greater measurement precision coupled with a lower test burden compared to conventional tests. Currently, there are no European pediatric HRQoL CATs available. This manuscript aims at describing the development of a HRQoL CAT for children and adolescents: the Kids-CAT, which was developed based on the established KIDSCREEN-27 HRQoL domain structure.

METHODS

The Kids-CAT was developed combining classical test theory and item response theory methods and using large archival data of European KIDSCREEN norm studies (n = 10,577-19,580). Methods were applied in line with the US PROMIS project. Item bank development included the investigation of unidimensionality, local independence, exploration of Differential Item Functioning (DIF), evaluation of Item Response Curves (IRCs), estimation and norming of item parameters as well as first CAT simulations.

RESULTS

The Kids-CAT was successfully built covering five item banks (with 26-46 items each) to measure physical well-being, psychological well-being, parent relations, social support and peers, and school well-being. The Kids-CAT item banks proved excellent psychometric properties: high content validity, unidimensionality, local independence, low DIF, and model conform IRCs. In CAT simulations, seven items were needed to achieve a measurement precision between .8 and .9 (reliability). It has a child-friendly design, is easy accessible online and gives immediate feedback reports of scores.

CONCLUSIONS

The Kids-CAT has the potential to advance pediatric HRQoL measurement by making it less burdensome and enhancing the patient-doctor communication.

Self-reported pediatric measures of physical activity, sedentary behavior, and strength impact for PROMIS: conceptual framework

PURPOSE

Children's physical activity (PA) levels are commonly assessed in pediatric clinical research, but rigorous self-report assessment tools for children are scarce, and computer adaptive test implementations are rare. Our objective was to improve pediatric self-report measures of activity using semistructured interviews with experts and children for conceptualization of a child-informed framework.

METHODS

Semistructured interviews were conducted to conceptualize PA, sedentary behaviors, and strengthening activities. We performed systematic literature reviews to identify item-level concepts used to assess these 3 domains.

RESULTS

We developed conceptual frameworks for each domain using words and phrases identified by children as relevant.

CONCLUSIONS

Semistructured interview methods provide valuable information of children's perspectives and the ways children recall previous activities. Conceptualized domains of PA are based on the literature and expert views, which also reflect children's experiences and understanding, providing a basis for pediatric self-report instruments.

Self-reported pediatric measures of physical activity, sedentary behavior, and strength impact for PROMIS: item development

BACKGROUND

Children's activity level is commonly assessed in clinical research, but rigorous assessment tools for children are scarce. Our objectives were to improve pediatric activity self-report measures using qualitative methods to develop item pools that measure these concepts.

METHODS

On the basis of the items generated from our conceptual framework development, we applied cognitive interviews and comprehensibility reviews to ensure children readily understood the items.

RESULTS

Our methods resulted in 129 unique items-physical activities (80 items), sedentary behaviors (23 items), and strengthening activities (26 items)-that were comprehensible to children between the ages of 8 and 18 years. Comprehensibility review resulted in the deletion of 4 items.

CONCLUSIONS

The resultant item pools reflect children's experiences and understanding of the concepts of physical activities, sedentary behaviors, and strengthening activities. The item pools will undergo calibration using item response theory to support computer-adaptive test administration of self- and proxy-reported outcomes.

Examining the validity and reliability of the Taita symptom checklist using Rasch analysis

BACKGROUND/PURPOSE

The Taita symptom checklist (TSCL) is a standardized self-rating psychiatric symptom scale for outpatients with mental illness in Taiwan. This study aimed to examine the validity and reliability of the TSCL using Rasch analysis.

METHODS

The TSCL was given to 583 healthy people and 479 people with mental illness. Rasch analysis was used to examine the appropriateness of the rating scale, the unidimensionality of the scale, the differential item functioning across sex and diagnosis, and the Rasch cut-off score of the scale.

RESULTS

Rasch analysis confirmed that the revised 37 items with a three-point rating scale of the TSCL demonstrated good internal consistency and met criteria for unidimensionality. The person and item reliability indices were high. The TSCL could reliably measure healthy participants and patients with mental illness. Differential item functioning due to sex or psychiatric diagnosis was evident for three items. A Rasch cut-off score for TSCL was produced for detecting participants' psychiatric symptoms based on an eight-level classification.

CONCLUSION

The TSCL is a reliable and valid assessment to evaluate the participants' perceived disturbance of psychiatric symptoms based on Rasch analysis.

Valid and reliable instruments for arm-hand assessment at ICF activity level in persons with hemiplegia: a systematic review

BACKGROUND

Loss of arm-hand performance due to a hemiparesis as a result of stroke or cerebral palsy (CP), leads to large problems in daily life of these patients. Assessment of arm-hand performance is important in both clinical practice and research. To gain more insight in e.g. effectiveness of common therapies for different patient populations with similar clinical characteristics, consensus regarding the choice and use of outcome measures is paramount. To guide this choice, an overview of available instruments is necessary. The aim of this systematic review is to identify, evaluate and categorize instruments, reported to be valid and reliable, assessing arm-hand performance at the ICF activity level in patients with stroke or cerebral palsy.

METHODS

A systematic literature search was performed to identify articles containing instruments assessing arm-hand skilled performance in patients with stroke or cerebral palsy. Instruments were identified and divided into the categories capacity, perceived performance and actual performance. A second search was performed to obtain information on their content and psychometrics.

RESULTS

Regarding capacity, perceived performance and actual performance, 18, 9 and 3 instruments were included respectively. Only 3 of all included instruments were used and tested in both patient populations. The content of the instruments differed widely regarding the ICF levels measured, assessment of the amount of use versus the quality of use, the inclusion of unimanual and/or bimanual tasks and the inclusion of basic and/or extended tasks.

CONCLUSIONS

Although many instruments assess capacity and perceived performance, a dearth exists of instruments assessing actual performance. In addition, instruments appropriate for more than one patient population are sparse. For actual performance, new instruments have to be developed, with specific focus on the usability in different patient populations and the assessment of quality of use as well as amount of use. Also, consensus about the choice and use of instruments within and across populations is needed.

A computerized adaptive version of the SF-36 is feasible for clinic and Internet administration in adults with HIV

DYNHA SF-36 is a computerized adaptive test version of the SF-36 Health Survey. The feasibility of administering a modified DYNHA SF-36 to adults with HIV was evaluated with Johns Hopkins University Moore (HIV) Clinic patients (N=100) and Internet consumer health panel members (N=101). Participants completed the DYNHA SF-36, modified to capture seven health domains [(physical function (PF), role function (RF, without physical or emotional attribution), bodily pain (BP), general health, vitality (VT), social function (SF), mental health (MH)], and scored to produce two summary components [Physical Component Summary (PCS), Mental Component Summary (MCS)]. Item-response theory-based response consistency, precision, mean scores, and discriminant validity were examined. A higher percentage of Internet participants responded consistently to the DYNHA SF-36. For each domain, three standard deviations were covered with five items (90% reliability); however, RF and SF scores were less precise at the upper end of measurement (better functioning). Mean scores were slightly higher for the Internet sample, with the exception of VT and MCS. Clinic and Internet participants reporting an AIDS diagnosis had significantly lower mean PCS and PF scores than those without a diagnosis. Additionally, significantly lower RF and BP scores were found for Internet participants reporting an AIDS diagnosis. The measure was well accepted by the majority of participants, although Internet respondents provided lower ratings for the tool's usefulness. The DYNHA SF-36 has promise for measuring the impact of HIV and its treatment in both the clinic setting and through telemonitoring.

Accuracy and precision of the Pediatric Evaluation of Disability Inventory computer-adaptive tests (PEDI-CAT)

AIM

The aims of the study were to: (1) build new item banks for a revised version of the Pediatric Evaluation of Disability Inventory (PEDI) with four content domains: daily activities, mobility, social/cognitive, and responsibility; and (2) use post-hoc simulations based on the combined normative and disability calibration samples to assess the accuracy and precision of the PEDI computer-adaptive tests (PEDI-CAT) compared with the administration of all items.

METHOD

Parents of typically developing children (n = 2205) and parents of children and adolescents with disabilities (n = 703) between the ages of 0 and 21 years, stratified by age and sex, participated by responding to PEDI-CAT surveys through an existing Internet opt-in survey panel in the USA and by computer tablets in clinical sites.

RESULTS

Confirmatory factor analyses supported four unidimensional content domains. Scores using the real data post hoc demonstrated excellent accuracy (intraclass correlation coefficients ≥ 0.95) with the full item banks. Simulations using item parameter estimates demonstrated relatively small bias in the 10-item and 15-item CAT versions; error was generally higher at the scale extremes.

INTERPRETATION

These results suggest the PEDI-CAT can be an accurate and precise assessment of children's daily performance at all functional levels.

Computer adaptive test performance in children with and without disabilities: prospective field study of the PEDI-CAT

PURPOSE

To examine the discriminant validity, test-retest reliability, administration time and acceptability of the pediatric evaluation of disability inventory computer adaptive test (PEDI-CAT).

METHODS

A sample of 102 parents of children 3 through 20 years of age with (n = 50) and without (n = 52) disabilities was recruited for this prospective field study. A sub-sample (n = 25) also completed the PEDI-CAT a second time within one month. Parents completed 15 items in each of the four PEDI-CAT domains (daily activities, mobility, social/cognitive, responsibility) using a laptop computer. Following completion, parents answered a four-question user evaluation survey.

RESULTS

PEDI-CAT scores based on parent responses differentiated between groups of children with and without disabilities in all four domains. Test-retest reliability estimates were high (ICC = 0.96-0.99) for all four domains. The mean time to complete 60 items for the full sample (n = 102) was 12.66 minutes (SD = 4.47). Parents reported favorable reactions to the PEDI-CAT.

CONCLUSIONS

The PEDI-CAT offers a valid and reliable assessment acceptable to parents.

Measuring mobility limitations in children with cerebral palsy: Rasch model fit of a mobility questionnaire, MobQues28

OBJECTIVE

To develop a Rasch-based version of a mobility questionnaire (MobQues28) for children with cerebral palsy (CP).

DESIGN

Cross-sectional study.

SETTING

Private physical therapy practices and outpatient departments of hospitals and rehabilitation centers.

PARTICIPANTS

Parents of 323 ambulatory children with CP (Gross Motor Function Classification System [GMFCS] level I-IV; aged, 2-13y).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE

The mobility questionnaire measures mobility limitations in children with CP by rating the difficulty of executing 47 mobility activities, as reported by the parents. Items for the Rasch-based version were selected based on fit to the Rasch measurement (partial credit) model and invariance of item characteristics across GMFCS level, age group, or sex.

RESULTS

Analysis revealed a fitting model when nonambulant and/or 2- and 3-year aged children were excluded (leaving a final sample of n=246) and answering categories were collapsed from 9 to 5. Thirteen items were removed from the questionnaire due to misfit to the model, 5 because of disordered thresholds, and 1 because of invariance across age group. Twenty-eight items out of the original 47 items showed good fit to the model.

CONCLUSIONS

The 28-item version of the mobility questionnaire (MobQues28) provides sound measurement properties for measuring mobility limitations in ambulant children with CP, aged 4 to 13 years, and shows promise as an instrument for research purposes.

Content range and precision of a computer adaptive test of upper extremity function for children with cerebral palsy

This article reports on the content range and measurement precision of an upper extremity (UE) computer adaptive testing (CAT) platform of physical function in children with cerebral palsy. Upper extremity items representing skills of all abilities were administered to 305 parents. These responses were compared with two traditional standardized measures: Pediatric Outcomes Data Collection Instrument and Functional Independence Measure for Children. The UE CAT correlated strongly with the upper extremity component of these measures and had greater precision when describing individual functional ability. The UE item bank has wider range with items populating the lower end of the ability spectrum. This new UE item bank and CAT have the capability to quickly assess children of all ages and abilities with good precision and, most importantly, with items that are meaningful and appropriate for their age and level of physical function.

Outcome measures of activity for children with cerebral palsy: a systematic review

PURPOSE

The purpose of this systematic review was to identify valid, reliable, and clinically practical measures of function/activity for children with cerebral palsy.

METHOD

Cochrane, MEDLINE, CINAHL, AMED, PEDro, and ScienceDirect were searched to identify relevant studies. Reference lists were hand-searched, and databases were searched specifically for outcome measures (OMs) identified. Studies were examined for bias based on published recommendations.

RESULTS

Seven studies on 6 OMs were included, which were of moderate or good methodological quality.

CONCLUSIONS

Most measures require further research to fully establish their psychometric properties. Only the Gross Motor Function Measure versions 88 and 66 and the Pediatric Evaluation of Disability Inventory were identified as potentially appropriate for this client group, but not all are valid for children of all ages and Gross Motor Function Classification System levels. The clinical application of these OMs is discussed in relation to capacity, capability, or performance; new developments are highlighted.

Item bank development for a revised pediatric evaluation of disability inventory (PEDI)

The Pediatric Evaluation of Disability Inventory (PEDI) is a useful clinical and research assessment, but it has limitations in content, age range, and efficiency. The purpose of this article is to describe the development of the item bank for a new computer adaptive testing version of the PEDI (PEDI-CAT). An expanded item set and response options were reviewed by clinician experts and examined at parent and clinician focus groups. Eleven parents participated in 32 cognitive interviews to examine content, format, and comprehension of items and responses. A set of 76 self-care, 78 mobility, and 64 social function items with pictures and a four-point "Difficulty" scale were developed. The PEDI's Caregiver Assistance scale was replaced by a "Responsibility Scale" with 53 items. Content validity was established incorporating input from clinicians and parents. The new item bank covers a broad range of functional activities for children of all ages and abilities.

Measuring physical function in children with airway support: a pilot study using computer adaptive testing

OBJECTIVE

To assess the responsiveness, examine the scoring range and determine the efficiency of a multidimensional computer adaptive testing version of the Pediatric Evaluation of Disability Inventory (PEDI-MCAT) for children admitted to inpatient pulmonary rehabilitation.

METHODS

The PEDI-MCAT was completed by clinician report for 30 infants and children. Mean self-care and mobility admission scores were compared with discharge scores for the total group and two diagnostic sub-groups (prematurity and congenital/neurological conditions). The scoring range of the mobility and self-care scales was examined to determine placement of the scores along the overall PEDI-MCAT scale. Efficiency was determined using an internal clock and average number of items required for score generation.

RESULTS

Mean changes for the total group and both sub-groups were significant for both self-care and mobility, except for the prematurity group's mobility scores. Effect sizes were small-to-moderate. Scores for both groups were at the low end of the scoring ranges. Average time to complete the PEDI-MCAT was 1.57 minutes. Average number of items administered was nine for self-care and 11 for mobility.

CONCLUSION

The PEDI-MCAT was responsive to change in physical function, although only low-ability items were needed. The PEDI-MCAT can potentially minimize clinician burden in inpatient settings.

PROMIS Pediatric Pain Interference Scale: an item response theory analysis of the pediatric pain item bank

An aim of the National Institutes of Health (NIH) Patient Reported Outcomes Measurement Information System (PROMIS) initiative is to develop item banks and computerized adaptive tests (CAT) that are applicable across a wide variety of chronic disorders. The PROMIS Pediatric Cooperative Group has concentrated on the development of pediatric self-report item banks for ages 8 through 17 years. The objective of the present study is to describe the Item Response Theory (IRT) analysis of the NIH PROMIS pediatric pain item bank and the measurement properties of the new unidimensional PROMIS Pediatric Pain Interference Scale. Test forms containing pediatric pain items were completed by a total of 3048 respondents. IRT analyses regarding scale dimensionality, item local dependence, and differential item functioning were conducted. A pain item pool was developed to yield scores on a T-score scale with a mean of 50 and standard deviation of 10. The recommended 8-item unidimensional short form for the PROMIS Pediatric Pain Interference Scale contains the item set which provides the maximum test information at the mean (50) on the T-score metric. A simulated CAT was computed that provides the most information at 5 possible score locations (30, 40, 50, 60, and 70 on the T-score metric).

PERSPECTIVE

The present study provides initial calibrations of the NIH PROMIS pediatric pain item bank and the creation of the PROMIS Pediatric Pain Interference Scale. It is anticipated that this new scale will have application in pediatric chronic and recurrent pain.

Lessons from use of the Pediatric Evaluation of Disability Inventory: where do we go from here?

PURPOSE

The purpose of this article is to review the innovations, applications, and effect of the original Pediatric Evaluation of Disability Inventory (PEDI) published in 1992 and to describe planned revisions.

SUMMARY OF KEY POINTS

During the past decade, the PEDI has helped to shift thinking from a developmental to a functional focus. Using the PEDI, researchers and clinicians worldwide have highlighted variations in functional skill acquisition in clinical populations, the importance of recognizing cultural differences, and the value of documenting functional progress in relation to interventions.

CONCLUSIONS

The PEDI has had a rich tradition in helping to document functional development. New methods are proposed for the next generation of the PEDI by using item banks and computer adaptive testing.

RECOMMENDATIONS FOR CLINICAL PRACTICE

The computer adaptive testing feature and the revised and expanded content of the new PEDI will enable therapists to more efficiently assess children's functioning to a broader age group of children.

Validity and reliability of physical functioning computer-adaptive tests for children with cerebral palsy

BACKGROUND

The purpose of this study was to assess the concurrent validity and reliability of scores from 4 new parent-report computer-adapted testing (CAT) programs developed to measure the physical functioning of children with cerebral palsy (CP). The Shriners Hospitals for Children CP-CAT battery includes upper-extremity skills, lower-extremity and mobility skills, activity, and global physical health.

METHODS

This was a prospective study of 91 children with CP who were tested cross-sectionally and 27 children with CP who were administered the CP-CAT programs twice within approximately a 1-month interval. We examined the concurrent validity of the 4 Shriners Hospitals for Children CP-CAT programs by Pearson correlations with comparative parent-report instruments. The scale reliability was tested by developing estimates of marginal reliability; test-retest reliability was assessed by intraclass correlations.

RESULTS

Pearson correlations were moderate to high in matching content domains of the CATs with the comparison measures. Marginal reliability estimates were always better for the CAT program than the comparative instruments. Average test-retest reliability using Intraclass correlations across the 4 CATs was ICC3,1=0.91 with a range of 0.88 to 0.94.

CONCLUSIONS

We found the CAT scores to be related to expected domains from external instruments, to have good scale reliability, and to have stable scores as determined by test-retest reliability. These results support the use of parent-report CATs in the assessment of physical functioning in children with CP.