Classical test theory and item response theory analyses of multi-item scales assessing parents' perceptions of their children's dental care

Item analysis on the quality of life scale for anxiety disorders QLICD-AD(V2.0) based on classical test theory and item response theory

BACKGROUND

Anxiety disorders can cause serious physical and psychological damage, so many anxiety scales have been developed internationally to measure anxiety disorders, but due to the cultural differences and cultural dependence of quality of life between Chinese and Western cultures, it is difficult to reflect the main characteristics of Chinese patients. Therefore, we developed a scale suitable for Chinese patients with anxiety disorders: the Anxiety Disorders Scale of the Quality of Life Instruments for Chronic Diseases (QLICD-AD), hoping to achieve satisfactory QOL assessments for anxiety disorders.

OBJECTIVES

Items from the Anxiety Disorders Scale of the Quality of Life in Chronic Disease Instrument QLICD-AD system were analyzed using CTT and IRT to lay the groundwork for further refinement of the scale to accurately measure anxiety disorders.

METHODS

120 patients with anxiety disorder were assessed using the QLICD-AD (V2.0). Descriptive statistics, variability method, correlation coefficient method, factor analysis and Cronbach's coefficient of CTT, and graded response model (GRM) of item response theory were used to analyze the items of the scale.

RESULT

CTT analysis showed that the standard deviation of each item was between 0.928 and 1.466; Pearson correlation coefficients of item-to-domain were generally greater than 0.5 and also greater than that of item-to-other domain; the Cronbach 's of the total scale was 0.931, α of each domain was between 0.706 and 0.865. IRT analysis showed that the discrimination was between 1.14 and 1.44. The difficulty parameter of all items increased with the increase of grade. But some items (GPH6,GPH8,GPS3,GSO2-GSO4,AD2,AD5) difficulty parameters were less than 4 or greater than 4. The average of information amount was between 0.022 and 0.910.

CONCLUSION

Based on CTT and IRT analysis, most items of the QLICD-AD (V2.0) scale have good performance and good differentiation, but a few items still need further revision. Suggests that the QLICD-AD (V2.0) appears to be a valid measure of anxiety disorders. It may effectively improve the diagnosticity of anxiety disorders, but due to the limitations of the current sample, further validation is needed in a broader population extrapolation trial.

The measurement of war-related trauma amongst internally displaced men and women in South Sudan: Psychometric analysis of the Harvard Trauma Questionnaire

BACKGROUND

Studies from armed conflict settings, including South Sudan, have revealed the deleterious mental health impact of exposure to war atrocities. However, there is little consensus on what is meant by war trauma, how it should be measured, and how levels of trauma vary across men and women.

METHODS

We used psychometric analyses to measure war trauma among 1178 internally displaced adults (mean age = 39 years, 50% women) in the Malakal region of South Sudan. We used cross-sectional survey data and applied classical test theory, factor analysis, item response theory, and differential item functioning with the war events subscale (17 items) of the Harvard Trauma Questionnaire (HTQ).

RESULTS

We found good validity and internal consistency reliability for the HTQ. We found evidence for unidimensionality using factor analyses, and item response theory models showed that some war events (like witnessing the killing of family or friends) were more sensitive to the underlying 'war-related trauma' trait than others (like abduction). Differential item functioning analyses revealed that the measure performed differently for men and women, indicating the need for sex-stratified analysis in the measurement of trauma.

LIMITATIONS

The use of self-report may lead to recall and response bias, and the study sample may not be representative of the broader population in South Sudan.

CONCLUSION

This study emphasizes the need for cultural adaptation and psychometric evaluation of commonly used measurement instruments, especially in humanitarian settings where survey data are used to set priorities for mental health and psychosocial support services.

Turkish version of the faces version of the Modified Child Dental Anxiety Scale (MCDASf): translation, reliability, and validity

OBJECTIVES

To cross-culturally adapt of MCDAS_f into Turkish and to evaluate reliability and validity of the Turkish version of MCDAS_f using explorative factor analysis (EFA), confirmatory factor analysis (CFA), and receiver operating characteristic (ROC) analysis to form a less time consuming and easy to understand tool for measuring dental anxiety.

MATERIALS AND METHODS

A total of 300 children (174 females, 126 males) aged 6-12 years who were recruited at the Department of Paediatric Dentistry in RTEU at the first visit participated in the present study. For the internal consistency reliability, Cronbach's alpha was calculated. Intraclass correlation coefficients (ICCs) were calculated for test-retest reliability (n=67). Construct validity was determined by comparing with the Venham Picture Test (VPT). The factor structure was examined using EFA. CFA was used for dimensionality. The cut-off points of the Turkish version of MCDAS_f were plotted using a ROC curve.

RESULTS

The Turkish version of MCDAS_f showed high internal consistency (0.703) and an excellent ICC value (0.827). Good correlations were found between the global scores of MCDAS_f and VPT (r=0.632). Three factors were identified from EFA and verified with CFA. Discriminant validity was supported by high scores of females and younger children (p<0.05). The cut-off point was assigned as >24.50. The area under the ROC curve (AUC) was 0.921 (95% CI: 0.873-0.969).

CONCLUSIONS

The Turkish version of the MCDAS_f is a reliable and valid scale that satisfied psychometric properties.

CLINICAL RELEVANCE

The Turkish version of MCDAS_f is a usable tool for the measurement of dental anxiety among Turkish speaking children.

Consumer Assessment of Healthcare Providers and Systems (CAHPS®) survey of experiences with ambulatory healthcare for Asians and non-Hispanic Whites in the United States

Background

Differences in experiences of care reported by Asian Americans (Asians) compared to non-Hispanic Whites (Whites) may be due to lack of measurement invariance.

Methods

We evaluated the three-factor structure and the equivalence of responses to the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinical and Group (CG-CAHPS) Adult Visit Survey 1.0 and compared care experiences of Asians and Whites. Thirteen questions were used to elicit reports about specific aspects of care and two questions assessed overall care perceptions.

This analysis of the CAHPS database included 769 providers and 266,327 respondents. Most surveys (98%) were administered by mail and the rest (2%) by phone. Only 0.5% of the surveys were administered in Spanish. The sample was 64% female, 89% White and 2% Asian, 39% 65 years or older, and 32% were high school graduates or less.

Results

A three-factor model was supported by categorical confirmatory factor analysis using weighted least squares with mean and variance adjustment: confirmatory fit index (CFI) = 0.99 and root mean squared error of approximation (RMSEA) = 0.03). A multi-group configural invariance model also fit the data well: (CFI = 0.993, RMSEA = 0.031). Regression models indicated that Asians reported worse access, lower scores on office staff courtesy and helpfulness and rating their doctors and were less likely to recommend their doctors to family/friends than did Whites.

Conclusions

Use of the CG-CAHPS Adult Visit Survey 1.0 to assess perceptions of care by Asians and Whites is supported. Quality improvement efforts are needed to address worse experiences of care among Asians in the United States.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41687-021-00303-3.

Development and testing of a patient-centered dental home assessment for low-income families

OBJECTIVES

The purpose of this study was to develop an instrument in English (ENG) and Spanish (SPA) for use in assessing perceptions of dental home characteristics among low-income adults.

METHODS

An expert panel selected 21 items from a pool of 81 items mapped to domains in the American Academy of Pediatrics' medical home. Pilot testing with mothers (n = 795) of children in Early Head Start (EHS) resulted in an expanded 36-item scale, which was field-tested in interviews with ENG- (NENG = 665) and SPA-speakers (NSPA = 116). An exploratory factor analysis of 23 usable items was carried out in an EHS reference group (NENG-EHS = 272). The selected model was evaluated by confirmatory factor analysis (CFA) in two other subgroups (NENG-NonEHS = 393; NSPA = 116). We tested measurement and structural invariances using a multiple-group approach. Known-groups validity was examined by testing factor mean differences between two groups defined by whether or not the mother used a single dental office.

RESULTS

A three-factor CFA model with 10 items was retained for its close fit for all three subgroups. Invariance tests found the two ENG subgroups to be homogenous and were combined. Several item and factor parameters in the SPA group differed from the combined ENG group. The proposed dental home measure demonstrated good known-group validity in that people who used the same dental office reported better dental care experiences.

CONCLUSIONS

An instrument to measure the dental home concept among low-income adults was developed with three domains (accessible-comprehensive, compassionate, and health literate care) that demonstrated good validity and reliability.

Development of the Sexual Minority Adolescent Stress Inventory

Although construct measurement is critical to explanatory research and intervention efforts, rigorous measure development remains a notable challenge. For example, though the primary theoretical model for understanding health disparities among sexual minority (e.g., lesbian, gay, bisexual) adolescents is minority stress theory, nearly all published studies of this population rely on minority stress measures with poor psychometric properties and development procedures. In response, we developed the Sexual Minority Adolescent Stress Inventory (SMASI) with N = 346 diverse adolescents ages 14–17, using a comprehensive approach to de novo measure development designed to produce a measure with desirable psychometric properties. After exploratory factor analysis on 102 candidate items informed by a modified Delphi process, we applied item response theory techniques to the remaining 72 items. Discrimination and difficulty parameters and item characteristic curves were estimated overall, within each of 12 initially derived factors, and across demographic subgroups. Two items were removed for excessive discrimination and three were removed following reliability analysis. The measure demonstrated configural and scalar invariance for gender and age; a three-item factor was excluded for demonstrating substantial differences by sexual identity and race/ethnicity. The final 64-item measure comprised 11 subscales and demonstrated excellent overall (α = 0.98), subscale (α range 0.75–0.96), and test–retest (scale r > 0.99; subscale r range 0.89–0.99) reliabilities. Subscales represented a mix of proximal and distal stressors, including domains of internalized homonegativity, identity management, intersectionality, and negative expectancies (proximal) and social marginalization, family rejection, homonegative climate, homonegative communication, negative disclosure experiences, religion, and work domains (distal). Thus, the SMASI development process illustrates a method to incorporate information from multiple sources, including item response theory models, to guide item selection in building a psychometrically sound measure. We posit that similar methods can be used to improve construct measurement across all areas of psychological research, particularly in areas where a strong theoretical framework exists but existing measures are limited.

Improving Measurement Efficiency of the Inner EAR Scale with Item Response Theory

Objectives

(1) To assess the 11-item Inner Effectiveness of Auditory Rehabilitation (Inner EAR) instrument with item response theory (IRT). (2) To determine whether the underlying latent ability could also be accurately represented by a subset of the items for use in high-volume clinical scenarios. (3) To determine whether the Inner EAR instrument correlates with pure tone thresholds and word recognition scores.

Design

IRT evaluation of prospective cohort data.

Setting

Tertiary care academic ambulatory otolaryngology clinic.

Subjects and Methods

Modern psychometric methods, including factor analysis and IRT, were used to assess unidimensionality and item properties. Regression methods were used to assess prediction of word recognition and pure tone audiometry scores.

Results

The Inner EAR scale is unidimensional, and items varied in their location and information. Information parameter estimates ranged from 1.63 to 4.52, with higher values indicating more useful items. The IRT model provided a basis for identifying 2 sets of items with relatively lower information parameters. Item information functions demonstrated which items added insubstantial value over and above other items and were removed in stages, creating a 8- and 3-item Inner EAR scale for more efficient assessment. The 8-item version accurately reflected the underlying construct. All versions correlated moderately with word recognition scores and pure tone averages.

Conclusion

The 11-, 8-, and 3-item versions of the Inner EAR scale have strong psychometric properties, and there is correlational validity evidence for the observed scores. Modern psychometric methods can help streamline care delivery by maximizing relevant information per item administered.

Reevaluation of the Amsterdam Inventory for Auditory Disability and Handicap Using Item Response Theory

PURPOSE

We reevaluated the psychometric properties of the Amsterdam Inventory for Auditory Disability and Handicap (AIADH; Kramer, Kapteyn, Festen, & Tobi, 1995) using item response theory. Item response theory describes item functioning along an ability continuum.

METHOD

Cross-sectional data from 2,352 adults with and without hearing impairment, ages 18-70 years, were analyzed. They completed the AIADH in the web-based prospective cohort study "Netherlands Longitudinal Study on Hearing." A graded response model was fitted to the AIADH data. Category response curves, item information curves, and the standard error as a function of self-reported hearing ability were plotted.

RESULTS

The graded response model showed a good fit. Item information curves were most reliable for adults who reported having hearing disability and less reliable for adults with normal hearing. The standard error plot showed that self-reported hearing ability is most reliably measured for adults reporting mild up to moderate hearing disability.

CONCLUSIONS

This is one of the few item response theory studies on audiological self-reports. All AIADH items could be hierarchically placed on the self-reported hearing ability continuum, meaning they measure the same construct. This provides a promising basis for developing a clinically useful computerized adaptive test, where item selection adapts to the hearing ability of individuals, resulting in efficient assessment of hearing disability.

Methodological and Substantive Issues in Conducting Multinational and Cross-Cultural Research

The rising importance of multinational and cross-cultural research has spurred the development of methodologies to adequately study human phenomena in organizations across culture and place. Disentangling culture and location from potential confounding variables is an underlying issue in conducting comparative studies. Research designs allow comparisons of countries and culture groups, as well as isolate the potential impact of cultural variables. Sampling methods must be chosen to control for differences that confound the cultural variables of interest with other characteristics of samples. Translation and back-translation are necessary when studies cross linguistic boundaries. Measures that compare cultural and national groups should be checked for measurement equivalence/invariance using one or more tools based on confirmatory factor analysis or item response theory. Some sources of invariance can be found in culturally determined response biases. In the same investigation, it is advisable to pair qualitative methods, which help characterize culture-specific factors, with quantitative methods, which help disentangle their confounds.

Psychometric properties of the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group Adult Visit Survey

BACKGROUND

The Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Clinician and Group Adult Visit Survey enables patients to report their experiences with outpatient medical offices.

OBJECTIVE

To evaluate the factor structure and reliability of the CAHPS Clinician and Group (CG-CAHPS) Adult Visit Survey.

DATA SOURCE

Data from 21,318 patients receiving care in 450 clinical practice sites collected from March 2010 to December 2010 were analyzed from the CG-CAHPS Database.

RESEARCH DESIGN AND PARTICIPANTS

Individual level and multilevel confirmatory factor analyses were used to examine CAHPS survey responses at the patient and practice site levels. We also estimated internal consistency reliability and practice site level reliability. Correlations among multi-item composites and correlations between the composites and 2 global rating items were examined.

MEASURES

Scores on CG-CAHPS composites assessing Access to Care, Doctor Communication, Courteous/Helpful Staff, and 2 global ratings of whether one would Recommend their Doctor, and an Overall Doctor Rating.

RESULTS

Analyses provide support for the hypothesized 3-factor model assessing Access to Care, Doctor Communication, and Courteous/Helpful Staff. In addition, the internal consistency reliabilities were ≥ 0.77 and practice site level reliabilities for sites with >4 clinicians were ≥ 0.75. All composites were positively and significantly correlated with the 2 global rating items, with Doctor Communication having the strongest relationship with the global ratings.

CONCLUSIONS

The CG-CAHPS Adult Visit Survey has acceptable psychometric properties at the individual level and practice site level. The analyses suggest that the survey items are measuring their intended concepts and yield reliable information.

Measurement invariance of the Functional Assessment of Cancer Therapy-Colorectal quality-of-life instrument among modes of administration

OBJECTIVES

To test for the measurement invariance of the Functional Assessment of Cancer Therapy-Colorectal (FACT-C) in patients with colorectal neoplasms between two modes of administration (self- and interviewer administrations). It is important to establish the measurement invariance of the FACT-C across different modes of administration to ascertain whether it is valid to pool FACT-C data collected by different modes or to assess each group separately.

METHODS

A cross-sectional sample of 391 Chinese patients with colorectal neoplasms was recruited from specialist outpatient clinics between September 2009 and July 2010. Confirmatory factor analysis (CFA) was used to test the original five-factor model of the FACT-C on data collected by self- and interviewer administrations in single-group analysis. Multiple-group CFA was then used to compare the factor structure between the two modes of administration using chi-square tests and other goodness-of-fit statistics.

RESULTS

The hypothesized five-factor model of FACT-C demonstrated good fit in each group. Configural invariance and metric invariance were fully supported in multiple-group CFA. Some item intercepts and their corresponding error variances were not identical between administration groups, suggesting evidence of partial strict factorial invariance.

CONCLUSIONS

Our results confirmed that the five-factor structure of FACT-C was invariant in Chinese patients using both self- and interviewer administrations. It is appropriate to pool or compare data in the emotional well-being and colorectal cancer subscale scores collected by both administrations. Measurement invariance in three items, one from each of the other subscales, may be contaminated by response bias between modes of administration.

Reducing patient burden to the FACT-Melanoma quality-of-life questionnaire

Respondent burden has been defined as the cumulative demand placed on study participants related to the use of questionnaires or measurement instruments. The aim of this study was to reduce respondent burden associated with the Functional Assessment of Cancer Therapy-Melanoma (FACT-M), a melanoma-specific quality-of-life questionnaire, through item reduction using multiple psychometric approaches. Data for this study were pooled from three institutional review board-approved protocols. Poorly performing items were identified through distributional and correlation analyses, confirmatory factor analysis, reliability estimation, and Rasch-based approaches in a developmental dataset, and the reduced scale was assessed in a separate testing cohort. Validity, reliability, goodness of fit, and Rasch-based testing were carried out for both the full and the reduced scales. The clinical characteristics of the development (n=198) and testing (n=204) cohorts were similar. Three items identified through classical psychometric approaches and three items identified by Rasch-based analyses were excluded from the FACT-M subscale. Two additional items were identified for potential reduction but were ultimately maintained due to the adverse consequences to the psychometric integrity of the reduced instrument. The reduced FACT-M module contains 18 items. In addition to psychometric assessment, expert consultation was essential when examining areas of content redundancy and was critical when considering specific items for removal. This methodological approach reduced respondent burden by 25% while maintaining the psychometric integrity of the FACT-M.

Cross - cultural adaptation and preliminary validation of the Turkish version of the early childhood oral health impact scale among 5-6-year-old children

Background

In Turkey, formal pre-primary education for children 5- 6 years old provides the ideal setting for school-based oral health promotion programs and oral health care services. To develop effective oral health promotion programs, there is a need to assess this target group's subjective oral health needs as well as clinical needs. The Early Childhood Oral Health Impact Scale (ECOHIS) is a well-known instrument for assessing oral health quality of life in children aged 0-5 years old and their families. This study aimed to adapt the ECOHIS for children 5-6 years old in a Turkish-speaking community and to undertake a preliminary investigation of its psychometric properties.

Methods

The Turkish version of the ECOHIS was obtained with forward/backward translations, expert panels and pre-testing and it was tested in a convenience sample of 121 parents of 5- 6 year-old children attending nursery classes of three public schools. Data were collected through clinical examinations and self-completed questionnaires. The main analyses were carried out on the imputed data set. The validity of content, face, construct, discriminant and convergent and as well as the reliability of internal and test-retest of the ECOHIS were evaluated. Sensitivity analysis was performed to examine the effect of the complete case analysis for managing "Don't know" responses on the validity and reliability of the ECOHIS.

Results

The analysis of the imputed data set showed that Cronbach's alphas for the child and family sections were 0.92 and 0.84 respectively, and for the whole scale was 0.93. The intraclass correlation coefficient for test-retest was 0.86. The scale scores on the child and parent sections indicating worse quality of life were significantly associated with poor parental ratings of their child's oral health, high caries experience, higher gingival index scores and problem-orientated dental attendance, supporting its construct, convergent and discriminant validity. Sensitivity analysis showed that the mean imputation method and the complete case analysis did not have differing effects on the validity and reliability of the ECOHIS.

Conclusions

This study provided preliminary evidence concerning validity and reliability of the Turkish version of the scale among 5-6-year-old children. Future studies should be conducted on the ECOHIS to evaluate fully its psychometric properties in both community- based and clinically-based studies among parents of children younger than five. This study provides initial evidence that the ECOHIS aimed at children aged 0-5 years may be a useful tool for assessing the oral health quality of life in 6 year - old preschool children.

Dentist-patient communication techniques used in the United States: the results of a national survey

BACKGROUND

The authors conducted a national survey to determine the communication techniques that dentists use routinely and variations in their use.

METHODS

American Dental Association Survey Center staff members mailed an 86-item questionnaire to a random sample of 6,300 U.S. dentists in private practice. Participants reported routine use ("most of the time" or "always") during a typical week of 18 communication techniques, of which seven are basic techniques. The authors used analysis of variance and ordinary least squares regression models to test the association of communication, provider and practice characteristics with the number of techniques.

RESULTS

Dentists routinely use an average of 7.1 of the 18 techniques and 3.1 of the seven basic techniques. Two-thirds or more of dentists used four of the techniques (hand out printed materials, speak slowly, use models or radiographs to explain, use simple language). Less than one-fourth of dentists used any of the techniques in the teach-back method or patient-friendly practice domains. A dentist's age, race/ethnicity, education outside the United States and area of dentistry affected use. Health literacy variables (awareness, education in communication, practice-level change, outcome expectancy) and lack of time were associated with the number of techniques used.

CONCLUSIONS

Routine use of all of the communication techniques is low among dentists, including some techniques thought to be most effective with patients with low literacy skills.

PRACTICE IMPLICATIONS

Professional education is needed to improve knowledge about communication techniques and to ensure that they are used effectively. A firm foundation for these efforts requires the development, evaluation and dissemination of communication guidelines for dental care professionals.

Rasch analysis of the Psychiatric Out-Patient Experiences Questionnaire (POPEQ)

BACKGROUND

The Psychiatric Out-Patient Experiences Questionnaire (POPEQ) is an 11-item core measure of psychiatric out-patients experiences of the perceived outcome of the treatment, the quality of interaction with the clinician, and the quality of information provision. The POPEQ was found to have evidence for reliability and validity following the application of classical test theory but has not previously been assessed by Rasch analysis.

METHODS

Two national postal surveys of psychiatric outpatients took place in Norway in 2004 and 2007. The performance of the POPEQ, including item functioning and differential item functioning, was assessed by Rasch analysis. Principal component analysis of item residuals was used to assess the presence of subdimensions.

RESULTS

6,677 (43.3%) and 11,085 (35.2%) psychiatric out patients responded to the questionnaire in 2004 and 2007, respectively. All items in the scale were retained after the Rasch analysis. The resulting scale had reasonably good fit to the Rasch model. The items performed the same for the two survey years and there was no differential item functioning relating to patient characteristics. Principal component analysis of the residuals confirmed that the measure to a high degree is unidimensional. However, the data also reflects three potential subscales, each relating to one of the three included aspects of health care.

CONCLUSIONS

The POPEQ had excellent psychometric properties and Rasch analysis further supported the construct validity of the scale by also identifying the three subdimensions originally included as components in the instrument development. The 11-item instrument is recommended in future research on psychiatric out-patient experiences. Future development may lead to the construction of more precise measures of the three subdomains that the POPEQ is based on.

Improving the quality and applicability of the Dutch scales of the Communication Profile for the Hearing Impaired using item response theory

PURPOSE

The aim of this study was to improve the quality and applicability of the 6 Dutch scales of the Communication Profile for the Hearing Impaired (CPHI; Demorest & Erdman, 1986, 1987, 1988) using item response theory (IRT). IRT modeling can produce precise, valid, and relatively brief instruments, resulting in minimal response burden (Edelen & Reeve, 2006).

METHOD

We investigated (a) whether items fit to the graded response model (GRM; Samejima, 1969) using Stone and Zhang's (2003) procedure to test goodness of fit and (b) whether items showed differential item functioning (DIF) with the IRT log-likelihood ratio approach for 5 group variables: gender, age, living arrangement, use of hearing aids, and degree of hearing impairment. We investigated the magnitude of DIF by calculating the maximum difference between expected item scores for subgroups of the group variables of items showing DIF.

RESULTS

In a cross-sectional study, 408 consecutive individuals with hearing impairment completed the Dutch CPHI scales. Twelve items were deleted because of misfit to GRMs, 14 items showed uniform DIF, and 2 showed nonuniform DIF. Five items showing DIF were deleted due to large magnitude of DIF. The short form of the Dutch CPHI scales contains 35 items.

CONCLUSIONS

DIF analyses showed that the short form was relatively free of DIF with respect to the 5 group variables. Hence, there is no need to make adjustments for the calculation of subgroup scores.

The classic measure of disability in activities of daily living is biased by age but an expanded IADL/ADL measure is not

OBJECTIVES

To evaluate, by age, the performance of 2 disability measures based on needing help: one using 5 classic activities of daily living (ADL) and another using an expanded set of 14 activities including instrumental activities of daily living (IADL), walking, getting outside, and ADL (IADL/ADL).

METHODS

Guttman and item response theory (IRT) scaling methods are used with a large (N = 25,470) nationally representative household survey of individuals aged 18 years and older.

RESULTS

Guttman scalability of the ADL items increases steadily with age, reaching a high level at ages 75 years and older. That is reflected in an IRT model by age-related differential item functioning (DIF) resulting in age-biased measurement of ADL. Guttman scalability of the IADL/ADL items also increases with age but is lower than the ADL. Although age-related DIF also occurs with IADL/ADL items, DIF is lower in magnitude and balances out without causing age bias.

DISCUSSION

An IADL/ADL scale measuring need for help is hierarchical, unidimensional, and unbiased by age. It has greater content validity for measuring need for help in the community and shows greater sensitivity by age than the classic ADL measure. As demand for community services is increasing among adults of all ages, an expanded IADL/ADL measure is more useful than ADL.

Measuring the ICF components of impairment, activity limitation and participation restriction: an item analysis using classical test theory and item response theory

BACKGROUND

The International Classification of Functioning, Disability and Health (ICF) proposes three main health outcomes, Impairment (I), Activity Limitation (A) and Participation Restriction (P), but good measures of these constructs are needed The aim of this study was to use both Classical Test Theory (CTT) and Item Response Theory (IRT) methods to carry out an item analysis to improve measurement of these three components in patients having joint replacement surgery mainly for osteoarthritis (OA).

METHODS

A geographical cohort of patients about to undergo lower limb joint replacement was invited to participate. Five hundred and twenty four patients completed ICF items that had been previously identified as measuring only a single ICF construct in patients with osteoarthritis. There were 13 I, 26 A and 20 P items. The SF-36 was used to explore the construct validity of the resultant I, A and P measures. The CTT and IRT analyses were run separately to identify items for inclusion or exclusion in the measurement of each construct. The results from both analyses were compared and contrasted.

RESULTS

Overall, the item analysis resulted in the removal of 4 I items, 9 A items and 11 P items. CTT and IRT identified the same 14 items for removal, with CTT additionally excluding 3 items, and IRT a further 7 items. In a preliminary exploration of reliability and validity, the new measures appeared acceptable.

CONCLUSION

New measures were developed that reflect the ICF components of Impairment, Activity Limitation and Participation Restriction for patients with advanced arthritis. The resulting Aberdeen IAP measures (Ab-IAP) comprising I (Ab-I, 9 items), A (Ab-A, 17 items), and P (Ab-P, 9 items) met the criteria of conventional psychometric (CTT) analyses and the additional criteria (information and discrimination) of IRT. The use of both methods was more informative than the use of only one of these methods. Thus combining CTT and IRT appears to be a valuable tool in the development of measures.

Development of EMPRO: a tool for the standardized assessment of patient-reported outcome measures

OBJECTIVE

This study was aimed to develop a tool for the standardized assessment of patient-reported outcomes (PROs) to assist the choice of instruments.

METHODS

An expert panel adapted the eight attributes proposed by the Medical Outcomes Trust as evaluation review criteria, created items to evaluate them, and included a response scale for each item. A pilot test was designed to test the new tool's feasibility and to obtain preliminary information concerning its psychometric properties. The Spanish versions of five measures were selected for assessment: the SF-36 Health Survey, the Nottingham Health Profile, the COOP-WONCA charts, the EuroQol-5D, and the Quality of Life Questionnaire EORTC-QLQ-C30. We assessed the new tool's reliability (Cronbach's alpha and intraclass correlation coefficient [ICC]) and construct validity.

RESULTS

The new EMPRO (Evaluating the Measurement of Patient-Reported Outcomes) tool has 39 items covering eight key attributes: conceptual and measurement model, reliability, validity, responsiveness, interpretability, burden, alternative modes of administration, and cross-cultural and linguistic adaptations. Internal consistency was high (alpha = 0.95) as was interrater concordance (ICC: 0.87-0.94). Positive associations consistent with a priori hypotheses were observed between EMPRO attribute scores and the number of articles identified for the measures, the years elapsed since the publication of the first article, and the number of citations.

CONCLUSION

A new tool for the standardized assessment of PRO measures is available. It has shown good preliminary reliability and validity and should be a useful aid to investigators who need to choose between alternative measures. Further assessment of the tool is necessary.

Toward conceptual clarity in a critical parenting construct: parental monitoring in youth with chronic illness

Parental monitoring has been defined as "a set of correlated parenting behaviors involving attention to and tracking of the child's whereabouts, activities, and adaptations." This construct is of significant interest due to its relatedness to a broad range of youth risk behaviors, including risky sexual behavior, substance abuse, and poor adherence. However, to date, measures of parental monitoring are largely absent from the chronic illness literature. The present article focuses upon two key problems in the operationalization of the monitoring construct to date: (a) poor conceptual specificity in parenting constructs such as monitoring, overprotection, and over-involvement when used to date among youth with chronic conditions and (b) the confounding of existing measures of parental monitoring with items evaluating parental knowledge of youth activities, which has resulted in a lack of data regarding the mechanisms by which parents obtain their information. Recommendations for the future development of monitoring measures are discussed.

Enhancing measurement in health outcomes research supported by Agencies within the US Department of Health and Human Services

Many of the Institutes, Agencies and Centers that make up the US Department of Health and Human Services (DHHS) have recognized the need for better instrumentation in health outcomes research, and provide support, both internally and externally, for research utilizing advances in measurement theory and computer technology (informatics). In this paper, representatives from several DHHS agencies and institutes will discuss their need for better instruments within their discipline and describe current or future initiatives for exploring the benefits of these technologies. Together, the perspectives underscore the importance of developing valid, precise, and efficient measures to capture the full burden of disease and treatment on patients. Initiatives, like the Patient-Reported Outcomes Measurement Information System (PROMIS) to create health-related quality of life item banks, represent a trans-DHHS effort to develop a standard set of measures for informing decision making in clinical research, practice, and health policy.

Overview of Quantitative Measurement Methods

BACKGROUND

Reviewed in this article are issues relating to the study of invariance and differential item functioning (DIF). The aim of factor analyses and DIF, in the context of invariance testing, is the examination of group differences in item response conditional on an estimate of disability. Discussed are parameters and statistics that are not invariant and cannot be compared validly in crosscultural studies with varying distributions of disability in contrast to those that can be compared (if the model assumptions are met) because they are produced by models such as linear and nonlinear regression.

OBJECTIVES

The purpose of this overview is to provide an integrated approach to the quantitative methods used in this special issue to examine measurement equivalence. The methods include classical test theory (CTT), factor analytic, and parametric and nonparametric approaches to DIF detection. Also included in the quantitative section is a discussion of item banking and computerized adaptive testing (CAT).

METHODS

Factorial invariance and the articles discussing this topic are introduced. A brief overview of the DIF methods presented in the quantitative section of the special issue is provided together with a discussion of ways in which DIF analyses and examination of invariance using factor models may be complementary.

CONCLUSIONS

Although factor analytic and DIF detection methods share features, they provide unique information and can be viewed as complementary in informing about measurement equivalence.