1
|
Qin H, Guo L. Priority attribute algorithm for Q-matrix validation: A didactic. Behav Res Methods 2024; 57:31. [PMID: 39738806 DOI: 10.3758/s13428-024-02547-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/06/2024] [Indexed: 01/02/2025]
Abstract
The Q-matrix is one of the core components of cognitive diagnostic assessment, which is a matrix describing the relationship between items and the attributes being assessed. Numerous studies have shown that inaccuracies in defining the Q-matrix can degrade parameter estimation and model fitting results. Currently, Q-matrix validation often involves exhaustive search algorithms (ESA), which traverse through all possible q -vectors and determine the optimal q -vector for items based on indicators or criteria corresponding to different validation methods. However, ESA methods are time-consuming, especially when the number of attributes is large, as the search complexity grows exponentially. This study proposes a more efficient search algorithm, the priority attribute algorithm (PAA), which conducts searches one by one according to the priority of attributes, greatly simplifying the search process. Simulation studies indicate that PAA can significantly enhance search efficiency while maintaining the same or even higher accuracy than ESA, particularly when dealing with a large number of attributes. Moreover, the Q-matrix validation method employing PAA demonstrates better applicability to small samples. A real-data analysis indicates that applying the PAA-based Q-matrix validation method may yield suggested Q-matrices with higher model-data fit and greater practical utility.
Collapse
Affiliation(s)
- Haijiang Qin
- Faculty of Psychology, Southwest University, Chongqing, China
| | - Lei Guo
- Faculty of Psychology, Southwest University, Chongqing, China.
- Southwest University Branch, Collaborative Innovation Center of Assessment toward Basic Education Quality, Chongqing, China.
| |
Collapse
|
2
|
Shan N, Xu PF. Bayesian Adaptive Lasso for Detecting Item-Trait Relationship and Differential Item Functioning in Multidimensional Item Response Theory Models. PSYCHOMETRIKA 2024; 89:1337-1365. [PMID: 39127801 DOI: 10.1007/s11336-024-09998-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/02/2023] [Revised: 07/16/2024] [Indexed: 08/12/2024]
Abstract
In multidimensional tests, the identification of latent traits measured by each item is crucial. In addition to item-trait relationship, differential item functioning (DIF) is routinely evaluated to ensure valid comparison among different groups. The two problems are investigated separately in the literature. This paper uses a unified framework for detecting item-trait relationship and DIF in multidimensional item response theory (MIRT) models. By incorporating DIF effects in MIRT models, these problems can be considered as variable selection for latent/observed variables and their interactions. A Bayesian adaptive Lasso procedure is developed for variable selection, in which item-trait relationship and DIF effects can be obtained simultaneously. Simulation studies show the performance of our method for parameter estimation, the recovery of item-trait relationship and the detection of DIF effects. An application is presented using data from the Eysenck Personality Questionnaire.
Collapse
Affiliation(s)
- Na Shan
- School of Psychology & Key Laboratory of Applied Statistics of MOE, Northeast Normal University, 5268 Renmin Street, Changchun, Jilin, China.
| | - Ping-Feng Xu
- Academy for Advanced Interdisciplinary Studies & Key Laboratory of Applied Statistics of MOE, Northeast Normal University, Changchun, China
- Shanghai Zhangjiang Institute of Mathematics, Shanghai, China
| |
Collapse
|
3
|
Halpin PF. Differential Item Functioning via Robust Scaling. PSYCHOMETRIKA 2024; 89:796-821. [PMID: 38704430 DOI: 10.1007/s11336-024-09957-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/20/2022] [Indexed: 05/06/2024]
Abstract
This paper proposes a method for assessing differential item functioning (DIF) in item response theory (IRT) models. The method does not require pre-specification of anchor items, which is its main virtue. It is developed in two main steps: first by showing how DIF can be re-formulated as a problem of outlier detection in IRT-based scaling and then tackling the latter using methods from robust statistics. The proposal is a redescending M-estimator of IRT scaling parameters that is tuned to flag items with DIF at the desired asymptotic type I error rate. Theoretical results describe the efficiency of the estimator in the absence of DIF and its robustness in the presence of DIF. Simulation studies show that the proposed method compares favorably to currently available approaches for DIF detection, and a real data example illustrates its application in a research context where pre-specification of anchor items is infeasible. The focus of the paper is the two-parameter logistic model in two independent groups, with extensions to other settings considered in the conclusion.
Collapse
Affiliation(s)
- Peter F Halpin
- University of North Carolina at Chapel Hill, 100 E Cameron Ave, Office 1070G, Chapel Hill, NC, 27514, USA.
| |
Collapse
|
4
|
Peng K, Liu G, Wang J, Chen T. Psychometric Properties of Fine Motor Function Measure in Children With Cerebral Palsy: A Rasch Analysis. Clin Pediatr (Phila) 2024:99228241274295. [PMID: 39183559 DOI: 10.1177/00099228241274295] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 08/27/2024]
Abstract
Our study utilized Rasch Analysis to examine the psychometric properties of 61-items fine motor function measure (FMFM) in children with cerebral palsy (CP). Partial credit model (PCM) was utilized to test the reliability and validity of FMFM. The response pattern of this samples displayed acceptable fitness to PCM. The analysis results supported the assumption of 1-dimensionality of FMFM. Disordered category thresholds were found in 30 items. Differential item functioning (DIF) was detected in 23 items. Participants with different CP subtypes in different age groups may perform in differently responses patterns. The Rasch analysis produces reliable evidence to support the clinical application of FMFM. Some items may produce inaccurate measurements originated from category structures. Difference in age groups and symptom topography may be associated with variation in fine motor ability among children with CP and leading to unnecessary assessment bias. Hence, FMFM items need modifications to calibrate the former item formulation.
Collapse
Affiliation(s)
| | | | | | - Turong Chen
- Shenzhen Children's Hospital, Shenzhen, China
| |
Collapse
|
5
|
Resnik LJ, Borgia M, Graczyk EL, Barth J, Ni P. Prosthesis usability experience is associated with extent of upper limb prosthesis adoption: A Structural Equation Modeling (SEM) analysis. PLoS One 2024; 19:e0299155. [PMID: 38917074 PMCID: PMC11198835 DOI: 10.1371/journal.pone.0299155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2023] [Accepted: 02/06/2024] [Indexed: 06/27/2024] Open
Abstract
Factors associated with upper limb prosthesis adoption are not well understood. In this study, we explored how prosthesis usability experience relates to the extent of prosthesis adoption through the development of a structural equation model (SEM). First, items related to prosthesis usability were developed and refined using cognitive testing and pilot testing and employed in a survey of 402 prosthesis users (mean age 61.7 (sd 14.4), 77.1% Veterans). The SEM examined two unidimensional latent constructs: Prosthesis Usability Experience and Prosthesis Adoption-and each had multiple measured indicators. SEMs tested direct as well as moderating and mediating effects between the latent constructs and covariates related to demographics and prosthesis type. SEM found a significant positive association between Prosthesis Usability Experience and Extent of Prosthesis Adoption. Several covariates had direct effects on prosthesis adoption: 1) Extent of Prosthesis Adoption was lower for those with transhumeral and shoulder amputation, and higher for those with bilateral amputation, compared to the reference group with unilateral transradial amputation and 2) Myoelectric multiple degree of freedom (multi-DOF) prosthesis use was associated with lower Extent of Prosthesis Adoption, compared to body-powered prosthesis use. Myoelectric multi-DOF use also modified the effect of Prosthesis Usability Experience on Extent of Prosthesis Adoption. For those with bilateral ULA, the strength of the relationship between Prosthesis Usability Experience and Extent of Prosthesis Adoption was reduced. Findings suggest that in order to increase prosthesis adoption, prosthetics developers and rehabilitation providers should focus on implementing strategies to improve prosthesis usability experience. New Prosthesis Usability Experience measures could be used to identify persons at greater risk for poor prosthesis adoption and target interventions to increase prosthesis use.
Collapse
Affiliation(s)
- Linda J. Resnik
- Providence VA Medical Center, Providence, Rhode Island, United States of America
- Department of Health Services, Policy and Practice, Brown University School of Public Health, Providence, Rhode Island, United States of America
| | - Matthew Borgia
- Providence VA Medical Center, Providence, Rhode Island, United States of America
| | - Emily L. Graczyk
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, Ohio, United States of America
- Louis Stokes Cleveland Department of Veterans Affairs Medical Center, Cleveland, Ohio, United States of America
| | - Jessica Barth
- Providence VA Medical Center, Providence, Rhode Island, United States of America
- Center for Innovation in Long-Term Services & Supports, Providence VA Medical Center, Providence, Rhode Island, United States of America
| | - Pengsheng Ni
- Biostatistics & Epidemiology Data Analytic Center, Department of Health Law, Policy and Management, Boston University School of Public Health, Boston, Massachusetts, United States of America
| |
Collapse
|
6
|
Wallin G, Chen Y, Moustaki I. DIF Analysis with Unknown Groups and Anchor Items. PSYCHOMETRIKA 2024; 89:267-295. [PMID: 38383880 PMCID: PMC11062998 DOI: 10.1007/s11336-024-09948-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Indexed: 02/23/2024]
Abstract
Ensuring fairness in instruments like survey questionnaires or educational tests is crucial. One way to address this is by a Differential Item Functioning (DIF) analysis, which examines if different subgroups respond differently to a particular item, controlling for their overall latent construct level. DIF analysis is typically conducted to assess measurement invariance at the item level. Traditional DIF analysis methods require knowing the comparison groups (reference and focal groups) and anchor items (a subset of DIF-free items). Such prior knowledge may not always be available, and psychometric methods have been proposed for DIF analysis when one piece of information is unknown. More specifically, when the comparison groups are unknown while anchor items are known, latent DIF analysis methods have been proposed that estimate the unknown groups by latent classes. When anchor items are unknown while comparison groups are known, methods have also been proposed, typically under a sparsity assumption - the number of DIF items is not too large. However, DIF analysis when both pieces of information are unknown has not received much attention. This paper proposes a general statistical framework under this setting. In the proposed framework, we model the unknown groups by latent classes and introduce item-specific DIF parameters to capture the DIF effects. Assuming the number of DIF items is relatively small, an L 1 -regularised estimator is proposed to simultaneously identify the latent classes and the DIF items. A computationally efficient Expectation-Maximisation (EM) algorithm is developed to solve the non-smooth optimisation problem for the regularised estimator. The performance of the proposed method is evaluated by simulation studies and an application to item response data from a real-world educational test.
Collapse
Affiliation(s)
- Gabriel Wallin
- Department of Mathematics and Statistics, Lancaster University, Umeå, Sweden
| | - Yunxiao Chen
- Department of Statistics, London School of Economics and Political Science, Columbia House, Room 5.16 Houghton Street, London, WC2A 2AE, UK.
| | - Irini Moustaki
- Department of Statistics, London School of Economics and Political Science, Columbia House, Room 5.16 Houghton Street, London, WC2A 2AE, UK
| |
Collapse
|
7
|
Resnik LJ, Borgia M, Clark MA, Ni P. Out-of-pocket costs and affordability of upper limb prostheses. Prosthet Orthot Int 2024; 48:108-114. [PMID: 36897203 DOI: 10.1097/pxr.0000000000000223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Accepted: 01/18/2023] [Indexed: 03/11/2023]
Abstract
BACKGROUND Given the funding policies in the Department of Veterans Affairs, the affordability of prostheses may be less of a concern among Veterans as compared to civilians. OBJECTIVES Compare rates of out-of-pocket prosthesis-related payments for Veterans and non-Veterans with upper limb amputation (ULA), develop and validate a measure of prosthesis affordability, and evaluate the impact of affordability on prosthesis nonuse. STUDY DESIGN Telephone survey of 727 persons with ULA; 76% Veterans and 24% non-Veterans. METHODS Odds of paying out-of-pocket costs for Veterans compared with non-Veterans were computed using logistic regression. Cognitive and pilot testing resulted in a new scale, evaluated using confirmatory factor and Rasch analysis. Proportions of respondents who cited affordability as a reason for never using or abandoning a prosthesis were calculated. RESULTS Twenty percent of those who ever used a prosthesis paid out-of-pocket costs. Veterans had 0.20 odds (95% confidence interval, 0.14-0.30) of paying out-of-pocket costs compared with non-Veterans. Confirmatory factor analysis supported unidimensionality of the 4-item Prosthesis Affordability scale. Rasch person reliability was 0.78. Cronbach alpha was 0.87. Overall, 14% of prosthesis never-users said affordability was a reason for nonuse; 9.6% and 16.5% of former prosthesis users said affordability of repairs or replacement, respectively, was a reason for abandonment. CONCLUSIONS Out-of-pocket prosthesis costs were paid by 20% of those sample, with Veterans less likely to incur costs. The Prosthesis Affordability scale developed in this study was reliable and valid for persons with ULA. Prosthesis affordability was a common reason for never using or abandoning prostheses.
Collapse
Affiliation(s)
- Linda J Resnik
- Research Department, Providence VA Medical Center, Providence, RI
- Department of Health Services, Policy and Practice, School of Public Health, Brown University, Providence, RI
| | - Matthew Borgia
- Research Department, Providence VA Medical Center, Providence, RI
| | - Melissa A Clark
- Department of Health Services, Policy and Practice, School of Public Health, Brown University, Providence, RI
- University of Massachusetts Medical School, Worcester, MA
| | - Pengsheng Ni
- Boston University School of Public Health, Boston, MA
| |
Collapse
|
8
|
Chen Y, Li C, Ouyang J, Xu G. DIF Statistical Inference Without Knowing Anchoring Items. PSYCHOMETRIKA 2023; 88:1097-1122. [PMID: 37550561 PMCID: PMC10656337 DOI: 10.1007/s11336-023-09930-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/24/2022] [Accepted: 07/05/2023] [Indexed: 08/09/2023]
Abstract
Establishing the invariance property of an instrument (e.g., a questionnaire or test) is a key step for establishing its measurement validity. Measurement invariance is typically assessed by differential item functioning (DIF) analysis, i.e., detecting DIF items whose response distribution depends not only on the latent trait measured by the instrument but also on the group membership. DIF analysis is confounded by the group difference in the latent trait distributions. Many DIF analyses require knowing several anchor items that are DIF-free in order to draw inferences on whether each of the rest is a DIF item, where the anchor items are used to identify the latent trait distributions. When no prior information on anchor items is available, or some anchor items are misspecified, item purification methods and regularized estimation methods can be used. The former iteratively purifies the anchor set by a stepwise model selection procedure, and the latter selects the DIF-free items by a LASSO-type regularization approach. Unfortunately, unlike the methods based on a correctly specified anchor set, these methods are not guaranteed to provide valid statistical inference (e.g., confidence intervals and p-values). In this paper, we propose a new method for DIF analysis under a multiple indicators and multiple causes (MIMIC) model for DIF. This method adopts a minimal [Formula: see text] norm condition for identifying the latent trait distributions. Without requiring prior knowledge about an anchor set, it can accurately estimate the DIF effects of individual items and further draw valid statistical inferences for quantifying the uncertainty. Specifically, the inference results allow us to control the type-I error for DIF detection, which may not be possible with item purification and regularized estimation methods. We conduct simulation studies to evaluate the performance of the proposed method and compare it with the anchor-set-based likelihood ratio test approach and the LASSO approach. The proposed method is applied to analysing the three personality scales of the Eysenck personality questionnaire-revised (EPQ-R).
Collapse
Affiliation(s)
- Yunxiao Chen
- London School of Economics and Political Science, London, UK.
| | | | | | | |
Collapse
|
9
|
Wijayanto F, Bucur IG, Mul K, Groot P, van Engelen BGM, Heskes T. Semi-automated Rasch analysis with differential item functioning. Behav Res Methods 2023; 55:3129-3148. [PMID: 36070131 PMCID: PMC10556135 DOI: 10.3758/s13428-022-01947-9] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/30/2022] [Indexed: 11/08/2022]
Abstract
Rasch analysis is a procedure to develop and validate instruments that aim to measure a person's traits. However, manual Rasch analysis is a complex and time-consuming task, even more so when the possibility of differential item functioning (DIF) is taken into consideration. Furthermore, manual Rasch analysis by construction relies on a modeler's subjective choices. As an alternative approach, we introduce a semi-automated procedure that is based on the optimization of a new criterion, called in-plus-out-of-questionnaire log likelihood with differential item functioning (IPOQ-LL-DIF), which extends our previous criterion. We illustrate our procedure on artificially generated data as well as on several real-world datasets containing potential DIF items. On these real-world datasets, our procedure found instruments with similar clinimetric properties as those suggested by experts through manual analyses.
Collapse
Affiliation(s)
- Feri Wijayanto
- Institute for Computing and Information Sciences, Radboud University Nijmegen, Nijmegen, The Netherlands.
- Department of Informatics, Universitas Islam Indonesia, Yogyakarta, Indonesia.
| | - Ioan Gabriel Bucur
- Institute for Computing and Information Sciences, Radboud University Nijmegen, Nijmegen, The Netherlands
| | - Karlien Mul
- Department of Neurology, Donders Institute for Brain, Cognition, and Behaviour, Nijmegen, The Netherlands
| | - Perry Groot
- Institute for Computing and Information Sciences, Radboud University Nijmegen, Nijmegen, The Netherlands
| | - Baziel G M van Engelen
- Department of Neurology, Donders Institute for Brain, Cognition, and Behaviour, Nijmegen, The Netherlands
| | - Tom Heskes
- Institute for Computing and Information Sciences, Radboud University Nijmegen, Nijmegen, The Netherlands
| |
Collapse
|
10
|
Dubuy Y, Hardouin JB, Blanchin M, Sébille V. Identification of sources of DIF using covariates in patient-reported outcome measures: a simulation study comparing two approaches based on Rasch family models. Front Psychol 2023; 14:1191107. [PMID: 37637889 PMCID: PMC10448192 DOI: 10.3389/fpsyg.2023.1191107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2023] [Accepted: 06/27/2023] [Indexed: 08/29/2023] Open
Abstract
When analyzing patient-reported outcome (PRO) data, sources of differential item functioning (DIF) can be multiple and there may be more than one covariate of interest. Hence, it could be of great interest to disentangle their effects. Yet, in the literature on PRO measures, there are many studies where DIF detection is applied separately and independently for each covariate under examination. With such an approach, the covariates under investigation are not introduced together in the analysis, preventing from simultaneously studying their potential DIF effects on the questionnaire items. One issue, among others, is that it may lead to the detection of false-positive effects when covariates are correlated. To overcome this issue, we developed two new algorithms (namely ROSALI-DIF FORWARD and ROSALI-DIF BACKWARD). Our aim was to obtain an iterative item-by-item DIF detection method based on Rasch family models that enable to adjust group comparisons for DIF in presence of two binary covariates. Both algorithms were evaluated through a simulation study under various conditions aiming to be representative of health research contexts. The performance of the algorithms was assessed using: (i) the rates of false and correct detection of DIF, (ii) the DIF size and form recovery, and (iii) the bias in the latent variable level estimation. We compared the performance of the ROSALI-DIF algorithms to the one of another approach based on likelihood penalization. For both algorithms, the rate of false detection of DIF was close to 5%. The DIF size and form influenced the rates of correct detection of DIF. Rates of correct detection was higher with increasing DIF size. Besides, the algorithm fairly identified homogeneous differences in the item threshold parameters, but had more difficulties identifying non-homogeneous differences. Over all, the ROSALI-DIF algorithms performed better than the penalized likelihood approach. Integrating several covariates during the DIF detection process may allow a better assessment and understanding of DIF. This study provides valuable insights regarding the performance of different approaches that could be undertaken to fulfill this aim.
Collapse
Affiliation(s)
- Yseulys Dubuy
- UMR INSERM 1246, MethodS in Patients-centered outcomes and HEalth ResEarch (SPHERE), Nantes Université, Nantes, France
| | - Jean-Benoit Hardouin
- UMR INSERM 1246, MethodS in Patients-centered outcomes and HEalth ResEarch (SPHERE), Nantes Université, Nantes, France
- Methodology and Biostatistics Unit, CHU Nantes, Nantes Université, Nantes, France
- Public Health Department, CHU Nantes, Nantes Université, Nantes, France
| | - Myriam Blanchin
- UMR INSERM 1246, MethodS in Patients-centered outcomes and HEalth ResEarch (SPHERE), Nantes Université, Nantes, France
| | - Véronique Sébille
- UMR INSERM 1246, MethodS in Patients-centered outcomes and HEalth ResEarch (SPHERE), Nantes Université, Nantes, France
- Methodology and Biostatistics Unit, CHU Nantes, Nantes Université, Nantes, France
| |
Collapse
|
11
|
Rodriguez RM, Silvia PJ, Kaufman JC, Reiter-Palmon R, Puryear JS. Taking Inventory of the Creative Behavior Inventory: An Item Response Theory Analysis of the CBI. CREATIVITY RESEARCH JOURNAL 2023. [DOI: 10.1080/10400419.2023.2183322] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/05/2023]
|
12
|
Wang C, Zhu R, Xu G. Using Lasso and Adaptive Lasso to Identify DIF in Multidimensional 2PL Models. MULTIVARIATE BEHAVIORAL RESEARCH 2023; 58:387-407. [PMID: 35086405 DOI: 10.1080/00273171.2021.1985950] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/26/2023]
Abstract
Differential item functioning (DIF) analysis refers to procedures that evaluate whether an item's characteristic differs for different groups of persons after controlling for overall differences in performance. DIF is routinely evaluated as a screening step to ensure items behave the same across groups. Currently, the majority DIF studies focus predominately on unidimensional IRT models, although multidimensional IRT (MIRT) models provide a powerful tool for enriching the information gained in modern assessment. In this study, we explore regularization methods for DIF detection in MIRT models and compare their performance to the classic likelihood ratio test. Regularization methods have recently emerged as a new family of methods for DIF detection due to their advantages: (1) they bypass the tedious iterative purification procedure that is often needed in other methods for identifying anchor items, and (2) they can handle multiple covariates simultaneously. The specific regularization methods considered in the study are: lasso with expectation-maximization (EM), lasso with expectation-maximization-maximization (EMM) algorithm, and adaptive lasso with EM. Simulation results show that lasso EMM and adaptive lasso EM hold great promise when the sample size is large, and they both outperform lasso EM. A real data example from PROMIS depression and anxiety scales is presented in the end.
Collapse
|
13
|
Resnik LJ, Stevens PM, Ni P, Borgia ML, Clark MA. Assessment of Patient-Reported Physical Function in Persons With Upper Extremity Amputation: Comparison of Short Form Instruments. Am J Phys Med Rehabil 2023; 102:120-129. [PMID: 35703194 PMCID: PMC9751229 DOI: 10.1097/phm.0000000000002044] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2023]
Abstract
OBJECTIVE The aim of the study was to compare psychometric properties of the Patient-Reported Outcomes Measurement Information System upper extremity measure (PROMIS UE) 7-item short form with 6- and 13-item versions for persons with upper limb amputation. DESIGN The study used a telephone survey of 681 persons with upper limb amputation. Versions were scored two ways: PROMIS health measure scoring (PROMIS UE HMSS) and sample-specific calibration (PROMIS UE AMP). Factor analyses and Rasch analyses evaluated unidimensionality, monotonicity, item fit, differential item functioning, and reliability. Known group validity was compared for all versions. RESULTS Model fit was acceptable for PROMIS-6 UE AMP and marginally acceptable for PROMIS-13 UE AMP and PROMIS-7 UE AMP. Item response categories were collapsed because of disordered categories. A total of 91.4% of participants had PROMIS-13 UE AMP scores with reliability greater than 0.8, compared with 70.4% for PROMIS-7 UE AMP, and 72.1% for PROMIS-6 UE AMP versions. No differences were observed by prosthesis use. Scores differed by amputation for all measures except the HMSS scored 13- and 7-item versions. CONCLUSIONS The PROMIS-13 UE AMP short form was superior to the health measures scoring system scored PROMIS-7 UE or PROMIS-6 UE, and to the PROMIS-7 UE AMP and PROMIS-6 UE AMP. Issues with known group validation suggest a need for a population-specific measure of upper extremity function for persons with upper limb amputation.
Collapse
Affiliation(s)
- Linda J. Resnik
- Research Department, Providence VA Medical Center, Providence, Rhode Island, United States of America
- Department of Health Services, Policy and Practice, Brown University School of Public Health, Providence, Rhode Island, United States of America
| | - Phillip M. Stevens
- Hanger Institute for Clinical Research and Education, Austin, Texas
- Department of Physical Medicine and Rehabilitation, University of Utah Health, Salt Lake City, UT
| | | | - Matthew L. Borgia
- Research Department, Providence VA Medical Center, Providence, Rhode Island, United States of America
| | - Melissa A. Clark
- Department of Health Services, Policy and Practice, Brown University School of Public Health, Providence, Rhode Island, United States of America
- Department of Population and Quantitative Health Sciences, University of Massachusetts Medical School, Worcester Massachusetts, United States of America
| |
Collapse
|
14
|
Soomro MA, Ali MH, Zailani S, Tseng ML, Makhbul ZM. Understanding barriers and motivations in solid waste management from Malaysian industries: a comparative analysis. ENVIRONMENTAL SCIENCE AND POLLUTION RESEARCH INTERNATIONAL 2023; 30:5717-5729. [PMID: 35978247 PMCID: PMC9385409 DOI: 10.1007/s11356-022-22558-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 01/06/2022] [Accepted: 08/11/2022] [Indexed: 06/15/2023]
Abstract
The objective of this study is to explore the similarities and differences in the barriers and motivations between the plastic and resins and food and beverages industries as these two industries are the major contributors of solid waste in Malaysia. Prior studies are lacking with regard to explaining the barriers and motivations in solid waste management from the Malaysian context. This study is focused on 10 firms from the plastics and resins industry and 9 from the food and beverages industry in Malaysia. Through Rasch measurement theory, the results indicate that the barriers of lack of skills and qualifications and lack of closed-loop control and the motivations of cost savings and a business model are performed differently. The findings further confirm that the lack of skills and qualifications is a more difficult barrier to overcome than the lack of closed-loop control, while the motivation factor of a business model is more difficult to achieve than cost savings. In terms of practical contribution, this study provides results that can help policy makers in Malaysia to close the gaps present regarding the adoption of solid waste management practices and to devise appropriate incentives. The study also supports managers of companies in regard to working on the most pressing hindering and promoting factors in the field of solid waste management.
Collapse
Affiliation(s)
- Mansoor Ahmed Soomro
- Teesside University International Business School, Middlesbrough, TS1 3BX Tees Valley UK
| | - Mohd Helmi Ali
- UKM-Graduate School of Business, Universiti Kebangsaan Malaysia, 43000 Bangi, Selangor Malaysia
| | - Suhaiza Zailani
- Department of Management, Faculty of Business and Economics, Universiti Malaya, 50403 Lembah Pantai, Kuala Lumpur, Malaysia
| | - Ming-Lang Tseng
- Institute of Innovation and Circular Economy, Asia University, Taichung City, Taiwan
- Department of Medical Research, China Medical University Hospital, China Medical University, Taichung City, Taiwan
- Ramon V. Del Rosario College of Business, De La Salle University, Manila, Philippines
| | - Zafir Mohd Makhbul
- UKM-Graduate School of Business, Universiti Kebangsaan Malaysia, 43000 Bangi, Selangor Malaysia
| |
Collapse
|
15
|
Regularized Mixture Rasch Model. INFORMATION 2022. [DOI: 10.3390/info13110534] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
The mixture Rasch model is a popular mixture model for analyzing multivariate binary data. The drawback of this model is that the number of estimated parameters substantially increases with an increasing number of latent classes, which, in turn, hinders the interpretability of model parameters. This article proposes regularized estimation of the mixture Rasch model that imposes some sparsity structure on class-specific item difficulties. We illustrate the feasibility of the proposed modeling approach by means of one simulation study and two simulated case studies.
Collapse
|
16
|
A Machine Learning Approach to Assess Differential Item Functioning of the KINDL Quality of Life Questionnaire Across Children with and Without ADHD. Child Psychiatry Hum Dev 2022; 53:980-991. [PMID: 33963488 DOI: 10.1007/s10578-021-01179-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 04/25/2021] [Indexed: 10/21/2022]
Abstract
This study aimed to investigate differential item functioning (DIF) of the child and parent reports of the KINDL measure across children with and without Attention-deficit/hyperactivity disorder (ADHD). The sample included 122 children with ADHD and 1086 healthy peers, alongside 127 and 1061 of their parents, respectively. The generalized partial credit model with lasso penalization, as a machine learning method, was used to assess DIF of the KINDL across the two groups. The findings showed that three out of 24 items of the child reports and seven out of 24 items of the parent reports of the KINDL exhibited DIF between children with and without ADHD. Accordingly, Iranian children with and without ADHD along with their parents perceive almost all items in the KINDL similarly. Hence, the observed difference in quality of life scores between children with and without ADHD is a real difference and not a reflection of measurement bias.
Collapse
|
17
|
A Machine Learning Approach to Assess Differential Item Functioning in Psychometric Questionnaires Using the Elastic Net Regularized Ordinal Logistic Regression in Small Sample Size Groups. BIOMED RESEARCH INTERNATIONAL 2021; 2021:6854477. [PMID: 34957307 PMCID: PMC8695002 DOI: 10.1155/2021/6854477] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/23/2021] [Accepted: 11/29/2021] [Indexed: 11/18/2022]
Abstract
Assessing differential item functioning (DIF) using the ordinal logistic regression (OLR) model highly depends on the asymptotic sampling distribution of the maximum likelihood (ML) estimators. The ML estimation method, which is often used to estimate the parameters of the OLR model for DIF detection, may be substantially biased with small samples. This study is aimed at proposing a new application of the elastic net regularized OLR model, as a special type of machine learning method, for assessing DIF between two groups with small samples. Accordingly, a simulation study was conducted to compare the powers and type I error rates of the regularized and nonregularized OLR models in detecting DIF under various conditions including moderate and severe magnitudes of DIF (DIF = 0.4 and 0.8), sample size (N), sample size ratio (R), scale length (I), and weighting parameter (w). The simulation results revealed that for I = 5 and regardless of R, the elastic net regularized OLR model with w = 0.1, as compared with the nonregularized OLR model, increased the power of detecting moderate uniform DIF (DIF = 0.4) approximately 35% and 21% for N = 100 and 150, respectively. Moreover, for I = 10 and severe uniform DIF (DIF = 0.8), the average power of the elastic net regularized OLR model with 0.03 ≤ w ≤ 0.06, as compared with the nonregularized OLR model, increased approximately 29.3% and 11.2% for N = 100 and 150, respectively. In these cases, the type I error rates of the regularized and nonregularized OLR models were below or close to the nominal level of 0.05. In general, this simulation study showed that the elastic net regularized OLR model outperformed the nonregularized OLR model especially in extremely small sample size groups. Furthermore, the present research provided a guideline and some recommendations for researchers who conduct DIF studies with small sample sizes.
Collapse
|
18
|
Resnik LJ, Borgia ML, Clark MA, Graczyk E, Segil J, Ni P. Structural validity and reliability of the patient experience measure: A new approach to assessing psychosocial experience of upper limb prosthesis users. PLoS One 2021; 16:e0261865. [PMID: 34962943 PMCID: PMC8714100 DOI: 10.1371/journal.pone.0261865] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2021] [Accepted: 12/10/2021] [Indexed: 11/18/2022] Open
Abstract
Recent advances in upper limb prosthetics include sensory restoration techniques and osseointegration technology that introduce additional risks, higher costs, and longer periods of rehabilitation. To inform regulatory and clinical decision making, validated patient reported outcome measures are required to understand the relative benefits of these interventions. The Patient Experience Measure (PEM) was developed to quantify psychosocial outcomes for research studies on sensory-enabled upper limb prostheses. While the PEM was responsive to changes in prosthesis experience in prior studies, its psychometric properties had not been assessed. Here, the PEM was examined for structural validity and reliability across a large sample of people with upper limb loss (n = 677). The PEM was modified and tested in three phases: initial refinement and cognitive testing, pilot testing, and field testing. Exploratory factor analysis (EFA) was used to discover the underlying factor structure of the PEM items and confirmatory factor analysis (CFA) verified the structure. Rasch partial credit modeling evaluated monotonicity, fit, and magnitude of differential item functioning by age, sex, and prosthesis use for all scales. EFA resulted in a seven-factor solution that was reduced to the following six scales after CFA: social interaction, self-efficacy, embodiment, intuitiveness, wellbeing, and self-consciousness. After removal of two items during Rasch analyses, the overall model fit was acceptable (CFI = 0.973, TLI = 0.979, RMSEA = 0.038). The social interaction, self-efficacy and embodiment scales had strong person reliability (0.81, 0.80 and 0.77), Cronbach's alpha (0.90, 0.80 and 0.71), and intraclass correlation coefficients (0.82, 0.85 and 0.74), respectively. The large sample size and use of contemporary measurement methods enabled identification of unidimensional constructs, differential item functioning by participant characteristics, and the rank ordering of the difficulty of each item in the scales. The PEM enables quantification of critical psychosocial impacts of advanced prosthetic technologies and provides a rigorous foundation for future studies of clinical and prosthetic interventions.
Collapse
Affiliation(s)
- Linda J. Resnik
- Research Department, Providence VA Medical Center, Providence, RI, United States of America
- Health Services, Policy and Practice, Brown University, Providence, RI, United States of America
| | - Mathew L. Borgia
- Research Department, Providence VA Medical Center, Providence, RI, United States of America
| | - Melissa A. Clark
- Health Services, Policy and Practice, Brown University, Providence, RI, United States of America
- University of Massachusetts Medical School, Worcester, Massachusetts, United States of America
| | - Emily Graczyk
- Department of Biomedical Engineering, Case Western Reserve University, Cleveland, OH, United States of America
- Research Department, Louis Stokes Cleveland VA Medical Center, Cleveland, OH, United States of America
| | - Jacob Segil
- Research Department, Rocky Mountain Regional VA Medical Center, Aurora, CO, United States of America
| | - Pengsheng Ni
- Boston University, Boston, Massachusetts, United States of America
| |
Collapse
|
19
|
Garcia JM, Gallagher MW, O’Bryant SE, Medina LD. Differential item functioning of the Beck Anxiety Inventory in a rural, multi-ethnic cohort. J Affect Disord 2021; 293:36-42. [PMID: 34166907 PMCID: PMC8349838 DOI: 10.1016/j.jad.2021.06.005] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 06/02/2021] [Accepted: 06/05/2021] [Indexed: 10/21/2022]
Abstract
BACKGROUND Evaluating measurement bias is vital to ensure equivalent assessment across diverse groups. One approach for evaluating test bias, differential item functioning (DIF), assesses item-level bias across specified groups by comparing item-level responses between groups that have the same overall score. Previous DIF studies of the Beck Anxiety Inventory (BAI) have only assessed bias across age, sex, and disease duration in monolingual samples. We expand this literature through DIF analysis of the BAI across age, sex, education, ethnicity, cognitive status, and test language. METHODS BAI data from a sample (n = 527, mean age=61.4 ± 12.7, mean education=10.9 ± 4.3, 69.3% female, 41.9% Hispanic/Latin American) from rural communities in West Texas, USA were analyzed. Item response theory (IRT) / logistic ordinal regression DIF was conducted across dichotomized demographic grouping factors. The Mann-Whitney U test and Hedge's g standardized mean differences were calculated before and after adjusting for the impact of DIF. RESULTS Significant DIF was demonstrated in 10/21 items. An adverse impact of DIF was not identified when demographics were assessed individually. Adverse DIF was identified for only one participant (1/527, 0.2%) when all demographics were aggregated. LIMITATIONS These results might not be generalizable to a sample with broader racial representation, more severe cognitive impairment, and higher levels of anxiety. CONCLUSIONS Minimal item-level bias was identified across demographic factors considered. These results support prior evidence that the BAI is valid for assessing anxiety across age and sex while contributing new evidence of its clinical relevance across education, ethnicity, cognitive status, and English/Spanish test language.
Collapse
Affiliation(s)
- Joshua M. Garcia
- University of Houston, Department of Psychology, Houston, TX, USA
| | | | - Sid E. O’Bryant
- University of North Texas Health Science Center, Graduate School of Biomedical Sciences, Fort Worth, TX, USA
| | - Luis D. Medina
- University of Houston, Department of Psychology, Houston, TX, USA,Corresponding Author. Luis D. Medina, PhD, Department of Psychology, University of Houston 3695 Cullen Blvd, Rm 126 Heyne, Houston, TX 77204-5022, Voice: 713.743.9318,
| |
Collapse
|
20
|
Yuan KH, Liu H, Han Y. Differential Item Functioning Analysis Without A Priori Information on Anchor Items: QQ Plots and Graphical Test. PSYCHOMETRIKA 2021; 86:345-377. [PMID: 33656627 DOI: 10.1007/s11336-021-09746-5] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/23/2019] [Revised: 12/30/2020] [Accepted: 01/08/2021] [Indexed: 06/12/2023]
Abstract
Differential item functioning (DIF) analysis is an important step in establishing the validity of measurements. Most traditional methods for DIF analysis use an item-by-item strategy via anchor items that are assumed DIF-free. If anchor items are flawed, these methods will yield misleading results due to biased scales. In this article, based on the fact that the item's relative change of difficulty difference (RCD) does not depend on the mean ability of individual groups, a new DIF detection method (RCD-DIF) is proposed by comparing the observed differences against those with simulated data that are known DIF-free. The RCD-DIF method consists of a D-QQ (quantile quantile) plot that permits the identification of internal references points (similar to anchor items), a RCD-QQ plot that facilitates visual examination of DIF, and a RCD graphical test that synchronizes DIF analysis at the test level with that at the item level via confidence intervals on individual items. The RCD procedure visually reveals the overall pattern of DIF in the test and the size of DIF for each item and is expected to work properly even when the majority of the items possess DIF and the DIF pattern is unbalanced. Results of two simulation studies indicate that the RCD graphical test has Type I error rate comparable to those of existing methods but with greater power.
Collapse
Affiliation(s)
- Ke-Hai Yuan
- Department of Psychology, University of Notre Dame, Notre Dame, IN, 46656, USA
| | - Hongyun Liu
- Faculty of Psychology, Beijing Normal University, No. 19, XinJieKouWai St., HaiDian District, Beijing , 100875, People's Republic of China.
| | - Yuting Han
- Faculty of Psychology, Beijing Normal University, No. 19, XinJieKouWai St., HaiDian District, Beijing , 100875, People's Republic of China
| |
Collapse
|
21
|
Wijayanto F, Mul K, Groot P, van Engelen BG, Heskes T. Semi-automated Rasch analysis using in-plus-out-of-questionnaire log likelihood. THE BRITISH JOURNAL OF MATHEMATICAL AND STATISTICAL PSYCHOLOGY 2021; 74:313-339. [PMID: 32857418 PMCID: PMC8246875 DOI: 10.1111/bmsp.12218] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/16/2019] [Revised: 07/09/2020] [Indexed: 06/11/2023]
Abstract
Rasch analysis is a popular statistical tool for developing and validating instruments that aim to measure human performance, attitudes and perceptions. Despite the availability of various software packages, constructing a good instrument based on Rasch analysis is still considered to be a complex, labour-intensive task, requiring human expertise and rather subjective judgements along the way. In this paper we propose a semi-automated method for Rasch analysis based on first principles that reduces the need for human input. To this end, we introduce a novel criterion, called in-plus-out-of-questionnaire log likelihood (IPOQ-LL). On artificial data sets, we confirm that optimization of IPOQ-LL leads to the desired behaviour in the case of multi-dimensional and inhomogeneous surveys. On three publicly available real-world data sets, our method leads to instruments that are, for all practical purposes, indistinguishable from those obtained by Rasch analysis experts through a manual procedure.
Collapse
Affiliation(s)
- Feri Wijayanto
- Department of InformaticsUniversitas Islam IndonesiaYogyakartaIndonesia
- Institute for Computing and Information SciencesRadboud UniversityNijmegenThe Netherlands
| | - Karlien Mul
- Department of NeurologyDonders Institute for BrainCognition, and BehaviourRadboud University Medical CenterNijmegenThe Netherlands
| | - Perry Groot
- Institute for Computing and Information SciencesRadboud UniversityNijmegenThe Netherlands
| | - Baziel G.M. van Engelen
- Department of NeurologyDonders Institute for BrainCognition, and BehaviourRadboud University Medical CenterNijmegenThe Netherlands
| | - Tom Heskes
- Institute for Computing and Information SciencesRadboud UniversityNijmegenThe Netherlands
| |
Collapse
|
22
|
Abstract
The four-parameter logistic model is an Item Response Theory model for dichotomous items that limit the probability of giving a positive response to an item into a restricted range, so that even people at the extremes of a latent trait do not have a probability close to zero or one. Despite the literature acknowledging the usefulness of this model in certain contexts, the difficulty of estimating the item parameters has limited its use in practice. In this paper we propose a regularized estimation approach for the estimation of the item parameters based on the inclusion of a penalty term in the log-likelihood function. Simulation studies show the good performance of the proposal, which is further illustrated through an application to a real-data set.
Collapse
|
23
|
Abstract
The comparison of group means in latent variable models plays a vital role in empirical research in the social sciences. The present article discusses an extension of invariance alignment and Haberman linking by choosing the robust power loss function ρ(x)=|x|p(p>0). This power loss function with power values p smaller than one is particularly suited for item responses that are generated under partial invariance. For a general class of linking functions, asymptotic normality of estimates is shown. Moreover, the theory of M-estimation is applied for obtaining linking errors (i.e., inference with respect to a population of items) for this class of linking functions. In a simulation study, it is shown that invariance alignment and Haberman linking have comparable performance, and in some conditions, the newly proposed robust Haberman linking outperforms invariance alignment. In three examples, the influence of the choice of a particular linking function on the estimation of group means is demonstrated. It is concluded that the choice of the loss function in linking is related to structural assumptions about the pattern of noninvariance in item parameters.
Collapse
|