Strengthening the assessment of factorial invariance across population subgroups: a commentary on Varni et al. (2013)

Children's Hope in South Africa: A Population-Based Study

A growing body of research has provided evidence for the cognitive motivational construct of hope as a psychological strength, particularly for children in adverse social circumstances. In children, hope is defined as a set of cognitions focused on children's agency to contemplate workable goals, to identify pathways to achieve those goals and the intrinsic beliefs about their capacity to activate sustained movement toward those goals. Using data from the third wave of the Children's Worlds International Survey on Children's Well-Being, the study aimed to explore children's hope amongst a random population-based sample of children in South Africa. The study further aimed to explore children's level of hope across the nine provincial regions of South Africa. Data were collected using Snyder et al.'s (1997) Children's Hope Scale (CHS). Confirmatory factor analysis (CFA) was used to analyze the data, with multi-group CFA used to analyze the data across provincial regions. The study found an appropriate fit structure for the CHS using the overall pooled sample. The mean score on the CHS for the national sample was of 4.781 (SD = 1.082). Measurement invariance demonstrated the tenability of scalar invariance, which indicates comparability across correlations, regressions and mean scores. Mean scores ranged from 4.511 (SD = 1.163) for the Northern Cape to 4.982. (SD = 0.974) for the Western Cape. Five provinces (Eastern Cape, Northern Cape, Free State, Mpumalanga, and KwaZulu Natal) scored below the national mean, while four provinces (North West, Western Cape, Limpopo, and Gauteng) scored above.

Quality of Communication and Trust in Patients With Serious Illness: An Exploratory Study of the Relationships of Race/Ethnicity, Socioeconomic Status, and Religiosity

CONTEXT

Better understanding of clinicians' skill communicating with their patients and of patients' trust in clinicians is necessary to develop culturally sensitive palliative care interventions. Race/ethnicity, socioeconomic status, and religiosity have been documented as factors influencing quality of communication and trust.

OBJECTIVES

The objective of this study was to explore associations of seriously ill patients' race/ethnicity, socioeconomic status, and religiosity with patients' ratings of the quality of clinicians' communication and trust in clinicians.

METHODS

An observational analysis was performed using baseline data from a multicenter cluster-randomized trial of a communication intervention. We enrolled consecutive patients with chronic, life-limiting illnesses (n = 537) cared for by primary and specialty care clinicians (n = 128) between 2014 and 2016 in outpatient clinics in Seattle, Washington. We assessed patient demographics (age, gender, race/ethnicity, education, income, and self-rated health status), Duke University Religion Index, Quality of Communication Scale, and Wake Forest Physician Trust Scale. We used probit and linear regression and path analyses to examine associations.

RESULTS

Patients providing higher ratings of clinician communication included those belonging to racial/ethnic minority groups (P = 0.001), those with lower income (P = 0.008), and those with high religiosity/spirituality (P = 0.004). Higher trust in clinicians was associated with minority status (P = 0.018), lower education (P = 0.019), and clinician skill in communication (P < 0.001).

CONCLUSION

Contrary to prior studies, racial/ethnic minorities and patients with lower income rated communication higher and reported higher trust in their clinicians than white and higher income patients. More research is needed to identify and understand factors associated with quality communication and trust between seriously ill patients and clinicians to guide development of patient-centered palliative care communication interventions.

Measuring health literacy in community agencies: a Bayesian study of the factor structure and measurement invariance of the health literacy questionnaire (HLQ)

BACKGROUND

The development of the Health Literacy Questionnaire (HLQ), reported in 2013, attracted widespread international interest. While the original study samples were drawn from clinical and home-based aged-care settings, the HLQ was designed for the full range of healthcare contexts including community-based health promotion and support services. We report a follow-up study of the psychometric properties of the HLQ with respondents from a diverse range of community-based organisations with the principal goal of contributing to the development of a soundly validated evidence base for its use in community health settings.

METHODS

Data were provided by 813 clients of 8 community agencies in Victoria, Australia who were administered the HLQ during the needs assessment stage of the Ophelia project, a health literacy-based intervention. Most analyses were conducted using Bayesian structural equation modelling that enables rigorous analysis of data but with some relaxation of the restrictive requirements for zero cross-loadings and residual correlations of 'classical' confirmatory factor analysis. Scale homogeneity was investigated with one-factor models that allowed for the presence of small item residual correlations while discriminant validity was studied using the inter-factor correlations and factor loadings from a full 9-factor model with similar allowance for small residual correlations and cross-loadings. Measurement invariance was investigated scale-by-scale using a model that required strict invariance of item factor loadings, thresholds, residual variances and co-variances.

RESULTS

All HLQ scales were found to be homogenous with composite reliability ranging from 0.80 to 0.89. The factor structure of the HLQ was replicated and 6 of the 9 scales were found to exhibit clear-cut discriminant validity. With a small number of exceptions involving non-invariance of factor loadings, strict measurement invariance was established across the participating organisations and the gender, language background, age and educational level of respondents.

CONCLUSIONS

The HLQ is highly reliable, even with only 4 to 6 items per scale. It provides unbiased mean estimates of group differences across key demographic indicators. While measuring relatively narrow constructs, the 9 dimensions are clearly separate and therefore provide fine-grained data on the multidimensional area of health literacy. These analyses provide researchers, program managers and policymakers with a range of robust evidence by which they can make judgements about the appropriate use of the HLQ for their community-based setting.

Measuring Depression-Severity in Critically Ill Patients' Families with the Patient Health Questionnaire (PHQ): Tests for Unidimensionality and Longitudinal Measurement Invariance, with Implications for CONSORT

CONTEXT

Families of intensive care unit patients are at risk for depression and are important targets for depression-reducing interventions. Multi-item scores for evaluating such interventions should meet criteria for unidimensionality and longitudinal measurement invariance. The Patient Health Questionnaire (PHQ), widely used for measuring depression severity, provides standard nine-, eight-, and two-item scores. However, published studies often report no (or weak) evidence of these scores' unidimensionality/invariance, and no tests have evaluated them as measures of depression severity in intensive care unit patients' families.

OBJECTIVES

To identify multi-item PHQ constructs with promise for evaluating change in depression severity among family members of critically ill patients.

METHODS

Structural equation models with rigorous fit criterion (χ(2), P ≥ 0.05) tested the standard nine-, eight-, and two-item PHQ, and other item subsets, for unidimensionality and longitudinal invariance, using data from a trial evaluating an intervention to reduce depressive symptoms in family members.

RESULTS

Neither the standard nine-item nor the eight-item PHQ construct showed longitudinal invariance, although the standard two-item construct and other item subsets did.

CONCLUSION

The longer eight- and nine-item PHQ scores appear inappropriate for assessing depression severity in this population, with constructs based on smaller subsets of items being more promising targets for future trials. The Consolidated Standards of Reporting Trials requirement for prespecified trial outcomes is problematic because unidimensionality/invariance testing must occur after trial completion. Consolidated Standards of Reporting Trials could be strengthened by endorsing rigorous assessment of composite scores and encouraging use of the most appropriate substitute, should trial-based evidence challenge the legitimacy of prespecified multi-item scores.

Reflections on Partial Least Squares Path Modeling

The purpose of the present article is to take stock of a recent exchange in Organizational Research Methods between critics and proponents of partial least squares path modeling (PLS-PM). The two target articles were centered around six principal issues, namely whether PLS-PM: (a) can be truly characterized as a technique for structural equation modeling (SEM), (b) is able to correct for measurement error, (c) can be used to validate measurement models, (d) accommodates small sample sizes, (e) is able to provide null hypothesis tests for path coefficients, and (f) can be employed in an exploratory, model-building fashion. We summarize and elaborate further on the key arguments underlying the exchange, drawing from the broader methodological and statistical literature to offer additional thoughts concerning the utility of PLS-PM and ways in which the technique might be improved. We conclude with recommendations as to whether and how PLS-PM serves as a viable contender to SEM approaches for estimating and evaluating theoretical models.

Commentary on strengthening the assessment of factorial invariance across population subgroups: a commentary on Varni et al. (2013)

In his commentary on Varni et al.'s (Qual Life Res. doi: 10.1007/s11136-013-0370-4 , 2013) article, McIntosh (Qual Life Res. doi: 10.1007/s11136-013-0465-y , 2013) has two main arguments. First, we should have paid more attention to statistical tests (i.e., χ 2 values) instead of approximate fit indexes for our analysis, especially with the baseline model. Second, Bayesian methods are better than the frequentist methods we used in determining the model's invariance across age and gender groups. We believe that statistical tests do have a place in assessing model fit, but overemphasis on them, especially with larger sample sizes, can lead to errant decisions. Second, while we agree that Bayesian methods have the potential to contribute much to the field of assessing invariance, more development needs to be conducted before they can be widely utilized in assessing factorial invariance across groups.