The reliability of psychological instruments in community samples: A cautionary note

German questionnaires assessing quality of life and psycho-social status in people with epilepsy: Reliable change and intercorrelations

OBJECTIVES

People with epilepsy (PWE) not only suffer from seizures but also from various psycho-social issues containing facets such as social functioning, anxiety, depression or stigmatization, and consequently quality of life. (1) Assessing reliable change of these issues is crucial to evaluate their course and potential treatment effects. As most psycho-social self-report questionnaires have been validated in separate samples, their clinical-socio-demographic differences may limit the comparability and generalizability of the scales' internal consistency, which is important for the reliable change index (RCI). Using a co-normalized approach, we provide the internal consistency and RCIs for a large set of questionnaires targeting quality of life (QOLIE-31-P), depressive symptoms (NDDI-E), anxiety (GAD-7), seizure severity (LSSS), subjective antiseizure medication adverse events (LAEP), stigma, epilepsy-related fear, and restrictions in daily life (PESOS), and subjective cognition (FLei). As for some German versions of these measures, psychometric data is still missing, we also add important information for the German language area. (2) In addition, knowledge about intercorrelations of these constructs is needed to shape questionnaire usage and treatment approaches. We thus investigate associations of these scales and compare weighted and unweighted subscales of the QOLIE-31-P.

METHODS

In our prospective study, 202 adult in-patients of the Epilepsy-Center Berlin-Brandenburg with a reliable diagnosis of epilepsy filled out a set of self-report questionnaires between 03/2018 and 03/2021. We calculated Cronbach's α, RCIs, and bivariate intercorrelations and compared the respective correlations of weighted and unweighted scales of the QOLIE-31-P.

RESULTS

For most of the scales, good to excellent internal consistency was identified. Furthermore, we found intercorrelations in the expected directions with strong links between scales assessing similar constructs (e.g., QOLIE-31-P Cognition and FLei), but weak relationships between measures for different constructs (e.g., QOLIE-31-P Seizure worry and FLei). The QOLIE-31-P Total score was highly correlated with most of the other scales. Some differences regarding their correlational patterns for weighted and unweighted QOLIE-31-P scales were identified.

CONCLUSIONS

Psycho-social constructs share a large amount of common variance, but still can be separated from each other. The QOLIE-31-P Total score represents an adequate measure of general psycho-social burden.

Health Literacy, Education, and Internal Consistency of Psychological Scales

Background:

Measurement error might lead to biased estimates, causing ineffective interventions and service delivery. Identifying measurement error of health-related instruments helps develop accurate assessment of health-related constructs.

Objective:

We compared the internal consistency of eight psychological scales used in health research in groups with adequate versus limited health literacy and in groups with higher versus lower education.

Methods:

Participants (N = 1,005) from a nationally representative internet panel completed eight self-report scales: (1) information avoidance, (2) cognitive causation, (3) unpredictability, (4) perceived severity, (5) time orientation, (6) internal health locus of control, (7) need for cognition, and (8) social desirability. The first four assess beliefs about diabetes and colon cancer. We used the Newest Vital Sign to categorize participants' health literacy (limited vs. adequate). We also categorized participants' education (high school or less vs. more than high school). We compared the Cronbach's alpha for each psychological scale between groups with different health literacy and education levels using the Feldt test.

Key Results:

Among all the 13 subscales, scale internal consistency was significantly lower among people with limited health literacy than those with adequate health literacy for five subscales: information avoidance for colon cancer (0.80 vs. 0.88), unpredictability of diabetes (0.84 vs. 0.88), perceived severity for diabetes (0.66 vs. 0.75), need for cognition (0.63 vs. 0.82), and social desirability (0.52 vs. 0.68). Internal consistency was significantly lower among people who had a high school education or less than among those with more than a high school education for four scales: perceived severity of diabetes (0.70 vs. 0.75), present orientation (0.60 vs. 0.66), need for cognition (0.73 vs. 0.80), and social desirability (0.61 vs. 0.70).

Conclusions:

Several psychological instruments demonstrated significantly lower internal consistency when used in a sample with limited health literacy or education. To advance health disparities research, we need to develop new scales with alternative conceptualizations of the constructs to produce a measure that is reliable among multiple populations. [HLRP: Health Literacy Research and Practice. 2021;5(3):e244–e255.]

Plain Language Summary:

We compared the internal consistency of several psychological scales in groups with adequate versus limited health literacy and higher versus lower education. For several scales, internal consistency was significantly lower among (1) people with limited health literacy compared those who have adequate health literacy and/or (2) people who had a high school education or less compared to those with more than a high school education.

Psychometric Properties of the Nine-Item Problematic Internet Use Questionnaire (PIUQ-9) in a Lithuanian Sample of Students

Objectives: To date, there is no reliable instrument which could be used to assess problematic Internet use (PIU) in Lithuania. The nine-item Problematic Internet Use Questionnaire (PIUQ-9) previously validated in multiple countries, could be a potential tool for measuring PIU severity. The main objective of the present study was to explore the psychometric properties of the Lithuanian version of the questionnaire. Methods: A total of 272 students (17% men, mean age 27 ± 9 years) completed the PIUQ-9, the Patient Health Questionnaire (PHQ) and answered questions about the impairment of daily functioning caused by PIU in an online survey. Results: A confirmatory factor analysis indicated that a bi-factor model with one general factor "general problem" and two-specific factors "obsession" and "neglect + control disorder" fitted the data well. The presence of a strong global factor was supported by the common variance index in the bi-factor model indicating that the "general problem" factor explained 67.7% of common variance. The multiple indicators multiple causes (MIMIC) model showed that psychiatric symptoms (β = 0.25) had a moderate, while impairment due to PIU (β = 0.41) had a moderate-to-strong direct effect on the factor "general problem" supporting the construct validity of the scale. Conclusion: The Lithuanian version of the PIUQ-9 has appropriate psychometric properties to be used in measuring PIU severity in student samples.

Threat sensitivity is associated with the healthcare source used most often: doctor's office, emergency room, or none at all

A significant proportion of American adults do not have a regular source of healthcare and the reasons for this shortfall are not fully understood. The objective of this study was to examine the relationship between individual differences in threat sensitivity and healthcare utilization in a survey of 483 African American men. Demographics, psychological characteristics, and health behaviors were assessed. The primary outcomes were: 1) most frequent source of healthcare utilization (doctor's office or clinic vs. emergency room vs. no place), and 2) frequency of healthcare utilization (one or more vs. no healthcare visits in the previous year). Data were analyzed with multivariable logistic regression. Results showed that threat sensitivity, insurance status, and age were associated with the most frequent source of healthcare utilization. Compared to men who most commonly used a doctor's office or clinic, men who tended to use an emergency room had higher levels of threat sensitivity and those with no usual healthcare source had lower levels of threat sensitivity. These findings fit with leading neurobiological theories of personality regarding threat sensitivity. From a pragmatic standpoint, these findings may also lend insight to the tailoring of health marketing messages designed to optimize utilization of healthcare resources.

Belief in a Loving Versus Punitive God and Behavior

We examined the extent to which beliefs in a loving God and punitive God correspond with self-reported behavior in an online, longitudinal survey involving three waves of data collection, each separated by 6 months. Adolescents (N = 760) reported the extent to which they believed in a loving God and a punitive God (Times 1 and 3) and reported their engagement in benevolent (helping and forgiveness) and aggressive behavior (Times 2 and 3). Participants strongly endorsed a loving God but not a punitive God. In addition, belief in a loving God corresponded with reports of less aggressive and more benevolent behavior, whereas belief in a punitive God corresponded with more aggressive and less benevolent behavior.

Regulatory Focus and Conspiratorial Perceptions: The Importance of Personal Control

We examine when and why people subscribe to conspiratorial beliefs, suggesting that promotion focus reduces conspiratorial perceptions by activating a sense of personal control. Study 1 established that individuals primed with promotion focus are less likely to perceive conspiracies than those in a baseline condition. However, individuals primed with prevention focus and those in a baseline condition did not differ in their levels of conspiratorial beliefs. Study 2 demonstrated that soldiers higher in promotion focus were less likely to endorse conspiracy theories because of their heightened sense of control; this relationship did not emerge for soldiers higher in prevention focus. Study 3 found that conspiratorial beliefs increased when individuals primed with promotion focus recalled personal control loss, whereas those primed with prevention focus were unaffected by personal control loss. Using measures and manipulations of regulatory focus and personal control, we establish when and why promotion focus reduces conspiracy theories.

Developing a scale to assess health regulatory focus

RATIONALE

Regulatory focus (i.e., focus on motivation to achieve gains or avoid losses and non-gains) is used to tailor health behavior change interventions, improving efficacy, but is currently assessed by scales that are not health-specific and may capture a version of the construct that is not ideally matched to the rationale for tailoring.

OBJECTIVE

We developed and validated a Health Regulatory Focus Scale (HRFS), which assesses tendencies to avoid negative health consequences (prevention focus) or achieve positive health outcomes (promotion focus).

METHODS

Across four studies (and a scale development study in supplementary online materials), we established convergent, discriminant, and predictive validity for the HRFS. In studies examining predictive validity, main outcome measures were health behavior intentions, including intentions to reduce alcohol use, quit smoking, eat a healthy diet, exercise, be screened for cancer, and engage in general cancer preventive behaviors.

RESULTS

The promotion and prevention sub-scales performed well in confirmatory factor analyses. Single-factor models had significantly poorer fit than models delineating promotion and prevention. The sub-scales were differentially (and only modestly) correlated with related constructs (anxiety, optimism, information avoidance, ambiguity/fatalism). Higher HRFS-Promotion focus generally corresponded with greater health behavior intentions. Conversely, higher HRFS-Prevention focus corresponded with lower health behavior intentions. Associations were largely maintained even when controlling for established regulatory focus measures, supporting the assertion that the HRFS would predict unique variance in health behavior intentions.

CONCLUSION

The HRFS has the potential to improve the precision with which framed health messages change behavior, as it may assess a version of regulatory focus that is more ideally matched to rationale for tailoring interventions.

Measurement invariance of intuitive cancer risk perceptions across diverse populations: The Cognitive Causation and Negative Affect in Risk scales

Intuitive cancer risk perceptions may inform strategies to motivate cancer prevention behaviors. This study evaluated factor structure and measurement invariance of two new measures of intuitive cancer risk, the Cognitive Causation and Negative Affect in Risk scales. Single- and multiple-group confirmatory factor analysis models were fit to responses from three diverse samples. The confirmatory factor analysis models fit the data well, with all comparative fit indices (CFI) ≥ 0.94. Items flagged by chi-square difference tests as potentially non-invariant were largely invariant between samples according to practical fit indices (e.g. ΔCFI). These novel scales may be particularly relevant in diverse, underserved populations.

Predicting Scheduling and Attending for an Oral Cancer Examination

BACKGROUND

Oral and pharyngeal cancer is highly treatable if diagnosed early, yet late diagnosis is commonplace apparently because of delays in undergoing an oral cancer examination.

PURPOSE

We explored predictors of scheduling and attending an oral cancer examination among a sample of Black and White men who were at high risk for oral cancer because they smoked.

METHODS

During an in-person interview, participants (N = 315) from rural Florida learned about oral and pharyngeal cancer, completed survey measures, and were offered a free examination in the next week. Later, participants received a follow-up phone call to explore why they did or did not attend their examination.

RESULTS

Consistent with the notion that scheduling and attending an oral cancer exam represent distinct decisions, we found that the two outcomes had different predictors. Defensive avoidance and exam efficacy predicted scheduling an examination; exam efficacy and having coping resources, time, and transportation predicted attending the examination. Open-ended responses revealed that the dominant reasons participants offered for missing a scheduled examination were conflicting obligations, forgetting, and confusion or misunderstanding about the examination.

CONCLUSIONS

The results suggest interventions to increase scheduling and attending an oral cancer examination.