The Use of Qualitative Methods to Improve Quantitative Measures of Health-Related Constructs

Unexpected lucidity in dementia: application of qualitative methods to develop an informant-reported lucidity measure

OBJECTIVES

Unexpected lucidity is a phenomenon of scientific, clinical, and psychological relevance to health professionals, to those who experience it, and their relatives. This paper describes qualitative methods used to develop an informant-based measure of lucidity episodes.

METHODS

The approach was refinement of the operationalization of the construct; review of seminal items, modification, and purification; and confirmation of the feasibility of reporting methodology. Modified focus groups were conducted with 20 staff and 10 family members using a web-based survey. Themes included reaction when hearing the term; words that come to mind; description of and first reaction to referenced or observed 'lucidity' events. Semi-structured cognitive interviews were conducted with 10 health professionals working with older adults with cognitive impairment. Data were extracted from Qualtrics or Microsoft 365 Word® for analysis using NVivo.

RESULTS

Conceptual issues, as well as issues regarding comprehension, interpretation, clarity, semantics, and standardization of definitions derived from an external advisory board, focus groups and cognitive interviews informed items' modification, and resulted in the final lucidity measure.

CONCLUSIONS

An obstacle to understanding the mechanisms and estimating the prevalence of lucid events among individuals with dementia and other neurological conditions is the scarcity of reliable and valid measures. The substantive and varied data gathered from multiple methods including the collaborative work of an External Advisory Board, modified focus groups with staff and family caregivers, and structured cognitive interviews with health professionals were central in creating the revised version of the lucidity measure.

Development and validation of preconception care improvement scale (PCIS) in a resource-limited setting

BACKGROUND

Preconception care helps to close the gaps in a continuum of care. It is of paramount importance to reduce maternal and child adverse pregnancy outcomes, increase the utilization of services such as antenatal care, skilled delivery care, and post-natal care, and improve the lives of future generations. Therefore, a validated instrument is required. The purpose of this study was to develop and validate the preconception care improvement scale (PCIS) in a resource-limited setting.

METHODS

A mixed-method study was carried out from 02, March to 10, April 2019 in Manna district, Oromia region, Ethiopia to test the reliability and validity of the scale. Items were generated from literatures review, in-depth interviews with different individuals, and focused group discussions with women of reproductive age groups. A pretested structured questionnaire was used and a survey was conducted among 623 pregnant women in the district. The collected data were entered into EPI-data version 3.1 software and exported to SPSS version 23 software and data were analyzed for internal consistency and validity using reliability analysis and factor analysis.

RESULTS

The PCIS has 17 items loaded into six factors: Substance-related behaviors, screening for common non-communicable and infectious diseases, micronutrient supplementation and vaccination, seeking advice, decision and readiness for conception, and screening for sexually transmitted diseases. Factor analysis accounted for 67.51% of the observed variance. The internal consistency (Cronbach's alpha) of the scale was 0.776. Diversified participants of the qualitative study and experts' discussions assured the face and content validity of the scale. Factor loading indicated the convergent validity of the scale. Three of the PCIS subscale scores had a positive and significant association with the practice of preconception care and antenatal care visits, which confirmed the predictive validity of the scale.

CONCLUSION

The PCIS exhibited good reliability, face validity, content validity, convergent validity, and predictive validity. Thus, the scale is valid and helps to improve preconception care, especially in resource-limited settings.

Heart Attacks, Bloody Noses, and Other "Emotional Problems": Cultural and Conceptual Issues With the Spanish Translation of Self-Report Emotional Health Items

This article examines how respondents understood items in the Spanish versions of the Short-Form 36 (SF-36v2). Cognitive interviews of the SF-36 were conducted in 2 phases with 46 Spanish speakers living in the United States. Roughly one-third (17/46) of respondents had difficulty understanding the Role Emotional items upon their initial reading, and almost half (21/46) provided examples that were inconsistent with the intended meaning of the items. The findings of this study underscore the importance of conducting cognitive testing to ensure conceptual equivalence of any instrument regardless of how well validated it appears to be.

A measure of expectancies for alcohol analgesia: Preliminary factor analysis, reliability, and validity

Rates of alcohol consumption are substantially higher among persons with pain, and recent research has focused on elucidating bidirectional pain-alcohol effects. Expectancies for alcohol analgesia could influence the degree to which alcohol confers acute pain-relieving effects, and may amplify the propensity to respond to pain with drinking behavior. However, no validated measures of expectancies for alcohol analgesia are available. Therefore, we developed a five-item measure of Expectancies for Alcohol Analgesia (EAA), which assesses the perceived likelihood that alcohol will reduce pain. The goal of this project was to examine psychometric properties of the EAA among a sample of 273 current alcohol users with chronic pain (M_age = 32.9; 34% female) who completed an online survey of pain and substance use. Confirmatory factor analysis (CFA) results indicated that the hypothesized single-factor structure of the EAA provided good model fit (Bollen-Stine bootstrap p = .13). The EAA also showed excellent internal consistency (α = 0.97), and scores were positively associated with quantity/frequency of alcohol use, alcohol outcome expectancies, coping-related drinking motives, and pain severity (ps < 0.01). These findings provide initial support regarding the single-factor structure, reliability, and validity of the EAA. Examination of predictive utility and further validation are important next steps.

Qualitative Methodology in Cardiovascular Outcomes Research: A Contemporary Look

Qualitative research offers unique opportunities to contribute to cardiovascular outcomes research. Despite the growth in qualitative research over the last decade, outcomes investigators in cardiology still have relatively little guidance on when and how best to implement these methods in their investigations, leaving the full potential of these methods unrealized. We offer a contemporary look at qualitative methods, including publication trends of qualitative studies in cardiology journals from 1998 to 2018, novel emerging data collection and analytic methods, and current use and examples of cardiovascular outcomes research that apply qualitative methods such as user-centered design, preimplementation evaluation, implementation evaluation, effectiveness evaluation, and policy analysis.

Development and Spanish translation of the Weinberg Center Risk and Abuse Prevention Screen (WC-RAPS)

Barriers for enhanced detection, identification, and reporting of elder abuse include the paucity of appropriate, valid, easily administered screening tools. This article describes the qualitative methods used in the development of the Weinberg Center Risk and Abuse Prevention Screen (WC-RAPS), and of its Spanish version. Focus groups and cognitive interviews were instrumental in identifying problematic items, underscoring potential response errors, and informing about putative causes for divergent interpretations of item-intent. Seven of the 11 original items were modified, a double-barreled item was segregated into two, one item deleted, and three additional items included to create the final 13 WC-RAPS items. The multi-step approach implemented for the Spanish conversion evidenced deviation from the original intended meaning for one item. The readability for English and Spanish versions was also assessed. Screening for elder abuse, if implemented systematically can be instrumental in identifying unrecognized abuse and preventing reoccurrence.

Using Qualitative Methods to Assess the Conceptual Equivalence of the Spanish and English Stroke Action Test (STAT)

Members of racial/ethnic minorities, in contrast to majority groups, experience disparities in stroke risk factors, recognition, evaluation, and treatment. This article describes the use of qualitative methods to examine the cultural and lifestyle appropriateness, clarity of item-wording, comprehension of item intent, and conceptual equivalence of the developed Spanish and the revised English Stroke Action Test (which includes three additional atypical stroke symptoms more common in women). Thirty in-depth cognitive interviews were conducted (10 in Spanish) using concurrent structured probes. The desired ultimate outcome was to obtain conceptually equivalent measures in both languages. Four sources of variability in comprehension were identified: unfamiliar and/or idiosyncratic Spanish words used in translation; phrases in the original (English) and in the literal (Spanish) translation that were not understood; and unclear intended meaning of the original (English) items. Cognitive interviews helped identify problematic items, highlighted potential response errors, and provided insight regarding putative causes for inconsistent interpretation.

Using qualitative methods to develop a measure of resident-to-resident elder mistreatment in nursing homes

BACKGROUND

Despite expansion of research on elder mistreatment, limited attention has been paid to the development of improved measurement instruments. This gap is particularly notable regarding measurement of mistreatment in long-term care facilities. This paper demonstrates the value of qualitative methods used in item development of a Resident-to-Resident Elder Mistreatment (R-REM) measure for use in nursing homes and other care facilities. It describes the development strategy and the modification and refinement of items using a variety of qualitative methods.

METHODS

A combination of qualitative methods was used to develop close-ended items to measure R-REM, including review by a panel of experts, focus groups, and in-depth cognitive interviews.

RESULTS

Information gathered from the multiple methods aided in flagging problematic items, helped to highlight the nature of the problems in measures, and provided suggestions for item modification and improvement.

CONCLUSIONS

The method employed is potentially useful for future attempts to develop better measures of elder mistreatment. The employment of previously established measurement items drawn from related fields, modified through an intensive qualitative research strategy, is an effective strategy to improve elder mistreatment measurement.

A framework for understanding modifications to measures for diverse populations

OBJECTIVE

Research on health disparities and determinants of health disparities among ethnic minorities and vulnerable older populations necessitates use of self-report measures. Most established instruments were developed on mainstream populations and may need adaptation for research with diverse populations. Although information is increasingly available on various problems using these measures in diverse groups, there is little guidance on how to modify the measures. A framework of issues to consider when modifying measures for diverse populations is presented.

METHODS

The authors describe reasons for considering modifications, the types of information that can be used as a basis for making modifications, and the types of modifications researchers have made. Recommendations are made to test modified measures to assure their appropriateness, and suggestions are provided on reporting modifications in publications using the measures.

DISCUSSION

The issues open a dialogue about what appropriate guidelines would be for researchers adapting measures in studies of ethnically diverse populations.

Modifying measures based on differential item functioning (DIF) impact analyses

OBJECTIVE

Measure modification can impact comparability of scores across groups and settings. Changes in items can affect the percent admitting to a symptom.

METHODS

Using item response theory (IRT) methods, well-calibrated items can be used interchangeably, and the exact same item does not have to be administered to each respondent, theoretically permitting wider latitude in terms of modification.

RESULTS

Recommendations regarding modifications vary, depending on the use of the measure. In the context of research, adjustments can be made at the analytic level by freeing and fixing parameters based on findings of differential item functioning (DIF). The consequences of DIF for clinical decision making depend on whether or not the patient's performance level approaches the scale decision cutpoint. High-stakes testing may require item removal or separate calibrations to ensure accurate assessment.

DISCUSSION

Guidelines for modification based on DIF analyses and illustrations of the impact of adjustments are presented.

Cognitive testing of physical activity and acculturation questions in recent and long-term Latino immigrants

Background

We ascertained the degree to which language (English versus Spanish), and residence time in the US influence responses to survey questions concerning two topics: self-reported acculturation status, and recent physical activity (PA). This topic is likely to be of general interest because of growing numbers of immigrants in countries worldwide.

Methods

We carried out qualitative (cognitive) interviews of survey items on acculturation and physical activity on 27 Latino subjects from three groups: (a) In Spanish, of those of low residence time (less than five years living in the U.S.) (n = 9); (b) In Spanish, of those of high residence time (15 or more years in the U.S) (n = 9); and (c) in English, of those of high residence time (n = 9).

Results

There were very few language translation problems; general question design defects and socio-cultural challenges to survey responses were more common. Problems were found for both acculturation and PA questions, with distinct problem types for the two question areas. Residence time/language group was weakly associated with overall frequency of problems observed: low residence time/Spanish (86%), high residence time/Spanish (67%), and English speaking groups (62%).

Conclusions

Standardized survey questions related to acculturation and physical activity present somewhat different cognitive challenges. For PA related questions, problems with such questions were similar regardless of subject residence time or language preference. For acculturation related questions, residence time/language or education level influenced responses to such questions. These observations should help in the interpretation of survey results for culturally diverse populations.

Overview of Quantitative Measurement Methods

BACKGROUND

Reviewed in this article are issues relating to the study of invariance and differential item functioning (DIF). The aim of factor analyses and DIF, in the context of invariance testing, is the examination of group differences in item response conditional on an estimate of disability. Discussed are parameters and statistics that are not invariant and cannot be compared validly in crosscultural studies with varying distributions of disability in contrast to those that can be compared (if the model assumptions are met) because they are produced by models such as linear and nonlinear regression.

OBJECTIVES

The purpose of this overview is to provide an integrated approach to the quantitative methods used in this special issue to examine measurement equivalence. The methods include classical test theory (CTT), factor analytic, and parametric and nonparametric approaches to DIF detection. Also included in the quantitative section is a discussion of item banking and computerized adaptive testing (CAT).

METHODS

Factorial invariance and the articles discussing this topic are introduced. A brief overview of the DIF methods presented in the quantitative section of the special issue is provided together with a discussion of ways in which DIF analyses and examination of invariance using factor models may be complementary.

CONCLUSIONS

Although factor analytic and DIF detection methods share features, they provide unique information and can be viewed as complementary in informing about measurement equivalence.