A Practical Guide for Item Generation in Measure Development: Insights From the Development of a Patient-Reported Experience Measure of Compassion

Initial Validation of a Patient-Reported Compassion Measure in a Mandarin-Speaking Long-Term Care Patient Population

Background and Purpose: Compassion is increasingly considered essential to quality nursing care and is a concept that is strongly embedded across cultures, including Chinese culture. The Patient Compassion Model (PCM) depicts the unique yet overlapping domains of compassion. The Sinclair Compassion Questionnaire (SCQ) was directly developed and validated from this empirical model. In this study, we sought to establish initial validation of a translated SCQ among Mandarin-speaking patients by assessing the transferability of the PCM and the clinical sensibility of the SCQ. Methods: Forward and back-translation of the PCM and SCQ were performed in accordance with World Health Organization guidelines. Qualitative interviews were used to assess the transferability of the PCM with conceptualizations of compassion within a Chinese context. Cognitive interviews were conducted to assess the clarity, readability, wording, questions, and response scales of the Mandarin translation of the SCQ. Qualitative data were analyzed using constant comparative analysis, and cognitive interviews were analyzed using framework analysis. Results: The original categories of the PCM were verified in this Mandarin-speaking patient population. Specifically, participants' understanding of compassion is described as consisting of healthcare provider virtues, emphasizing the importance of a virtuous response that sought to understand the individual and their unique needs, to relationally communicate from a place of shared humanity and to ameliorate suffering. Participants were able to answer, comprehend, and endorse all 15 Mandarin SCQ items, resulting in no modifications to the Mandarin SCQ. Conclusions: This study provides initial validation of the Mandarin SCQ and PCM. Future studies should consider further establishing the validity and reliability of the Mandarin SCQ among a larger Chinese patient population.

Adaptation and validation of a patient-reported compassion measure in the Spanish population: The Spanish version of the Sinclair Compassion Questionnaire (SCQesp)

International practice guidelines and policies recognize compassion as a fundamental dimension of quality care. A key element in enhancing compassion in healthcare settings is having reliable patient-reported experience measures. In the Spanish context, there is a need for a valid Spanish patient-reported compassion measure for use in both research and clinical practice. The Sinclair Compassion Questionnaire (SCQ) represents the gold standard for patient-reported compassion measures in English-speaking settings. The primary aim of this study is to culturally adapt and validate the SCQ in a Spanish population. A Spanish version of the SCQ (SCQesp) was used to collect data from 303 Spanish patients (in two contexts: hospitalized and medical visit). Confirmatory factor analysis confirmed a one-factor solution in the 15-item (SCQesp) and five-item (SCQesp-SF) short form version. The SCQesp showed excellent values of reliability: Cronbach's α = 0.98; composite variance = 0.98 (0.905-0.854); and stratified variance = 0.78. The SCQesp-SF showed similar values of reliability. The SCQesp has excellent psychometric properties, making it a valid and reliable measure for assessing compassion in healthcare research and clinical care. This scientifically rigorous and psychometrically robust compassion measure in Spanish could allow healthcare providers, researchers, and leaders to routinely assess compassion.

Developing and exploring the validity of a patient reported experience measure for adult inpatient diabetes care

AIM

To develop and explore the validity of a Patient Reported Experience Measure (PREM) for adult inpatient diabetes care.

METHOD

27 in-depth interviews were conducted to inform the development of the 42-item PREM which was cognitively tested with 10 people. A refined 38-item PREM was piloted with 228 respondents completing a paper (n = 198) or online (n = 30) version. The performance of the PREM was evaluated by exploring (i) uptake/number of responses and (ii) survey validity by investigating whether the PREM data were of adequate quality and delivered useful information.

RESULTS

The PREM had low drop-out or missing data rates suggesting it was appropriately constructed. Analysis of item frequencies and variances, and problem score calculations concluded that questions provided sufficient score differentiation.

CONCLUSIONS

This new PREM allows for experiences of inpatient diabetes care to be measured, understood and reported on to help identify priority areas for improving care quality.

Tools for measuring client experiences and satisfaction with healthcare in low- and middle-income countries: a systematic review of measurement properties

BACKGROUND

Perspectives of patients as clients on healthcare offer unique insights into the process and outcomes of care and can facilitate improvements in the quality of services. Differences in the tools used to measure these perspectives often reflect differences in the conceptualization of quality of care and personal experiences. This systematic review assesses the validity and reliability of instruments measuring client experiences and satisfaction with healthcare in low- and middle-income countries (LMICs).

METHODS

We performed a systematic search of studies published in PubMed, SCOPUS, and CINAHL. This review was reported according to the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guidelines. Studies describing the development and psychometric properties of client experience and satisfaction with general health care were included in the review. Critical appraisal of study design was undertaken using the Appraisal tool for Cross-Sectional Studies (AXIS). The Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist and Terwee's criteria were used to appraise the psychometric properties of the included studies. A narrative synthesis approach was used in the interpretation of the findings.

RESULTS

Of the 7470 records identified, 12 studies with 14 corresponding instruments met the inclusion criteria and were included in the final review. No study assessed all the psychometric properties highlighted by the COSMIN criteria. In most instruments, we found evidence that initial development work incorporated client participation. The most evaluated measurement properties were content validity, internal consistency, and structural validity. Measurement error and responsiveness were not reported in any study.

CONCLUSION

Reliability and validity should be considered important elements when choosing or developing an instrument for professionals seeking an effective instrument for use within the population. Our review identified limitations in the psychometric properties of patient experience and satisfaction instruments, and none met all methodological quality standards. Future studies should focus on further developing and testing available measures for their effectiveness in clinical practice. Furthermore, the development of new instruments should incorporate clients' views and be rigorously tested or validated in studies with high methodological quality.

TRIAL REGISTRATION

CRD42020150438.

What is the Most Valid and Reliable Compassion Measure in Healthcare? An Updated Comprehensive and Critical Review

Our previous review of compassion measures in healthcare between 1985 and 2016 concluded that no available measure assessed compassion in healthcare in a comprehensive or methodologically rigorous fashion. The present study provided a comparative review of the design and psychometric properties of recently updated or newly published compassion measures. The search strategy of our previous review was replicated. PubMed, MEDLINE, CINAHL, and PsycINFO databases and grey literature were searched to identify studies that reported information on instruments that measure compassion or compassionate care in clinicians, physicians, nurses, healthcare students, and patients. Textual qualitative descriptions of included studies were prepared. Instruments were evaluated using the Evaluating Measures of Patient-Reported Outcomes (EMPRO) tool. Measures that underwent additional testing since our last review included the Compassion Competence Scale (CCS), the Compassionate Care Assessment Tool (CCAT)^©, and the Schwartz Center Compassionate Care Scale (SCCCS)™. New compassion measures included the Sussex-Oxford Compassion for Others Scale (SOCS-O), a self-report measure of compassion for others; the Bolton Compassion Strengths Indicators (BSCI), a self-report measure of the characteristics (strengths) associated with a compassionate nurse; a five-item Tool to Measure Patient Assessment of Clinician Compassion (TMPACC); and the Sinclair Compassion Questionnaire (SCQ). The SCQ was the only measure that adhered to measure development guidelines, established initial construct validity by first defining the concept of interest, and included the patient perspective across all stages of development. The SCQ had the highest EMPRO overall score at 58.1, almost 9 points higher than any other compassion measure, and achieved perfect EMPRO subscale scores for internal consistency, reliability, validity, and respondent burden, which were up to 43 points higher than any other compassion measure. These findings establish the SCQ as the ‘gold standard’ compassion measure, providing an empirical basis for evaluations of compassion in routine care.

Detecting the patient’s need for help with machine learning based on expressions

Background

Developing machine learning models to support health analytics requires increased understanding about statistical properties of self-rated expression statements used in health-related communication and decision making. To address this, our current research analyzes self-rated expression statements concerning the coronavirus COVID-19 epidemic and with a new methodology identifies how statistically significant differences between groups of respondents can be linked to machine learning results.

Methods

A quantitative cross-sectional study gathering the “need for help” ratings for twenty health-related expression statements concerning the coronavirus epidemic on an 11-point Likert scale, and nine answers about the person’s health and wellbeing, sex and age. The study involved online respondents between 30 May and 3 August 2020 recruited from Finnish patient and disabled people’s organizations, other health-related organizations and professionals, and educational institutions (n = 673). We propose and experimentally motivate a new methodology of influence analysis concerning machine learning to be applied for evaluating how machine learning results depend on and are influenced by various properties of the data which are identified with traditional statistical methods.

Results

We found statistically significant Kendall rank-correlations and high cosine similarity values between various health-related expression statement pairs concerning the “need for help” ratings and a background question pair. With tests of Wilcoxon rank-sum, Kruskal-Wallis and one-way analysis of variance (ANOVA) between groups we identified statistically significant rating differences for several health-related expression statements in respect to groupings based on the answer values of background questions, such as the ratings of suspecting to have the coronavirus infection and having it depending on the estimated health condition, quality of life and sex. Our new methodology enabled us to identify how statistically significant rating differences were linked to machine learning results thus helping to develop better human-understandable machine learning models.

Conclusions

The self-rated “need for help” concerning health-related expression statements differs statistically significantly depending on the person’s background information, such as his/her estimated health condition, quality of life and sex. With our new methodology statistically significant rating differences can be linked to machine learning results thus enabling to develop better machine learning to identify, interpret and address the patient’s needs for well-personalized care.

Development and validation of a patient-reported measure of compassion in healthcare: the Sinclair Compassion Questionnaire (SCQ)

OBJECTIVES

Compassion is a key indicator of quality care that is reportedly eroding from patients' care experience. While the need to assess compassion is recognised, valid and reliable measures are lacking. This study developed and validated a clinically informed, psychometrically rigorous, patient-reported compassion measure.

DESIGN

Data were collected from participants living with life-limiting illnesses over two study phases across four care settings (acute care, hospice, long term care (LTC) and homecare). In phase 1, data were analysed through exploratory factor analysis (EFA), with the final items analysed via confirmatory factor analysis (CFA) in phase 2. The Schwartz Center Compassionate Care Scale (SCCCS), the revised Edmonton Symptom Assessment Scale (ESAS-r) and Picker Patient Experience Questionnaire (PPEQ) were also administered in phase 2 to assess convergent and divergent validity.

SETTING AND PARTICIPANTS

633 participants were recruited over two study phases. In the EFA phase, a 54-item version of the measure was administered to 303 participants, with 330 participants being administered the final 15-item measure in the CFA phase.

RESULTS

Both EFA and CFA confirmed compassion as a single factor construct with factor loadings for the 15-item measure ranging from 0.76 to 0.86, with excellent test-retest reliability (intraclass correlation coefficient range: 0.74-0.89) and excellent internal reliability (Cronbach's alpha of 0.96). The measure was positively correlated with the SCCCS (r=0.75, p<0.001) and PPEQ (r=0.60, p<0.001). Participants reporting higher experiences of compassion had significantly greater well-being and lower depression on the ESAS-r. Patients in acute care and hospice reported significantly greater experiences of compassion than LTC residents.

CONCLUSIONS

There is strong initial psychometric evidence for the Sinclair Compassion Questionnaire (SCQ) as a valid and reliable patient-reported compassion measure. The SCQ provides healthcare providers, settings and administrators the means to routinely measure patients experiences of compassion, while providing researchers a robust measure to conduct high-quality research.

Initial Validation of a Patient-Reported Measure of Compassion: Determining the Content Validity and Clinical Sensibility among Patients Living with a Life-Limiting and Incurable Illness

BACKGROUND

Although compassionate care is considered a cornerstone of quality palliative care, there is a paucity of valid and reliable measures to study, assess, and evaluate how patients experience compassion/compassionate care in their care.

OBJECTIVE

The aim was to develop a patient-reported compassion measure for use in research and clinical practice with established content-related validity evidence for the items, question stems, and response scale.

METHODS

Content validation for an initial 109 items was conducted through a two-round modified Delphi technique, followed by cognitive interviews with patients. A panel of international Subject Matter Experts (SMEs) and a Patient Advisory Group (PAG) assessed the items for their relevancy to their associated domain of compassion, yielding an Item-level Content Validity Index (I-CVI), which was used to determine content modifications. The SMEs and the PAG also provided narrative feedback on the clarity, flow, and wording of the instructions, questions, and response scale, with items being modified accordingly. Cognitive interviews were conducted with 16 patients to further assess the clarity, comprehensibility, and readability of each item within the revised item pool.

RESULTS

The first round of the Delphi review produced an overall CVI of 72% among SMEs and 80% among the PAG for the 109 items. Delphi panelists then reviewed a revised measure containing 84 items, generating an overall CVI of 84% for SMEs and 86% for the PAG. Sixty-eight items underwent further testing via cognitive interviews with patients, resulting in an additional 14 items being removed.

CONCLUSIONS

Having established this initial validity evidence, further testing to assess internal consistency, test-retest reliability, factor structure, and relationships to other variables is required to produce the first valid, reliable, and clinically informed patient-reported measure of compassion.