Patient engagement in designing, conducting, and disseminating clinical pain research: IMMPACT recommended considerations

ABSTRACT

In the traditional clinical research model, patients are typically involved only as participants. However, there has been a shift in recent years highlighting the value and contributions that patients bring as members of the research team, across the clinical research lifecycle. It is becoming increasingly evident that to develop research that is both meaningful to people who have the targeted condition and is feasible, there are important benefits of involving patients in the planning, conduct, and dissemination of research from its earliest stages. In fact, research funders and regulatory agencies are now explicitly encouraging, and sometimes requiring, that patients are engaged as partners in research. Although this approach has become commonplace in some fields of clinical research, it remains the exception in clinical pain research. As such, the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials convened a meeting with patient partners and international representatives from academia, patient advocacy groups, government regulatory agencies, research funding organizations, academic journals, and the biopharmaceutical industry to develop consensus recommendations for advancing patient engagement in all stages of clinical pain research in an effective and purposeful manner. This article summarizes the results of this meeting and offers considerations for meaningful and authentic engagement of patient partners in clinical pain research, including recommendations for representation, timing, continuous engagement, measurement, reporting, and research dissemination.

Researchers, patients, and other stakeholders' perspectives on challenges to and strategies for engagement

BACKGROUND

There is growing interest in patient and stakeholder engagement in research, yet limited evidence about effective methods. Since 2012, the Patient-Centered Outcomes Research Institute (PCORI) has funded patient-centered comparative effectiveness research with a requirement for engaging patients and other stakeholders as research partners in study planning, conduct, and dissemination. This requirement, unique among large healthcare research funders in the US, provides an opportunity to learn about challenges encountered and specific strategies used by PCORI-funded study teams. The primary objective of this study is to describe -- from the perspective of PCORI investigators and research partners-the most common engagement challenges encountered in the first two years of the projects and promising strategies to prevent and overcome these challenges.

METHODS

Descriptive information about investigators, partners, and their engagement was collected from investigators via annual (N = 235) and mid-year (N = 40) project progress reporting to PCORI, and from their partners (N = 260) via voluntary survey. Qualitative data were analyzed using content and thematic analyses.

RESULTS

Investigators and partners most often described engagement challenges in three domains: (1) infrastructure to support engagement, (2) building relationships, and (3) maintaining relationships. Infrastructure challenges related to financial and human resources, including funding support and dedicated staff, identifying diverse groups of partners, and partners' logistical needs. Challenges for both building and maintaining relationships encompass a variety of aspects of authentic, positive interactions that facilitate mutual understanding, full participation, and genuine influence on the projects. Strategies to prevent or mitigate engagement challenges also corresponded overall to the same three domains. Both groups typically described strategies more generally, with applicability to a range of challenges rather than specific actions to address only particular challenges.

CONCLUSION

Meaningful engagement of patients and other stakeholders comes with challenges, as does any innovation in the research process. The challenges and promising practices identified by these investigators and partners, related to engagement infrastructure and the building and maintenance of relationships, reveal actionable areas to improve engagement, including organizational policies and resources, training, new engagement models, and supporting engagement by viewing it as an investment in research uptake and impact.

What motivates patients and caregivers to engage in health research and how engagement affects their lives: Qualitative survey findings

Background

US research organizations increasingly are supporting patient and stakeholder engagement in health research with a goal of producing more useful, relevant and patient‐centered evidence better aligned with real‐world clinical needs. The Patient‐Centered Outcomes Research Institute (PCORI) engages patients, family caregivers and other health‐care stakeholders, including clinicians, payers and policymakers, as active partners in prioritizing, designing, conducting and disseminating research as a key strategy to produce useful evidence for health‐care decision making.

Objective

To inform effective engagement practices and policies, we sought to understand what motivates patients and caregivers to engage as partners on PCORI‐funded research projects and how such engagement changed their lives.

Methods

We conducted thematic analysis of open‐ended survey responses from 255 patients, family caregivers and individuals from advocacy and community‐based organizations who engaged as partners on 139 PCORI‐funded research projects focusing on a range of health conditions.

Results

Partners' motivations for engaging in research were oriented primarily towards benefiting others, including a desire to improve patients' lives and to support effective health‐care interventions. In addition to feeling they made a positive difference, many partners reported direct benefits from engagement, such as new relationships and improved health habits.

Discussion and Conclusions

By identifying patient and caregiver motivations for engaging in research partnerships and what they get out of the experience, our study may help research teams and organizations attract partners and foster more satisfying and sustainable partnerships. Our findings also add to evidence that engagement benefits the people involved as partners, strengthening the case for more widespread engagement.

Examining the role of funders in ensuring value and reducing waste in research: An organizational case-study of the Patient-Centered Outcomes Research Institute

International experts have recommended actions that funders can take to improve the value of research investments. They state that self-assessment and public sharing are the basis for accountability and improvement. We examined our policies and practice to determine the extent to which the Patient-Centered Outcomes Research Institute's (PCORI) policies and practices as a research funder align with international best practice recommendations. A self-audit of current policies and practice against 17 recommendations and 35 sub-recommendations representing five major stages of research production, based on adapted methods used for self-assessment by another funder, was performed.  Fit of existing PCORI policies and practices with 35 sub-recommendations, qualitative assessment of adequacy (area of strength; area of partial strength; area of growth; not applicable) for 17 recommendations for five stages of research production was assessed. Of the 17 recommendations, 15 were applicable to PCORI's research mission and focus.  PCORI has policies and practices in place for all elements of six recommendations ("area of strength") and policies that address each element but with some still in active development for three ("area of partial strength"). PCORI is partially addressing six of the 15 relevant recommendations ("area of growth"). Areas for growth include making study protocols publicly available, improving policies on data sharing, and enhancing collaboration with other funders to reduce redundant funding. A voluntary consortium of international funders is underway to encourage further progress, including additional self-assessment and public sharing for accountability. These findings indicate PCORI has undertaken efforts to align its funding practices with international recommendations to ensure the value of public dollars invested in research.  Further efforts will likely require additional coordination and collaboration between funders and stakeholders.

Examining the role of funders in ensuring value and reducing waste in research: An organizational case-study of the Patient-Centered Outcomes Research Institute

International experts have recommended actions that funders can take to improve the value of research investments. They state that self-assessment and public sharing are the basis for accountability and improvement. We examined our policies and practice to determine the extent to which the Patient-Centered Outcomes Research Institute's (PCORI) policies and practices as a research funder align with international best practice recommendations. A self-audit of current policies and practice against 17 recommendations and 35 sub-recommendations representing five major stages of research production, based on adapted methods used for self-assessment by another funder, was performed.  Fit of existing PCORI policies and practices with 35 sub-recommendations, qualitative assessment of adequacy (area of strength; area of partial strength; area of growth; not applicable) for 17 recommendations for five stages of research production was assessed. Of the 17 recommendations, 15 were applicable to PCORI's research mission and focus.  PCORI has policies and practices in place for all elements of six recommendations ("area of strength") and policies that address each element but with some still in active development for three ("area of partial strength"). PCORI is partially addressing six of the 15 relevant recommendations ("area of growth"). Areas for growth include making study protocols publicly available, improving policies on data sharing, and enhancing collaboration with other funders to reduce redundant funding. A voluntary consortium of international funders is underway to encourage further progress, including additional self-assessment and public sharing for accountability. These findings indicate PCORI has undertaken efforts to align its funding practices with international recommendations to ensure the value of public dollars invested in research.  Further efforts will likely require additional coordination and collaboration between funders and stakeholders.

Researchers, Patients, and Stakeholders Evaluating Comparative-Effectiveness Research: A Mixed-Methods Study of the PCORI Reviewer Experience

OBJECTIVES

The Patient-Centered Outcomes Research Institute (PCORI) includes patients and stakeholders alongside scientists in reviewing research applications using unique review criteria including patient-centeredness and patient and/or stakeholder engagement. To support extension of this unique collaborative model to other funders, information from the reviewers on the review process is needed to understand how scientists and nonscientists evaluate research proposals together. Thus, this study aimed to describe reviewers' perspectives of the interactions during the in-person review panel; to examine the value and challenges of including scientists, patients, and stakeholders together; and to understand the perceived importance of PCORI's review criteria.

METHODS

This study utilized anonymous, cross-sectional surveys (N = 925 respondents from 5 funding cycles: 470 scientists, 217 patients, 238 stakeholders; survey completion rates by cycle: 70-89%) and group interviews (N = 18).

RESULTS

Reviewers of all types describe PCORI Merit Review as respectful, balanced, and one of reciprocal influence among different reviewer types. Reviewers indicate strong support and value of input from all reviewer types, receptivity to input from others, and the panel chair's incorporation of all views. Patients and stakeholders provide real-world perspectives on importance to patients, research partnership plans, and study feasibility. Challenges included concerns about a lack of technical expertise of patient/stakeholder reviewers and about scientists dominating conversations. The most important criterion for assigning final review scores was technical merit-either alone or in conjunction with patient-centeredness or patient/ stakeholder engagement.

CONCLUSIONS

PCORI Merit Reviewers' self-reports indicate that the perspectives of different reviewer types are influential in panel discussions and Merit Review outcomes.

Unique Review Criteria and Patient and Stakeholder Reviewers: Analysis of PCORI's Approach to Research Funding

OBJECTIVE

The Patient-Centered Outcomes Research Institute (PCORI) uses a unique approach to Merit Review that includes patients and stakeholders as reviewers with scientists, and includes unique review criteria (patient-centeredness and active engagement of end users in the research). This study assessed the extent to which different reviewer types influence review scores and funding outcomes, the emphasis placed on technical merit compared to other criteria by a multistakeholder panel, and the impact of the in-person discussion on agreement among different reviewer types.

METHODS

Cross-sectional analysis of administrative data from PCORI online and in-person Merit Review (N = 1312 applications from the five funding cycles from November 2013 to August 2015). Linear and logistic regression models were used to analyze the data.

RESULTS

For all reviewer types, final review scores were associated with at least one review criterion score from each of the three reviewer types. The strongest predictor of final overall scores for all reviewer types was scientists' prediscussion ratings of technical merit. All reviewers' prediscussion ratings of the potential to improve health care and outcomes, and scientists' ratings of technical merit and patient-centeredness, were associated with funding success. For each reviewer type, overall impact scores from the online scoring were changed on at least half of the applications at the in-person panel discussion. Score agreement across reviewer types was greater after panel discussion.

CONCLUSIONS

Scientist, patient, and stakeholder views all contribute to PCORI Merit Review of applications for research funding. Technical merit is critical to funding success but patient and stakeholder ratings of other criteria also influence application disposition.

Patient, caregiver and clinician use of comparative effectiveness research findings in care decisions: results from a national study

AIM

To assess awareness, use and attitudes concerning comparative effectiveness research (CER) findings.

MATERIALS & METHODS

Online surveys of patients and caregivers managing rare (n = 560 patients, n = 609 caregivers) or chronic conditions (n = 762 patients, n = 776 caregivers), and practicing clinicians (n = 638).

RESULTS

Less than half of patients and caregivers reported exposure to any type of CER findings in the past 12 months. Of those, over half identify healthcare professionals as the information source, yet only 10% of clinicians indicated they were 'very familiar' with CER. Clinicians were concerned about the time required to find relevant evidence and the appropriateness of comparisons in available health research.

CONCLUSION

Clinicians, patients and caregivers indicate unmet need for incorporating CER research findings into clinical decision-making.

Health researcher views on comparative effectiveness research and research engagement

AIM

To understand researcher capability for and interest in patient-centered comparative effectiveness research (PC-CER), particularly related to engaging with patients/caregivers.

MATERIALS & METHODS

Web-based survey of 508 health researchers recruited via professional health research organizations.

RESULTS

Most respondents (94%) were familiar with CER and many (69%) reported having previously conducting some form of CER. Most respondents were familiar with (81%) and interested in (87%) partnering with patients and/or caregivers in research. Resources to assist in training, coordination of partners, guidance in apply for funding and improved infrastructure were commonly cited factors that would help researchers conduct PC-CER.

CONCLUSION

There is a significant opportunity for researchers to engage patients and caregivers as partners in CER. Researchers recognize the need for additional training and expertise to leverage those opportunities.

Patient, caregiver and clinician views on engagement in comparative effectiveness research

AIM

Describe patient, caregiver and clinician views toward engagement as partners in health research.

MATERIALS & METHODS

Online surveys of patients and caregivers managing rare (n = 560 patients, n = 609 caregivers) or chronic conditions (n = 762 patients, n = 776 caregivers) and practicing clinicians (n = 638).

RESULTS

Over half of respondents were unfamiliar with the concept of partnering with researchers but most expressed interest in working in a research partnership. Potential facilitators endorsed were ensuring research is meaningful, applying results in an understandable way, and sharing results. Lack of time is a potential barrier. Clinicians were most interested in helping researchers decide on intervention comparisons and identifying implications for clinical practice.

CONCLUSION

Patients, caregivers and clinicians are interested in research roles that emphasize usefulness and understandability of research.

Patient and Stakeholder Engagement in the PCORI Pilot Projects: Description and Lessons Learned

BACKGROUND

Patients and healthcare stakeholders are increasingly becoming engaged in the planning and conduct of biomedical research. However, limited research characterizes this process or its impact.

OBJECTIVE

We aimed to characterize patient and stakeholder engagement in the 50 Pilot Projects funded by the Patient-Centered Outcomes Research Institute (PCORI), and identify early contributions and lessons learned.

DESIGN

A self-report instrument was completed by researchers between 6 and 12 months following project initiation.

PARTICIPANTS

Forty-seven principal investigators or their designees (94 % response rate) participated in the study. MAIN MEASURES Self-report of types of stakeholders engaged, stages and levels of engagement, facilitators and barriers to engagement, lessons learned, and contributions from engagement were measured.

KEY RESULTS

Most (83 %) reported engaging more than one stakeholder in their project. Among those, the most commonly reported groups were patients (90 %), clinicians (87 %), health system representatives (44 %), caregivers (41 %), and advocacy organizations (41 %). Stakeholders were commonly involved in topic solicitation, question development, study design, and data collection. Many projects engaged stakeholders in data analysis, results interpretation, and dissemination. Commonly reported contributions included changes to project methods, outcomes or goals; improvement of measurement tools; and interpretation of qualitative data. Investigators often identified communication and shared leadership strategies as "critically important" facilitators (53 and 44 % respectively); lack of stakeholder time was the most commonly reported challenge (46 %). Most challenges were only partially resolved. Early lessons learned included the importance of continuous and genuine partnerships, strategic selection of stakeholders, and accommodation of stakeholders' practical needs.

CONCLUSIONS

PCORI Pilot Projects investigators report engaging a variety of stakeholders across many stages of research, with specific changes to their research attributed to engagement. This study identifies early lessons and barriers that should be addressed to facilitate engagement. While this research suggests potential impact of stakeholder engagement, systematic characterization and evaluation of engagement at multiple stages of research is needed to build the evidence base.

Cancer survivors' receipt of treatment summaries and implications for patient-centered communication and quality of care

OBJECTIVE

The Institute of Medicine recommends cancer survivors completing treatment be provided with a treatment summary to facilitate delivery of patient-centered survivorship care. However, the relationship between treatment summary receipt and patient-centered communication (PCC) and overall quality of care (QOC) are not well understood.

METHODS

Cancer survivors responding to the Health Information National Trends Survey reported treatment summary receipt, QOC, and experiences of six core functions of PCC. Multivariable logistic regression assessed the relationship between treatment summary receipt and PCC. The prevalence of survivors' treatment summary receipt and demographic/clinical characteristics predictive of treatment summary receipt were also assessed.

RESULTS

Of 359 respondents with a cancer history, 34.5% reported receiving a treatment summary. Greater treatment burden was associated with increased treatment summary receipt. Treatment summary receipt was associated with higher QOC and more PCC, both overall and for five of the six PCC functions.

CONCLUSION

The receipt of cancer treatment summaries may improve PCC and QOC for survivors.

PRACTICE IMPLICATIONS

The positive relationship between treatment summary receipt and survivors' PCC experience substantiates continued efforts to provide treatment summaries to survivors transitioning from active treatment to survivorship care. Future research should characterize mechanisms by which treatment summary provision may enhance PCC.

Patient and clinician views on comparative effectiveness research and engagement in research

AIMS

Describe and compare patient and primary care clinician attitudes about comparative effectiveness research (CER) and inclusion of patients and clinicians as partners in clinical research.

MATERIALS & METHODS

Online survey of patients (n = 900) and primary care clinicians (n = 750) from opt-in survey panels.

RESULTS

Nearly half (45%) of clinicians were not familiar with CER, but after reviewing a definition of CER, most (76%) agreed that it can improve patient care. Patients and clinicians indicated interest in engagement as research partners (66% and 55%, respectively). Lack of time was the most commonly endorsed barrier.

CONCLUSION

Attitudes toward CER and engagement in clinical research were generally favorable. Identified facilitators and barriers point to strategies for enhancing patient and clinician engagement in research.

Association between serious psychological distress and health care use and expenditures by cancer history

BACKGROUND

Serious psychological distress (SPD) is associated with adverse health outcomes such as poor quality of life and shorter survival in cancer survivors, but to the authors' knowledge, the relationship between SPD and health care use and medical expenditures is not clear.

METHODS

A total of 4326 cancer survivors and 57,109 noncancer participants were identified from the 2008 through 2010 Medical Expenditure Panel Survey, a nationwide population-based survey, and their psychological distress was assessed with the 6-item Kessler Psychological Distress Scale (SPD defined by a score ≥13). The association between SPD and use and medical expenditures of various types of health care (office-based, outpatient, hospital inpatient, emergency department, dental, and prescriptions) was examined using a 2-part modeling approach that adjusted for demographic, personal, and comorbidity factors. The marginal effects of SPD on health care use and expenditures were calculated for cancer survivors and were compared with those of noncancer participants.

RESULTS

The weighted prevalence of SPD in cancer survivors was 8.2% compared with 4.8% in the noncancer participants. SPD was significantly associated with higher use of all care types except dental care in cancer survivors. Cancer survivors with SPD spent $4431 (95% confidence interval, $3419-$5443) more than survivors without SPD on medical services each year, whereas this extra expenditure associated with SPD for participants without cancer was $2685 (95% confidence interval, $2099-$3271).

CONCLUSIONS

In a national representative sample of cancer survivors, SPD was found to be associated with higher health care use and medical expenditures. Distress screening and psychosocial care in cancer survivors may help reduce the economic burden of cancer in the United States.

Racial and ethnic disparities in patient-provider communication, quality-of-care ratings, and patient activation among long-term cancer survivors

PURPOSE

We examined racial and ethnic disparities in patient-provider communication (PPC), perceived care quality, and patient activation among long-term cancer survivors.

METHODS

In 2005 to 2006, survivors of breast, prostate, colorectal, ovarian, and endometrial cancers completed a mailed survey on cancer follow-up care. African American, Asian/Pacific Islander (Asian), Hispanic, and non-Hispanic white (white) survivors who had seen a physician for follow-up care in the past 2 years (n = 1,196) composed the analytic sample. We conducted linear and logistic regression analyses to identify racial and ethnic differences in PPC (overall communication and medical test communication), perceived care quality, and patient activation in clinical care (self-efficacy in medical decisions and perceived control). We further examined the potential contribution of PPC to racial and ethnic differences in perceived care quality and patient activation.

RESULTS

Compared with white survivors (mean score, 85.16), Hispanic (mean score, 79.95) and Asian (mean score, 76.55) survivors reported poorer overall communication (P = .04 and P < .001, respectively), and Asian survivors (mean score, 79.97) reported poorer medical test communication (P = .001). Asian survivors were less likely to report high care quality (odds ratio, 0.47; 95% CI, 0.30 to 0.72) and reported lower self-efficacy in medical decisions (mean score, 74.71; P < .001) compared with white survivors (mean score, 84.22). No disparity was found in perceived control. PPC was positively associated with care quality (P < .001) and self-efficacy (P < .001). After adjusting for PPC and other covariates, when compared with whites, Asian disparities remained significant.

CONCLUSION

Asian survivors report poorer follow-up care communication and care quality. More research is needed to identify contributing factors beyond PPC, such as cultural influences and medical system factors.

Engaging patients and stakeholders in research proposal review: the patient-centered outcomes research institute

The inaugural round of merit review for the Patient-Centered Outcomes Research Institute (PCORI) in November 2012 included patients and other stakeholders, as well as scientists. This article examines relationships among scores of the 3 reviewer types, changes in scoring after in-person discussion, and the effect of inclusion of patient and stakeholder reviewers on the review process. In the first phase, 363 scientists scored 480 applications. In the second phase, 59 scientists, 21 patients, and 31 stakeholders provided a "prediscussion" score and a final "postdiscussion" score after an in-person meeting for applications. Bland-Altman plots were used to characterize levels of agreement among and within reviewer types before and after discussion. Before discussion, there was little agreement among average scores given by the 4 lead scientific reviewers and patient and stakeholder reviewers. After discussion, the 4 primary reviewers showed mild convergence in their scores, and the 21-member panel came to a much stronger agreement. Of the 25 awards with the best (and lowest) scores after phase 2, only 13 had ranked in the top 25 after the phase 1 review by scientists. Five percent of the 480 proposals submitted were funded. The authors conclude that patient and stakeholder reviewers brought different perspectives to the review process but that in-person discussion led to closer agreement among reviewer types. It is not yet known whether these conclusions are generalizable to future rounds of peer review. Future work would benefit from additional data collection for evaluation purposes and from long-term evaluation of the effect on the funded research.

Follow-up care experiences and perceived quality of care among long-term survivors of breast, prostate, colorectal, and gynecologic cancers

INTRODUCTION

Post-treatment follow-up represents a crucial aspect of quality cancer care; however, data are lacking regarding follow-up care experiences, perception of provider involvement in care, and perceived quality of care from diverse samples of long-term survivors diagnosed as adults.

METHODS

Questionnaires were mailed in 2005 to 2006 to breast, prostate, colorectal, endometrial, and ovarian cancer survivors (4 to 14 years after diagnosis), sampled from California SEER cancer registries.

RESULTS

Most survivors (n = 1,490) reported recent follow-up care (68.7%), generally from oncology specialists only (47.4%) or shared between oncology and primary care providers (PCPs; 27.6%). Most survivors reported follow-up care advice (79.9%); fewer reported late-effects advice or receipt of a treatment summary (41.7% and 19.9%, respectively). Survivors who identified a PCP as their main follow-up care physician were as likely as those identifying an oncology specialist to rate their care as high quality (odds ratio [OR], 2.56; 95% CI, 0.98 to 6.74); however, survivors who could not identify a main follow-up care provider were less likely to report high-quality care (OR, 0.20; 95% CI, 0.08 to 0.50). Compared with follow-up care by an oncology specialist only, care by a PCP only was associated with a lower quality-of-care rating (OR, 0.34; 95% CI, 0.13 to 0.91), but there was no significant difference in quality rating by survivors when care was shared by an oncology specialist and PCP compared with an oncology specialist only.

CONCLUSION

Long-term survivors commonly report follow-up care years after their diagnosis; however, many patients' follow-up lacks important components. Care is more likely to be rated as high quality when one main provider is identified and an oncology specialist is involved.

Provision and discussion of survivorship care plans among cancer survivors: results of a nationally representative survey of oncologists and primary care physicians

PURPOSE

Survivorship care planning should involve discussions between providers and cancer survivors to address survivors' needs and optimize adherence. We examined the frequency and factors associated with oncologists' and primary care physicians' (PCPs) reports of provision of written survivorship care plans (SCPs) and discussion of survivorship care recommendations with survivors.

METHODS

A nationally representative sample of 1,130 oncologists and 1,020 PCPs was surveyed about survivorship care practices with survivors. Logistic regression models predicted multilevel factors associated with providing SCPs or discussing recommendations with survivors.

RESULTS

Although a majority of oncologists (64%) reported always/almost always discussing survivorship care recommendations with survivors, fewer also discussed who survivors should see for cancer-related and other follow-up care (32%); fewer still also provided a written SCP to the survivor (< 5%). Survivorship care recommendations and provider responsibility were not regularly discussed by PCPs and survivors (12%). Oncologists who reported detailed training about late and long-term effects of cancer were more likely to provide written SCPs (odds ratio [OR], 1.73; 95% CI, 1.22 to 2.44) and discuss survivorship care planning with survivors (OR, 2.02; 95% CI, 1.51 to 2.70). PCPs who received SCPs from oncologists were 9× more likely (95% CI, 5.74 to 14.82) to report survivorship discussions with survivors.

CONCLUSION

A minority of both PCPs and oncologists reported consistently discussing and providing SCPs to cancer survivors. Training and knowledge specific to survivorship care and coordinated care between PCPs and oncologists were associated with increased survivorship discussions with survivors. These nationally representative data provide a useful benchmark to assess implementation of new efforts to improve the follow-up care of survivors.

Social support, self-efficacy for decision-making, and follow-up care use in long-term cancer survivors

OBJECTIVE

Cancer survivors play an important role in coordinating their follow-up care and making treatment-related decisions. Little is known about how modifiable factors such as social support are associated with active participation in follow-up care. This study tests associations between social support, cancer-related follow-up care use, and self-efficacy for participation in decision-making related to follow-up care (SEDM). We also identified sociodemographic and clinical factors associated with social support among long-term survivors.

METHODS

The FOllow-up Care Use among Survivors study is a cross-sectional, population-based survey of breast, prostate, colon, and gynecologic cancer survivors (n=1522) 4-14 years post-diagnosis. Multivariable regression models were used to test associations between perceived social support (tangible and emotional/informational support modeled separately), follow-up care use (past 2 years), and SEDM, as well as to identify factors associated with perceived support.

RESULTS

Neither support type was associated with follow-up care use (all p>0.05), although marital status was uniquely, positively associated with follow-up care use (p<0.05). Both tangible support (B for a standard deviation increase (SE)=9.75(3.15), p<0.05) and emotional/informational support (B(SE)=12.61(3.05), p<0.001) were modestly associated with SEDM. Being married, having adequate financial resources, history of recurrence, and better perceived health status were associated with higher perceived tangible and emotional support (all p<0.05).

CONCLUSIONS

While perceived social support may facilitate survivor efficacy for participation in decision-making during cancer follow-up care, other factors, including marital satisfaction, appear to influence follow-up care use. Marital status and social support may be important factors to consider in survivorship care planning.

Role of oncologists and primary care physicians in providing follow-up care to non-Hodgkin lymphoma survivors within 5 years of diagnosis: a population-based study

PURPOSE

The growing population of non-Hodgkin lymphoma (NHL) survivors living longer with high physical and psychological treatment burden, in combination with the projected shortage of medical professionals, necessitates redesigning cancer follow-up care. This study examined follow-up care patterns, factors associated with follow-up care, and attitudes towards follow-up care among NHL survivors.

METHODS

We surveyed survivors of aggressive NHL 2 to 5 years post-diagnosis (N = 363) using a population-based sample from the Los Angeles County Surveillance Epidemiology and End Results registry.

RESULTS

Most survivors (82 %) received cancer-related follow-up care in the past year from an oncologist. History of recurrence, more comorbidities, more symptoms, and a shorter survivor-oncologist relationship were associated with high-frequency care with the oncologist [(≥5 visits in the past year), p < 0.05]. Many survivors followed up by oncologists (71 %) also saw a primary care provider (PCP) and 47 % also saw both a PCP and other specialists. Factors associated with seeing a PCP in addition to an oncologist included more symptoms, more health information needs, no history of recurrence, perceived excellent quality of cancer follow-up care, and fewer visits with the oncologist (p < 0.05). Survivors generally reported high reassurance from, and low negative anticipation towards, follow-up care.

CONCLUSIONS

The high proportion of NHL survivors receiving care from multiple physicians, and the sizable proportion (∼30 %) who have not recently seen a PCP, suggests that coordinating care across specialties is critical to ensure comprehensive, non-duplicative care. Understanding factors associated with cancer-related follow-up is a first step towards more effective, efficient, patient-centered care.

Use of survivorship care plans in the United States: associations with survivorship care

BACKGROUND

Survivorship care plans (SCPs), including a treatment summary and follow-up plan, intend to promote coordination of posttreatment cancer care; yet, little is known about the provision of these documents by oncologists to primary care physicians (PCPs). This study compared self-reported oncologist provision and PCP receipt of treatment summaries and follow-up plans, characterized oncologists who reported consistent provision of these documents to PCPs, and examined associations between PCP receipt of these documents and survivorship care.

METHODS

A nationally representative sample of medical oncologists (n = 1130) and PCPs (n = 1020) were surveyed regarding follow-up care for breast and colon cancer survivors. All statistical tests were two-sided. Multivariable regression models identified factors associated with oncologist provision of treatment summaries and SCPs to PCPs (always/almost always vs less frequent).

RESULTS

Nearly half of oncologists reported always/almost always providing treatment summaries, whereas 20.2% reported always/almost always providing SCPs (treatment summary + follow-up plan). Approximately one-third of PCPs indicated always/almost always receiving treatment summaries; 13.4% reported always/almost always receiving SCPs. Oncologists who reported training in late- and long-term effects of cancer and use of electronic medical records were more likely to report SCP provision (P < .05). PCP receipt of SCPs was associated with better PCP-reported care coordination, physician-physician communication, and confidence in survivorship care knowledge compared to receipt of neither treatment summaries nor SCPs (P < .05).

CONCLUSIONS

Providing SCPs to PCPs may enhance survivorship care coordination, physician-physician communication, and PCP confidence. However, considerable progress will be necessary to achieve implementation of sharing SCPs among oncologists and PCPs.

Are survivors who report cancer-related financial problems more likely to forgo or delay medical care?

BACKGROUND

Financial problems caused by cancer and its treatment can substantially affect survivors and their families and create barriers to seeking health care.

METHODS

The authors identified cancer survivors diagnosed as adults (n=1556) from the nationally representative 2010 National Health Interview Survey. Using multivariable logistic regression analyses, the authors report sociodemographic, clinical, and treatment-related factors associated with perceived cancer-related financial problems and the association between financial problems and forgoing or delaying health care because of cost. Adjusted percentages using the predictive marginals method are presented.

RESULTS

Cancer-related financial problems were reported by 31.8% (95% confidence interval, 29.3%-34.5%) of survivors. Factors found to be significantly associated with cancer-related financial problems in survivors included younger age at diagnosis, minority race/ethnicity, history of chemotherapy or radiation treatment, recurrence or multiple cancers, and shorter time from diagnosis. After adjustment for covariates, respondents who reported financial problems were more likely to report delaying (18.3% vs 7.4%) or forgoing overall medical care (13.8% vs 5.0%), prescription medications (14.2% vs 7.6%), dental care (19.8% vs 8.3%), eyeglasses (13.9% vs 5.8%), and mental health care (3.9% vs 1.6%) than their counterparts without financial problems (all P<.05).

CONCLUSIONS

Cancer-related financial problems are not only disproportionately represented in survivors who are younger, members of a minority group, and have a higher treatment burden, but may also contribute to survivors forgoing or delaying medical care after cancer.

Lost productivity and burden of illness in cancer survivors with and without other chronic conditions

BACKGROUND

Cancer survivors may experience long-term and late effects from treatment that adversely affect health and limit functioning. Few studies examine lost productivity and disease burden in cancer survivors compared with individuals who have other chronic conditions or by cancer type.

METHODS

We identified 4960 cancer survivors and 64,431 other individuals from the 2008-2010 Medical Expenditure Panel Survey and compared multiple measures of disease burden, including health status and lost productivity, between conditions and by cancer site for cancer survivors. All analyses controlled for the effects of age, sex, race/ethnicity, and number of comorbid conditions.

RESULTS

Overall, in adjusted analyses in multiple models, cancer survivors with another chronic disease (heart disease or diabetes) experienced higher levels of burden compared with individuals with a history of cancer only, chronic disease only, and neither cancer, heart disease, nor diabetes across multiple measures (P < .05). Among cancer survivors, individuals with short survival cancers and multiple cancers consistently had the highest levels of burden across multiple measures (P < .0001).

CONCLUSIONS

Cancer survivors who have another chronic disease experience more limitations and higher levels of burden across multiple measures. Limitations are particularly severe in cancer survivors with short survival cancer and multiple cancers.

Changes among US Cancer Survivors: Comparing Demographic, Diagnostic, and Health Care Findings from the 1992 and 2010 National Health Interview Surveys

Background. Differences in healthcare and cancer treatment for cancer survivors in the United States (US) have not been routinely examined in nationally representative samples or studied before and after important Institute of Medicine (IOM) recommendations calling for higher quality care provision and attention to comprehensive cancer care for cancer survivors. Methods. To assess differences between survivor characteristics in 1992 and 2010, we conducted descriptive analyses of 1992 and 2010 National Health Interview Survey (NHIS) data. Our study sample consisted of 1018 self-reported cancer survivors from the 1992 NHIS and 1718 self-reported cancer survivors from the 2010 NHIS who completed the Cancer Control (CCS) and Cancer Epidemiology (CES) Supplements. Results. The prevalence of reported survivors increased from 1992 to 2010 (4.2% versus 6.3%). From 1992 to 2010, there was an increase in long-term cancer survivors and a drop in multiple malignancies, and surgery remained the most widely used treatment. Significantly fewer survivors (<10 years after diagnosis) were denied insurance coverage. Survivors continue to report low participation in counseling or support groups. Conclusions. As the prevalence of cancer survivors continues to grow, monitoring differences in survivor characteristics can be useful in evaluating the effects of policy recommendations and the quality of clinical care.

Receipt of psychosocial care among cancer survivors in the United States

PURPOSE

Given the importance of psychosocial care for cancer survivors, this study used population-based data to characterize survivors who reported a discussion with health care provider(s) about the psychosocial effects of cancer and who reported using professional counseling or support groups (PCSG) and tested associations between receipt of psychosocial care and satisfaction with care.

PATIENTS AND METHODS

We examined survivors of adult cancers from the 2010 National Health Interview Survey (N = 1,777). Multivariable logistic regression models examined factors associated with receipt of and satisfaction with psychosocial care.

RESULTS

Most survivors (55.1%) reported neither provider discussions nor use of PCSG; 31.4% reported provider discussion only, 4.4% reported use of PCSG only, and 8.9% reported both. Non-Hispanic blacks (v non-Hispanic whites), married survivors, survivors of breast cancer (v prostate or less prevalent cancers), those treated with chemotherapy, and survivors reporting past research study/clinical trial participation were more likely to report provider discussion(s) (P < .01). Hispanics (v non-Hispanic whites), survivors age 40 to 49 years (v ≤ 39 years), survivors of breast cancer (v melanoma or less prevalent cancers), those diagnosed ≤ 1 year ago (v > 5 years ago), survivors treated with radiation, and past research participants were more likely to report use of PCSG (P < .05). Survivors reporting any psychosocial care were more likely to be "very satisfied" with how their needs were met (P < .001).

CONCLUSION

Many survivors do not report a discussion with providers about the psychosocial effects of cancer, which reflects a missed opportunity to connect survivors to psychosocial services. These data can benchmark the success of efforts to improve access to cancer-related psychosocial care.

Cancer survivorship research in Europe and the United States: where have we been, where are we going, and what can we learn from each other?

The growing number of cancer survivors worldwide has led to of the emergence of diverse survivorship movements in the United States and Europe. Understanding the evolution of cancer survivorship within the context of different political and health care systems is important for identifying the future steps that need to be taken and collaborations needed to promote research among and enhance the care of those living after cancer. The authors first review the history of survivorship internationally and important related events in both the United States and Europe. Lessons learned from survivorship research are then broadly discussed, followed by examination of the infrastructure needed to sustain and advance this work, including platforms for research, assessment tools, and vehicles for the dissemination of findings. Future perspectives concern the identification of collaborative opportunities for investigators in Europe and the United States to accelerate the pace of survivorship science going forward.

Receipt of cancer treatment summaries and follow-up instructions among adult cancer survivors: results from a national survey

PURPOSE

The purpose of this study is to examine reporting of treatment summaries and follow-up instructions among cancer survivors.

METHODS

Using the 2010 National Health Interview Survey, we created logistic regression models among cancer survivors not in treatment (n = 1,345) to determine characteristics associated with reporting treatment summaries and written follow-up instructions, adjusting for sociodemographic, access, and cancer-related factors. Findings are presented for all survivors and those recently diagnosed (≤4 years). We also examined unadjusted associations between written instructions and subsequent surveillance and screening.

RESULTS

Among those recently diagnosed, 38 % reported receiving treatment summaries and 58 % reported written instructions. Among all survivors, approximately one third reported summaries and 44 % reported written instructions. After adjustment, lower reporting of summaries was associated with cancer site, race, and number of treatment modalities among those recently diagnosed, and white vs. black or Hispanic race/ethnicity, breast vs. colorectal cancer, >10 vs. ≤5 years since diagnosis, no clinical trials participation, and better than fair health among all survivors. For instructions, lower reporting was associated with no trials participation and lower income among those recently diagnosed, and increasing age, white vs. black race, lower income, >10 vs. ≤5 years since diagnosis, 1 vs. ≥2 treatment modalities, no trials participation, and at least good vs. fair/poor health among all survivors. Written instructions were associated with reporting provider recommendations for breast and cervical cancer surveillance, and recent screening mammograms.

CONCLUSION

Many recently diagnosed cancer survivors did not report receiving treatment summaries and written follow-up instructions. Opportunities exist to examine associations between use of these documents and recommended care and outcomes, and to facilitate their adoption.

IMPLICATIONS FOR CANCER SURVIVORS

Cancer survivors who have completed therapy should ask their providers for treatment summaries and written follow-up instructions, and discuss with them how their cancer and therapy impact their future health care.

Pain in long-term breast cancer survivors: the role of body mass index, physical activity, and sedentary behavior

Although pain is common among post-treatment breast cancer survivors, studies that are longitudinal, identify a case definition of clinically meaningful pain, or examine factors contributing to pain in survivors are limited. This study describes longitudinal patterns of pain in long-term breast cancer survivors, evaluating associations of body mass index (BMI), physical activity, sedentary behavior with mean pain severity and above-average pain. Women newly diagnosed with stages 0-IIIA breast cancer (N = 1183) were assessed, on average, 6 months (demographic/clinical characteristics), 30 months (demographics), 40 months (demographics, pain), 5 years (BMI, physical activity, and sedentary behavior), and 10 years (demographics, pain, BMI, physical activity, and sedentary behavior) post-diagnosis. This analysis includes survivors who completed pain assessments 40 months post-diagnosis (N = 801), 10 years post-diagnosis (N = 563), or both (N = 522). Above-average pain was defined by SF-36 bodily pain scores ≥1/2 standard deviation worse than age-specific population norms. We used multiple regression models to test unique associations of BMI, physical activity, and sedentary behavior with pain adjusting for demographic and clinical factors. The proportion of survivors reporting above-average pain was higher at 10 years than at 40 months (32.3 vs. 27.8 %, p < 0.05). Approximately one-quarter of survivors reported improved pain, while 9.0 % maintained above-average pain and 33.1 % reported worsened pain. Cross-sectionally at 10 years, overweight and obese survivors reported higher pain than normal-weight survivors and women meeting physical activity guidelines were less likely to report above-average pain than survivors not meeting these guidelines (p < 0.05). Longitudinally, weight gain (>5 %) was positively associated, while meeting physical activity guidelines was inversely associated, with above-average pain (OR, 95 % CI = 1.76, 1.03-3.01 and 0.40, 0.20-0.84, respectively) (p < 0.05). Weight gain and lack of physical activity place breast cancer survivors at risk for pain long after treatment ends. Weight control and exercise interventions should be tested for effects on long-term pain in these women.

Attachment style is associated with perceived spouse responses and pain-related outcomes

PURPOSE/OBJECTIVE

Attachment theory can provide a heuristic model for examining factors that may influence the relationship of social context to adjustment in chronic pain. This study examined the associations of attachment style with self-reported pain behavior, pain intensity, disability, depression, and perceived spouse responses to pain behavior. We also examined whether attachment style moderates associations between perceived spouse responses and self-reported pain behavior and depressive symptoms, as well as perceived spouse responses as a mediator of these associations.

METHOD

Individuals with chronic pain (N = 182) completed measures of self-reported attachment style, perceived spouse responses, and pain-related criterion variables.

RESULTS

Secure attachment was inversely associated with self-reported pain behaviors, pain intensity, disability, depressive symptoms, and perceived negative spouse responses; preoccupied and fearful attachment scores were positively associated with these variables. In multivariable regression models, both attachment style and perceived spouse responses were uniquely associated with self-reported pain behavior and depressive symptoms. Attachment style did not moderate associations between perceived spouse responses to self-reported pain behavior and pain criterion variables, but negative spouse responses partially mediated some relationships between attachment styles and pain outcomes.

CONCLUSIONS/IMPLICATIONS

Findings suggest that attachment style is associated with pain-related outcomes and perceptions of spouse responses. The hypothesized moderation effects for attachment were not found; however, mediation analyses showed that perceived spouse responses may partially explain associations between attachment and adjustment to pain. Future research is needed to clarify how attachment style and the social environment affect the pain experience.

Health information needs and health-related quality of life in a diverse population of long-term cancer survivors

OBJECTIVE

To investigate health information needs and their association with health-related quality of life (HRQOL) in a diverse, population-based sample of long-term cancer survivors.

METHODS

We analyzed health information needs from 1197 cancer survivors 4-14 years post-diagnosis drawn from two cancer registries in California. Multivariable regression models were used to identify factors associated with endorsement of total number and different categories of needs. The relationship between number of needs and HRQOL and effect modification by confidence for obtaining information was examined.

RESULTS

Survivors reported a high prevalence of unmet information needs in the following categories: side effects & symptoms: 75.8%; tests & treatment: 71.5%; health promotion: 64.5%; interpersonal & emotional: 60.2%; insurance: 39.0%; and sexual functioning & fertility: 34.6%. Survivors who were younger, non-White, and did not receive but wanted a written treatment summary reported a higher number of needs. Number of information needs was inversely related to mental well-being, particularly for those with low confidence for obtaining information (P<0.05).

CONCLUSION

These patterns suggest disparities in access to important health information in long-term survivors and that affect HRQOL.

PRACTICE IMPLICATIONS

Findings suggest a need for tailored interventions to equip survivors with comprehensive health information and to bolster skills for obtaining information.

Mental and physical health-related quality of life among U.S. cancer survivors: population estimates from the 2010 National Health Interview Survey

BACKGROUND

Despite extensive data on health-related quality of life (HRQOL) among cancer survivors, we do not yet have an estimate of the percentage of survivors with poor mental and physical HRQOL compared with population norms. HRQOL population means for adult-onset cancer survivors of all ages and across the survivorship trajectory also have not been published.

METHODS

Survivors (N = 1,822) and adults with no cancer history (N = 24,804) were identified from the 2010 National Health Interview Survey. The PROMIS® Global Health Scale was used to assess HRQOL. Poor HRQOL was defined as 1 SD or more below the PROMIS® population norm.

RESULTS

Poor physical and mental HRQOL were reported by 24.5% and 10.1% of survivors, respectively, compared with 10.2% and 5.9% of adults without cancer (both P < 0.0001). This represents a population of approximately 3.3 million and 1.4 million U.S. survivors with poor physical and mental HRQOL. Adjusted mean mental and physical HRQOL scores were similar for breast, prostate, and melanoma survivors compared with adults without cancer. Survivors of cervical, colorectal, hematologic, short-survival, and other cancers had worse physical HRQOL; cervical and short-survival cancer survivors reported worse mental HRQOL.

CONCLUSION

These data elucidate the burden of cancer diagnosis and treatment among U.S. survivors and can be used to monitor the impact of national efforts to improve survivorship care and outcomes.

IMPACT

We present novel data on the number of U.S. survivors with poor HRQOL. Interventions for high-risk groups that can be easily implemented are needed to improve survivor health at a population level.

Who provides psychosocial follow-up care for post-treatment cancer survivors? A survey of medical oncologists and primary care physicians

PURPOSE

Addressing psychosocial needs, including key components of psychologic distress, physical symptoms, and health promotion, is vital to cancer follow-up care. Yet little is known about who provides psychosocial care. This study examined physician-reported practices regarding care of post-treatment cancer survivors. We sought to characterize physicians who reported broad involvement in (ie, across key components of care) and shared responsibility for psychosocial care.

METHODS

A nationally representative sample of medical oncologists (n = 1,130) and primary care physicians (PCPs; n = 1,021) were surveyed regarding follow-up care of breast and colon cancer survivors.

RESULTS

Approximately half of oncologists and PCPs (52%) reported broad involvement in psychosocial care. Oncologist and PCP confidence, beliefs about who is able to provide psychosocial support, and preferences for shared responsibility for care predicted broad involvement. However, oncologists' and PCPs' perceptions of who provides specific aspects of psychosocial care differed (P < .001); both groups saw themselves as the main providers. Oncologists' confidence, PCPs' beliefs about who is able to provide psychosocial support, and oncologist and PCP preference for models other than shared care were inversely associated with a shared approach to care.

CONCLUSION

Findings that some providers are not broadly involved in psychosocial care and that oncologists and PCPs differ in their beliefs regarding who provides specific aspects of care underscore the need for better care coordination, informed by the respective skills and desires of physicians, to ensure needs are met. Interventions targeting physician confidence, beliefs about who is able to provide psychosocial support, and preferred models for survivorship care may improve psychosocial care delivery.

The Medical Expenditure Panel Survey (MEPS) experiences with cancer survivorship supplement

INTRODUCTION

The prevalence of cancer survivorship in the USA is expected to increase in the future because the US population is increasing in size and is aging and because survival following diagnosis is improving for many types of cancer. Medical care costs associated with cancer are also projected to increase dramatically. However, currently available data for estimating medical care costs and other important aspects of the burden of cancer, including time spent receiving medical care, productivity loss due to morbidity for patients and their families, and financial hardship, are limited, particularly in the population under the age of 65.

METHODS

We describe selected publicly available data sources for estimating the burden of cancer in the USA and a new collaborative effort to improve the quality of these data: the nationally representative Medical Expenditure Panel Survey (MEPS) Experiences with Cancer Survivorship Supplement.

CONCLUSIONS

Data from this effort can be used to address key gaps in cancer survivorship research related to medical care costs, employment patterns, financial hardship, and other aspects of the burden of illness for cancer survivors and their families.

IMPLICATIONS FOR CANCER SURVIVORS

Research using the MEPS Experiences with Cancer Survivorship Supplement can inform efforts by health care policy makers, healthcare systems, providers, and employers to improve the cancer survivorship experience in the USA.

Enhancing a cancer prevention and control curriculum through interactive group discussions

The Principles and Practice of Cancer Prevention and Control course (Principles course) is offered annually by the National Cancer Institute Cancer Prevention Fellowship Program. This 4-week postgraduate course covers the spectrum of cancer prevention and control research (e.g., epidemiology, laboratory, clinical, social, and behavioral sciences) and is open to attendees from medical, academic, government, and related institutions across the world. In this report, we describe a new addition to the Principles course syllabus, which was exclusively a lecture-based format for over 20 years. In 2011, cancer prevention fellows and staff designed and implemented small group discussion sessions as part of the curriculum. The goals of these sessions were to foster an interactive environment, discuss concepts presented during the Principles course, exchange ideas, and enhance networking among the course participants and provide a teaching and leadership opportunity to current cancer prevention fellows. Overall, both the participants and facilitators who returned the evaluation forms (n=61/87 and 8/10, respectively) reported a high satisfaction with the experience for providing both an opportunity to explore course concepts in a greater detail and to network with colleagues. Participants (93%) and facilitators (100%) stated that they would like to see this component remain a part of the Principles course curriculum, and both groups provided recommendations for the 2012 program. The design, implementation, and evaluation of this initial discussion group component of the Principles course are described herein. The findings in this report will not only inform future discussion group sessions in the Principles course but may also be useful to others planning to incorporate group learning into large primarily lecture-based courses.

Daytime sleepiness and sleep duration in long-term cancer survivors and non-cancer controls: results from a registry-based survey study

PURPOSE

Sleep-related complaints are common in cancer survivors. Although daytime sleepiness and sleep duration are associated with poor functional status, quality of life, and mortality in the general population, little is known about these issues in long-term cancer survivors. This study examined differences in daytime sleepiness and sleep duration between long-term cancer survivors and non-cancer controls.

METHODS

Survey data were analyzed from individuals diagnosed with cancer ≥2 years in the past (n = 1,171, mean age = 64.30, 80.8% white, 22.8% male) and spouse/friends controls (n = 250, mean age = 60.78, 88.0% white, 64.8% male). Daytime sleepiness was assessed using the Epworth Sleepiness Scale. Associations between sleep-related variables and history of cancer were estimated with multivariable logistic regression, adjusting for potential confounders. Stratified analyses were conducted to identify subgroups of survivors most at risk for sleep problems.

RESULTS

Cancer survivors were more likely than controls to report excessive daytime sleepiness (OR = 1.64, 95% CI: 1.07, 2.50). A cancer diagnosis was associated with longer sleep duration among males (OR = 1.25, 95% CI: 1.02, 1.53), but not in females (OR = 0.87, 95% CI: 0.37, 1.05). All other associations were similar regardless of cancer site, histology, time since diagnosis, treatment history, and history of multiple cancers.

CONCLUSIONS

Disturbances in daytime sleepiness and sleep duration persist among long-term cancer survivors and should be monitored in routine survivorship care. More research is needed to identify cancer survivors who are at increased risk for daytime sleepiness and disturbed sleep duration, as well as to identify causal mechanisms for, and interventions to mitigate, persistent differences.