Integrating mixed methods in health services and delivery system research

Assessment of the quality of inpatient meals and nutrition and dietetics service processes in select Philippine public hospitals

AIMS

The nutrition and dietetics service in Philippine public hospitals was implemented by the Department of Health in 2016 to standardise the daily allowance and nutritional content of inpatient meals. Five years later, it is timely for the Department of Health to assess the quality of inpatient meals and mandated nutrition processes in areas such as staffing, food service, and outcomes monitoring.

METHODS

A mixed-method sequential explanatory design was employed using (1) quantitative assessment through a facility survey (n = 193 hospitals) and (2) qualitative exploration of quantitative results through 6 focus group discussions (n = 36 hospitals).

RESULTS

Philippine public hospitals were unequipped with the inputs necessary to implement processes that produce high-quality meals for patients. The hospitals were unable to comply with the required minimum meal allowance (51%), nutritional content of meals (40%), and food service standards. Moreover, they had insufficient human resources and inefficient food procurement practices.

CONCLUSIONS

The quality of nutrition care and inpatient meals in Philippine public hospitals, who serve mostly people on low incomes, is a neglected problem in the Philippines. Moving forward, a systems approach involving the Department of Health, its regional offices, and hospital management is necessary to equip Philippine public hospitals with the inputs and structures necessary to provide high-quality nutrition care and inpatient meals that will facilitate patient recovery and overall patient health.

Audit of documentation accompanying referred maternity cases to a referral hospital in northern Ghana: a mixed-methods study

Background

Effective referral of maternity cases, which cannot be managed at the primary healthcare level, with detailed referral forms is important for reducing possible delays in the provision of higher-level healthcare. This is the first study to audit documentation or referral forms that accompany referred maternity cases to a referral hospital in the northern region of Ghana.

Materials and methods

This study employed an explanatory sequential mixed-method design, starting with a quantitative review of referral forms that accompanied all patients referred to four units (antenatal, antenatal emergency, labour and neonatal intensive care) of a referral hospital in northern Ghana. In-depth interviews were held with the heads of the four units afterwards. Descriptive statistics were computed for the quantitative data. The qualitative data was subjected to content analysis. Integration of the data occurred at the data interpretation/discussion level.

Results

A total of 217 referral forms were analysed. Nearly half of the cases were referred from the Tamale Metropolis (46.5%) and 83.9% were referred for advanced care, whilst 8.3% were referred due to a lack of medical logistics and equipment such as oxygen and skilled personnel (6%). Completion rates of the referral forms were as follows: < 50% completion (n = 81; 37.3%), 50–75% completion (n = 112; 51.6%) above 75% completion (n = 24; 11.1%). Some of the handwriting were not legible and were quite difficult to read. The key informants stated that incomplete forms sometimes delay treatment. The head of the antenatal care unit at the referral hospital suggested professional development sessions as a strategy for supporting clinicians to fill the forms as expected.

Conclusion

The Ghana Health Service should conduct regular audits, develop job aides and provide incentives for health professionals who accurately complete referral forms. Completing forms and digitizing health records can help ensure further efficiencies in the health information system and sustain good maternity referral documentation practices.

Choosing Wisely: An idea worth sustaining

OBJECTIVES

To evaluate the sustainability potential of Choosing Wisely (CW) to address unnecessary medical care at Ontario community hospitals.

DATA SOURCES/STUDY SETTING

Ontario community hospitals and their affiliated family health teams (FHTs).

STUDY DESIGN

A mixed-methods study involving the administration of a validated sustainability survey to CW implementation teams followed by their participation in focus groups.

DATA COLLECTION/EXTRACTION METHODS

Survey data were collected using an Excel file with an embedded, automated scoring system. We collated individual survey scores and generated aggregate team scores. We also performed descriptive statistics for quantitative data (frequencies, means). Qualitative data were triangulated with quantitative assessments to support data interpretations using the meta-matrix method.

PRINCIPAL FINDINGS

Fifteen CW implementation teams across four Ontario community hospitals and six affiliated primary care FHTs participated. CW priority areas investigated were de-prescribing of proton pump inhibitors (PPIs) and reducing Pre-Op testing and BUN/Urea lab testing. Survey results showed steady improvements in sustainability scores from baseline to final follow-up among most implementation teams: 10% increase for PPI de-prescribing (six FHTs) and 2% increase (three hospital teams); 18% increase in BUN/Urea lab testing (three hospital teams). Regardless of site or CW priority area, common facilitators were fit with existing processes and workflows, leadership support, and optimized team communication; common challenges were lack of awareness and buy-in, leadership engagement or a champion, and lack of fit with existing workflow and culture. All teams identified at least one challenge for which they co-designed and implemented a plan to maximize the sustainability potential of their CW initiative.

CONCLUSIONS

Evaluating the sustainability potential of an innovation such as Choosing Wisely is critical to ensuring that they have the best potential for impact. Our work highlights that implementation teams can be empowered to influence implementation efforts and to realize positive outcomes for their health care services and patients.

Content of antenatal care and perception about services provided by primary hospitals in Nepal: a convergent mixed methods study

BACKGROUND

Nepal has made significant strides in maternal and neonatal mortality over the last three decades. However, poor quality of care can threaten the gains, as maternal and newborn services are particularly sensitive to quality of care. Our study aimed to understand current gaps in the process and the outcome dimensions of the quality of antenatal care (ANC), particularly at the sub-national level. We assessed these dimensions of the quality of ANC in 17 primary, public hospitals across Nepal. We also assessed the variation in the ANC process across the patients' socio-economic gradient.

METHODS

We used a convergent mixed methods approach, whereby we triangulated qualitative and quantitative data. In the quantitative component, we observed interactions between providers (17 hospitals from all 7 provinces) and 198 women seeking ANC and recorded the tasks the providers performed, using the Service Provision Assessments protocol available from the Demographic and Health Survey program. The main outcome variable was the number of tasks performed by the provider during an ANC consultation. The tasks ranged from identifying potential signs of danger to providing counseling. We analyzed the resulting data descriptively and assessed the relationship between the number of tasks performed and users' characteristics. In the qualitative component, we synthesized users' and providers' narratives on perceptions of the overall quality of care obtained through focus group discussions and in-depth interviews.

RESULTS

Out of the 59 tasks recommended by the World Health Organization, providers performed only 22 tasks (37.3%) on average. The number of tasks performed varied significantly across provinces, with users in province 3 receiving significantly higher quality care than those in other provinces. Educated women were treated better than those with no education. Users and providers agreed that the overall quality of care was inadequate, although providers mentioned that the current quality was the best they could provide given the constraints they faced.

CONCLUSION

The quality of ANC in Nepal's primary hospitals is poor and inequitable across education and geographic gradients. While current efforts, such as the provision of 24/7 birthing centers, can mitigate gaps in service availability, additional equipment, infrastructure and human resources will be needed to improve quality. Providers also need additional training focused on treating patients from different backgrounds equally. Our study also points to the need for additional research, both to document the quality of care more objectively and to establish key determinants of quality to inform policy.

Implementing eScreening for suicide prevention in VA post-9/11 transition programs using a stepped-wedge, mixed-method, hybrid type-II implementation trial: a study protocol

BACKGROUND

Post-9/11 veterans who enroll in VA health care frequently present with suicidal ideation and/or recent suicidal behavior. Most of these veterans are not screened on their day of enrollment and their risk goes undetected. Screening for suicide risk, and associated mental health factors, can lead to early detection and referral to effective treatment, thereby decreasing suicide risk. eScreening is an innovative Gold Standard Practice with evidence to support its effectiveness and implementation potential in transition and care management (TCM) programs. We will evaluate the impact of eScreening to improve the rate and speed of suicide risk screening and referral to mental health care compared to current screening methods used by transition care managers. We will also evaluate the impact of an innovative, multicomponent implementation strategy (MCIS) on the reach, adoption, implementation, and sustained use of eScreening.

METHODS

This is an eight-site 4-year, stepped-wedge, mixed-method, hybrid type-II implementation trial comparing eScreening to screening as usual while also evaluating the potential impact of the MCIS focusing on external facilitation and Lean/SixSigma rapid process improvement workshops in TCM. The aims will address: 1) whether using eScreening compared to oral and/or paper-based methods in TCM programs is associated with improved rates and speed of PTSD, depression, alcohol, and suicide screening & evaluation, and increased referral to mental health treatment; 2) whether and to what degree our MCIS is feasible, acceptable, and has the potential to impact adoption, implementation, and maintenance of eScreening; and 3) how contextual factors influence the implementation of eScreening between high- and low-eScreening adopting sites. We will use a mixed methods approach guided by the RE-AIM outcomes of the Practical Robust Implementation and Sustainability Model (PRISM). Data to address Aim 1 will be collected via medical record query while data for Aims 2 and 3 will be collected from TCM staff questionnaires and qualitative interviews.

DISCUSSION

The results of this study will help identify best practices for screening in suicide prevention for Post-9/11 veterans enrolling in VA health care and will provide information on how best to implement technology-based screening into real-world clinical care programs.

TRIAL REGISTRATION

ClinicalTrials.gov : NCT04506164; date registered: August 20, 2020; retrospectively registered.

Evaluating context-specific evidence-based quality improvement intervention on lymphatic filariasis mass drug administration in Northern Ghana using the RE-AIM framework

Background

Over a decade of implementing a global strategy to eliminate lymphatic filariasis in Ghana through mass drug administration, the disease is still being transmitted in 11 districts out of an initial 98 endemic districts identified in 2000. A context-specific evidence-based quality improvement intervention was implemented in the Bole District of Northern Ghana after an initial needs assessment to improve the lymphatic filariasis mass drug administration towards eliminating the disease. Therefore, this study aimed to evaluate the process and impact of the lymphatic filariasis context-specific evidence-based quality improvement intervention in the Bole District of Northern Ghana.

Method

A cross-sectional mixed methods study using the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework to evaluate the context-specific evidence-based quality improvement intervention was employed. Quantitative secondary data was extracted from the neglected tropical diseases database. A community survey was conducted with 446 randomly selected participants. Qualitative data were collected from 42 purposively selected health workers, chiefs/opinion leaders and community drug distributors in the study area.

Results

The evaluation findings showed an improvement in social mobilisation and sensitisation, knowledge about lymphatic filariasis and mass drug administration process, willingness to ingest the medication and adherence to the direct observation treatment strategy. We observed an increase in coverage ranging from 0.1 to 12.3% after implementing the intervention at the sub-district level and reducing self-reported adverse drug reaction. The level of reach, effectiveness and adoption at the district, sub-district and individual participants’ level suggest that the context-specific evidence-based quality improvement intervention is feasible to implement in lymphatic filariasis hotspot districts based on initial context-specific needs assessment.

Conclusion

The study provided the groundwork for future application of the RE-AIM framework to evaluate the implementation of context-specific evidence-based quality improvement intervention to improve lymphatic filariasis mass drug administration towards eliminating the disease as a public health problem.

A Longitudinal Mixed Method Approach for Assessing Implementation Context and Process Factors: Comparison of Three Sites from a Housing First Implementation Strategy Pilot

Background

Implementation science's focus on establishing implementation strategy effectiveness has overshadowed the need to understand differential performance of such strategies under various conditions. Methods allowing for assessment between implementation context and process can help address this gap. This paper provides a detailed description of a mixed method procedure for assessing factors related to the implementation context and process intersection, which was developed as part of the pilot study of the Housing First Technical Assistance and Training (HFTAT) Program, a multifaceted strategy designed to support Housing First model implementation.

Methods

The HFTAT was pilot tested among a sample of three organizations. Our mixed method approach combines two tools often used in implementation research-the Stages of Implementation Completion and the Consolidated Framework for Implementation Research-in a novel way. Several stages to analysis were completed, starting with a separate analysis of data pertaining to each measure and then two levels of mixed method analysis.

Results

The approach provided a better understanding of the issues that impacted the implementation guided by the HFTAT, suggesting: (1) individual determinants seemed to have a bigger impact based on the number of SIC phases they affected, (2) implementation context and process were connected through climate-related factors in the inner setting that made the sites more or less responsive to addressing identified barriers, and (3) there is a need to better assess context factors to identify areas where implementation drivers should be better targeted to facilitate change, and this is supported by prior research.

Conclusions

Understanding the underlying factors impacting a setting's performance related to a specific implementation strategy has potential to improve decision-making and optimize future implementation efforts. The approach likely be as successful combining the SIC with other determinant frameworks and should be utilized at the onset of an implementation project to maximize its usefulness.

Day centres for older people - attender characteristics, access routes and outcomes of regular attendance: findings of exploratory mixed methods case study research

BACKGROUND

Social prescribing is encouraged to promote well-being, reduce isolation and loneliness. Traditional, generalist day centres for older people could be suggested by social prescribing, but little is known about their clientele or their outcomes. As part of a larger study of the role, outcomes and commissioning of generalist English day centres for older people, the characteristics of attenders at 4 day centres, their reasons for attendance and outcomes were explored.

METHODS

This mixed-methods study used qualitative interviews and standardised tools within an embedded multiple-case study design. Semi-structured interviews with older day centre attenders (n = 23, 62% of eligible attenders) of 4 day centres in south-east England, recruited purposively to reflect organisational differences, were analysed.

RESULTS

Participants reported non-elective withdrawal from socialisation following health or mobility decline, or losses. Apart from living arrangements and marital status, attenders' profiles differed between centres. Access had been mostly facilitated by others. Day centre attendance enhanced quality of life for this group of socially isolated people with mobility restrictions and at risk of declining independence and wellbeing. The positive impact on attenders' social participation and involvement and on meaningful occupation was significant (p-value < 0.001, 99% CI), with an average ASCOT gain score of 0.18. Ten outcome themes were identified.

CONCLUSION

Outcomes of day centre attendance are those targeted by social care and health policy. Centres were communities that 'enabled' and offset loss or isolation, thus supporting ageing in place through wellbeing and contributed something unique to their attenders' lives. By monitoring attenders' health and wellbeing and providing practical support, information and facilitating access to other services, centres offered added value. Attendance needs to be set in the context of other social engagement and care provision which may not overlap or duplicate centre support. Professionals may wish to explore the benefits of social prescriptions to day centres but should map local centres' provision, engage with their organisers, and seek information on attenders, who may differ from those in this study.

Role of Health Information Technology in Addressing Health Disparities: Patient, Clinician, and System Perspectives

Over the last decade, health information technology (IT) has dramatically transformed medical practice in the United States. On May 11-12, 2017, the National Institute on Minority Health and Health Disparities, in partnership with the National Science Foundation and the National Health IT Collaborative for the Underserved, convened a scientific workshop, "Addressing Health Disparities with Health Information Technology," with the goal of ensuring that future research guides potential health IT initiatives to address the needs of health disparities populations. The workshop examined patient, clinician, and system perspectives on the potential role of health IT in addressing health disparities. Attendees were asked to identify and discuss various health IT challenges that confront underserved communities and propose innovative strategies to address them, and to involve these communities in this process. Community engagement, cultural competency, and patient-centered care were highlighted as key to improving health equity, as well as to promoting scalable, sustainable, and effective health IT interventions. Participants noted the need for more research on how health IT can be used to evaluate and address the social determinants of health. Expanding public-private partnerships was emphasized, as was the importance of clinicians and IT developers partnering and using novel methods to learn how to improve health care decision-making. Finally, to advance health IT and promote health equity, it will be necessary to record and capture health disparity data using standardized terminology, and to continuously identify system-level deficiencies and biases.

Beyond exploratory: a tailored framework for designing and assessing qualitative health research

The objective of this commentary is to develop a framework for assessing the rigour of qualitative approaches that identifies and distinguishes between the diverse objectives of qualitative health research, guided by a narrative review of the published literature on qualitative guidelines and standards from peer-reviewed journals and national funding organisations that support health services research, patient-centered outcomes research and other applied health research fields. In this framework, we identify and distinguish three objectives of qualitative studies in applied health research: exploratory, descriptive and comparative. For each objective, we propose methodological standards that may be used to assess and improve rigour across all study phases-from design to reporting. Similar to hierarchies of quality of evidence within quantitative studies, we argue that standards for qualitative rigour differ, appropriately, for studies with different objectives and should be evaluated as such. Distinguishing between different objectives of qualitative health research improves the ability to appreciate variation in qualitative studies and to develop appropriate evaluations of the rigour and success of qualitative studies in meeting their stated objectives. Researchers, funders and journal editors should consider how further developing and adopting the framework for assessing qualitative rigour outlined here may advance the rigour and potential impact of this important mode of inquiry.

Participant Engagement in and Perspectives on a Web-Based Mindfulness Intervention for 9-1-1 Telecommunicators: Multimethod Study

Background

Demanding working conditions and secondary exposure to trauma may contribute to a high burden of stress among 9-1-1 telecommunicators, decreasing their ability to work effectively and efficiently. Web-based mindfulness-based interventions (MBIs) can be effective in reducing stress in similar populations. However, low engagement may limit the effectiveness of the intervention.

Objective

The aim of this study was to assess participant engagement in a Web-based MBI designed for 9-1-1 telecommunicators. Specifically, we sought to describe the following: (1) participant characteristics associated with intervention engagement, (2) participant perspectives on engaging with the intervention, and (3) perceived challenges and facilitators to engaging.

Methods

We used qualitative and quantitative data from participant surveys (n=149) that were collected to assess the efficacy of the intervention. We conducted descriptive and bivariate analyses to identify associations between demographic, psychosocial, and workplace characteristics and engagement. We conducted a thematic analysis of qualitative survey responses to describe participant experiences with the MBI.

Results

We found that no individual participant characteristics were associated with the level of engagement (low vs high number of lessons completed). Participant engagement did vary by the call center (P<.001). We identified the following overarching qualitative themes: (1) the participants perceived benefits of mindfulness practice, (2) the participants perceived challenges to engage with mindfulness and the intervention, and (3) intervention components that facilitated engagement. The participants expressed positive beliefs in the perceived benefits of practicing mindfulness, including increased self-efficacy in coping with stressors and increased empathy with callers. The most commonly cited barriers were work-related, particularly not having time to participate in the intervention at work. Facilitators included shorter meditation practices and the availability of multiple formats and types of intervention content.

Conclusions

The findings of this study suggest that efforts to improve intervention engagement should focus on organizational-level factors rather than individual participant characteristics. Future research should explore the effect of mindfulness practice on the efficiency and effectiveness of 9-1-1 telecommunicators at work.

Trial Registration

ClinicalTrials.gov NCT02961621; https://clinicaltrials.gov/ct2/show/NCT02961621

A conceptual framework and metrics for evaluating multicomponent interventions to increase colorectal cancer screening within an organized screening program

BACKGROUND

Multicomponent, evidence-based interventions are viewed increasingly as essential for increasing the use of colorectal cancer (CRC) screening to meet national targets. Multicomponent interventions involve complex care pathways and interactions across multiple levels, including the individual, health system, and community.

METHODS

The authors developed a framework and identified metrics and data elements to evaluate the implementation processes, effectiveness, and cost effectiveness of multicomponent interventions used in the Centers for Disease Control and Prevention's Colorectal Cancer Control Program.

RESULTS

Process measures to evaluate the implementation of interventions to increase community and patient demand for CRC screening, increase patient access, and increase provider delivery of services are presented. In addition, performance measures are identified to assess implementation processes along the continuum of care for screening, diagnosis, and treatment. Series of intermediate and long-term outcome and cost measures also are presented to evaluate the impact of the interventions.

CONCLUSIONS

Understanding the effectiveness of multicomponent, evidence-based interventions and identifying successful approaches that can be replicated in other settings are essential to increase screening and reduce CRC burden. The use of common framework, data elements, and evaluation methods will allow the performance of comparative assessments of the interventions implemented across CRCCP sites to identify best practices for increasing colorectal screening, particularly among underserved populations, to reduce disparities in CRC incidence and mortality.

Interprofessional training and team function in patient-centred medical home: Findings from a mixed method study of interdisciplinary provider perspectives

Transitioning from profession-specific to interprofessional (IP) models of care requires major change. The Veterans Assessment and Improvement Laboratory (VAIL), is an initiative based in the United States that supports and evaluates the Veterans Health Administration's (VAs) transition of its primary care practices to an IP team based patient-centred medical home (PCMH) care model. We postulated that modifiable primary care practice organizational climate factors impact PCMH implementation. VAIL administered a survey to 322 IP team members in primary care practices in one VA administrative region during early implementation of the PCMH and interviewed 79 representative team members. We used convergent mixed methods to study modifiable organizational climate factors in relationship to IP team functioning. We found that leadership support and job satisfaction were significantly positively associated with team functioning. We saw no association between team functioning and either role readiness or team training. Qualitative interview data confirmed survey findings and explained why the association with IP team training might be absent. In conclusion, our findings demonstrate the importance of leadership support and individual job satisfaction in producing highly functioning PCMH teams. Based on qualitative findings, we hypothesize interprofessional training is important, however, inconsistencies in IP training delivery compromise its potential benefit. Future implementation efforts should improve standardization of training process and train team members together. Interprofessional leadership coordination of interprofessional training is warranted.

Deliberative Engagement Methods for Patient-Centered Outcomes Research

There is growing emphasis on eliciting and incorporating stakeholder perspectives into health research and public policy development. The deliberative engagement session (DES) method provides one approach to elicit informed preferences from patients and other stakeholders on policy issues. DES involves day-long interaction with participants, including short plenary presentations followed by small group discussion. While interest in DES methods is expanding, practical guidance for researchers on this method remains limited. In this paper, we describe the DES method and its contemporary relevance for health policy research, illustrate how to conduct a DES using an example of a recent patient-centered outcomes research (PCOR) study with which we were involved, and discuss strengths and challenges of using this approach. DES methods generate rich data, reduce the risk of eliciting uniformed preferences or non-attitudes, and increase the likelihood of eliciting informed, reflective preferences. However, they are resource-intensive, and thus generally require trading away a larger, more representative sample. Despite these limitations, the DES method, when carefully designed, is well-suited for engaging stakeholders in research on complex health policy issues.

Qualitative approaches to use of the RE-AIM framework: rationale and methods

BACKGROUND

There have been over 430 publications using the RE-AIM model for planning and evaluation of health programs and policies, as well as numerous applications of the model in grant proposals and national programs. Full use of the model includes use of qualitative methods to understand why and how results were obtained on different RE-AIM dimensions, however, recent reviews have revealed that qualitative methods have been used infrequently. Having quantitative and qualitative methods and results iteratively inform each other should enhance understanding and lessons learned.

METHODS

Because there have been few published examples of qualitative approaches and methods using RE-AIM for planning or assessment and no guidance on how qualitative approaches can inform these processes, we provide guidance on qualitative methods to address the RE-AIM model and its various dimensions. The intended audience is researchers interested in applying RE-AIM or similar implementation models, but the methods discussed should also be relevant to those in community or clinical settings.

RESULTS

We present directions for, examples of, and guidance on how qualitative methods can be used to address each of the five RE-AIM dimensions. Formative qualitative methods can be helpful in planning interventions and designing for dissemination. Summative qualitative methods are useful when used in an iterative, mixed methods approach for understanding how and why different patterns of results occur.

CONCLUSIONS

In summary, qualitative and mixed methods approaches to RE-AIM help understand complex situations and results, why and how outcomes were obtained, and contextual factors not easily assessed using quantitative measures.

Qualitative Methods in Patient-Centered Outcomes Research

The Patient-Centered Outcomes Research Institute (PCORI), created to fund research guided by patients, caregivers, and the broader health care community, offers a new research venue. Many (41 of 50) first funded projects involved qualitative research methods. This study was completed to examine the current state of the science of qualitative methodologies used in PCORI-funded research. Principal investigators participated in phenomenological interviews to learn (a) how do researchers using qualitative methods experience seeking funding for, implementing and disseminating their work; and (b) how may qualitative methods advance the quality and relevance of evidence for patients? Results showed the experience of doing qualitative research in the current research climate as "Being a bona fide qualitative researcher: Staying true to research aims while negotiating challenges," with overlapping patterns: (a) researching the elemental, (b) expecting surprise, and (c) pushing boundaries. The nature of qualitative work today was explicitly described and is rendered in this article.

Role confusion as a barrier to effective carer involvement for people with intellectual disabilities in acute hospitals: findings from a mixed-method study

AIMS

To understand issues around carer roles that affect carer involvement for people with intellectual disabilities in acute hospitals.

BACKGROUND

There is evidence that a lack of effective carer involvement can lead to poorer health outcomes for people with intellectual disabilities, but there is a lack of insight into the reasons for poor carer involvement in acute hospitals.

DESIGN

Mixed methods in six acute hospital trusts in England (2011-2013).

METHODS

Electronic hospital staff survey (n = 990), carer questionnaires (n = 88), semi-structured interviews with hospital staff (n = 68) and carers (n = 37). Data were triangulated and analysed using a conceptual framework.

RESULTS

There was strong support for carer involvement among hospital staff, and most carers indicated that they felt welcomed and supported. However, an investigation of negative experiences showed that there were discrepancies in the perspectives of hospital staff and carers on the scope of 'carer involvement'. An important contributory factor to the effectiveness of carer involvement was the degree to which staff understood the importance of carer expertise (rather than simply carer work) and welcomed it. Carers' contributions to basic nursing care tasks could be taken for granted by hospital staff, sometimes erroneously.

CONCLUSION

The roles and contributions of carers should be clarified on an individual basis by hospital staff. The authors propose a new model to support this clarification. Further research is needed to assess the suitability of the model for patients with intellectual disabilities and other vulnerable patient groups.

Recommendations for a mixed methods approach to evaluating the patient-centered medical home

PURPOSE

There is a strong push in the United States to evaluate whether the patient-centered medical home (PCMH) model produces desired results. The explanatory and contextually based questions of how and why PCMH succeeds in different practice settings are often neglected. We report the development of a comprehensive, mixed qualitative-quantitative evaluation set for researchers, policy makers, and clinician groups.

METHODS

To develop an evaluation set, the Brown Primary Care Transformation Initiative convened a multidisciplinary group of PCMH experts, reviewed the PCMH literature and evaluation strategies, developed key domains for evaluation, and selected or created methods and measures for inclusion.

RESULTS

The measures and methods in the evaluation set (survey instruments, PCMH meta-measures, patient outcomes, quality measures, qualitative interviews, participant observation, and process evaluation) are meant to be used together. PCMH evaluation must be sufficiently comprehensive to assess and explain both the context of transformation in different primary care practices and the experiences of diverse stakeholders. In addition to commonly assessed patient outcomes, quality, and cost, it is critical to include PCMH components integral to practice culture transformation: patient and family centeredness, authentic patient activation, mutual trust among practice employees and patients, and transparency, joy, and collaboration in delivering and receiving care in a changing environment.

CONCLUSIONS

This evaluation set offers a comprehensive methodology to enable understanding of how PCMH transformation occurs in different practice settings. This approach can foster insights about how transformation affects critical outcomes to achieve meaningful, patient-centered, high-quality, and cost-effective sustainable change among diverse primary care practices.

Ethnographic process evaluation in primary care: explaining the complexity of implementation

Background

The recent growth of implementation research in care delivery systems has led to a renewed interest in methodological approaches that deliver not only intervention outcome data but also deep understanding of the complex dynamics underlying the implementation process. We suggest that an ethnographic approach to process evaluation, when informed by and integrated with quantitative data, can provide this nuanced insight into intervention outcomes. The specific methods used in such ethnographic process evaluations are rarely presented in detail; our objective is to stimulate a conversation around the successes and challenges of specific data collection methods in health care settings. We use the example of a translational clinical trial among 11 community clinics in Portland, OR that are implementing an evidence-based, health-information technology (HIT)-based intervention focused on patients with diabetes.

Discussion

Our ethnographic process evaluation employed weekly diaries by clinic-based study employees, observation, informal and formal interviews, document review, surveys, and group discussions to identify barriers and facilitators to implementation success, provide insight into the quantitative study outcomes, and uncover lessons potentially transferable to other implementation projects. These methods captured the depth and breadth of factors contributing to intervention uptake, while minimizing disruption to clinic work and supporting mid-stream shifts in implementation strategies. A major challenge is the amount of dedicated researcher time required.

Summary

The deep understanding of the ‘how’ and ‘why’ behind intervention outcomes that can be gained through an ethnographic approach improves the credibility and transferability of study findings. We encourage others to share their own experiences with ethnography in implementation evaluation and health services research, and to consider adapting the methods and tools described here for their own research.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-014-0607-0) contains supplementary material, which is available to authorized users.

Data collection challenges in community settings: insights from two field studies of patients with chronic disease

PURPOSE

Collecting information about health and disease directly from patients can be fruitfully accomplished using contextual approaches, ones that combine more and less structured methods in home and community settings. This paper's purpose was to describe and illustrate a framework of the challenges of contextual data collection.

METHODS

A framework is presented based on prior work in community-based participatory research and organizational science, comprised of ten types of challenges across four broader categories. Illustrations of challenges and suggestions for addressing them are drawn from two mixed method, contextual studies of patients with chronic disease in two regions of the USA.

RESULTS

The first major category of challenges was concerned with the researcher-participant partnership, for example, the initial lack of mutual trust and understanding between researchers, patients, and family members. The second category concerned patient characteristics such as cognitive limitations and a busy personal schedule that created barriers to successful data collection. The third concerned research logistics and procedures such as recruitment, travel distances, and compensation. The fourth concerned scientific quality and interpretation, including issues of validity, reliability, and combining data from multiple sources. The two illustrative studies faced both common and diverse research challenges and used many different strategies to address them.

CONCLUSION

Collecting less structured data from patients and others in the community is potentially very productive but requires the anticipation, avoidance, or negotiation of various challenges. Future work is necessary to better understand these challenges across different methods and settings, as well as to test and identify strategies to address them.