Measuring organizational and individual factors thought to influence the success of quality improvement in primary care: a systematic review of instruments

Strategies that facilitate the delivery of exceptionally good patient care in general practice: a qualitative study with patients and primary care professionals

BACKGROUND

In recent years, proactive strengths-based approaches to improving quality of care have been advocated. The positive deviance approach seeks to identify and learn from those who perform exceptionally well. Central to this approach is the identification of the specific strategies, behaviours, tools and contextual strategies used by those positive deviants to perform exceptionally well. This study aimed to: identify and collate the specific strategies, behaviours, processes and tools used to support the delivery of exceptionally good care in general practice; and to abstract the identified strategies into an existing framework pertaining to excellence in general practice; the Identifying and Disseminating the Exceptional to Achieve Learning (IDEAL) framework.

METHODS

This study comprised a secondary analysis of data collected during semi-structured interviews with 33 purposively sampled patients, general practitioners, practice nurses, and practice managers. Discussions explored the key factors and strategies that support the delivery of exceptional care across five levels of the primary care system; the patient, provider, team, practice, and external environment. For analysis, a summative content analysis approach was undertaken whereby data were inductively analysed and summated to identify the key strategies used to achieve the delivery of exceptionally good general practice care, which were subsequently abstracted as a new level of the IDEAL framework.

RESULTS

In total, 222 individual factors contributing to exceptional care delivery were collated and abstracted into the framework. These included specific behaviours (e.g., patients providing useful feedback and personal history to the provider), structures (e.g., using technology effectively to support care delivery (e.g., electronic referrals & prescriptions)), processes (e.g., being proactive in managing patient flow and investigating consistently delayed wait times), and contextual factors (e.g., valuing and respecting contributions of every team member).

CONCLUSION

The addition of concrete and contextual strategies to the IDEAL framework has enhanced its practicality and usefulness for supporting improvement in general practices. Now, a multi-level systems approach is needed to embed these strategies and create an environment where excellence is supported. The refined framework should be developed into a learning tool to support teams in general practice to measure, reflect and improve care within their practice.

Developing a model for primary care quality improvement success: a comparative case study in rural, urban and Kaupapa Māori organisations

Introduction Primary care is under pressure to achieve accessible, equitable, quality health care, while being increasingly under resourced. There is a need to understand factors that influence quality improvement (QI) to support a high-performing primary care system. Literature highlights the impact of context on QI but there is little primary care research on this topic. Aim This qualitative case study research seeks to discover the contextual factors influencing QI in primary care, and how the relationships between contextual factors, the QI initiative, and the implementation process influence outcomes. Methods The Consolidated Framework for Implementation Research was used to frame this qualitative study exploring primary care experiences in depth. Six sites were selected to provide a sample of rural, urban and Kaupapa Māori settings. Qualitative data was collected via semi-structured interviews and compared and contrasted with the organisational documents and data provided by participants. Results Cases reported success in achieving improved outcomes for patients, practices, and staff. Strong internal cultures of 'Clan' and 'Adhocracy' typologies supported teamwork, distributed leadership, and a learning climate to facilitate iterative sensemaking activities. To varying degrees, external network relationships provided resources, knowledge, and support. Discussion Organisations were motivated by a combination of patient/community need and organisational culture. Network relationships assisted to varying degrees depending on need. Engaged and distributed leadership based on teamwork was observed, where leadership was shared and emerged at different levels and times as the need arose. A learning climate was supported to enable iterative sensemaking activities to achieve success.

Impact of externally facilitated continuous quality improvement cohorts on Advanced Access to support primary healthcare teams: protocol for a quasi-randomized cluster trial

BACKGROUND

Improving access to primary health care is among top priorities for many countries. Advanced Access (AA) is one of the most recommended models to improve timely access to care. Over the past 15 years, the AA model has been implemented in Canada, but the implementation of AA varies substantially among providers and clinics. Continuous quality improvement (CQI) approaches can be used to promote organizational change like AA implementation. While CQI fosters the adoption of evidence-based practices, knowledge gaps remain, about the mechanisms by which QI happens and the sustainability of the results. The general aim of the study is to analyse the implementation and effects of CQI cohorts on AA for primary care clinics. Specific objectives are: 1) Analyse the process of implementing CQI cohorts to support PHC clinics in their improvement of AA. 2) Document and compare structural organisational changes and processes of care with respect to AA within study groups (intervention and control). 3) Assess the effectiveness of CQI cohorts on AA outcomes. 4) Appreciate the sustainability of the intervention for AA processes, organisational changes and outcomes.

METHODS

Cluster-controlled trial allowing for a comprehensive and rigorous evaluation of the proposed intervention 48 multidisciplinary primary care clinics will be recruited to participate. 24 Clinics from the intervention regions will receive the CQI intervention for 18 months including three activities carried out iteratively until the clinic's improvement objectives are achieved: 1) reflective sessions and problem priorisation; 2) plan-do-study-act cycles; and 3) group mentoring. Clinics located in the control regions will receive an audit-feedback report on access. Complementary qualitative and quantitative data reflecting the quintuple aim will be collected over a period of 36 months.

RESULTS

This research will contribute to filling the gap in the generalizability of CQI interventions and accelerate the spread of effective AA improvement strategies while strengthening local QI culture within clinics. This research will have a direct impact on patients' experiences of care.

CONCLUSION

This mixed-method approach offers a unique opportunity to contribute to the scientific literature on large-scale CQI cohorts to improve AA in primary care teams and to better understand the processes of CQI.

TRIAL REGISTRATION

Clinical Trials: NCT05715151.

Stakeholder Perspectives of Attributes and Features of Context Relevant to Knowledge Translation in Health Settings: A Multi-Country Analysis

BACKGROUND

Context is recognized as important to successful knowledge translation (KT) in health settings. What is meant by context, however, is poorly understood. The purpose of the current study was to elicit tacit knowledge about what is perceived to constitute context by conducting interviews with a variety of health system stakeholders internationally so as to compile a comprehensive list of contextual attributes and their features relevant to KT in healthcare.

METHODS

A descriptive qualitative study design was used. Semi-structured interviews were conducted with health system stakeholders (change agents/KT specialists and KT researchers) in four countries: Australia, Canada, the United Kingdom, and the United States. Interview transcripts were analyzed using inductive thematic content analysis in four steps: (1) selection of utterances describing context, (2) coding of features of context, (3) categorizing of features into attributes of context, (4) comparison of attributes and features by: country, KT experience, and role.

RESULTS

A total of 39 interviews were conducted. We identified 66 unique features of context, categorized into 16 attributes. One attribute, Facility Characteristics, was not represented in previously published KT frameworks. We found instances of all 16 attributes in the interviews irrespective of country, level of experience with KT, and primary role (change agent/KT specialist vs. KT researcher), revealing robustness and transferability of the attributes identified. We also identified 30 new context features (across 13 of the 16 attributes).

CONCLUSION

The findings from this study represent an important advancement in the KT field; we provide much needed conceptual clarity in context, which is essential to the development of common assessment tools to measure context to determine which context attributes and features are more or less important in different contexts for improving KT success.

Contextual antecedents of quality improvement: a comparative case study in rural, urban and Kaupapa Māori general practice

Introduction The impact of contextual factors on primary health-care quality improvement is significant. In-depth research is required to identify the key contextual factors that influence quality improvement initiatives to develop high-performing primary health-care systems. Aim This research seeks to answer two questions; what are the contextual factors influencing primary care improvement initiatives?; and how do contextual factors, the quality improvement initiative and the implementation process influence one another and the overall improvement outcomes? Methods A multi-case study methodology was used to explore the complexities of the phenomena in situ . Three sites where successful quality improvement had occurred were selected by purposeful theoretical sampling to provide a sample of rural, urban and Kaupapa Māori general practice settings typical of the New Zealand environment. Semi-structured interviews were conducted with team members and triangulated with secondary data provided by the organisations. Results The quality improvement topic and the approach taken were intrinsically linked to context. Sites reported success in achieving the desired outcomes benefitting the patients, practice and staff. Teams did not use formal improvement methods, instead relying on established relationships and elements of change management methods. The culture in all three cases was a large component of why and how these initiatives were successful. Discussion Intrinsic motivation was generated by community connections and networks. This combined with a learning climate generated by distributed leadership and teamwork enabled success. Iterative reflection and sensemaking processes were able to deliver quality improvement success in primary care without the use of formal improvement methods.

Association of Implementation and Social Network Factors With Patient Safety Culture in Medical Homes: A Coincidence Analysis

OBJECTIVES

The patient-centered medical home (PCMH) may provide a key model for ambulatory patient safety. Our objective was to explore which PCMH and patient safety implementation and social network factors may be necessary or sufficient for higher patient safety culture.

METHODS

This was a cross-case analysis study in 25 diverse U.S. PCMHs. Data sources included interviews of a clinician and an administrator in each PCMH, surveys of clinicians and staff, and existing data on the PCMHs' characteristics. We used coincidence analysis, a novel method based on set theory and Boolean logic, to evaluate relationships between factors and the implementation outcome of patient safety culture.

RESULTS

The coincidence analysis identified 5 equally parsimonious solutions (4 factors), accounting for all practices with higher safety culture. Three solutions contained the same core minimally sufficient condition: the implementation factor leadership priority for patient safety and the social network factor reciprocity in advice-seeking network ties (advice-seeking relationships). This minimally sufficient condition had the highest coverage (5/7 practices scoring higher on the outcome) and best performance across solutions; all included leadership priority for patient safety. Other key factors included self-efficacy and job satisfaction and quality improvement climate. The most common factor whose absence was associated with the outcome was a well-functioning process for behavioral health.

CONCLUSIONS

Our findings suggest that PCMH safety culture is higher when clinicians and staff perceive that leadership prioritizes patient safety and when high reciprocity among staff exists. Interventions to improve patient safety should consider measuring and addressing these key factors.

Organizational contextual factors that predict success of a quality improvement collaborative approach to enhance integrated HIV-tuberculosis services: a sub-study of the Scaling up TB/HIV Integration trial

Background

A quality improvement (QI) collaborative approach to enhancing integrated HIV-Tuberculosis (TB) services may be effective in scaling up and improving the quality of service delivery. Little is known of the role of organizational contextual factors (OCFs) in influencing the success of QI collaboratives. This study aims to determine which OCFs were associated with improvement in a QI collaborative intervention to enhance integrated HIV-TB services delivery.

Methods

This is a nested sub-study embedded in a cluster-randomized controlled trial. Sixteen nurse supervisors (clusters) overseeing 40 clinics were randomized (1:1) to receive QI training and mentorship, or standard of care support (SOC). In the QI arm, eight nurse supervisors and 20 clinics formed a “collaborative” which aimed to improve HIV-TB process indicators, namely HIV testing, TB screening, isoniazid preventive therapy (IPT) initiations, viral load testing, and antiretroviral therapy for TB patients. OCFs measured at baseline were physical infrastructure, key staff, flexibility of clinic hours, monitoring data for improvement (MDI), and leadership support. Surveys were administered to clinic staff at baseline and month 12 to assess perceptions of supportiveness of contexts for change, and clinic organization for delivering integrated HIV-TB services. Linear mixed modelling was used to test for associations between OCFs and HIV-TB process indicators.

Results

A total of 209 clinic staff participated in the study; 97 (46.4%) and 112 (53.6%) from QI and SOC arms, respectively. There were no differences between the QI and SOC arms scores achieved for physical infrastructure (78.9% vs 64.7%; p = 0.058), key staff (95.8 vs 92; p = 0.270), clinic hours (66.9 vs 65.5; p = 0.900), MDI (63.3 vs 65; p = 0.875, leadership support (46.0 vs 57.4; p = 0.265), and perceptions of supportiveness of contexts for change (76.2 vs 79.7; p = 0.128 and clinic organization for delivering integrated HIV-TB services (74.1 vs 80.1; p = 0.916). IPT initiation was the only indicator that was significantly improved in the parent study. MDI was a significantly associated with increasing IPT initiation rates [beta coefficient (β) = 0.004; p = 0.004].

Discussion

MDI is a practice that should be fostered in public health facilities to increase the likelihood of success of future QI collaboratives to improve HIV-TB service delivery.

Trial registration

Clinicaltrials.gov, NCT02654613. Registered 01 June 2015.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01155-7.

Quality management systems in Aboriginal Community Controlled Health Services: a review of the literature

BACKGROUND

A national accreditation policy for the Australian primary healthcare (PHC) system was initiated in 2008. While certification standards are mandatory, little is known about their effects on the efficiency and sustainability of organisations, particularly in the Aboriginal Community Controlled Health Service (ACCHS) sector.

AIM

The literature review aims to answer the following: to what extent does the implementation of the International Organisation for Standardization 9001:2008 quality management system (QMS) facilitate efficiency and sustainability in the ACCHS sector?

METHODS

Thematic analysis of peer-reviewed and grey literature was undertaken from Australia and New Zealand PHC sector with a focus on First Nations people. The databases searched included Medline, Scopus and three Informit sites (AHB-ATSIS, AEI-ATSIS and AGIS-ATSIS). The initial search strategy included quality improvement, continuous quality improvement, efficiency and sustainability.

RESULTS

Sixteen included studies were assessed for quality using the McMaster criteria. The studies were ranked against the criteria of credibility, transferability, dependability and confirmability. Three central themes emerged: accreditation (n=4), quality improvement (n=9) and systems strengthening (n=3). The accreditation theme included effects on health service expenditure and clinical outcomes, consistency and validity of accreditation standards and linkages to clinical governance frameworks. The quality improvement theme included audit effectiveness and value for specific population health. The theme of systems strengthening included prerequisite systems and embedded clinical governance measures for innovative models of care.

CONCLUSION

The ACCHS sector warrants reliable evidence to understand the value of QMSs and enhancement tools, particularly given ACCHS (client-centric) services and their specialist status. Limited evidence exists for the value of standards on health system sustainability and efficiency in Australia. Despite a mandatory second certification standard, no studies reported on sustainability and efficiency of a QMS in PHC.

Patient feedback for safety improvement in primary care: results from a feasibility study

OBJECTIVES

Patient involvement in safety improvement is a developing area of research. The aim of this study was to investigate the feasibility of a patient feedback on safety intervention in primary care. Specifically, the intervention acceptability, fidelity, implementation enablers and barriers, scalability, and process of systematically collecting safety data were examined.

DESIGN, SETTING AND PARTICIPANTS

Mixed-methods feasibility trial with six purposively selected Australian primary care practices.

INTERVENTION

The intervention comprised an iterative process with a cycle of measurement, learning, feedback, action planning and implementation period of 6 months.

PRIMARY AND SECONDARY OUTCOMES

Qualitative and quantitative data relating to feasibility measures (acceptability, fidelity, enablers, barriers, scalability and process of collecting safety data) were collected and analysed.

RESULTS

A total of n=1750 patients provided feedback on safety. There was a statistically significant increase in mean patient safety scores indicating improved safety (4.30-4.37, p=0.002). Staff deemed the intervention acceptable, with minor recommendations for improvement. Intervention fidelity was high and implementation enablers were attributed to the intervention structure and framework, use of intuitive problem-solving approaches, and multidisciplinary team involvement. Practice-based safety interventions resulted in sustainable and measurable changes to systems for safety, such as increased access to care and improved patient information accuracy.

CONCLUSIONS

The findings indicate that this innovative patient feedback on safety intervention is feasible for scale-up to a larger effectiveness trial and further spread into policy and practice. This intervention complements existing safety improvement strategies and activities, and integrates into current patient feedback service requirements for Australian primary care. Further research is needed to examine the intervention effects on safety incident reduction.

Evaluating implementation and impact of a provincial quality improvement collaborative for the management of chronic diseases in primary care: the COMPAS+ study protocol

BACKGROUND

Chronic conditions such as diabetes and chronic obstructive pulmonary disease (COPD) are common and burdensome diseases primarily managed in primary care. Yet, evidence points to suboptimal quality of care for these conditions in primary care settings. Quality improvement collaboratives (QICs) are organized, multifaceted interventions that can be effective in improving chronic disease care processes and outcomes. In Quebec, Canada, the Institut national d'excellence en santé et en services sociaux (INESSS) has developed a large-scale QIC province-wide program called COMPAS+ that aims to improve the prevention and management of chronic diseases in primary care. This paper describes the protocol for our study, which aims to evaluate implementation and impact of COMPAS+ QICs on the prevention and management of targeted chronic diseases like diabetes and COPD.

METHODS

This is a mixed-methods, integrated knowledge translation study. The quantitative component involves a controlled interrupted time series involving nine large integrated health centres in the province. Study sites will receive one of two interventions: the multifaceted COMPAS+ intervention (experimental condition) or a feedback only intervention (control condition). For the qualitative component, a multiple case study approach will be used to achieve an in-depth understanding of individual, team, organizational and contextual factors influencing implementation and effectiveness of the COMPAS+ QICs.

DISCUSSION

COMPAS+ is a QI program that is unique in Canada due to its integration within the governance of the Quebec healthcare system and its capacity to reach many primary care providers and people living with chronic diseases across the province. We anticipate that this study will address several important gaps in knowledge related to large-scale QIC projects and generate strong and useful evidence (e.g., on leadership, organizational capacity, patient involvement, and implementation) having the potential to influence the design and optimisation of future QICs in Canada and internationally.

Conceptualizing the Association Between Community Participation and CQI in Aboriginal and Torres Strait Islander PHC Services

Drawing from Australian Aboriginal and Torres Strait Islander perspectives, we conceptualize the association between community participation and continuous quality improvement (CQI) processes in Indigenous primary health care (PHC) services. Indigenous experiences of community participation were drawn from our study identifying contextual factors affecting CQI processes in high-improving PHC services. Using case study design, we collected quantitative and qualitative data at the micro-, meso-, and macro-health system level in 2014 and 2015 in six services in northern Australia. Analyzing qualitative data, we found community participation was an important contextual factor in five of the six services. Embedded in cultural foundations, cultural rules, and expectations, community participation involved interacting elements of trusting relationships in metaphorically safe spaces, and reciprocated learning about each other's perspectives. Foregrounding Indigenous perspectives on community participation might assist more effective participatory processes in Indigenous PHC including in CQI processes.

Assessing improvement capability in healthcare organisations: a qualitative study of healthcare regulatory agencies in the UK

Objectives

Healthcare regulatory agencies are increasingly concerned not just with assessing the current performance of the organisations they regulate, but with assessing their improvement capability to predict their future performance trajectory. This study examines how improvement capability is conceptualised and assessed by healthcare UK regulatory agencies.

Design

Qualitative analysis of data from six UK healthcare regulatory agencies was conducted. Three data sources were analysed using an a priori framework of eight dimensions of improvement capability identified from an extensive literature review.

Setting

The focus of the research study was the regulation of hospital-based care, which accounts for the majority of UK healthcare expenditure. Six UK regulatory agencies that review hospital care participated.

Participants

Data sources included interviews with regulatory staff (n = 48), policy documents (n = 90) and assessment reports (n = 30).

Intervention

None-this was a qualitative, observational study.

Results

This research study finds that of eight dimensions of improvement capability, process improvement and learning, and strategy and governance, dominate regulatory assessment practices. The dimension of service-user focus receives the least frequency of use. It may be that dimensions which are relatively easy to 'measure', such as documents for strategy and governance, dominate assessment processes, or there may be gaps in regulatory agencies' assessment instruments, deficits of expertise in improvement capability, or practical difficulties in operationalising regulatory agency intentions to reliably assess improvement capability.

Conclusions

The UK regulatory agencies seek to assess improvement capability to predict performance trajectories, but out of eight dimensions of improvement capability, two dominate assessment. Furthermore, the definition and meaning of assessment instruments requires development. This would strengthen the validity and reliability of agencies' assessment, diagnosis and prediction of performance trajectories, and support development of more appropriate regulatory performance interventions.

Contextual factors that influence quality improvement implementation in primary care: The role of organizations, teams, and individuals

BACKGROUND

Recent emphasis on value-based health care has highlighted the importance of quality improvement (QI) in primary care settings. QI efforts, which require providers and staff to work in cross-functional teams, may be implemented with varying levels of success, with implementation being affected by factors at the organizational, teamwork, and individual levels.

PURPOSE

The purpose of our study was to (a) identify contextual factors (organizational, teamwork, and individual) that affect implementation effectiveness of QI interventions in primary care settings and (b) compare perspectives about these factors across roles (health care administrators, physician and nonphysician clinicians, and administrative staff).

METHODS/APPROACH

We conducted semistructured interviews with 24 health care administrators, physician and nonphysician primary care providers, and administrative staff representing 10 primary care practices affiliated with one integrated delivery system.

RESULTS

Participants across all roles identified similar organizational- and team-level factors that influence QI implementation including organizational capacity to take on new initiatives (e.g., time availability of physicians), technical capability for QI (e.g., data analysis skills), and team climate (e.g., how well staff work together). There was greater variation in terms of individual-level factors, particularly perceived meaning and purpose of QI. Perceptions about value of QI ranged from positive impacts on patient care and practice competitiveness to decreased efficiency and distractions from patient care, but differences did not appear attributable to role.

CONCLUSIONS

Successful QI implementation requires effective collaboration within cross-functional teams. Additional research is needed to assess how best to employ implementation strategies that promote cross-understanding of QI among team members and, ultimately, effective implementation of QI programs.

PRACTICE IMPLICATIONS

Health care managers in primary care settings should strive to create a strong teamwork climate, reinforced by opportunities for staff in various roles to discuss QI as a collective.

Conceptualizing and assessing improvement capability: a review

Purpose

The literature is reviewed to examine how ‘improvement capability’ is conceptualized and assessed and to identify future areas for research.

Data sources

An iterative and systematic search of the literature was carried out across all sectors including healthcare. The search was limited to literature written in English.

Data extraction

The study identifies and analyses 70 instruments and frameworks for assessing or measuring improvement capability. Information about the source of the instruments, the sectors in which they were developed or used, the measurement constructs or domains they employ, and how they were tested was extracted.

Results of data synthesis

The instruments and framework constructs are very heterogeneous, demonstrating the ambiguity of improvement capability as a concept, and the difficulties involved in its operationalisation. Two-thirds of the instruments and frameworks have been subject to tests of reliability and half to tests of validity. Many instruments have little apparent theoretical basis and do not seem to have been used widely.

Conclusion

The assessment and development of improvement capability needs clearer and more consistent conceptual and terminological definition, used consistently across disciplines and sectors. There is scope to learn from existing instruments and frameworks, and this study proposes a synthetic framework of eight dimensions of improvement capability. Future instruments need robust testing for reliability and validity. This study contributes to practice and research by presenting the first review of the literature on improvement capability across all sectors including healthcare.

Evaluating for impact: what type of data can assist a health promoting school approach?

There is debate within the health promoting school (HPS) movement on whether schools should monitor health behaviour outcomes as part of an evaluation or rely more on process type measures, such as changes to school policies and the physical and social environment which yield information about (in)effective implementation. The debate is often framed around ideological considerations of the role of schools and there is little empirical work on how these indicators of effective implementation can influence change at a policy and practice level in real world settings. Information has potentially powerful effects in motivating a change process, but this will vary according to the type of information and the type of organizational culture into which it is presented. The current predominant model relies on process data, policy and environmental audit monitoring and benchmarking approaches, and there is little evidence of whether this engages school communities. Theoretical assertions on the importance of monitoring data to motivate change need to be empirically tested and, in doing so, we can learn which types of data influence adoption of HPS in which types of school and policy contexts.

Effects of external inspection on sepsis detection and treatment: a study protocol for a quasiexperimental study with a stepped-wedge design

INTRODUCTION

Inspections are widely used in health care as a means to improve the health services delivered to patients. Despite their widespread use, there is little evidence of their effect. The mechanisms for how inspections can promote change are poorly understood. In this study, we use a national inspection campaign of sepsis detection and initial treatment in hospitals as case to: (1) Explore how inspections affect the involved organizations. (2) Evaluate what effect external inspections have on the process of delivering care to patients, measured by change in indicators reflecting how sepsis detection and treatment is carried out. (3) Evaluate whether external inspections affect patient outcomes, measured as change in the 30-day mortality rate and length of hospital stay.

METHODS AND ANALYSIS

The intervention that we study is inspections of sepsis detection and treatment in hospitals. The intervention will be rolled out sequentially during 12 months to 24 hospitals. Our effect measures are change on indicators related to the detection and treatment of sepsis, the 30-day mortality rate and length of hospital stay. We collect data from patient records at baseline, before the inspections, and at 8 and 14 months after the inspections. We use logistic regression models and linear regression models to compare the various effect measurements between the intervention and control periods. All the models will include time as a covariate to adjust for potential secular changes in the effect measurements during the study period. We collect qualitative data before and after the inspections, and we will conduct a thematic content analysis to explore how inspections affect the involved organisations.

ETHICS AND DISSEMINATION

The study has obtained ethical approval by the Regional Ethics Committee of Norway Nord and the Norwegian Data Protection Authority. It is registered at www.clinicaltrials.gov (Identifier: NCT02747121). Results will be reported in international peer-reviewed journals.

TRIAL REGISTRATION

NCT02747121; Pre-results.

Assessment for improving the performance of NICUs: The Italian experience

This paper describes neonatal intensive care units (NICU) in Italy using a set of organizational variables identifying management profile. The correlations between variables, and how these are impacted by structural aspects of the department or hospital, are investigated. The research was conducted within the 5-year far-reaching and complex SONAR study run by the Italian Neonatal Network, which maps NICU, monitors outcomes of member centres, defines organizational models, and identifies best practices to improve care quality. Seven variables relating to activities, organization processes, and behaviour models used in the SONAR study were used here to assess NICU. Data from 54 Italian NICUs, 1601 nursing staff, and 643 doctors were used. We identified high levels of variation in NICUs for all aspects of organization. We also identified important opportunities for improvement, especially in the areas of performance measurement, quality improvement, and learning for healthcare staff. In terms of structural characteristics, we identified big differences between NICUs in the north and south of Italy. The findings provide a description of NICUs in Italy and identify a set of variables useful for management in assessing NICU, which are among the most complex and costly operational units in a hospital.

Psychometric properties of implementation measures for public health and community settings and mapping of constructs against the Consolidated Framework for Implementation Research: a systematic review

BACKGROUND

Recent reviews have synthesised the psychometric properties of measures developed to examine implementation science constructs in healthcare and mental health settings. However, no reviews have focussed primarily on the properties of measures developed to assess innovations in public health and community settings. This review identified quantitative measures developed in public health and community settings, examined their psychometric properties, and described how the domains of each measure align with the five domains and 37 constructs of the Consolidated Framework for Implementation Research (CFIR).

METHODS

MEDLINE, PsycINFO, EMBASE, and CINAHL were searched to identify publications describing the development of measures to assess implementation science constructs in public health and community settings. The psychometric properties of each measure were assessed against recommended criteria for validity (face/content, construct, criterion), reliability (internal consistency, test-retest), responsiveness, acceptability, feasibility, and revalidation and cross-cultural adaptation. Relevant domains were mapped against implementation constructs defined by the CFIR.

RESULTS

Fifty-one measures met the inclusion criteria. The majority of these were developed in schools, universities, or colleges and other workplaces or organisations. Overall, most measures did not adequately assess or report psychometric properties. Forty-six percent of measures using exploratory factor analysis reported >50 % of variance was explained by the final model; none of the measures assessed using confirmatory factor analysis reported root mean square error of approximation (<0.06) or comparative fit index (>0.95). Fifty percent of measures reported Cronbach's alpha of <0.70 for at least one domain; 6 % adequately assessed test-retest reliability; 16 % of measures adequately assessed criterion validity (i.e. known-groups); 2 % adequately assessed convergent validity (r > 0.40). Twenty-five percent of measures reported revalidation or cross-cultural validation. The CFIR constructs most frequently assessed by the included measures were relative advantage, available resources, knowledge and beliefs, complexity, implementation climate, and other personal resources (assessed by more than ten measures). Five CFIR constructs were not addressed by any measure.

CONCLUSIONS

This review highlights gaps in the range of implementation constructs that are assessed by existing measures developed for use in public health and community settings. Moreover, measures with robust psychometric properties are lacking. Without rigorous tools, the factors associated with the successful implementation of innovations in these settings will remain unknown.

Organizational Capabilities for Integrating Care: A Review of Measurement Tools

The success of integrated care interventions is highly dependent on the internal and collective capabilities of the organizations in which they are implemented. Yet, organizational capabilities are rarely described, understood, or measured with sufficient depth and breadth in empirical studies or in practice. Assessing these capabilities can contribute to understanding why some integrated care interventions are more effective than others. We identified, organized, and assessed survey instruments that measure the internal and collective organizational capabilities required for integrated care delivery. We conducted an expert consultation and searched Medline and Google Scholar databases for survey instruments measuring factors outlined in the Context and Capabilities for Integrating Care Framework. A total of 58 instruments were included in the review and assessed based on their psychometric properties, practical considerations, and applicability to integrated care efforts. This study provides a bank of psychometrically sound instruments for describing and comparing organizational capabilities. Greater use of these instruments across integrated care interventions and studies can enhance standardized comparative analyses and inform change management. Further research is needed to build an evidence base for these instruments and to explore the associations between organizational capabilities and integrated care processes and outcomes.

Combining administrative data feedback, reflection and action planning to engage primary care professionals in quality improvement: qualitative assessment of short term program outcomes

BACKGROUND

Improving primary care for chronic disease management requires a coherent, integrated approach to quality improvement. Evidence in the continuing professional development (CPD) field suggests the importance of using strategies such as feedback delivery, reflective practice and action planning to facilitate recognition of gaps and service improvement needs. Our study explored the outcomes of a CPD intervention, named the COMPAS Project, which consists of a three-hour workshop composed of three main activities: feedback, critical reflection and action planning. The feedback intervention is delivered face-to-face and presents performance indicators extracted from clinical-administrative databases. This aim of this study was to assess the short term outcomes of this intervention to engage primary care professional in continuous quality improvement (QI).

METHODS

In order to develop an understanding of our intervention and of its short term outcomes, a program evaluation approach was used. Ten COMPAS workshops on diabetes management were directly observed and qualitative data was collected to assess the intervention short term outcomes. Data from both sources were combined to describe the characteristics of action plans developed by professionals. Two independent coders analysed the content of these plans to assess if they promoted engagement in QI and interprofessional collaboration.

RESULTS

During the ten workshops held, 26 interprofessional work teams were formed. Twenty-two of them developed a QI project they could implement themselves and that targeted aspects of their own practice they perceived in need of change. Most frequently prioritized strategies for change were improvement of systematic clientele follow-up, medication compliance, care pathway and support to improve adoption of healthier life habits. Twenty-one out of 22 action plans were found to target some level of improvement of interprofessional collaboration in primary care.

DISCUSSION

Our study results demonstrate that the COMPAS intervention enabled professionals to target priorities for practice improvements and to develop action plans that promote interprofessional collaboration. The COMPAS intervention aims to increase capability for continuous QI, readiness to implement process of care changes and team shared goals but available resources, climate and culture for change and leadership, are also important required conditions to successfully implement these practice changes.

CONCLUSION

We think that the proposed approach can be very useful to support and engage primary care professionals in the planning stage of quality improvement projects since it combines key successful ingredients: feedback, reflection and planning of action.

Taking the Pulse of a Target Population: Assessing Change Readiness

OBJECTIVE

Implementing change in a health care organization poses structural and cultural challenges.

DESIGN

We developed a survey derived from an existing organizational change model and distributed it to the target population of an impending change initiative.

RESULTS

The respondents (59.2% response rate) indicated that changes in communication protocols were necessary, believed that change would be possible, reported feeling supported during previous change efforts, and tended to think that their feedback had been valued in the past.

CONCLUSIONS

The survey revealed that our target population was ready to change the ways in which it communicates about patient needs and care protocols. To increase the probability of successful implementation of an initiative to improve communication protocols on the unit, we will design our implementation strategy to provide both the process support and the training support that participants perceived to be lacking.

IMPLICATIONS

By taking the "pulse" of the target population, change agents can identify more effective implementation strategies.

Facility characteristics and quality of lung cancer care in an integrated health care system

INTRODUCTION

In a national, integrated health care system, we sought to identify facility-level attributes associated with better quality of lung cancer care.

METHODS

Adherence to 23 quality indicators across four domains (Diagnosis and Staging, Treatment, Supportive Care, End-of-Life Care) was assessed through abstraction of electronic records from 4804 lung cancer patients diagnosed in 2007 at 131 Veterans Health Administration facilities. Performance was reported as proportions of eligible patients fulfilling adherence criteria. With stratification of patients by stage, generalized estimating equations identified facility-level characteristics associated with performance by domain.

RESULTS

Overall performance was high for the older (mean age 67.7 years, SD 9.4 years), predominantly male (98%) veterans. However, no facility did well on every measure, and range of adherence across facilities was large; 9% of facilities were in the highest quartile for one or more domain of care, more than 30% for two, and 65% for three. No facility performed consistently well across all domains. Less than 1% performed in the lowest quartile for all. Few facility-level characteristics were associated with care quality. For End-of-Life Care, diagnosis and treatment within the same facility, availability of cancer psychiatry/psychology consultation services, and availability of both inpatient and outpatient palliative care consultation services were associated with better adherence.

CONCLUSIONS

Quality of Veterans Health Administration lung cancer care is generally high, though substantial variation exists across facilities. With the exception of the salutary impact of palliative care consultation services on end-of-life quality of care, observed facility-level characteristics did not consistently predict adherence to indicators, suggesting quality may be determined by complex local factors that are difficult to measure.

One size does not fit all: a qualitative content analysis of the importance of existing quality improvement capacity in the implementation of Releasing Time to Care: the Productive Ward™ in Saskatchewan, Canada

Background

Releasing Time to Care: The Productive Ward™ (RTC) is a method for conducting continuous quality improvement (QI). The Saskatchewan Ministry of Health mandated its implementation in Saskatchewan, Canada between 2008 and 2012. Subsequently, a research team was developed to evaluate its impact on the nursing unit environment. We sought to explore the influence of the unit’s existing QI capacity on their ability to engage with RTC as a program for continuous QI.

Methods

We conducted interviews with staff from 8 nursing units and asked them to speak about their experience doing RTC. Using qualitative content analysis, and guided by the Organizing for Quality framework, we describe the existing QI capacity and impact of RTC on the unit environment.

Results

The results focus on 2 units chosen to highlight extreme variation in existing QI capacity. Unit B was characterized by a strong existing environment. RTC was implemented in an environment with a motivated manager and collaborative culture. Aided by the structural support provided by the organization, the QI capacity on this unit was strengthened through RTC. Staff recognized the potential of using the RTC processes to support QI work. Staff on unit E did not have the same experience with RTC. Like unit B, they had similar structural supports provided by their organization but they did not have the same existing cultural or political environment to facilitate the implementation of RTC. They did not have internal motivation and felt they were only doing RTC because they had to. Though they had some success with RTC activities, the staff did not have the same understanding of the methods that RTC could provide for continuous QI work.

Conclusions

RTC has the potential to be a strong tool for engaging units to do QI. This occurs best when RTC is implemented in a supporting environment. One size does not fit all and administrative bodies must consider the unique context of each environment prior to implementing large-scale QI projects. Use of an established framework, like Organizing for Quality, could highlight the distinctive supports needed in particular care environments to increase the likelihood of successful engagement.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-014-0642-x) contains supplementary material, which is available to authorized users.

Understanding context in knowledge translation: a concept analysis study protocol

AIM

To conduct a concept analysis of clinical practice contexts (work environments) that facilitate or militate against the uptake of research evidence by healthcare professionals in clinical practice. This will involve developing a clear definition of context by describing its features, domains and defining characteristics.

BACKGROUND

The context where clinical care is delivered influences that care. While research shows that context is important to knowledge translation (implementation), we lack conceptual clarity on what is context, which contextual factors probably modify the effect of knowledge translation interventions (and hence should be considered when designing interventions) and which contextual factors themselves could be targeted as part of a knowledge translation intervention (context modification).

DESIGN

Concept analysis.

METHODS

The Walker and Avant concept analysis method, comprised of eight systematic steps, will be used: (1) concept selection; (2) determination of aims; (3) identification of uses of context; (4) determination of defining attributes of context; (5) identification/construction of a model case of context; (6) identification/construction of additional cases of context; (7) identification/construction of antecedents and consequences of context; and (8) definition of empirical referents of context. This study is funded by the Canadian Institutes of Health Research (January 2014).

DISCUSSION

This study will result in a much needed framework of context for knowledge translation, which identifies specific elements that, if assessed and used to tailor knowledge translation activities, will result in increased research use by nurses and other healthcare professionals in clinical practice, ultimately leading to better patient care.

Development and pilot study of the Primary Care Practice Improvement Tool (PC-PIT): an innovative approach

OBJECTIVE

To assess the usability and validity of the Primary Care Practice Improvement Tool (PC-PIT), a practice performance improvement tool based on 13 key elements identified by a systematic review. It was co-created with a range of partners and designed specifically for primary health care.

DESIGN

This pilot study examined the PC-PIT using a formative assessment framework and mixed-methods research design.

SETTING AND PARTICIPANTS

Six high-functioning general practices in Queensland, Australia, between February and July 2013. A total of 28 staff participated - 10 general practitioners, six practice or community nurses, 12 administrators (four practice managers; one business manager and eight reception or general administrative staff).

MAIN OUTCOME MEASURES

Readability, content validity and staff perceptions of the PC-PIT.

RESULTS

The PC-PIT offers an appropriate and acceptable approach to internal quality improvement in general practice. Quantitative assessment scores and qualitative data from all staff identified two areas in which the PC-PIT required modification: a reduction in the indicative reading age, and simplification of governance-related terms and concepts.

CONCLUSION

The PC-PIT provides an innovative approach to address the complexity of organisational improvement in general practice and primary health care. This initial validation will be used to develop a suite of supporting, high-quality and free-to-access resources to enhance the use of the PC-PIT in general practice. Based on these findings, a national trial is now underway.

Key elements of high-quality practice organisation in primary health care: a systematic review

OBJECTIVES

To identify elements that are integral to high-quality practice and determine considerations relating to high-quality practice organisation in primary care.

STUDY DESIGN

A narrative systematic review of published and grey literature.

DATA SOURCES

Electronic databases (PubMed, CINAHL, the Cochrane Library, Embase, Emerald Insight, PsycInfo, the Primary Health Care Research and Information Service website, Google Scholar) were searched in November 2013 and used to identify articles published in English from 2002 to 2013. Reference lists of included articles were searched for relevant unpublished articles and reports.

DATA SYNTHESIS

Data were configured at the study level to allow for the inclusion of findings from a broad range of study types. Ten elements were most often included in the existing organisational assessment tools. A further three elements were identified from an inductive thematic analysis of descriptive articles, and were noted as important considerations in effective quality improvement in primary care settings.

CONCLUSION

Although there are some validated tools available to primary care that identify and build quality, most are single-strategy approaches developed outside health care settings. There are currently no validated organisational improvement tools, designed specifically for primary health care, which combine all elements of practice improvement and whose use does not require extensive external facilitation.

A qualitative study of contextual factors' impact on measures to reduce surgery cancellations

BACKGROUND

Contextual factors influence quality improvement outcomes. Understanding this influence is important when adapting and implementing interventions and translating improvements into new settings. To date, there is limited knowledge about how contextual factors influence quality improvement processes. In this study, we explore how contextual factors affected measures to reduce surgery cancellations, which are a persistent problem in healthcare. We discuss the usefulness of the theoretical framework provided by the model for understanding success in quality (MUSIQ) for this kind of research.

METHOD

We performed a qualitative case study at Førde Hospital, Norway, where we had previously demonstrated a reduction in surgery cancellations. We interviewed 20 clinicians and performed content analysis to explore how contextual factors affected measures to reduce cancellations of planned surgeries.

RESULTS

We identified three common themes concerning how contextual factors influenced the change process: 1) identifying a need to change, 2) facilitating system-wide improvement, and 3) leader involvement and support. Input from patients helped identify a need to change and contributed to the consensus that change was necessary. Reducing cancellations required improving the clinical system. This improvement process was based on a strategy that emphasized the involvement of frontline clinicians in detecting and improving system problems. Clinicians shared information about their work by participating in improvement teams to develop a more complete understanding of the clinical system and its interdependencies. This new understanding allowed clinicians to detect system problems and design adequate interventions. Middle managers' participation in the improvement teams and in regular work processes was important for successfully implementing and adapting interventions.

CONCLUSION

Contextual factors interacted with one another and with the interventions to facilitate changes in the clinical system, reducing surgery cancellations. The MUSIQ framework is useful for exploring how contextual factors influence the improvement process and how they influence one another. Discussing data in relation to a theoretical framework can promote greater uniformity in reporting findings, facilitating knowledge-building across studies.

How can we recognize continuous quality improvement?

Objective

Continuous quality improvement (CQI) methods are foundational approaches to improving healthcare delivery. Publications using the term CQI, however, are methodologically heterogeneous, and labels other than CQI are used to signify relevant approaches. Standards for identifying the use of CQI based on its key methodological features could enable more effective learning across quality improvement (QI) efforts. The objective was to identify essential methodological features for recognizing CQI.

Design

Previous work with a 12-member international expert panel identified reliably abstracted CQI methodological features. We tested which features met rigorous a priori standards as essential features of CQI using a three-phase online modified-Delphi process.

Setting

Primarily United States and Canada.

Participants

119 QI experts randomly assigned into four on-line panels.

Intervention(s)

Participants rated CQI features and discussed their answers using online, anonymous and asynchronous discussion boards. We analyzed ratings quantitatively and discussion threads qualitatively.

Main outcome measure(s)

Panel consensus on definitional CQI features.

Results

Seventy-nine (66%) panelists completed the process. Thirty-three completers self-identified as QI researchers, 18 as QI practitioners and 28 as both equally. The features ‘systematic data guided activities,’ ‘designing with local conditions in mind’ and ‘iterative development and testing’ met a priori standards as essential CQI features. Qualitative analyses showed cross-cutting themes focused on differences between QI and CQI.

Conclusions

We found consensus among a broad group of CQI researchers and practitioners on three features as essential for identifying QI work more specifically as ‘CQI.’ All three features are needed as a minimum standard for recognizing CQI methods.

Measuring team factors thought to influence the success of quality improvement in primary care: a systematic review of instruments

BACKGROUND

Measuring team factors in evaluations of Continuous Quality Improvement (CQI) may provide important information for enhancing CQI processes and outcomes; however, the large number of potentially relevant factors and associated measurement instruments makes inclusion of such measures challenging. This review aims to provide guidance on the selection of instruments for measuring team-level factors by systematically collating, categorizing, and reviewing quantitative self-report instruments.

METHODS

DATA SOURCES

We searched MEDLINE, PsycINFO, and Health and Psychosocial Instruments; reference lists of systematic reviews; and citations and references of the main report of instruments.

STUDY SELECTION

To determine the scope of the review, we developed and used a conceptual framework designed to capture factors relevant to evaluating CQI in primary care (the InQuIRe framework). We included papers reporting development or use of an instrument measuring factors relevant to teamwork. Data extracted included instrument purpose; theoretical basis, constructs measured and definitions; development methods and assessment of measurement properties. Analysis and synthesis: We used qualitative analysis of instrument content and our initial framework to develop a taxonomy for summarizing and comparing instruments. Instrument content was categorized using the taxonomy, illustrating coverage of the InQuIRe framework. Methods of development and evidence of measurement properties were reviewed for instruments with potential for use in primary care.

RESULTS

We identified 192 potentially relevant instruments, 170 of which were analyzed to develop the taxonomy. Eighty-one instruments measured constructs relevant to CQI teams in primary care, with content covering teamwork context (45 instruments measured enabling conditions or attitudes to teamwork), team process (57 instruments measured teamwork behaviors), and team outcomes (59 instruments measured perceptions of the team or its effectiveness). Forty instruments were included for full review, many with a strong theoretical basis. Evidence supporting measurement properties was limited.

CONCLUSIONS

Existing instruments cover many of the factors hypothesized to contribute to QI success. With further testing, use of these instruments measuring team factors in evaluations could aid our understanding of the influence of teamwork on CQI outcomes. Greater consistency in the factors measured and choice of measurement instruments is required to enable synthesis of findings for informing policy and practice.