Deepening our understanding of quality improvement in Europe (DUQuE): overview of a study of hospital quality management in seven countries

A multiphase, multicentre development and validation of two maturity tools assessing the implementation of the FlaQuM co-creation roadmap

As part of the new Flanders Quality Model (FlaQuM) towards sustainable quality management systems, a co-creation roadmap with 6 primary drivers and 19 building blocks that guides healthcare organizations has been developed. Currently, no assessment tool is available to monitor hospitals' quality management systems implementation according to this co-creation roadmap. Therefore, we aimed to measure the maturity of the implementation of the FlaQuM co-creation roadmap in hospitals. A three-phase approach in co-design with 19 hospitals started with defining the scope, followed by establishing content validity through a literature review, involvement of content experts (n = 47), 20 focus groups with content experts (n = 79), and a Delphi round with healthcare quality managers (n = 19) to test the content validity index. Construct validity was assessed by confirmatory factor analyses and convergent validity by Spearman's ρ correlation coefficients. Based on 17 included existing maturity instruments and subcomponents of content experts, two maturity tools were developed according to the implementation of the FlaQuM co-creation roadmap: (i) a maturity matrix with 52 subcomponents and (ii) a co-creation scan with 19 statements. The overall scale-content validity index varied between 93.3% and 90.0% in terms of relevance and clarity, respectively. In a sample of 119 healthcare professionals, factor analyses revealed a six-factor structure and 16 (84.2%) of the 19 hypothesis for testing convergent validity between both maturity tools were statistically significant. Measuring the implementation of the FlaQuM co-creation roadmap and monitoring its maturity over time should be feasible by using these comprehensive maturity tools in hospitals. Results of both tools should be able to describe the current state of hospitals' implementation of the co-creation roadmap as basis for strategic improvement plans and next steps.

Correlation Between Teamwork and Patient Safety in a Tertiary Hospital in Cyprus

Background Over time, the multidimensional nature of the safety culture in the healthcare field has led to great efforts to improve quality and create tools aiming at enhancing safety. In particular, emphasis has been placed on teamwork and the safety climate. There is a strong relationship between these two complex elements, which interact to improve the safety climate and reduce patient-safety issues. In this study, "teamwork" includes the perceptions of the health professionals collaborating within a health team to provide safe patient care, and "safety climate" refers to the professional commitment to patient safety. Objective This article assesses health professionals' perceptions of both patient-safety issues and teamwork in their hospital work environment after the development and implementation of a comprehensive quality-assurance system. Methods This descriptive correlation study is based on anonymous and self-completed questionnaires obtained after the development and implementation of a comprehensive quality assurance system in the wards and departments of Nicosia General Hospital. The research sample consisted of the health professionals who participated in the working groups that implemented the quality assurance system. We used the questionnaire's sociodemographic data and the Safety Attitudes Questionnaire (SAQ) developed in the Deepening our Understanding of Quality Improvement in Europe program, focusing on two factors: Teamwork and the safety climate. Results While teamwork received a positive score (>75%), the same did not occur for the safety climate (68.60%). Women typically rated the safety climate more positively than men, who mostly gave negative ratings (p = 0.005). There was a statistically significant difference (p = 0.011) in the scores between participants aged 24-44 and those aged 45-54, with the latter reporting higher teamwork scores. The participants' educational levels also played important roles in their responses, with university graduates (BSc) providing more positive teamwork scores than those with a master's degree (p = 0.018). Conclusions Our research revealed that the health professionals of Nicosia General Hospital perceived the teamwork climate as positive, in contrast to the safety climate. The results highlight the need not only to intervene in all the areas covered by the SAQ to improve the safety climate but also to keep encouraging teamwork to obtain better results.

Exploring the actionability of healthcare performance indicators for quality of care: a qualitative analysis of the literature, expert opinion and user experience

BACKGROUND

This study explores the meaning of actionable healthcare performance indicators for quality of care-related decisions. To do so, we analyse the constructs of fitness for purpose and fitness for use across healthcare systems and in practice based on the literature, expert opinion and user experience.

METHODS

A multiphase qualitative study was undertaken. Phases included a literature review, a first round of one-on-one interviews with a panel of academics and thought leaders in the field (n=16), and a second round of interviews with real-world users of performance indicators (n=16). Thematic analysis was conducted between phases in order to triangulate findings in a stepwise process.

RESULTS

Common uses of healthcare performance indicators were differentiated within micro-meso-macro contexts of healthcare systems. Each purpose of use signals different decision-making tasks, and in effect information needs. An indicator's fitness for use can be appraised by three clusters of considerations: methodological, contextual and managerial. Methodological considerations gauge an indicator's perceived importance, engagement potential, interpretability, standardisation, feasibility of remedial actions, alignment to care models and sensitivity to change. Information infrastructure, system governance, workforce capacity and learning culture were found as enabling contextual considerations. Managerial considerations influencing an indicator's use in practice were found to span the selection of indicators, data collection, analysis, display of results and delivery of information to decision-makers.

CONCLUSION

The actionability of a healthcare performance indicator should be appraised by its alignment with the intended purpose of use beyond aggregate healthcare system levels, in combination with the extent to which methodological, contextual and managerial fitness for use considerations are met. Striking a better balance between the importance weighted to an indicator's statistical merits and emphasis put to its fitness for purpose and use is needed for indicators that are ultimately actionable for quality of care-related decision-making.

Safety climate, leadership and patient views associated with hip fracture care quality and clinician perceptions of hip fracture care performance

BACKGROUND

Hip fracture is a major public health concern for older adults, requiring surgical treatment for patients presenting at hospitals across Australia. Although guidelines have been developed to drive appropriate care of hip fracture patients in hospitals, data on health outcomes suggest these are not well-followed.

OBJECTIVE

This study aims to examine whether clinician measures of safety, teamwork and leadership, and patient perceptions of care are associated with key indicators of hip fracture care and the extent to which there is agreement between clinician perceptions of hip fracture care performance and actual hospital performance of hip fracture care.

METHODS

Retrospective analysis was performed on a series of questionnaires used to assess hospital department- and patient-level measures from the Deepening our Understanding of Quality in Australia study. Data were analysed from 32 public hospitals that encompassed 23 leading hip fracture clinicians, 716 patient medical records and 857 patients from orthopaedic public hospital wards.

RESULTS

Aggregated across all hospitals, only 5 of 12 of the key hip fracture indicators had ≥50% adherence. Adherence to indicators requiring actions to be performed within a recommended time period was poor (7.2-25.6%). No Patient Measure of Safety or clinician-based measures of teamwork, safety climate or leadership were associated with adherence to key indicators of hip fracture care. Simple proportionate agreement between clinician perceptions and actual hospital performance was generally strong, but few agreement coefficients were compelling.

CONCLUSION

The development of strong quality management processes requires ongoing effort. The findings of this study provide important insights into the relationship between hospital care and outcomes for hip fracture patients and could drive the design of targeted interventions for improved quality assurance of hip fracture care.

Measuring social capital of hospital management boards in European hospitals: A validation study on psychometric properties of a questionnaire for Chief Executive Officers

Background

The commitment of hospital managers plays a key role in decisions regarding investments in quality improvement (QI) and the implementation of quality improvement systems (QIS). With regard to the concept of social capital, successful cooperation and coordination among hospital management board members is strongly influenced by commonly shared values and mutual trust. The purpose of this study is to investigate the reliability and validity of a survey scale designed to assess Social Capital within hospital management boards (SOCAPO-B) in European hospitals.

Methods

Data were collected as part of the EU funded mixed-method project “Deepening our understanding of quality improvement in Europe (DUQuE)” from 210 hospitals in 7 European countries (France, Poland, Czech Republic, Germany, Portugal, Spain, and Turkey). The Chief Executive Officers (CEOs) completed the SOCAPO-B scale (six-item survey, numeric scale, 1=‘strongly disagree’ to 4=‘strongly agree’) regarding their perceptions of social capital within the hospital management board. We investigated the factor structure of the social capital scale using exploratory and confirmatory factor analyses. Internal consistency was assessed using Cronbach’s alpha, while construct validity was assessed through Pearson’s correlation coefficients between the scale items.

Results

A total of 188 hospitals participated in the DUQuE-study. Of these, 177 CEOs completed the questionnaire(172 observations for social capital) Hospital CEOs perceive relatively high social capital among hospital management boards (average SOCAPO-B mean of 3.2, SD = 0.61). The exploratory factor analysis resulted in a 1-factor-model with Cronbach’s alpha of 0.91. Pearson’s correlation coefficients between the single scale items ranged from 0.48 to 0.68.

Conclusions

The SOCAPO-B₋scale can be used to obtain reliable and valid measurements of social capital in European hospital management boards, at least from the CEO’s point of view. The brevity of the scale enables it to be a cost-effective and tool for measuring social capital in hospital management boards.

Trial registration

This validation study was not registered.

Patient-Reported Experience Measures in Stroke Care: A Systematic Review

BACKGROUND AND PURPOSE

Patient-reported experience measures (PREMs) assess patients' perception of health care. We aimed to identify all reported PREMs for stroke care and critically appraise psychometric properties of PREMs validated for patients with stroke.

METHODS

Studies on the development, validation, or utilization of PREMs for adult patients with stroke were systematically identified. The Consensus-Based Standards for the Selection of Health Measurement Instruments criteria were used to appraise psychometric performance.

RESULTS

We included 18 studies, examining 13 PREMs. Two PREMs had been developed for stroke care: Consumer Quality Index: Cerebrovascular Accident and Riksstroke. Consumer Quality Index: Cerebrovascular Accident was given a positive psychometric assessment, but its length and limited language applicability impede clinical implementation. Riksstroke was appraised as doubtful. Eleven PREMs were generic. The psychometric performance of 5 generic PREMS, validated for patients with stroke, received conflicting assessments. Six generic PREMs had not been validated in patients with stroke and were therefore not assessed for instrument performance.

CONCLUSIONS

Thirteen PREMs have been published for use in stroke care. The stroke-specific Consumer Quality Index: Cerebrovascular Accident has favorable psychometric performance but lacks practical feasibility. Other PREMs have inadequate or unknown psychometric properties. This indicates the need for developing stroke-specific PREMs to support quality improvement and enhance patient-centered care.

Social determinants of the impact of hospital management boards on quality management: a study of 109 European hospitals using a parsonian approach

Background

The consolidated framework for implementation research states that personal leadership matters in quality management implementation. However, it remains to be answered which characteristics of plural leadership in hospital management boards make them impactful. The present study focuses on social determinants of implementation power of hospital boards using Talcott Parsons’ sociological concept of adaptation, goal attainment, integration, and latency (AGIL), focusing on the G (goal attainment) and I (integration) factors of this concept. The study aims to test the hypothesis that hospitals with management boards that are oriented toward the quality goal (G) and socially integrated (I) (GI boards) are better at implementing quality management than hospitals with boards lacking these characteristics (non-GI boards).

Methods

A cross-sectional mixed-method design was used for data collection in 109 randomly selected hospitals in seven European countries. Data is based on the study “Deepening our understanding of quality improvement in Europe” (DUQUE). We used responses from (a) hospitals’ chief executive officers to measure the variable social integration and the variable quality orientation of the board and (b) responses from quality managers to measure the degree of implementation of the quality management system. We developed the GI index measuring the combination of goal-orientation and integration. A multiple linear regression analysis was performed.

Results

Hospitals with management boards that are quality oriented and socially integrated (GI boards) had significantly higher scores on the quality management system index than hospitals with boards scoring low on these features, when controlled for several context factors.

Conclusions

Our findings suggest that the implementation power of hospital management boards is higher if there is a sense of unity and purpose within the boards. Thus, to improve quality management, it could be worthwhile to increase boards’ social capital and to increase time designated for quality management in board meetings.

Contribution of Healthcare Professionals in Issues that Relate to Quality Management

Background:

The health sector should adopt integrated quality systems because of the need to survive and develop in a highly competitive environment. Inefficiency of mechanistic procedures, along with inadequate administrative infrastructure, impose innovative appoaches to improve operations and increase revenues by reducing quality feilures.

Objective:

A health system that relies on quality healthcare services can directly benefit the entire society, may reduce mortality, disease severity, and increase life expectancy. The following literature review constitutes an attempt to assess the contribution of healthcare professionals in issues that relate to quality management over the course of recent years.

Methods:

This systematic review took place between May 2019 and June 2020 in the databases PubMed, Cochrane Library, Wiley Online Library, Web of Science, Google Scholar and Scopus search engine databases. Study Selection and Data Extraction: This review includes articles written in English language, which contain quantitative and qualitative analysis of healthcare professionals’ involvement in quality activities. Correspondingly, the exclusion criteria were: languages other than English, secondary surveys (general and systematic reviews or post-analyses), letters to the publisher, and editorials or articles that did not illuminate the subject under study. After an extensive literature review, a standardised Excel spreadsheet was developed for data extraction from the included studies. The main characteristics of the studies were recorded (author’s name, place and time of work, the article under study and the methodology) so that all research articles corresponding to the review could be included. 31 articles were included.

Results:

Healthcare professionals are engaged in quality improvement activities and there is high association between quality management strategies and clinical processes. A systematic approach on healthcare activities based on the input of healthcare professionals can help increase business performance, reduce errors, improve patient safety, and contribute to a more proactive care.

Conclusion:

Health professionals’ contribution in the strategic planning of healthcare organisations that address quality activities can lead to better output, both in patient satisfaction and safety.

Using patient feedback to drive quality improvement in hospitals: a qualitative study

OBJECTIVES

Although different forms of patient feedback are available, their use in hospital management is still limited. The objective of this study is to explore how patient feedback is currently used in hospitals to improve quality.

DESIGN

This is a qualitative exploratory multiple case study. Data collection included nine interviews, of an average duration of 50 min, conducted between March and June 2019. Additionally, a document and secondary data analysis were performed.

SETTING

This study was conducted in three Brazilian hospitals selected for their solid patient feedback practises.

PARTICIPANTS

Managers from the customer service, quality, nursing, operations, projects and patient experience departments of the three hospitals.

RESULTS

Despite literature suggesting that organisational objectives regarding patient feedback are not clear, data show that there is managerial concern regarding the promotion of an environment capable of changing according to patient feedback. In these instances, organisational processes were structured to focus on patients' feedback and its receipt by the staff, including a non-punitive culture. Several patient feedback forms are available: voluntary events, patient surveys and informal feedback. Instruments to measure patient feedback focused on specific aspects of healthcare, to identify and clarify the problems for addressal by the management. The net promoter score was the main strategic indicator of patient feedback, used to assess the impact of improvement action.

CONCLUSIONS

The hospitals had established objectives that valued the patient's perspective. Involvement of the health team, availability of different channels for feedback and the use of quality tools are considered a good basis for using patient feedback to drive quality improvement.

Managerial features and outcome in neonatal intensive care units: results from a cluster analysis

Background

Healthcare organisations differ in performance even if they are located in the same country or region. Suitable managerial practices and organisational processes can lead to better health outcomes. As a result, hospitals are constantly looking for managerial arrangements that can improve outcomes and keep costs down. This study aims to identify different managerial models in neonatal intensive care units (NICUs) and their impact on a large number of outcomes.

Methods

The research was conducted in Italy, within the SONAR project. SONAR’s aim was to identify the characteristics of NICUs, monitor outcomes and promote best practices. This study includes 51 of the 63 NICUs that took part in the SONAR project. Questionnaires on the activities and managerial features were administered to doctors and nurses working in NICUs. A total of 643 questionnaires were analysed from doctors and a total of 1601 from nurses. A cluster analysis was performed to identify managerial models of NICUs.

Results

Three managerial models emerged from cluster analysis: traditional, collaborative and individualistic. In the “traditional” model the doctor is above the nurse in the hierarchy, and the nurse therefore has exclusively operational autonomy. The “collaborative” model has as key elements professional specialisation and functional coordination. The “individualistic” model considers only individual professional skills and does not concern the organisational conditions necessary to generate organisational effectiveness.

The results also showed that there is an association between managerial model and neonatal outcomes. The collaborative model shows best results in almost all outcomes considered, and the traditional model has the worst. The individualistic model is in the middle, although its values are very close to those of traditional model.

Conclusions

Health management needs to assess NICU strategically in order to develop models to improve outcomes. This study provides insights for management useful for designing managerial characteristics of NICUs in order to achieve better results. NICUs characterised by a collaborative model in fact show better neonatal outcomes.

Deepening our Understanding of Quality in Australia (DUQuA): An overview of a nation-wide, multi-level analysis of relationships between quality management systems and patient factors in 32 hospitals

OBJECTIVE

The Deepening our Understanding of Quality in Australia (DUQuA) project is a multisite, multi-level, cross-sectional study of 32 of the largest hospitals in Australia. This overview examines relationships between (i) organization-level quality management systems and department-level quality management strategies and (ii) patient-level measures (clinical treatment processes, patient-reported perceptions of care and clinical outcomes) within Australian hospitals.

DESIGN

We examined hospital quality improvement structures, processes and outcomes, collecting data at organization, department and patient levels for acute myocardial infarction (AMI), hip fracture and stroke. Data sources included surveys of quality managers, clinicians and patients, hospital visits, medical record reviews and national databases. Outcomes data and patient admissions data were analysed. Relationships between measures were evaluated using multi-level models. We based the methods on the Deepening our Understanding of Quality Improvement in Europe (DUQuE) framework, extending that work in parts and customizing the design to Australian circumstances.

SETTING, PARTICIPANTS AND OUTCOME MEASURES

The 32 hospitals, containing 119 participating departments, provided wide representation across metropolitan, inner and outer regional Australia. We obtained 31 quality management, 1334 clinician and 857 patient questionnaires, and conducted 2401 medical record reviews and 151 external assessments. External data via a secondary source comprised 14 460 index patient admissions across 14 031 individual patients. Associations between hospital, Emergency Department (ED) and department-level systems and strategies and five patient-level outcomes were assessed: 19 of 165 associations (11.5%) were statistically significant, 12 of 79 positive associations (15.2%) and 7 of 85 negative associations (8.2%).

RESULTS

We did not find clear relationships between hospital-level quality management systems, ED or department quality strategies and patient-level outcomes. ED-level clinical reviews were related to adherence to clinical practice guidelines for AMI, hip fracture and stroke, but in different directions. The results, when considered alongside the DUQuE results, are suggestive that front line interventions may be more influential than department-level interventions when shaping quality of care and that multi-pronged strategies are needed. Benchmark reports were sent to each participating hospital, stimulating targeted quality improvement activities.

CONCLUSIONS

We found no compelling relationships between the way care is organized and the quality of care across three targeted patient-level outcome conditions. The study was cross-sectional, and thus we recommend that the relationships studied should be assessed for changes across time. Tracking care longitudinally so that quality improvement activities are monitored and fed back to participants is an important initiative that should be given priority as health systems strive to develop their capacity for quality improvement over time.

Organization quality systems and department-level strategies: refinement of the Deepening our Understanding in Quality in Australia (DUQuA) organization and department-level scales

OBJECTIVE

The aim of this study was to develop and refine indices to measure organization and care pathway-level quality management systems in Australian hospitals.

DESIGN

A questionnaire survey and audit tools were derived from instruments validated as part of the Deepening Our Understanding of Quality improvement in Europe (DUQuE) study, adapted for Australian hospitals through expert opinion. Statistical processes were used to explore the factor structure, reliability and non-redundancy and descriptive statistics of the scales.

SETTING

Thirty-two large Australian public hospitals.

PARTICIPANTS

Audit of quality management processes at organization-level and care pathway processes at department level for three patient conditions (acute myocardial infarction (AMI), hip fracture and stroke) and senior quality manager, at each of the 32 participating hospitals.

MAIN OUTCOME MEASURE(S)

The degree of quality management evident at organization and care pathway levels.

RESULTS

Analysis yielded seven quality systems and strategies scales. The three hospital-level measures were: the Quality Management Systems Index (QMSI), the Quality Management Compliance Index (QMCI) and the Clinical Quality Implementation Index (CQII). The four department-level measures were: Specialised Expertise and Responsibility (SER), Evidence-Based Organisation of Pathways (EBOP), Patient Safety Strategies (PSS) and Clinical Review (CR). For QMCI, and for seven out of eight subscales in QMSI, adequate internal consistency (Cronbach's $\alpha$ >0.8) was achieved. For CQII, lack of variation and ceiling effects in the data resulted in very low internal consistency scores, but items were retained for theoretical reasons. Internal consistency was high for CR (Cronbach's $\alpha$ 0.74-0.88 across the three conditions), and this was supported by all item-total correlations exceeding the desired threshold. For EBOP, Cronbach's $\alpha$ was acceptable for hip fracture (0.80) and stroke (0.76), but only moderate for AMI (0.52). PSS and SER scales were retained for theoretical reasons, although internal consistencies were only moderate (SER) to poor (PSS).

CONCLUSIONS

The Deepening our Understanding of Quality in Australia (DUQuA) organization and department scales can be used by Australian hospital managers to assess and measure improvement in quality management at organization and department levels within their hospitals and are readily modifiable for other health systems depending on their needs.

The clinician safety culture and leadership questionnaire: refinement and validation in Australian public hospitals

OBJECTIVE

This study aimed to refine and validate a tool to measure safety culture and leadership in Australian hospitals.

DESIGN

The clinician safety culture and leadership questionnaire was constructed by combining and refining the following two previously validated scales: Safety Attitudes Questionnaire and the Leadership Effectiveness Survey. Statistical processes were used to explore the factor structure, reliability, validity and descriptive statistics of the new instrument.

SETTING

Thirty-two large Australian public hospitals.

PARTICIPANTS

1382 clinicians (doctors, nurses and allied health professionals).

MAIN OUTCOME MEASURE(S)

Descriptive statistics, structure and validity of clinician safety culture and leadership scale.

RESULTS

We received 1334 valid responses from participants. The distribution of ratings was left-skewed, with a small ceiling effect, meaning that scores were clustered toward the high end of the scale. Using confirmatory factor analysis, we confirmed the structure of the three scales as a combined measure of safety culture and leadership. The data were divided into equal calibration and validation datasets. For the calibration dataset, the Chi-square: df ratio was 4.4, the root mean square error of approximation RMSEA (a measure of spread of the data) was 0.071, the standardized root mean square residual SRMR (an absolute measure of the fit of the data) was 0.058 and the Confirmatory Fit Index (CFI) (another test confirming the fit of the data) was 0.82; while none of the indices suggested good fit, all but CFI fell within acceptable thresholds. All factors demonstrated adequate internal consistency and construct reliability, as desired. All three domains achieved discriminant validity through cross-loadings, meaning that the three domains were determined to be independent constructs. Results for the validation dataset were effectively identical to those found in the calibration dataset.

CONCLUSIONS

While the model may benefit from additional refinement, we have validated the tool for measuring clinician safety culture and leadership in our Australian sample. The DUQuA safety culture and leadership scale can be used by Australian hospitals to assess clinician safety culture and leadership, and is readily modifiable for other health systems depending on their needs.

Implementation and data-related challenges in the Deepening our Understanding of Quality in Australia (DUQuA) study: implications for large-scale cross-sectional research

Healthcare organisations vary in the degree to which they implement quality and safety systems and strategies. Large-scale cross-sectional studies have been implemented to explore whether this variation is associated with outcomes relevant at the patient level. The Deepening our Understanding of Quality in Australia (DUQuA) study draws from earlier research of this type, to examine these issues in 32 Australian hospitals. This paper outlines the key implementation and analysis challenges faced by DUQuA. Many of the logistical difficulties of implementing DUQuA derived from compliance with the administratively complex and time-consuming Australian ethics and governance system designed principally to protect patients involved in clinical trials, rather than for low-risk health services research. The complexity of these processes is compounded by a lack of organizational capacity for multi-site health services research; research is expected to be undertaken in addition to usual work, not as part of it. These issues likely contributed to a relatively low recruitment rate for hospitals (41% of eligible hospitals). Both sets of issues need to be addressed by health services researchers, policymakers and healthcare administrators, if health services research is to flourish. Large-scale research also inevitably involves multiple measurements. The timing for applying these measures needs to be coherent, to maximise the likelihood of finding real relationships between quality and safety systems and strategies, and patient outcomes; this timing was less than ideal in DUQuA, in part due to administrative delays. Other issues that affected our study include low response rates for measures requiring recruitment of clinicians and patients, missing data and a design that necessarily included multiple statistical comparisons. We discuss how these were addressed. Successful completion of these projects relies on mutual and ongoing commitment, and two-way communication between the research team and hospital staff at all levels. This will help to ensure that enthusiasm and engagement are established and maintained.

Bending the quality curve

With this paper, we initiate the Supplement on Deepening our Understanding of Quality in Australia (DUQuA). DUQuA is an at-scale, cross-sectional research programme examining the quality activities in 32 large hospitals across Australia. It is based on, with suitable modifications and extensions, the Deepening our Understanding of Quality improvement in Europe (DUQuE) research programme, also published as a Supplement in this Journal, in 2014. First, we briefly discuss key data about Australia, the health of its population and its health system. Then, to provide context for the work, we discuss previous activities on the quality of care and improvement leading up to the DUQuA studies. Next, we present a selection of key interventional studies and policy and institutional initiatives to date. Finally, we conclude by outlining, in brief, the aims and scope of the articles that follow in the Supplement. This first article acts as a framing vehicle for the DUQuA studies as a whole. Aggregated, the series of papers collectively attempts an answer to the questions: what is the relationship between quality strategies, both hospital-wide and at department level? and what are the relationships between the way care is organised, and the actual quality of care as delivered? Papers in the Supplement deal with a multiplicity of issues including: how the DUQuA investigators made progress over time, what the results mean in context, the scales designed or modified along the way for measuring the quality of care, methodological considerations and provision of lessons learnt for the benefit of future researchers.

Can benchmarking Australian hospitals for quality identify and improve high and low performers? Disseminating research findings for hospitals

This paper examines the principles of benchmarking in healthcare and how benchmarking can contribute to practice improvement and improved health outcomes for patients. It uses the Deepening our Understanding of Quality in Australia (DUQuA) study published in this Supplement and DUQuA’s predecessor in Europe, the Deepening our Understanding of Quality improvement in Europe (DUQuE) study, as models. Benchmarking is where the performances of institutions or individuals are compared using agreed indicators or standards. The rationale for benchmarking is that institutions will respond positively to being identified as a low outlier or desire to be or stay as a high performer, or both, and patients will be empowered to make choices to seek care at institutions that are high performers. Benchmarking often begins with a conceptual framework that is based on a logic model. Such a framework can drive the selection of indicators to measure performance, rather than their selection being based on what is easy to measure. A Donabedian range of indicators can be chosen, including structure, process and outcomes, created around multiple domains or specialties. Indicators based on continuous variables allow organizations to understand where their performance is within a population, and their interdependencies and associations can be understood. Benchmarking should optimally target providers, in order to drive them towards improvement. The DUQuA and DUQuE studies both incorporated some of these principles into their design, thereby creating a model of how to incorporate robust benchmarking into large-scale health services research.

Validation of the patient measure of safety (PMOS) questionnaire in Australian public hospitals

Objective

Patients can provide a unique perspective on the safety of care in hospitals. Understanding that the extent to which the way hospitals are organized for quality and safety is associated with patient perceptions of care is becoming increasingly valued and necessary for the direction of targeted interventions across healthcare systems. The UK-developed patient measure of safety (PMOS) assesses eight domains of ward safety from the patient point of view and has recently been adapted and piloted in Australia. The aim of this study is to test the psychometric properties of PMOS-Australia (PMOS-A) amongst a large cohort of hospitalized patients.

Design

Cross-sectional questionnaire validation assessment.

Setting and participants

As part of the DUQuA project, the PMOS-A survey was distributed within acute myocardial infarction, hip fracture and stroke departments across 32 large public hospitals in Australia. Patients could complete the PMOS-A independently, or request the assistance of a family member/guardian, or staff on the wards—space was included to record mode of completion.

Main outcome measures

Confirmatory factor analysis (CFA) was undertaken on a calibration sample to generate the model, and a validation sample was used to cross-validate the model. A subset of only those participants who received assistance for PMOS-A completion was also tested using CFA on a calibration and validation sample. Model fit indices (chi-square to degrees of freedom ratio [Chi-square:DF], root mean square error of approximation [RMSEA], comparative fit indices [CFI], standardized root mean squared residual [SRMR]), Cronbach’s α, average inter-item correlations, construct reliability and cross-loadings were examined with reference to recommended thresholds to establish the extent of convergent validity and discriminant validity. A marker of criterion validity was assessed through testing associations between the PMOS-A and adherence to clinical guidelines.

Results

Across the calibration and validation samples of the full (N = 911) and assisted completers only subset (N = 490), three (Chi-square:DF, SRMR, RMSEA) of the four indices consistently or almost always met thresholds for acceptable model fit. CFI indices did not meet the recommended limits (0.72–0.78, against a target > 0.9). Positive relationships were found for all tests between PMOS-A and adherence to clinical guidelines, and these were significant when assessed in the calibration datasets for the full and assisted completion samples.

Conclusion

A sufficiently reliable and valid measure of patient perceptions of safety has been developed. These findings should provide adequate support to justify the use of this measure to assess patient perceptions of safety in Australian hospitals and can be modified for use elsewhere.

Use of WHO standards to improve quality of maternal and newborn hospital care: a study collecting both mothers' and staff perspective in a tertiary care hospital in Italy

Background

WHO developed a list of standards for improving maternal and newborn hospital care. However, there is little experience on their use, and no precise guidance on their implementation. This study aimed at documenting the use of the WHO standards for improving the quality of maternal and neonatal care (QMNC) in a tertiary hospital, Northeast Italy.

Methods

The study was conducted between May 2016 and May 2018, in three phases: phase I—sensitisation and training of health professionals; phase II—data collection on the WHO standards through a survey among service users and providers; phase III—based on the findings of phase II, development of recommendations for improving the QMNC.

Results

Overall, 101 health professionals were successfully trained. 1050 mothers and 105 hospital staff participated in the survey. Key indicators of QMNC (and related prevalence) from the mothers survey included: caesarean section (23.1%); episiotomy (18.3%); restrictions to free movements during labour (46.5%), lithotomy position for staff choice (69.3%); skin to skin (80.8%); early breast feeding (67.2%); information on newborn danger signs (47.2%); high satisfaction with QMNC (68.8%). Only 1.2% and 0.7% of women respectively reported discrimination or abuse. Key indicators (and prevalence) reported from staff included: availability of clinical protocols (37%); regular training (14%); health information system used for quality improvement (16.3%); training on effective communication (9.7%) and on emotional support (19.6%); protocols to prevent mistreatment and abuse (6.9%). On several indicators, the opinions of mothers on QMNC was better than those of staff. Overall, 55 quality improvement recommendations were agreed.

Conclusions

Information on the WHO standards can be collected from both services users and providers and can be proactively used for planning improvements on QMNC.

Reshaping public hospitals: an agenda for reform in Asia and the Pacific

Hospitals in the Asia-Pacific today face the 'triple aim' challenge, proposed by the Institute for Healthcare Improvement, of how to improve quality of care and population health, while at the same time controlling healthcare costs. Yet, pursuing these challenges in combination is presently a remote prospect for many hospitals and, indeed, in a majority of countries in the region. The roles and functions of the public hospital sector within local health systems need redefinition and reform in the context of demographic and epidemiological transitions. Policymakers, managers and health professionals have an obligation to reshape the future of public hospitals. This article outlines actions for how public hospitals can be reshaped from a health system perspective. First, hospitals should be integrated into the fabric of the local health system; they can lead in this through working in alliances with other healthcare facilities, including primary care and private hospitals. Policymakers have a role in facilitating this as it contributes to health improvement of the population. Second, investments in system innovation, management improvement and information systems are required and their impact assessed. Such investments can contribute to cost control and efficiency. Public hospital sector investments should be strategic, efficient and should not bias investment in broader determinants of health. Third, reorienting health workforce competencies and appropriate skills should be central to hospital sector reforms, from policy to frontline services delivery. Creative thinking is needed to build and support flexible care delivery arrangements for services designed to respond to patients ' and providers' needs. Pivotal to achievement of each of these three areas of reform is good governance and leadership.

Recommended care received by geriatric hip fracture patients: where are we now and where are we heading?

INTRODUCTION

Despite the availability of clinical guidelines on the prevention and treatment of geriatric hip fractures, the percentage of recommended care received by patients is low. We conducted an importance-performance analysis for prioritizing interventions to improve the in-hospital management of these patients.

MATERIALS AND METHODS

A secondary data analysis was conducted on the in-hospital treatment of 540 geriatric hip fracture patients in 34 hospitals in Belgium, Italy, and Portugal. First, we assessed the level of expert consensus on the process indicators composing international guidelines on hip fracture treatment. Second, guideline adherence on in-hospital care was evaluated within and across hospitals. Third, an importance-performance analysis was conducted, linking expert consensus to guideline adherence.

RESULTS

Level of expert consensus was high (above 75%) for 12 of 22 process indicators identified from the literature. There is large between and within hospital variation in guideline adherence for these indicators and for none of the 540 patients were all 22 process indicators adhered to. Importance-performance analysis demonstrated that three indicators that had a high level of expert consensus also had a high level of adherence (above 80%). Nine indicators, most of which have been previously linked to patient outcomes, had a high level of expert consensus but a consistently low level of adherence across hospitals and are identified as priority areas for improvement.

CONCLUSIONS

Guideline adherence for the treatment of geriatric hip fracture patients is remarkably suboptimal. Importance-performance analysis is a useful strategic approach to assist practitioners and healthcare managers to improve the quality of care.

Factors Affecting the Referral Rate of the Hoarding Disorder at Primary Mental Health Care in Quebec

Hoarding disorder (HD) places an important burden on people with HD, on their family members and society. In this paper we evaluate help-seeking in HD at primary mental health, measured in referral rate, together with its individual, environmental and structural correlates. We conducted an aggregate study by combining existing official data with our own survey data at the catchment area level. We found a mean annual referral rate of 1.58 (SD = 1.79) cases of HD in primary mental health facilities per 10,000 of adult population. The referrals rate correlated with socio-demographic characteristics of the catchment area, the availability of tools for clinical management of HD, and affiliation to a University Medical school. We also found that: (1) family members, neighbours, municipal workers and health professionals are the primary source of complaints for HD; (2) 72% of primary mental health facilities worked with HD in crisis situations, 52% expressed difficulties in obtaining the consent of people with HD for an intervention (3) health/social services professionals lack HD clinical management tools, training and formal collaboration with municipal (housing, building security, fire prevention) specialists. Improvement of the readiness of the health-system to deal with HD will improve help-seeking for formal medical counselling on the part of people with HD. We can improve this readiness by providing primary mental-health facilities with training, clinical management tools and by helping them to establish formal collaboration with municipalities and community organisations. University medical schools can take a leadership role and become centers catalysing the change in HD clinical management.

Conceptualizing and assessing improvement capability: a review

Purpose

The literature is reviewed to examine how ‘improvement capability’ is conceptualized and assessed and to identify future areas for research.

Data sources

An iterative and systematic search of the literature was carried out across all sectors including healthcare. The search was limited to literature written in English.

Data extraction

The study identifies and analyses 70 instruments and frameworks for assessing or measuring improvement capability. Information about the source of the instruments, the sectors in which they were developed or used, the measurement constructs or domains they employ, and how they were tested was extracted.

Results of data synthesis

The instruments and framework constructs are very heterogeneous, demonstrating the ambiguity of improvement capability as a concept, and the difficulties involved in its operationalisation. Two-thirds of the instruments and frameworks have been subject to tests of reliability and half to tests of validity. Many instruments have little apparent theoretical basis and do not seem to have been used widely.

Conclusion

The assessment and development of improvement capability needs clearer and more consistent conceptual and terminological definition, used consistently across disciplines and sectors. There is scope to learn from existing instruments and frameworks, and this study proposes a synthetic framework of eight dimensions of improvement capability. Future instruments need robust testing for reliability and validity. This study contributes to practice and research by presenting the first review of the literature on improvement capability across all sectors including healthcare.

Investigation of the conditions affecting the joining of Hungarian hospitals to an accreditation programme: a cross-sectional study

OBJECTIVE

Quantitative studies have shown the various benefits for having accreditation in hospitals. However, neither of these explored the general conditions before applying for an accreditation. To close this gap, this study aimed to investigate the possible association between joining an accreditation programme with various hospital characteristics.

DESIGN

A cross-sectional study was implemented using the databases of the 2013 Hungarian hospital survey and of the Hungarian State Treasury.

SETTING

Public general hospitals in Hungary.

PARTICIPANTS

The analysis involved 44 public general hospitals, 14 of which joined the preparatory project for a newly developed accreditation programme.

MAIN OUTCOME MEASURES

The outcomes included the percentage of compliance in quality management, patient information and identification, internal professional regulation, safe surgery, pressure sore prevention, infection control, the opinions of the heads of quality management regarding the usefulness of quality management and clinical audits, and finally, the total debt of the hospital per bed and per discharged patient.

RESULTS

According to our findings, the general hospitals joining the preparatory project of the accreditation programme performed better in four of the six investigated activities, the head of quality management had a better opinion on the usefulness of quality management, and both the debt per bed number and the debt per discharged patient were lower than those who did not join. However, no statistically significant differences between the two groups were found in any of the examined outcomes.

CONCLUSIONS

The findings suggest that hospitals applying for an accreditation programme do not differ significantly in characteristics from those which did not apply. This means that if in the future the accredited hospitals become better than other hospitals, then the improvement could be solely contributed to the accreditation.

De-freezing frozen patient management

Objective

To compare the effectiveness of two methods in encouraging the consideration of a leap from one patient management routine to another: (i) real-time review of the facts by an external medical team (ii) implementation of the 're-thinking-protocol' ('de-Freezing') by both treating and external medical teams.

Design

Students accompanied doctors, nurses and patients as non-interrupting observers. When an obvious gap between the expected and actual findings occurred, it was discussed four times: by two teams (treating team, external medical team) in two discussion modes (real-time review, de-Freezing-questionnaire). The students then recorded if a leap was considered for each discussion.

Setting

The study was conducted in the emergency department of the Baruch Padeh Medical Centre, Poriya, Israel.

Participants

All patients were included during times when both medical teams (treating, external) were present.

Intervention(s)

During 14 periods of 5-7 h each, 459 patients were sampled. In 183 patients, 200 gaps were discovered.

Results

The external team considered a leap 76 times, compared with 47 by the treating team (P < 0.001). Using the de-Freezing-protocol, the treating team considered a leap 133 times. Interestingly, even the external team benefited from the de-Freezing protocol and considered a leap 140 times (NS compared to the treating team).

Conclusions

While the importance of timely leaping from one patient management routine to another is emphasized in the training of physicians, medical teams too often fail to do so. The de-Freezing-protocol inexpensively encourages the consideration of a leap beyond what is evoked by the involvement of an external team. The protocol is applicable to all medical processes and should be incorporated into medical practice and education.

Towards healthy learning climates in postgraduate medical education: exploring the role of hospital-wide education committees

BACKGROUND

Postgraduate medical education prepares residents for delivery of high quality patient care during training as well as for later practice, which makes high quality residency training programs crucial to safeguard patient care. Healthy learning climates contribute to high quality postgraduate medical education. In several countries, modernization of postgraduate medical education has resulted in hospital-wide responsibilities for monitoring learning climates. This study investigates the association between the actions undertaken by hospital-wide education committees and learning climates in postgraduate medical education.

METHODS

Research conducted in December 2010 invited 57 chairs of hospital-wide education committees to complete a questionnaire on their implemented level of quality improvement policies. We merged the survey data from 21 committees that oversaw training programs and used the Dutch Residency Educational Climate Test (D-RECT) instrument in 2012 to measure their training programs' learning climate. We used descriptive statistics and linear mixed models to analyse associations between the functioning of hospital-wide education committees and corresponding learning climates.

RESULTS

In total, 812 resident evaluations for 99 training programs in 21 teaching hospitals were available for analysis. The implementation level of the internal quality management systems as adopted by the hospital-wide education committees varied from 1.6 to 2.6 on a 5 point Likert-scale (ranging from 1 (worst) to 5 (best)). No significant associations were found between the functioning of the committees and corresponding learning climates.

CONCLUSIONS

The contribution of hospital-wide committees to creating healthy learning climates is yet to be demonstrated. The absence of such an association could be due to the lack of a Plan-Do-Check-Act cycle guiding the policy as implemented by the committees and the lack of involvement of departmental leadership. Insight into the impact of these strategies on learning climates will benefit the quality of postgraduate medical education and, hopefully, patient care.

Are the registry data reliable? An audit of a regional trauma registry in the Netherlands

Objective

Data in trauma registries need to be reliable when used for evaluation of injury management, trauma protocols and hospital statistics. The aim of this audit was to analyse the reliability of the data in the Trauma Centre West Netherlands (TCWN) region.

Design

Routinely registered trauma patients from all nine hospitals in the TCWN region were re-registered by a registrar for analysis.

Setting

Nine hospitals in the TCWN region in the Netherlands.

Participants

A randomly selected representative trauma population sample of 350 patients and a sample of 100 polytrauma patients were re-registered and used for analysis.

Intervention

Re-registration of trauma patients in the Trauma Registry.

Main Outcome Measure(s)

The inter-rater agreement on Injury Severity Score (ISS), number of Abbreviated Injury Scale (AIS) codes, identical codes and survival status were analysed using Kappa's coefficient and intraclass correlation coefficients.

Results

The inter-rater agreement on ISS and number of AIS codes were, respectively, almost perfect (ICC = 0.81) and substantial (ICC = 0.76) in the trauma population sample, and substantial (ICC = 0.70) and fair (ICC = 0.33) in the polytrauma sample. For patients with serious injuries (AIS ≥ 2) in the population sample, the inter-rater agreement on ISS (ICC = 0.87) and number of AIS codes (ICC = 0.84) were almost perfect.

Conclusions

These results confirm that the Dutch regional registry system works well and may serve as a reliable basis for prospective analysis of national and international trauma care. Particular attention should be paid to the coding of polytrauma patients as discrepancies are more likely to occur in this group.

Evaluating the impact of continuous quality improvement methods at hospitals in Tanzania: a cluster-randomized trial

Objective

To evaluate the impact of implementing continuous quality improvement (CQI) methods on patient's experiences and satisfaction in Tanzania.

Design

Cluster-randomized trial, which randomly allocated district-level hospitals into treatment group and control group, was conducted.

Setting

Sixteen district-level hospitals in Kilimanjaro and Manyara regions of Tanzania.

Participants

Outpatient exit surveys targeting totally 3292 individuals, 1688 in the treatment and 1604 in the control group, from 3 time-points between September 2011 and September 2012.

Intervention

Implementation of the 5S (Sort, Set, Shine, Standardize, Sustain) approach as a CQI method at outpatient departments over 12 months.

Main outcome measures

Cleanliness, waiting time, patient's experience, patient's satisfaction.

Results

The 5S increased cleanliness in the outpatient department, patients' subjective waiting time and overall satisfaction. However, negligible effects were confirmed for patient's experiences on hospital staff behaviours.

Conclusions

The 5S as a CQI method is effective in enhancing hospital environment and service delivery; that are subjectively assessed by outpatients even during the short intervention period. Nevertheless, continuous efforts will be needed to connect CQI practices with the further improvement in the delivery of quality health care.

Perceptions of quality and safety and experience of adverse events in 27 European Union healthcare systems, 2009-2013

Objective

To assess trends in the perception of quality and safety between 2009 and 2013 in the European Union (EU).

Design

We analysed data from waves 72.2 and 80.2 of the Eurobarometer survey. Multilevel logistic regression models adjusted for sociodemographic factors and country-level health expenditure were fitted to assess changes between 2009 and 2013 in each of the assessed outcomes.

Setting

Twenty-seven EU member states.

Participants

A total of n = 26 663 (2009) and n = 26 917 (2013) individuals aged ≥15 years.

Main outcome measure(s)

Outcomes included the perception of being harmed in hospital and non-hospital care; rating of the overall quality of the healthcare system; and personal or family experience of adverse events.

Results

Respondents in 2013 were more likely to think that it was likely to be harmed in hospital (Odds Ratio [OR] = 1.09; 95% Confidence Interval [CI]: 1.05-1.13; P < 0.001) and non-hospital care (OR = 1.11; 95% CI: 1.07-1.15; P < 0.001), compared to 2009. However, they were more likely to rate the quality of their country's healthcare system as good (OR = 1.26; 95% CI: 1.21-1.32; P < 0.001) and no significant change over time was identified in reported experience of adverse events (OR = 1.00; 95% CI: 0.95-1.05; P = 0.929). Lower health expenditure and decrease in health expenditure between the two waves were associated with worse outcomes in overall quality and perceptions of harm. There was significant variation between and within countries in all indicators.

Conclusions

The public's perception of safety in European healthcare systems declined in recent years, which highlights that there are safety issues that could be addressed.

Evaluation of the organisation and effectiveness of internal audits to govern patient safety in hospitals: a mixed-methods study

OBJECTIVES

Hospital boards are legally responsible for safe healthcare. They need tools to assist them in their task of governing patient safety. Almost every Dutch hospital performs internal audits, but the effectiveness of these audits for hospital governance has never been evaluated. The aim of this study is to evaluate the organisation of internal audits and their effectiveness for hospitals boards to govern patient safety.

DESIGN AND SETTING

A mixed-methods study consisting of a questionnaire regarding the organisation of internal audits among all Dutch hospitals (n=89) and interviews with stakeholders regarding the audit process and experienced effectiveness of audits within six hospitals.

RESULTS

Response rate of the questionnaire was 76% and 43 interviews were held. In every responding hospital, the internal audits followed the plan-do-check-act cycle. Every hospital used interviews, document analysis and site visits as input for the internal audit. Boards stated that effective aspects of internal audits were their multidisciplinary scope, their structured and in-depth approach, the usability to monitor improvement activities and to change hospital policy and the fact that results were used in meetings with staff and boards of supervisors. The qualitative methods (interviews and site visits) used in internal audits enable the identification of soft signals such as unsafe culture or communication and collaboration problems. Reported disadvantages were the low frequency of internal audits and the absence of soft signals in the actual audit reports.

CONCLUSION

This study shows that internal audits are regarded as effective for patient safety governance, as they help boards to identify patient safety problems, proactively steer patient safety and inform boards of supervisors on the status of patient safety. The description of the Dutch internal audits makes these audits replicable to other healthcare organisations in different settings, enabling hospital boards to complement their systems to govern patient safety.

Assessment for improving the performance of NICUs: The Italian experience

This paper describes neonatal intensive care units (NICU) in Italy using a set of organizational variables identifying management profile. The correlations between variables, and how these are impacted by structural aspects of the department or hospital, are investigated. The research was conducted within the 5-year far-reaching and complex SONAR study run by the Italian Neonatal Network, which maps NICU, monitors outcomes of member centres, defines organizational models, and identifies best practices to improve care quality. Seven variables relating to activities, organization processes, and behaviour models used in the SONAR study were used here to assess NICU. Data from 54 Italian NICUs, 1601 nursing staff, and 643 doctors were used. We identified high levels of variation in NICUs for all aspects of organization. We also identified important opportunities for improvement, especially in the areas of performance measurement, quality improvement, and learning for healthcare staff. In terms of structural characteristics, we identified big differences between NICUs in the north and south of Italy. The findings provide a description of NICUs in Italy and identify a set of variables useful for management in assessing NICU, which are among the most complex and costly operational units in a hospital.

Keys to successful implementation of a French national quality indicator in health care organizations: a qualitative study

Background

Several countries have launched public reporting systems based on quality indicators (QIs) to increase transparency and improve quality in health care organizations (HCOs). However, a prerequisite to quality improvement is successful local QI implementation. The aim of this study was to explore the pathway through which a mandatory QI of the French national public reporting system, namely the quality of the anesthesia file (QAF), was put into practice.

Method

Seven ethnographic case studies in French HCOs combining in situ observations and 37 semi-structured interviews.

Results

A significant proportion of potential QAF users, such as anesthetists or other health professionals were often unaware of quality data. They were, however, involved in improvement actions to meet the QAF criteria. In fact, three intertwined factors influenced QAF appropriation by anesthesia teams and impacted practice. The first factor was the action of clinical managers (chief anesthetists and head of department) who helped translate public policy into local practice largely by providing legitimacy by highlighting the scientific evidence underlying QAF, achieving consensus among team members, and pointing out the value of QAF as a means of work recognition. The two other factors related to the socio-material context, namely the coherence of information systems and the quality of interpersonal ties within the department.

Conclusions

Public policy tends to focus on the metrological validity of QIs and on ranking methods and overlooks QI implementation. However, effective QI implementation depends on local managerial activity that is often invisible, in interaction with socio-material factors. When developing national quality improvement programs, health authorities might do well to specifically target these clinical managers who act as invaluable mediators. Their key role should be acknowledged and they ought to be provided with adequate resources.

Design and validation of a questionnaire to assess organizational culture in French hospital wards

BACKGROUND

Although many organizational culture questionnaires have been developed, there is a lack of any validated multidimensional questionnaire assessing organizational culture at hospital ward level and adapted to health care context. Facing the lack of an appropriate tool, a multidisciplinary team designed and validated a dimensional organizational culture questionnaire for healthcare settings to be administered at ward level.

METHODS

A database of organizational culture items and themes was created after extensive literature review. Items were regrouped into dimensions and subdimensions (classification validated by experts). Pre-test and face validation was conducted with 15 health care professionals. In a stratified cluster random sample of hospitals, the psychometric validation was conducted in three phases on a sample of 859 healthcare professionals from 36 multidisciplinary medicine services: 1) the exploratory phase included a description of responses' saturation levels, factor and correlations analyses and an internal consistency analysis (Cronbach's alpha coefficient); 2) confirmatory phase used the Structural Equation Modeling (SEM); 3) reproducibility was studied by a test-retest.

RESULTS

The overall response rate was 80 %; the completion average was 97 %. The metrological results were: a global Cronbach's alpha coefficient of 0.93, higher than 0.70 for 12 sub-dimensions; all Dillon-Goldstein's rho coefficients higher than 0.70; an excellent quality of external model with a Goodness of Fitness (GoF) criterion of 0.99. Seventy percent of the items had a reproducibility ranging from moderate (Intra-Class Coefficient between 50 and 70 % for 25 items) to good (ICC higher than 70 % for 33 items).

CONCLUSIONS

COMEt (Contexte Organisationnel et Managérial en Etablissement de Santé) questionnaire is a validated multidimensional organizational culture questionnaire made of 6 dimensions, 21 sub-dimensions and 83 items. It is the first dimensional organizational culture questionnaire, specific to healthcare context, for a unit level assessment showing robust psychometric properties (validity and reliability). This tool is suited for research purposes, especially for assessing organizational context in research analysing the effectiveness of hospital quality improvement strategies. Our tool is also suited for an overall assessment of ward culture and could be a powerful trigger to improve management and clinical performance. Its psychometric properties in other health systems need to be tested.

The effectiveness and variation of acute medical units: a systematic review

PURPOSE

To evaluate the evidence for the effectiveness of acute medical units (AMUs) compared with other models of care and compare the components of AMU models.

DATA SOURCES

Six electronic databases and grey literature sources searched between 1990 and 2014.

STUDY SELECTION

Studies reporting on AMUs as an intervention for unplanned medical presentations to hospital with the inclusion of all outcome measures/study designs/comparators.

DATA EXTRACTION

Data on study characteristics/outcomes/AMU components were extracted by one author and confirmed by a second.

DATA SYNTHESIS

Seventeen studies of 12 AMUs across five countries were included. The AMU model was associated with a reduction in-hospital length of stay (LOS) in all analyses ranging from 0.3 to 2.6 days; and a reduction in mortality in 12 of the 14 analyses with the change ranging from a 0.1% increase to a 8.8% reduction. Evidence relating to readmissions and patient/staff satisfaction was less conclusive. There was variation in the following components of AMUs: admission criteria, entry sources, functions and consultant work patterns.

CONCLUSION

This review provides evidence that AMUs are associated with reductions in-hospital LOS and, less convincingly, mortality compared with other models of care when implemented in European and Australasian settings. Reported estimates may be affected by residual confounding. This review reports heterogeneity in components of the AMU model. Further work to identify what constitutes the key components of an AMU is needed to improve the quality and effectiveness of acute medical care. This is of particular importance given the escalating demand on acute services.

Older, vulnerable patient view: a pilot and feasibility study of the patient measure of safety (PMOS) with patients in Australia

OBJECTIVES

The UK-developed patient measure of safety (PMOS) is a validated tool which captures patient perceptions of safety in hospitals. We aimed (1) to investigate the extent to which the PMOS is appropriate for use with stroke, acute myocardial infarction (AMI) and hip fracture patients in Australian hospitals and (2) to pilot the PMOS for use in a large-scale, national study 'Deepening our Understanding of Quality in Australia' (DUQuA).

PARTICIPANTS

Stroke, AMI and hip fracture patients (n=34) receiving care in 3 wards in 1 large hospital.

METHODS

2 phases were conducted. First, a 'think aloud' study was used to determine the validity of PMOS with this population in an international setting, and to make amendments based on patient feedback. The second phase tested the revised measure to establish the internal consistency reliability of the revised subscales, and piloted the recruitment and administration processes to ensure feasibility of the PMOS for use in DUQuA.

RESULTS

Of the 43 questions in the PMOS, 13 (30%) were amended based on issues patients highlighted for improvement in phase 1. In phase 2, a total of 34 patients were approached and 29 included, with a mean age of 71.3 years (SD=16.39). Internal consistency reliability was established using interitem correlation and Cronbach's α for all but 1 subscale. The most and least favourably rated aspects of safety differed between the 3 wards. A study log was categorised into 10 key feasibility factors, including liaising with wards to understand operational procedures and identify patterns of patient discharge.

CONCLUSIONS

Capturing patient perceptions of care is crucial in improving patient safety. The revised PMOS is appropriate for use with vulnerable older adult groups. The findings from this study have informed key decisions made for the deployment of this measure as part of the DUQuA study.

The involvement of medical doctors in hospital governance and implications for quality management: a quick scan in 19 and an in depth study in 7 OECD countries

Background

Hospital governance is broadening its orientation from cost and production controls towards ‘improving performance on clinical outcomes’. Given this new focus one might assume that doctors are drawn into hospital management across OECD countries. Hospital performance in terms of patient health, quality of care and efficiency outcomes is supposed to benefit from their involvement. However, international comparative evidence supporting this idea is limited. Just a few studies indicate that there may be a positive relationship between medical doctors being part of hospital boards, and overall hospital performance. More importantly, the assumed relationship between these so-called doctor managers and hospital performance has remained a ‘black-box’ thus far. However, there is an increasing literature on the implementation of quality management systems in hospitals and their relation with improved performance. It seems therefore fair to assume that the relation between the involvement of doctors in hospital management and improved hospital performance is partly mediated via quality management systems. The threefold aim of this paper is to 1) perform a quick scan of the current situation with regard to doctor managers in hospital management in 19 OECD countries, 2) explore the phenomenon of doctor managers in depth in 7 OECD countries, and 3) investigate whether doctor involvement in hospital management is associated with more advanced implementation of quality management systems.

Methods

This study draws both on a quick scan amongst country coordinators in OECD’s Health Care Quality Indicator program, and on the DUQuE project which focused on the implementation of quality management systems in European hospitals.

Results

This paper reports two main findings. First, medical doctors fulfil a broad scope of managerial roles at departmental and hospital level but only partly accompanied by formal decision making responsibilities. Second, doctor managers having more formal decision making responsibilities in strategic hospital management areas is positively associated with the level of implementation of quality management systems.

Conclusions

Our findings suggest that doctors are increasingly involved in hospital management in OECD countries, and that this may lead to better implemented quality management systems, when doctors take up managerial roles and are involved in strategic management decision making.

Monitoring mental healthcare on a system level: Country profiles and status from EU countries

AIMS

Routinely collected data can be used to monitor the performance and improve the quality of mental healthcare systems. Data-based and system-level Quality Monitoring Programmes in Mental Health Care (QMP-MHC) are increasingly being implemented in EU countries. They are believed to be indispensable for the sustainable improvement of the quality of mental healthcare. However, there is a paucity of comparative research on national strategies in quality monitoring. This study explores the status of system-level Quality Monitoring Programmes in Mental Health Care (QMP-MHC) in EU countries. It aims to provide a descriptive overview, which is intended to be the first step for comparative research in this field.

METHODS

Case studies of system-level QMP-MHCs were gathered from eight EU countries. Experts from each country were asked to describe their approach using a template. These experts were all members of a Europe-wide network of researchers and members of public institutions involved in quality assessment and performance monitoring of mental healthcare.

RESULTS

Country profiles were gathered from England, Denmark, France, Germany, Italy, the Netherlands, Portugal, and Sweden. All these countries have - or are in the process of implementing - system-level QMP-MHCs. Implementation seems to be facilitated when a national performance monitoring programme for general healthcare is already in place, although in the Netherlands, a QMP-MHC was established without being attached to a general monitoring programme. All the monitoring programmes described use various quality and performance indicators; some systems enhance this methodology with the addition of qualitative assessment methods such as peer reviews linked to accreditation processes.

CONCLUSIONS

Research is needed to better understand the historical, political and technical backgrounds of the different national cases and to explore the effectiveness of different improvement mechanisms on the actual quality of healthcare. Policy makers and those designing performance measurement programmes are recommended to look across the borders of their own healthcare systems as there are many ways to assess performance and many ways to feedback results to service providers. No evidence is available whether one of these methods is superior to the others in improving the quality of mental healthcare.

Deepening our Understanding of Quality in Australia (DUQuA): a study protocol for a nationwide, multilevel analysis of relationships between hospital quality management systems and patient factors

INTRODUCTION

Despite the growing body of research on quality and safety in healthcare, there is little evidence of the association between the way hospitals are organised for quality and patient factors, limiting our understanding of how to effect large-scale change. The 'Deepening our Understanding of Quality in Australia' (DUQuA) study aims to measure and examine relationships between (1) organisation and department-level quality management systems (QMS), clinician leadership and culture, and (2) clinical treatment processes, clinical outcomes and patient-reported perceptions of care within Australian hospitals.

METHODS AND ANALYSIS

The DUQuA project is a national, multilevel, cross-sectional study with data collection at organisation (hospital), department, professional and patient levels. Sample size calculations indicate a minimum of 43 hospitals are required to adequately power the study. To allow for rejection and attrition, 70 hospitals across all Australian jurisdictions that meet the inclusion criteria will be invited to participate. Participants will consist of hospital quality management professionals; clinicians; and patients with stroke, acute myocardial infarction and hip fracture. Organisation and department-level QMS, clinician leadership and culture, patient perceptions of safety, clinical treatment processes, and patient outcomes will be assessed using validated, evidence-based or consensus-based measurement tools. Data analysis will consist of simple correlations, linear and logistic regression and multilevel modelling. Multilevel modelling methods will enable identification of the amount of variation in outcomes attributed to the hospital and department levels, and the factors contributing to this variation.

ETHICS AND DISSEMINATION

Ethical approval has been obtained. Results will be disseminated to individual hospitals in de-identified national and international benchmarking reports with data-driven recommendations. This ground-breaking national study has the potential to influence decision-making on the implementation of quality and safety systems and processes in Australian and international hospitals.

Implementation of Departmental Quality Strategies Is Positively Associated with Clinical Practice: Results of a Multicenter Study in 73 Hospitals in 7 European Countries

BACKGROUND

Given the amount of time and resources invested in implementing quality programs in hospitals, few studies have investigated their clinical impact and what strategies could be recommended to enhance its effectiveness.

OBJECTIVE

To assess variations in clinical practice and explore associations with hospital- and department-level quality management systems.

DESIGN

Multicenter, multilevel cross-sectional study.

SETTING AND PARTICIPANTS

Seventy-three acute care hospitals with 276 departments managing acute myocardial infarction, deliveries, hip fracture, and stroke in seven countries.

INTERVENTION

None.

MEASURES

Predictor variables included 3 hospital- and 4 department-level quality measures. Six measures were collected through direct observation by an external surveyor and one was assessed through a questionnaire completed by hospital quality managers. Dependent variables included 24 clinical practice indicators based on case note reviews covering the 4 conditions (acute myocardial infarction, deliveries, hip fracture and stroke). A directed acyclic graph was used to encode relationships between predictors, outcomes, and covariates and to guide the choice of covariates to control for confounding.

RESULTS AND LIMITATIONS

Data were provided on 9021 clinical records by 276 departments in 73 hospitals. There were substantial variations in compliance with the 24 clinical practice indicators. Weak associations were observed between hospital quality systems and 4 of the 24 indicators, but on analyzing department-level quality systems, strong associations were observed for 8 of the 11 indicators for acute myocardial infarction and stroke. Clinical indicators supported by higher levels of evidence were more frequently associated with quality systems and activities.

CONCLUSIONS

There are significant gaps between recommended standards of care and clinical practice in a large sample of hospitals. Implementation of department-level quality strategies was significantly associated with good clinical practice. Further research should aim to develop clinically relevant quality standards for hospital departments, which appear to be more effective than generic hospital-wide quality systems.

Quality management and perceptions of teamwork and safety climate in European hospitals

OBJECTIVE

This study aimed to investigate the associations of quality management systems with teamwork and safety climate, and to describe and compare differences in perceptions of teamwork climate and safety climate among clinical leaders and frontline clinicians.

METHOD

We used a multi-method, cross-sectional approach to collect survey data of quality management systems and perceived teamwork and safety climate. Our data analyses included descriptive and multilevel regression methods.

SETTING AND PARTICIPANTS

Data on implementation of quality management system from seven European countries were evaluated including patient safety culture surveys from 3622 clinical leaders and 4903 frontline clinicians.

MAIN OUTCOME MEASURES

Perceived teamwork and safety climate.

RESULTS

Teamwork climate was reported as positive by 67% of clinical leaders and 43% of frontline clinicians. Safety climate was perceived as positive by 54% of clinical leaders and 32% of frontline clinicians. We found positive associations between implementation of quality management systems and teamwork and safety climate.

CONCLUSIONS

Our findings, which should be placed in a broader clinical quality improvement context, point to the importance of quality management systems as a supportive structural feature for promoting teamwork and safety climate. To gain a deeper understanding of this association, further qualitative and quantitative studies using longitudinally collected data are recommended. The study also confirms that more clinical leaders than frontline clinicians have a positive perception of teamwork and safety climate. Such differences should be accounted for in daily clinical practice and when tailoring initiatives to improve teamwork and safety climate.

Patient involvement in quality management: rationale and current status

Purpose

– The purpose of this paper is to review how patient and public involvement (PPI) can contribute to quality improvement functions and describe the levels of PPI in quality improvement functions at hospital and departmental level in a sample of European hospitals.

Design/methodology/approach

– Literature review and cross-sectional study.

Findings

– PPI takes multiple forms in health care and there is not a single strategy or method that can be considered to reflect best practice. The literature reveals that PPI can serve important functions to support quality improvement efforts. In contrast, the assessment of actual PPI in quality improvement shows that PPI is low.

Research limitations/implications

– Findings are not representative of hospitals in the EU.

Practical implications

– A diverse set of methods and tools that can be employed to realize PPI. Service providers should consider PPI at all stages, in particular in setting quality standards and criteria and in evaluating the results.

Originality/value

– Contextualization of empirical findings with case studies from the literature that inform further practice and research on PPI.

Patient Experience Shows Little Relationship with Hospital Quality Management Strategies

OBJECTIVES

Patient-reported experience measures are increasingly being used to routinely monitor the quality of care. With the increasing attention on such measures, hospital managers seek ways to systematically improve patient experience across hospital departments, in particular where outcomes are used for public reporting or reimbursement. However, it is currently unclear whether hospitals with more mature quality management systems or stronger focus on patient involvement and patient-centered care strategies perform better on patient-reported experience. We assessed the effect of such strategies on a range of patient-reported experience measures.

MATERIALS AND METHODS

We employed a cross-sectional, multi-level study design randomly recruiting hospitals from the Czech Republic, France, Germany, Poland, Portugal, Spain, and Turkey between May 2011 and January 2012. Each hospital contributed patient level data for four conditions/pathways: acute myocardial infarction, stroke, hip fracture and deliveries. The outcome variables in this study were a set of patient-reported experience measures including a generic 6-item measure of patient experience (NORPEQ), a 3-item measure of patient-perceived discharge preparation (Health Care Transition Measure) and two single item measures of perceived involvement in care and hospital recommendation. Predictor variables included three hospital management strategies: maturity of the hospital quality management system, patient involvement in quality management functions and patient-centered care strategies. We used directed acyclic graphs to detail and guide the modeling of the complex relationships between predictor variables and outcome variables, and fitted multivariable linear mixed models with random intercept by hospital, and adjusted for fixed effects at the country level, hospital level and patient level.

RESULTS

Overall, 74 hospitals and 276 hospital departments contributed data on 6,536 patients to this study (acute myocardial infarction n = 1,379, hip fracture n = 1,503, deliveries n = 2,088, stroke n = 1,566). Patients admitted for hip fracture and stroke had the lowest scores across the four patient-reported experience measures throughout. Patients admitted after acute myocardial infarction reported highest scores on patient experience and hospital recommendation; women after delivery reported highest scores for patient involvement and health care transition. We found no substantial associations between hospital-wide quality management strategies, patient involvement in quality management, or patient-centered care strategies with any of the patient-reported experience measures.

CONCLUSION

This is the largest study so far to assess the complex relationship between quality management strategies and patient experience with care. Our findings suggest absence of and wide variations in the institutionalization of strategies to engage patients in quality management, or implement strategies to improve patient-centeredness of care. Seemingly counterintuitive inverse associations could be capturing a scenario where hospitals with poorer quality management were beginning to improve their patient experience. The former suggests that patient-centered care is not yet sufficiently integrated in quality management, while the latter warrants a nuanced assessment of the motivation and impact of involving patients in the design and assessment of services.

The associations between organizational culture, organizational structure and quality management in European hospitals

OBJECTIVE

To better understand associations between organizational culture (OC), organizational management structure (OS) and quality management in hospitals.

DESIGN

A multi-method, multi-level, cross-sectional observational study.

SETTING AND PARTICIPANTS

As part of the DUQuE project (Deepening our Understanding of Quality improvement in Europe), a random sample of 188 hospitals in 7 countries (France, Poland, Turkey, Portugal, Spain, Germany and Czech Republic) participated in a comprehensive questionnaire survey and a one-day on-site surveyor audit. Respondents for this study (n = 158) included professional quality managers and hospital trustees.

MAIN OUTCOME MEASURES

Extent of implementation of quality management systems, extent of compliance with existing management procedures and implementation of clinical quality activities.

RESULTS

Among participating hospitals, 33% had a clan culture as their dominant culture type, 26% an open and developmental culture type, 16% a hierarchical culture type and 25% a rational culture type. The culture type had no statistically significant association with the outcome measures. Some structural characteristics were associated with the development of quality management systems.

CONCLUSION

The type of OC was not associated with the development of quality management in hospitals. Other factors (not culture type) are associated with the development of quality management. An OS that uses fewer protocols is associated with a less developed quality management system, whereas an OS which supports innovation in care is associated with a more developed quality management system.

Evidence-based organization and patient safety strategies in European hospitals

OBJECTIVE

To explore how European hospitals have implemented patient safety strategies (PSS) and evidence-based organization of care pathway (EBOP) recommendations and examine the extent to which implementation varies between countries and hospitals.

DESIGN

Mixed-method multilevel cross-sectional design in seven countries as part of the European Union-funded project 'Deepening our Understanding of Quality improvement in Europe' (DUQuE).

SETTING AND PARTICIPANTS

Seventy-four acute care hospitals with 292 departments managing acute myocardial infarction (AMI), hip fracture, stroke, and obstetric deliveries. Main outcome measure Five multi-item composite measures-one generic measure for PSS and four pathway-specific measures for EBOP.

RESULTS

Potassium chloride had only been removed from general medication stocks in 9.4-30.5% of different pathways wards and patients were adequately identified with wristband in 43.0-59.7%. Although 86.3% of areas treating AMI patients had immediate access to a specialist physician, only 56.0% had arrangements for patients to receive thrombolysis within 30 min of arrival at the hospital. A substantial amount of the total variance observed was due to between-hospital differences in the same country for PSS (65.9%). In EBOP, between-country differences play also an important role (10.1% in AMI to 57.1% in hip fracture).

CONCLUSIONS

There were substantial gaps between evidence and practice of PSS and EBOP in a sample of European hospitals and variations due to country differences are more important in EBOP than in PSS, but less important than within-country variations. Agencies supporting the implementation of PSS and EBOP should closely re-examine the effectiveness of their current strategies.

Involvement of patients or their representatives in quality management functions in EU hospitals: implementation and impact on patient-centred care strategies

OBJECTIVE

The objective of this study was to describe the involvement of patients or their representatives in quality management (QM) functions and to assess associations between levels of involvement and the implementation of patient-centred care strategies.

DESIGN

A cross-sectional, multilevel

STUDY DESIGN

that surveyed quality managers and department heads and data from an organizational audit.

SETTING

Randomly selected hospitals (n = 74) from seven European countries (The Czech Republic, France, Germany, Poland, Portugal, Spain and Turkey).

PARTICIPANTS

Hospital quality managers (n = 74) and heads of clinical departments (n = 262) in charge of four patient pathways (acute myocardial infarction, stroke, hip fracture and deliveries) participated in the data collection between May 2011 and February 2012.

MAIN OUTCOME MEASURES

Four items reflecting essential patient-centred care strategies based on an on-site hospital visit: (1) formal survey seeking views of patients and carers, (2) written policies on patients' rights, (3) patient information literature including guidelines and (4) fact sheets for post-discharge care. The main predictors were patient involvement in QM at the (i) hospital level and (ii) pathway level.

RESULTS

Current levels of involving patients and their representatives in QM functions in European hospitals are low at hospital level (mean score 1.6 on a scale of 0 to 5, SD 0.7), but even lower at departmental level (mean 0.6, SD 0.7). We did not detect associations between levels of involving patients and their representatives in QM functions and the implementation of patient-centred care strategies; however, the smallest hospitals were more likely to have implemented patient-centred care strategies.

CONCLUSIONS

There is insufficient evidence that involving patients and their representatives in QM leads to establishing or implementing strategies and procedures that facilitate patient-centred care; however, lack of evidence should not be interpreted as evidence of no effect.

The use of on-site visits to assess compliance and implementation of quality management at hospital level

OBJECTIVE

Stakeholders of hospitals often lack standardized tools to assess compliance with quality management strategies and the implementation of clinical quality activities in hospitals. Such assessment tools, if easy to use, could be helpful to hospitals, health-care purchasers and health-care inspectorates. The aim of our study was to determine the psychometric properties of two newly developed tools for measuring compliance with process-oriented quality management strategies and the extent of implementation of clinical quality strategies at the hospital level.

DESIGN

We developed and tested two measurement instruments that could be used during on-site visits by trained external surveyors to calculate a Quality Management Compliance Index (QMCI) and a Clinical Quality Implementation Index (CQII). We used psychometric methods and the cross-sectional data to explore the factor structure, reliability and validity of each of these instruments.

SETTING AND PARTICIPANTS

The sample consisted of 74 acute care hospitals selected at random from each of 7 European countries.

MAIN OUTCOME MEASURES

The psychometric properties of the two indices (QMCI and CQII).

RESULTS

Overall, the indices demonstrated favourable psychometric performance based on factor analysis, item correlations, internal consistency and hypothesis testing. Cronbach's alpha was acceptable for the scales of the QMCI (α: 0.74-0.78) and the CQII (α: 0.82-0.93). Inter-scale correlations revealed that the scales were positively correlated, but distinct. All scales added sufficient new information to each main index to be retained.

CONCLUSION

This study has produced two reliable instruments that can be used during on-site visits to assess compliance with quality management strategies and implementation of quality management activities by hospitals in Europe and perhaps other jurisdictions.