Feasibility first: Developing public performance indicators on patient safety and clinical effectiveness for Dutch hospitals

Developing a quality and safety assessment framework for Iran's military hospitals

BACKGROUND

The first crucial step towards military hospitals performance improvement is to develop a local and scientific tool to assess quality and safety based on the context and aims of military hospitals. This study introduces a Quality and Safety Assessment Framework (Q&SAF) for Iran's military hospitals.

METHODS

This is a literature review which continued with a qualitative study. The Q&SAF for Iran's military hospitals was developed initially, through a review of the WHO's framework for hospital performance, literature review (other related framework), review of military hospital-related local documents, consultations with a national and sub-national expert. Finally, the Delphi technique used to finalize the framework.

RESULTS

Based on the literature review results; 13 hospital Q&SAF were identified. After reviewing literature review results and expert opinions; Iran's military hospitals Q&SAF was developed with 58 indictors in five dimensions including clinical effectiveness, safety, efficiency, patient-centeredness, and Responsive Management (Command and Control). The efficiency dimension had the highest number of indictors (19 indictors), whereas the patient-centered dimension had the lowest number of indices (4 indictors).

CONCLUSION

Regarding the comprehensiveness of the developed assessment framework due to its focus on the majority of quality dimensions and important components of the hospital's performance, it can be used as a useful tool for assessing and continuously improving the quality of hospitals, particularly military hospitals.

Analysis of Medico-Legal Complaint Data: A Retrospective Study of Three Large Italian University Hospitals

(1) Background: Identifying hospital-related critical, and excellent, areas represents the main goal of this paper, in both a national and local setting. Information was collected and organized for an internal company's reports, regarding civil litigation that has been affecting the hospital, to relate the obtained results with the phenomenon of medical malpractice on a national scale. This is for the development of targeted improvement strategies, and for investing available resources in a proficient way. (2) Methods: In the present study, data from claims management in Umberto I General Hospital, Agostino Gemelli University Hospital Foundation and Campus Bio-Medico University Hospital Foundation, from 2013 to 2020 were collected. A total of 2098 files were examined, and a set of 13 outcome indicators in the assessment of "quality of care" was proposed. (3) Results: From the total number, only 779 records (37.1%) were attributable to the categories indexable for the present analysis. This data highlights how, following a correct and rigorous categorization of hospital events, it is possible to analyze these medico-legal aspects using a small number of indicators. Furthermore, it is important to consider how a consistent percentage of remaining events was difficult to index, and was also of poor scientific interest. (4) Conclusions: The proposed indicators do not require standards to be compared to, but provide a useful instrument for comparative purposes. In fact, in addition to comparative assessment between different business realities distributed throughout the territory, the use of outcome indicators allows for a longitudinal analysis evaluating the performance of an individual structure over time.

Big Data Health Care Innovations: Performance Dashboarding as a Process of Collective Sensemaking

Big data is poised to revolutionize health care, and performance dashboards can be an important tool to manage big data innovations. Dashboards show the progress being made and provide critical management information about effectiveness and efficiency. However, performance dashboards are more than just a clear and straightforward representation of performance in the health care context. Instead, the development and maintenance of informative dashboards can be more productively viewed as an interactive and iterative process involving all stakeholders. We refer to this process as dashboarding and reflect on our learnings within a large European Union–funded project. Within this project, multiple big data applications in health care are being developed, piloted, and scaled up. In this paper, we discuss the ways in which we cope with the inherent sensitivities and tensions surrounding dashboarding in such a dynamic environment.

A public health value-based healthcare paradigm for HIV

Background

HIV patients face considerable acute and chronic healthcare needs and battling the HIV epidemic remains of the utmost importance. By focusing on health outcomes in relation to the cost of care, value-based healthcare (VBHC) proposes a strategy to optimize quality of care and cost-efficiency. Its implementation may provide an answer to the increasing pressure to optimize spending in healthcare while improving patient outcomes. This paper describes a pragmatic value-based healthcare framework for HIV care.

Methods

A value-based HIV healthcare framework was developed during a series of roundtable discussions bringing together 16 clinical stakeholder representatives from the Belgian HIV reference centers and 2 VBHC specialists. Each round of discussions was focused on a central question translating a concept or idea to the next level of practical implementation: 1) how can VBHC principles be translated into value-based HIV care drivers; 2) how can these value-based HIV care divers be translated into value-based care objectives and activities; and 3) how can value-based HIV care objectives and activities be translated into value-based care indicators. Value drivers were linked to concrete objectives and activities using a logical framework approach. Finally, specific, measurable, and acceptable structure, process and outcomes indicators were defined to complement the framework.

Results

Our framework identifies 4 core value areas where HIV care would benefit most from improvements: Prevention, improvement of the cascade of care, providing patient-centered HIV care and sustaining a state-of-the-art HIV disease management context. These 4 core value areas were translated into 12 actionable core value objectives. For each objective, example activities were proposed. Indicators are suggested for each level of the framework (outcome indicators for value areas and objectives, process indicators for suggested activities).

Conclusions

This framework approach outlines how to define a patient- and public health centered value-based HIV care paradigm. It proposes how to translate core value drivers to practical objectives and activities and suggests defining indicators that can be used to track and improve the framework’s implementation in practice.

What makes a good quality indicator set? A systematic review of criteria

BACKGROUND

While single indicators measure a specific aspect of quality (e.g. timely support during labour), users of these indicators, such as patients, providers and policy-makers, are typically interested in some broader construct (e.g. quality of maternity care) whose measurement requires a set of indicators. However, guidance on desirable properties of indicator sets is lacking.

OBJECTIVE

Based on the premise that a set of valid indicators does not guarantee a valid set of indicators, the aim of this review is 2-fold: First, we introduce content validity as a desirable property of indicator sets and review the extent to which studies in the peer-reviewed health care quality literature address this criterion. Second, to obtain a complete inventory of criteria, we examine what additional criteria of quality indicator sets were used so far.

METHODS

We searched the databases Web of Science, Medline, Cinahl and PsycInfo from inception to May 2021 and the reference lists of included studies. English- or German-language, peer-reviewed studies concerned with desirable characteristics of quality indicator sets were included. Applying qualitative content analysis, two authors independently coded the articles using a structured coding scheme and discussed conflicting codes until consensus was reached.

RESULTS

Of 366 studies screened, 62 were included in the review. Eighty-five per cent (53/62) of studies addressed at least one of the component criteria of content validity (content coverage, proportional representation and contamination) and 15% (9/62) addressed all component criteria. Studies used various content domains to structure the targeted construct (e.g. quality dimensions, elements of the care pathway and policy priorities), providing a framework to assess content validity. The review revealed four additional substantive criteria for indicator sets: cost of measurement (21% [13/62] of the included studies), prioritization of 'essential' indicators (21% [13/62]), avoidance of redundancy (13% [8/62]) and size of the set (15% [9/62]). Additionally, four procedural criteria were identified: stakeholder involvement (69% [43/62]), using a conceptual framework (44% [27/62]), defining the purpose of measurement (26% [16/62]) and transparency of the development process (8% [5/62]).

CONCLUSION

The concept of content validity and its component criteria help assessing whether conclusions based on a set of indicators are valid conclusions about the targeted construct. To develop a valid indicator set, careful definition of the targeted construct including its (sub-)domains is paramount. Developers of quality indicators should specify the purpose of measurement and consider trade-offs with other criteria for indicator sets whose application may reduce content validity (e.g. costs of measurement) in light thereof.

A Hospital Performance Assessment Model Using the IPOCC Approach

BACKGROUND

Developing a practical model to assess hospital performance improves the quality of services and leads to patient satisfaction. This study aims to develop and present such a model using the IPOCC (Input, Process, Output, Control and Context) approach.

METHODS

This study used a mixed-method research. The statistical population of the qualitative part included 27 experts who were purposefully selected and the sampling process was continued by the snowball method until the data saturation was reached. The quantitative part included 334 managers at different levels within a hospital, who were selected by a random sampling method based on Cochran's formula.

RESULTS

The hospital evaluation model has 5 dimensions with 20 factors: input (human, financial, physical, information and equipment), process (treatment, para-clinical, prevention, management, and leadership processes), outcome (patient, staff and community outcomes and key performance index), control (internal control, external control), context (hospital culture, hospital status, the role of evaluators and community conditions). The value of chi-square was 4689.154, the degree of freedom was 2385, and the ratio of chi-square to the degree of freedom in the model was 1.966, which is an acceptable value. The values obtained from CFI, GFI, and IFI fit indices were acceptable. The SRMR index was 0.1130.

CONCLUSIONS

Using a performance assessment model along with the IPOCC approach evaluates hospital processes and the output obtained from the proper implementation of these processes in all areas. The areas include the hospital provided services like the control and context, or the traditional perspectives like physical, human, financial, and equipment resources.

Steering by their own lights: Why regulators across Europe use different indicators to measure healthcare quality

•

Indicator sets differ in how they define, measure, and assess healthcare quality.

•

National sets shaped by varying governance traditions and healthcare system configuration.

•

Targeting of quality dimensions and hospital activities shaped by system-specific ‘demand-side’ pressures.

•

Measurement styles shaped by ‘supply-side’ constraints on data access and indicator construction.

•

International benchmarking is easier when healthcare systems and governance traditions are similar.

Despite widespread faith that quality indicators are key to healthcare improvement and regulation, surprisingly little is known about what is actually measured in different countries, nor how, nor why. To address that gap, this article compares the official indicator sets--comprising some 1100 quality measures-- used by statutory hospital regulators in England, Germany, France, and the Netherlands. The findings demonstrate that those countries’ regulators strike very different balances in: the dimensions of quality they assess (e.g. between safety, effectiveness, and patient-centredness); the hospital activities they target (e.g. between clinical and non-clinical activities and management); and the ‘Donabedian’ measurement style of their indicators (between structure, process and outcome indicators). We argue that these contrasts reflect: i) how the distinctive problems facing each country’s healthcare system create different ‘demand-side’ pressures on what national indicator sets measure; and ii) how the configuration of national healthcare systems and governance traditions create ‘supply-side’ constraints on the kinds of data that regulators can use for indicator construction. Our analysis suggests fundamental differences in the meaning of quality and its measurement across countries that are likely to impede international efforts to benchmark quality and identify best practice.

Walking the tightrope: how rebels “do” quality of care in healthcare organizations

Purpose

The purpose of this paper is to empirically explore and conceptualize how healthcare professionals and managers give shape to the increasing call for compassionate care as an alternative for system-based quality management systems. The research demonstrates how quality rebels craft deviant practices of good care and how they account for them.

Design/methodology/approach

Ethnographic research was conducted in three Dutch hospitals, studying clinical groups that were identified as deviant: a nursing ward for infectious diseases, a mother–child department and a dialysis department. The research includes over 120 h of observation, 41 semi-structured interviews and 2 focus groups.

Findings

The research shows that rebels’ quality practices are an emerging set of collaborative activities to improving healthcare and meeting (individual) patient needs. They conduct “contexting work” to achieve their quality aims by expanding their normative work to outside domains. As rebels deviate from hospital policies, they are sometimes forced to act “under the radar” causing the risk of groupthink and may undermine the aim of public accounting.

Practical implications

The research shows that in order to come to more compassionate forms of care, organizations should allow for more heterogeneity accompanied with ongoing dialogue(s) on what good care yields as this may differ between specific fields or locations.

Originality/value

This is the first study introducing quality rebels as a concept to understanding social deviance in the everyday practices of doing compassionate and good care.

Identification and ranking of specific balanced scorecard performance measures for hospitals: A case study of private hospitals in the Klang Valley area, Malaysia

Performance measurement is a necessity for private hospitals as they need to be efficient, attract customers, increase profitability, and survive in the competitive environment of the health care industry. Hospitals typically struggle to identify appropriate performance measures because of lack of reliable source of performance measures for private hospitals. Despite numerous studies on performance measurement, few studies have focused on performance measures in private hospitals. This paper aims to fill that gap by identifying and ranking a specific set of performance measures that are feasible and relevant for private hospitals. Forty-four health care performance measures in four balanced scorecard (BSC) performance perspectives (financial, customer, internal business processes, and learning and growth) were compiled and filtered based on "feasibility" and "relevance" criteria using a questionnaire survey in private hospitals in the Klang Valley area, Malaysia. Since all collected data were in numeric format, data analysis was performed quantitatively. Consequently, 31 BSC performance measures were identified for private hospitals. Next, the 31 performance measures went through a ranking survey in Klang Valley private hospitals. Therefore, a weight between 0 and 1 with a range of 0.095 to 0.207 was obtained for each performance measure to help hospitals quantify their overall performance more accurately.

Contractual health services performance agreements for responsive health systems: from conception to implementation in the case of Qatar

Objective

Despite their use worldwide, strategy-based performance management is limited in the Eastern Mediterranean Region. This article explores Qatar's experience, the first from the Region, in implementing contractual agreements between healthcare providers and the regulator-Ministry of Public Health-to align strategy, performance and accountabilities.

Design

mixed-methods including tools development and pilot-testing, guided by performance management cycle with a focus on knowledge translation and key principles: feasibility; mandatory participation; participatory approach through Steering Committee.

Setting

All public, private and semi-governmental hospitals and primary healthcare centers.

Intervention(s)

(i) semi-structured interviews; (ii) review of 4982 indicators; (iii) Delphi technique for selecting indicators with > 80% agreement on importance and > 60% agreement on feasibility; (iv) capacity-building of providers and Ministry staff and 2-month pilot assessed by questionnaire with indicators scoring > 3 considered valid, reliable and feasible; and (v) 1-year grace period assessed by questionnaire.

Main Outcome Measure(s)

Approach strengths and challenges; Data collection and healthcare quality improvements.

Results

Contracts mandate reporting 25 hospital and 15 primary healthcare indicators to the regulator, which delivers confidential benchmarking reports to providers. Scorecards were discussed with the regulator for evidence-informed policymaking. The approach uncovered system-related challenges and learning for public and private sectors: providers commended the participatory approach (82%) and indicated that contracts enabled collecting valid and timely data (64%) and improved healthcare quality (55%).

Conclusion

This experience provides insights for countries implementing performance management, responsive regulation and public-private partnerships. It suggests that contractual agreements can be useful, despite their mandatory nature, if clear principles are applied early on.

Translating value-based health care: an experiment into healthcare governance and dialogical accountability

This article analyses an experiment into healthcare governance in Denmark inspired by principles of value-based health care and intended to re-orient the focus of healthcare governance from 'productivity' to 'value for the patient'. The region in charge of the experiment exempted nine hospital departments from activity-based financing and accountability based on diagnosis-related groups, which allegedly incentivised hospitals in 'perverse' and counterproductive ways. Instead, the departments were to develop new indicators from their local practices to support and account for quality and value for the patient. Drawing on the actor-network theory concept of 'translation', this article analyses how the experiment was received and put into practice in the nine departments, and how it established new kinds of accountability relations. We argue that the experiment provides fruitful inspiration for future governance schemes in healthcare to embrace the local complexities of clinical practices. In particular, we argue that the locally developed indicators facilitated what we call 'dialogical accountability', and we discuss whether this represents a feasible way forward for value-based health care.

Developing a hospital-wide quality and safety dashboard: a qualitative research study

Background

Several countries have national policies and programmes requiring hospitals to use quality and safety (QS) indicators. To present an overview of these indicators, hospital-wide QS (HWQS) dashboards are designed. There is little evidence how these dashboards are developed. The challenges faced to develop these dashboards in Dutch hospitals were retrospectively studied.

Methods

24 focus group interviews were conducted: 12 with hospital managers (n=25; 39.7%) and 12 support staff (n=38; 60.3%) in 12 of the largest Dutch hospitals. Open and axial codings were applied consecutively to analyse the data collected.

Results

A heuristic tool for the general development process for HWQS dashboards containing five phases was identified. In phase 1, hospitals make inventories to determine the available data and focus too much on quantitative data relevant for accountability. In phase 2, hospitals develop dashboard content by translating data into meaningful indicators for different users, which is not easy due to differing demands. In phase 3, hospitals search for layouts that depict the dashboard content suited for users with different cognitive abilities and analytical skills. In phase 4, hospitals try to integrate dashboards into organisational structures to ensure that data are systematically reviewed and acted on. In phase 5, hospitals want to improve the flexibility of their dashboards to make this adaptable under differing circumstances.

Conclusion

The literature on dashboards addresses the technical and content aspects of dashboards, but overlooks the organisational development process. This study shows how technical and organisational aspects are relevant in development processes.

Exploring Dutch surgeons' views on volume-based policies: a qualitative interview study

Objective In many countries, the evidence for volume-outcome associations in surgery has been transferred into policy. Despite the large body of research that exists on the topic, qualitative studies aimed at surgeons' views on, and experiences with, these volume-based policies are lacking. We interviewed Dutch surgeons to gain more insight into the implications of volume-outcome policies for daily clinical practice, as input for effective surgical quality improvement. Methods Semi-structured interviews were conducted with 20 purposively selected surgeons from a stratified sample for hospital type and speciality. The interviews were recorded, transcribed verbatim and underwent inductive content analysis. Results Two overarching themes were inductively derived from the data: (1) minimum volume standards and (2) implications of volume-based policies. Although surgeons acknowledged the premise 'more is better', they were critical about the validity and underlying evidence for minimum volume standards. Patients often inquire about caseload, which is met with both understanding and discomfort. Surgeons offered many examples of controversies surrounding the process of determining thresholds as well as the ways in which health insurers use volume as a purchasing criterion. Furthermore, being held accountable for caseload may trigger undesired strategic behaviour, such as unwarranted operations. Volume-based policies also have implications for the survival of low-volume providers and affect patient travel times, although the latter is not necessarily problematic in the Dutch context. Conclusions Surgeons in this study acknowledged that more volume leads to better quality. However, validity issues, undesired strategic behaviour and the ways in which minimum volume standards are established and applied have made surgeons critical of current policy practice. These findings suggest that volume remains a controversial quality measure and causes polarization that is not conducive to a collective effort for quality improvement. We recommend enforcing thresholds that are based on the best achievable level of consensus and assessing additional criteria when passing judgement on quality of care.

Benchmarking operating room departments in the Netherlands

Purpose

– Benchmarking is increasingly considered a useful management instrument to improve performance in healthcare. The purpose of this paper is to assess if a nationwide long-term benchmarking collaborative between operating room (OR) departments of university medical centres in the Netherlands leads to benefits in OR management and to evaluate if the initiative meets the requirements of the 4P-model.

Design/methodology/approach

– The evaluation was based on the 4P-model (purposes, performance indicators, participating organisations, performance management system), developed in former studies. A mixed-methods design was applied, consisting of document study, observations, interviews as well as analysing OR performance data using SPSS statistics.

Findings

– Collaborative benchmarking has benefits different from mainly performance improvement and identification of performance gaps. It is interesting that, since 2004, the OR benchmarking initiative still endures after already existing for ten years. A key benefit was pointed out by all respondents as “the purpose of networking”, on top of the purposes recognised in the 4P-model. The networking events were found to make it easier for participants to contact and also visit one another. Apparently, such informal contacts were helpful in spreading knowledge, sharing policy documents and initiating improvement. This benchmark largely met all key conditions of the 4P-model.

Research limitations/implications

– The current study has the limitations accompanied with any qualitative research and particularly related to interviewing. Qualitative research findings must be viewed within the context of the conducted case study. The experiences in this university hospital context in the Netherlands might not be transferable to other (general) hospital settings or other countries. The number of conducted interviews is restricted; nevertheless, all other data sources are extensive.

Originality/value

– A collaborative approach in benchmarking can be effective because participants use its knowledge-sharing infrastructure which enables operational, tactical and strategic learning. Organisational learning is to the advantage of overall OR management. Benchmarking seems a useful instrument in enabling hospitals to learn from each other, to initiate performance improvements and catalyse knowledge-sharing.

Diabetic retinopathy care--an international quality comparison

PURPOSE

The purpose of this paper is to study the quality indicator appropriateness and use it for international quality comparison on diabetic retinopathy (DR) patient care process in one American and one Dutch eye hospital.

DESIGN/METHODOLOGY/APPROACH

A 17-item DR quality indicator set was composed based on a literature review and systematically applied in two hospitals. Qualitative analysis entailed document study and 12 semi-structured face-to-face interviews with ophthalmologists, managers, and board members of the two hospitals.

FINDINGS

While the medical-clinical approach to DR treatment in both hospitals was similar, differences were found in quality of care perception and operationalization. Neither hospital systematically used outcome indicators for DR care. On the process level, the authors found larger differences. Similarities and differences were found in the structure of both hospitals. The hospitals' particular contexts influenced the interpretation and use of quality indicators.

PRACTICAL IMPLICATIONS

Although quality indicators and quality comparison between hospitals are increasingly used in international settings, important local differences influence their application. Context should be taken into account. Since that context is locally bound and directly linked to hospital setting, caution should be used interpreting the results of quality comparison studies.

ORIGINALITY/VALUE

International quality comparison is increasingly suggested as a useful way to improve healthcare. Little is known, however, about the appropriateness and use of quality indicators in local hospital care practices.

Validation of quality indicators for the organization of palliative care: a modified RAND Delphi study in seven European countries (the Europall project)

BACKGROUND

Validated quality indicators can help health-care professionals to evaluate their medical practices in a comparative manner to deliver optimal clinical care. No international set of quality indicators to measure the organizational aspects of palliative care settings exists.

AIM

To develop and validate a set of structure and process indicators for palliative care settings in Europe.

DESIGN

A two-round modified RAND Delphi process was conducted to rate clarity and usefulness of a previously developed set of 110 quality indicators.

SETTING/PARTICIPANTS

In total, 20 multi-professional palliative care teams of centers of excellence from seven European countries.

RESULTS

In total, 56 quality indicators were rated as useful. These valid quality indicators concerned the following domains: the definition of a palliative care service (2 quality indicators), accessibility to palliative care (16 quality indicators), specific infrastructure to deliver palliative care (8 quality indicators), symptom assessment tools (1 quality indicator), specific personnel in palliative care services (9 quality indicators), documentation methodology of clinical data (14 quality indicators), evaluation of quality and safety procedures (1 quality indicator), reporting of clinical activities (1 quality indicator), and education in palliative care (4 quality indicator).

CONCLUSION

The modified RAND Delphi process resulted in 56 international face-validated quality indicators to measure and compare organizational aspects of palliative care. These quality indicators, aimed to assess and improve the organization of palliative care, will be pilot tested in palliative care settings all over Europe and be used in the EU FP7 funded IMPACT project.

Using quality measures for quality improvement: the perspective of hospital staff

Research objective

This study examines the perspectives of a range of key hospital staff on the use, importance, scientific background, availability of data, feasibility of data collection, cost benefit aspects and availability of professional personnel for measurement of quality indicators among Iranian hospitals. The study aims to facilitate the use of quality indicators to improve quality of care in hospitals.

Study design

A cross-sectional study was conducted over the period 2009 to 2010. Staff at Iranian hospitals completed a self-administered questionnaire eliciting their views on organizational, clinical process, and outcome (clinical effectiveness, patient safety and patient centeredness) indicators.

Population studied

93 hospital frontline staff including hospital/nursing managers, medical doctors, nurses, and quality improvement/medical records officers in 48 general and specialized hospitals in Iran.

Principal findings

On average, only 69% of respondents reported using quality indicators in practice at their affiliated hospitals. Respondents varied significantly in their reported use of organizational, clinical process and outcome quality indicators. Overall, clinical process and effectiveness indicators were reported to be least used. The reported use of indicators corresponded with their perceived level of importance. Quality indicators were reported to be used among clinical staff significantly more than among managerial staff. In total, 74% of the respondents reported to use obligatory indicators, while this was 68% for voluntary indicators (p<0.05).

Conclusions

There is a general awareness of the importance and usability of quality indicators among hospital staff in Iran, but their use is currently mostly directed towards external accountability purposes. To increase the formative use of quality indicators, creation of a common culture and feeling of shared ownership, alongside an increased uptake of clinical process and effectiveness indicators is needed to support internal quality improvement processes at hospital level.

Performance evaluation of public non-profit hospitals using a BP artificial neural network: the case of Hubei Province in China

To provide a reference for evaluating public non-profit hospitals in the new environment of medical reform, we established a performance evaluation system for public non-profit hospitals. The new “input-output” performance model for public non-profit hospitals is based on four primary indexes (input, process, output and effect) that include 11 sub-indexes and 41 items. The indicator weights were determined using the analytic hierarchy process (AHP) and entropy weight method. The BP neural network was applied to evaluate the performance of 14 level-3 public non-profit hospitals located in Hubei Province. The most stable BP neural network was produced by comparing different numbers of neurons in the hidden layer and using the “Leave-one-out” Cross Validation method. The performance evaluation system we established for public non-profit hospitals could reflect the basic goal of the new medical health system reform in China. Compared with PLSR, the result indicated that the BP neural network could be used effectively for evaluating the performance public non-profit hospitals.

The use of quality information by general practitioners: does it alter choices? A randomized clustered study

BACKGROUND

Following the introduction of elements of managed competition in the Netherlands in 2006, General Practitioners (GPs) and patients were given the role to select treatment hospital using public quality information. In this study we investigate to what extent hospital preferences of GP's are affected by performance indicators on medical effectiveness and patient experiences. We selected three conditions: breast cancer, cataract surgery, and hip and knee replacement.

METHODS

After an inquiry 26 out of 226 GPs in the region signed up to participate in our study. After a 2:1 randomization, we analyzed the referral patterns in the region using three groups of GPs: GPs (n=17) who used the report cards and received personal clarification, GPs that signed up for the study but were assigned to the control group (n=9), and the GPs outside the study (n=200).We conducted a difference in differences analysis where the choice for a particular hospital was the dependent variable and time (2009 or 2010), the sum score of the CQI, the sum score of the PI's and dummy variables for the individual hospitals were used as independent variables.

RESULTS

The analysis of the conditions together and cataract surgery and hip and knee replacement separately, showed no significant relationships between the scores on the report cards and the referral patterns of the GPs. For breast cancer our analysis revealed that GPs in the intervention group refer 1.0% (p=0.01) more to hospitals that score one percent point better on the indicators for medical effectiveness.

CONCLUSION

Our study provides empirical evidence that GP referral patterns were unaffected by the available quality information, except for the outcome indicators for breast cancer care that were presented. This finding was surprising since our study was designed to identify changes in hospital preference (1) amongst the most motivated GP's, (2) that received personal clarification of the performance indicators, and (3) selected indicators/conditions from a large set of indicators that they believed were most important. This finding may differ when quality information is based on outcome indicators with a clinically relevant difference, as shown by our indicators for breast cancer treatment. We believe that the current set of (largely process) hospital quality indicators do not serve the GP's information needs and consequently quality plays little role in the selection of hospitals for treatment.

Hospital benchmarking: are U.S. eye hospitals ready?

BACKGROUND

Benchmarking is increasingly considered a useful management instrument to improve quality in health care, but little is known about its applicability in hospital settings.

PURPOSE

The aims of this study were to assess the applicability of a benchmarking project in U.S. eye hospitals and compare the results with an international initiative.

METHODOLOGY

We evaluated multiple cases by applying an evaluation frame abstracted from the literature to five U.S. eye hospitals that used a set of 10 indicators for efficiency benchmarking. Qualitative analysis entailed 46 semistructured face-to-face interviews with stakeholders, document analyses, and questionnaires.

FINDINGS

The case studies only partially met the conditions of the evaluation frame. Although learning and quality improvement were stated as overall purposes, the benchmarking initiative was at first focused on efficiency only. No ophthalmic outcomes were included, and clinicians were skeptical about their reporting relevance and disclosure. However, in contrast with earlier findings in international eye hospitals, all U.S. hospitals worked with internal indicators that were integrated in their performance management systems and supported benchmarking. Benchmarking can support performance management in individual hospitals. Having a certain number of comparable institutes provide similar services in a noncompetitive milieu seems to lay fertile ground for benchmarking. International benchmarking is useful only when these conditions are not met nationally.

PRACTICE IMPLICATIONS

Although the literature focuses on static conditions for effective benchmarking, our case studies show that it is a highly iterative and learning process. The journey of benchmarking seems to be more important than the destination. Improving patient value (health outcomes per unit of cost) requires, however, an integrative perspective where clinicians and administrators closely cooperate on both quality and efficiency issues. If these worlds do not share such a relationship, the added "public" value of benchmarking in health care is questionable.

Improving the quality of emergency medicine care by developing a quality requirement framework: a study from The Netherlands

Background

In The Netherlands, mainly inexperienced physicians work in the ED on all shifts, including the evening and night shifts, when no direct supervision is available. In 2004 a report of the Dutch Health Care Inspectorate revealed that quality of care at Emergency Departments (EDs) was highly variable. Based on this report and international studies showing significant potential for quality improvement, stakeholders felt the need to improve the quality of EM care. Based on the literature, a baseline measurement and a panel of experts, The Netherlands recently developed a nationwide quality requirement framework (QRF) for EM. This article describes the content of and path to this QRF.

Methods

To conduct a baseline measurement, the panel needed to identify measurable entities related to EM care at EDs. This was done by formulating both qualitative and partly quantitative questions related to the following competence areas: triage system, training of personnel (physicians and nurses), facilities and supervision of physicians.

27 out of 104 Dutch EDs were sampled via a cross-sectional study design, using an online survey and standardized follow-up interview in which the answers of the survey were reviewed.

Results

In the QRF, EM care is divided into a basic level of EM care and six competence certification areas (CCAs): (acute) abdominal aortic aneurysm, acute coronary syndrome, acute psychiatric behavioral disorder, cerebral vascular accident, pediatric critical care and infants with low birth weight. For the basic level of EM care and for every CCA minimum prerequisites for medical devices and training of personnel are established. The factors selected for the QRF can be regarded as minimum quality standards for EM care. A major finding of this study was that in The Netherlands, none of the 27 sampled EDs demonstrated compliance with these factors.

Conclusion

Our study shows that Dutch EDs fall short of what the expert consensus panelists considered minimum prerequisites for adequate EM care. The process of systematic enquiry allowed this information to come to light for the first time, which resulted in the implementation of a QRF for Dutch ED personnel, that is intended improve quality of EM care over time. This is an important development for the worldwide EM community as the QRF shows a way to generate interim standards to improve the chances of appropriate delivery of EM care when the gold standard of providing fully qualified EPs is not initially achievable.

Design, implementation and scaling up of the balanced scorecard for hospitals in Lebanon: policy coherence and application lessons for low and middle income countries

OBJECTIVES

This paper describes the development and implementation of the first national hospital performance indicators in Lebanon including its institutionalization within existing policy framework and the initiation of independent governance structure for sustainability.

METHODS

Guided by the Ontario Acute Care Balanced Scorecard framework, a step-wise approach was used. Guiding principles were non-punitive reporting, anonymity, voluntary participation, stakeholder involvement, consensus and feasibility. Modified Delphi technique was used, readiness assessment surveys in 52 hospitals were conducted, pilot testing and evaluation were completed in 14 hospitals.

RESULTS

Initial balanced set of 21 indicators were selected. Findings showed wide variations in indicators' measurement in hospitals including formulas and tools. Barriers to measurement included lack of quality culture, physician resistance and resources. A gradual implementation strategy was developed and selected indicators were divided into two levels. Most piloted indicators proved to be valid, feasible and reliable. The initiative was linked to the national hospital accreditation system resulting in a balanced set of 40 indicators. An independent, not-for-profit, arm's-length organization was established.

CONCLUSIONS

This is among the first attempts made in the East Mediterranean Region to adapt the BSC approach and translate the experience of its development to addresses local needs and contextual reality.

A simplified method of performance indicators development for epidemiological surveillance networks--application to the RESAPATH surveillance network

BACKGROUND

Develop and calculate performance indicators allows to continuously follow the operation of an epidemiological surveillance network. This is an internal evaluation method, implemented by the coordinators in collaboration with all the actors of the network. Its purpose is to detect weak points in order to optimize management. A method for the development of performance indicators of epidemiological surveillance networks was developed in 2004 and was applied to several networks. Its implementation requires a thorough description of the network environment and all its activities to define priority indicators. Since this method is considered to be complex, our objective consisted in developing a simplified approach and applying it to an epidemiological surveillance network.

METHODS

We applied the initial method to a theoretical network model to obtain a list of generic indicators that can be adapted to any surveillance network.

RESULTS

We obtained a list of 25 generic performance indicators, intended to be reformulated and described according to the specificities of each network. It was used to develop performance indicators for RESAPATH, an epidemiological surveillance network of antimicrobial resistance in pathogenic bacteria of animal origin in France.

CONCLUSION

This application allowed us to validate the simplified method, its value in terms of practical implementation, and its level of user acceptance. Its ease of use and speed of application compared to the initial method argue in favor of its use on broader scale.

Relationships between technical efficiency and the quality and costs of health care in Italy

OBJECTIVE

This paper reports the measurement of technical efficiency of Tuscan Local Health Authorities and its relationship with quality and appropriateness of care.

DESIGN

First, a bias-corrected measure of technical efficiency was developed using the bootstrap technique applied to data envelopment analysis. Then, correlation analysis was used to investigate the relationships among technical efficiency, quality and appropriateness of care.

SETTING AND PARTICIPANTS

These analyses have been applied to the Local Health Authorities of Tuscany Region (Italy), which provide not only hospital inpatient services, but also prevention and primary care. All top managers of Tuscan Local Health Authorities were involved in selection of the inputs and outputs for calculating technical efficiency.

MAIN OUTCOME MEASURES

The main measure used in this study are volume, quality and appropriateness indicators monitored by the multidimensional performance evaluation system developed in the Tuscany Region.

RESULTS

On average, Tuscan Local Health Authorities experienced 14(%) of bias-corrected inefficiency in 2007. Correlation analyses showed a significant negative correlation between per capita costs and overall performance. No correlation was found in 2007 between technical efficiency and overall performance or between technical efficiency and per capita costs.

CONCLUSIONS

Technical efficiency cannot be considered as an extensive measure of healthcare performance, but evidence shows that Tuscan Local Health Authorities have room for improvement in productivity levels. Indeed, correlation findings suggest that, to pursue financial sustainability, Local Health Authorities mainly have to improve their performance in terms of quality and appropriateness.

Understanding organisational development, sustainability, and diffusion of innovations within hospitals participating in a multilevel quality collaborative

Background

Between 2004 and 2008, 24 Dutch hospitals participated in a two-year multilevel quality collaborative (MQC) comprised of (a) a leadership programme for hospital executives, (b) six quality-improvement collaboratives (QICs) for healthcare professionals and other staff, and (c) an internal programme organisation to help senior management monitor and coordinate team progress. The MQC aimed to stimulate the development of quality-management systems and the spread of methods to improve patient safety and logistics. The objective of this study is to describe how the first group of eight MQC hospitals sustained and disseminated improvements made and the quality methods used.

Methods

The approach followed by the hospitals was described using interview and questionnaire data gathered from eight programme coordinators.

Results

MQC hospitals followed a systematic strategy of diffusion and sustainability. Hospital quality-management systems are further developed according to a model linking plan-do-study-act cycles at the unit and hospital level. The model involves quality norms based on realised successes, performance agreements with unit heads, organisational support, monitoring, and quarterly accountability reports.

Conclusions

It is concluded from this study that the MQC contributed to organisational development and dissemination within participating hospitals. Organisational learning effects were demonstrated. System changes affect the context factors in the theory of organisational readiness: organisational culture, policies and procedures, past experience, organisational resources, and organisational structure. Programme coordinator responses indicate that these factors are utilised to manage spread and sustainability. Further research is needed to assess long-term effects.

Effects of purchaser competition in the Dutch health system: is the glass half full or half empty?

In 2006, the Dutch health insurance system was radically reformed to strengthen competition among health insurers as purchasers of health services. This article considers whether purchaser competition has improved efficiency in health-care provision. Although supply and price regulation still dominates the allocation of health services, purchaser competition has already significantly affected the provision of hospital care, pharmaceuticals and primary care, as well as efforts to gather and disseminate information about quality of care. From this perspective, the glass is half full. However, based on the crude performance indicators available, the reforms have not yet demonstrated significant effects on the performance of the Dutch health system. From this perspective the glass is half empty. The article concludes that the effectiveness of purchaser competition depends crucially on the success of ongoing efforts to improve performance indicators, product classification and the risk equalisation scheme.

Defining antimicrobial prescribing quality indicators: what is a new prescription?

 

Since guidelines on antibiotic drug treatment often focus on appropriate first choice drugs, assessment of guideline adherence should only concentrate on the first drug prescribed, and not on subsequent antibiotics prescribed after failure of the first one.

Purpose

To determine a valid cut-off point for a definition of “first” or “new” prescription in indicators for the assessment of the quality of antibiotic drug treatment on the basis of pharmaceutical data.

Methods

Three possible definitions for the term “new prescription” were compared, based on three different periods of time, viz. more than 35, 28, or 21 days after starting a previous antibiotic. In an observational study, 1,225 antimicrobial prescriptions from the medical files of five family practices were audited (“clinical classification”) and compared with a classification based on the three definitions (“technical classification”). Agreement between these clinical and technical classifications was determined by calculating Cohen’s kappa. The technical classification was analyzed as a diagnostic test, using the clinical classification as the gold standard, and sensitivity, specificity, likelihood ratios, and post-test probabilities were calculated.

Results

Defining “new prescription” as “more than 35 days after a previous prescription was issued” resulted in a Cohen’s kappa of 0.93 (95% CI 0.92–0.98). The diagnostic value of this definition was extremely high, with a sensitivity of 0.976, specificity of 0.987, positive likelihood ratio of 77.7, and negative likelihood ratio of 0.02.

Conclusion

We recommend using a cut-off value of 35 days since the last antimicrobial prescription as the definition of a “new prescription” when no diagnostic information is available, i.e., when using pharmaceutical data to assess the quality of antibiotic prescribing behavior.

Indicadores de segurança do paciente para hospitais de pacientes agudos: revisão sistemática

Foi realizada uma revisão sistemática para identificar as estratégias utilizadas no desenvolvimento de indicadores de segurança do paciente para hospitais de pacientes agudos. As fontes de dados utilizadas foram: MEDLINE, EMBASE, sítios na Internet e referências bibliográficas dos documentos selecionados. Foram incluídos 14 projetos de desenvolvimento de indicadores. O uso de diversos termos relacionados à qualidade e segurança do paciente foi observado com definições variadas. A revisão da literatura e a participação de especialistas e outras representações caracterizaram os projetos. Dos 285 indicadores identificados, 125 foram classificados em mais de uma dimensão da qualidade. A combinação mais freqüente foi segurança e efetividade. Identificou-se um número maior de indicadores sobre medicamentos, e a maioria representa informações de resultado. Observou-se a importância de considerar no desenvolvimento dos indicadores variações culturais, da prática clínica, a disponibilidade dos sistemas de informação e a capacidade de hospitais para implementar sistemas de monitoramento efetivos.

How do healthcare consumers process and evaluate comparative healthcare information? A qualitative study using cognitive interviews

BACKGROUND

To date, online public healthcare reports have not been effectively used by consumers. Therefore, we qualitatively examined how healthcare consumers process and evaluate comparative healthcare information on the Internet.

METHODS

Using semi-structured cognitive interviews, interviewees (n = 20) were asked to think aloud and answer questions, as they were prompted with three Dutch web pages providing comparative healthcare information.

RESULTS

We identified twelve themes from consumers' thoughts and evaluations. These themes were categorized under four important areas of interest: (1) a response to the design; (2) a response to the information content; (3) the use of the information, and (4) the purpose of the information.

CONCLUSION

Several barriers to an effective use of comparative healthcare information were identified, such as too much information and the ambiguity of terms presented on websites. Particularly important for future research is the question of how comparative healthcare information can be integrated with alternative information, such as patient reviews on the Internet. Furthermore, the readability of quality of care concepts is an issue that needs further attention, both from websites and communication experts.

Exploring the black box of quality improvement collaboratives: modelling relations between conditions, applied changes and outcomes

INTRODUCTION

Despite the popularity of quality improvement collaboratives (QICs) in different healthcare settings, relatively little is known about the implementation process. The objective of the current study is to learn more about relations between relevant conditions for successful implementation of QICs, applied changes, perceived successes, and actual outcomes.

METHODS

Twenty-four Dutch hospitals participated in a dissemination programme based on QICs. A questionnaire was sent to 237 leaders of teams who joined 18 different QICs to measure changes in working methods and activities, overall perceived success, team organisation, and supportive conditions. Actual outcomes were extracted from a database with team performance indicator data. Multi-level analyses were conducted to test a number of hypothesised relations within the cross-classified hierarchical structure in which teams are nested within QICs and hospitals.

RESULTS

Organisational and external change agent support is related positively to the number of changed working methods and activities that, if increased, lead to higher perceived success and indicator outcomes scores. Direct and indirect positive relations between conditions and perceived success could be confirmed. Relations between conditions and actual outcomes are weak. Multi-level analyses reveal significant differences in organisational support between hospitals. The relation between perceived successes and actual outcomes is present at QIC level but not at team level.

DISCUSSION

Several of the expected relations between conditions, applied changes and outcomes, and perceived successes could be verified. However, because QICs vary in topic, approach, complexity, and promised advantages, further research is required: first, to understand why some QIC innovations fit better within the context of the units where they are implemented; second, to assess the influence of perceived success and actual outcomes on the further dissemination of projects over new patient groups.

Exploring the black box of quality improvement collaboratives: modelling relations between conditions, applied changes and outcomes

Introduction

Despite the popularity of quality improvement collaboratives (QICs) in different healthcare settings, relatively little is known about the implementation process. The objective of the current study is to learn more about relations between relevant conditions for successful implementation of QICs, applied changes, perceived successes, and actual outcomes.

Methods

Twenty-four Dutch hospitals participated in a dissemination programme based on QICs. A questionnaire was sent to 237 leaders of teams who joined 18 different QICs to measure changes in working methods and activities, overall perceived success, team organisation, and supportive conditions. Actual outcomes were extracted from a database with team performance indicator data. Multi-level analyses were conducted to test a number of hypothesised relations within the cross-classified hierarchical structure in which teams are nested within QICs and hospitals.

Results

Organisational and external change agent support is related positively to the number of changed working methods and activities that, if increased, lead to higher perceived success and indicator outcomes scores. Direct and indirect positive relations between conditions and perceived success could be confirmed. Relations between conditions and actual outcomes are weak. Multi-level analyses reveal significant differences in organisational support between hospitals. The relation between perceived successes and actual outcomes is present at QIC level but not at team level.

Discussion

Several of the expected relations between conditions, applied changes and outcomes, and perceived successes could be verified. However, because QICs vary in topic, approach, complexity, and promised advantages, further research is required: first, to understand why some QIC innovations fit better within the context of the units where they are implemented; second, to assess the influence of perceived success and actual outcomes on the further dissemination of projects over new patient groups.

Quantitative data management in quality improvement collaboratives

Background

Collaborative approaches in quality improvement have been promoted since the introduction of the Breakthrough method. The effectiveness of this method is inconclusive and further independent evaluation of the method has been called for. For any evaluation to succeed, data collection on interventions performed within the collaborative and outcomes of those interventions is crucial. Getting enough data from Quality Improvement Collaboratives (QICs) for evaluation purposes, however, has proved to be difficult. This paper provides a retrospective analysis on the process of data management in a Dutch Quality Improvement Collaborative. From this analysis general failure and success factors are identified.

Discussion

This paper discusses complications and dilemma's observed in the set-up of data management for QICs. An overview is presented of signals that were picked up by the data management team. These signals were used to improve the strategies for data management during the program and have, as far as possible, been translated into practical solutions that have been successfully implemented.

The recommendations coming from this study are:

From our experience it is clear that quality improvement programs deviate from experimental research in many ways. It is not only impossible, but also undesirable to control processes and standardize data streams. QIC's need to be clear of data protocols that do not allow for change. It is therefore minimally important that when quantitative results are gathered, these results are accompanied by qualitative results that can be used to correctly interpret them.

Monitoring and data acquisition interfere with routine. This makes a database collecting data in a QIC an intervention in itself. It is very important to be aware of this in reporting the results. Using existing databases when possible can overcome some of these problems but is often not possible given the change objective of QICs.

Introducing a standardized spreadsheet to the teams is a very practical and helpful tool in collecting standardized data within a QIC. It is vital that the spreadsheets are handed out before baseline measurements start.

Longitudinal analysis on the development of hospital quality management systems in the Netherlands

OBJECTIVE

Many changes have been initiated in the Dutch hospital sector to optimize health-care delivery: national agenda-setting, increased competition and transparency, a new system of hospital reimbursement based on diagnosis-treatment combinations, intensified monitoring of quality and a multi-layered organizational development programme based on quality improvement collaboratives. The objective is to answer the question as to whether these changes were accompanied by a further development of hospital quality management systems and to what extent did the development within the multi-layered programme hospitals differ from that in other hospitals.

DESIGN

Longitudinal data were collected in 1995, 2000, 2005 and 2007 using a validated questionnaire. Descriptive analyses and multi-level modelling were applied to test whether: (1) quality management system development stages in hospitals differ over time, (2) development stages and trends differ between hospitals participating or not participating in the multi-layered programme and (3) hospital size has an effect on development stage.

SETTING

Dutch hospital sector between 1995 and 2007.

PARTICIPANTS

Hospital organizations.

INTERVENTION

Changes through time.

MAIN OUTCOME MEASURE

Quality management system development stage.

RESULTS

Since 1995, hospital quality management systems have reached higher development levels. Programme participants have developed their quality management system more rapidly than have non-participants. However, this effect is confounded by hospital size.

CONCLUSIONS

Study results suggest that the combination of policy measures at macro level was accompanied by an increase in hospital size and the further development of quality management systems. Hospitals are entering the stage of systematic quality improvement.

Performance evaluation of inpatient service in Beijing: a horizontal comparison with risk adjustment based on Diagnosis Related Groups

BACKGROUND

The medical performance evaluation, which provides a basis for rational decision-making, is an important part of medical service research. Current progress with health services reform in China is far from satisfactory, without sufficient regulation. To achieve better progress, an effective tool for evaluating medical performance needs to be established. In view of this, this study attempted to develop such a tool appropriate for the Chinese context.

METHODS

Data was collected from the front pages of medical records (FPMR) of all large general public hospitals (21 hospitals) in the third and fourth quarter of 2007. Locally developed Diagnosis Related Groups (DRGs) were introduced as a tool for risk adjustment and performance evaluation indicators were established: Charge Efficiency Index (CEI), Time Efficiency Index (TEI) and inpatient mortality of low-risk group cases (IMLRG), to reflect respectively work efficiency and medical service quality. Using these indicators, the inpatient services' performance was horizontally compared among hospitals. Case-mix Index (CMI) was used to adjust efficiency indices and then produce adjusted CEI (aCEI) and adjusted TEI (aTEI). Poisson distribution analysis was used to test the statistical significance of the IMLRG differences between different hospitals.

RESULTS

Using the aCEI, aTEI and IMLRG scores for the 21 hospitals, Hospital A and C had relatively good overall performance because their medical charges were lower, LOS shorter and IMLRG smaller. The performance of Hospital P and Q was the worst due to their relatively high charge level, long LOS and high IMLRG. Various performance problems also existed in the other hospitals.

CONCLUSION

It is possible to develop an accurate and easy to run performance evaluation system using Case-Mix as the tool for risk adjustment, choosing indicators close to consumers and managers, and utilizing routine report forms as the basic information source. To keep such a system running effectively, it is necessary to improve the reliability of clinical information and the risk-adjustment ability of Case-Mix.

One-year mortality after a first visit to a cardiology outpatient clinic: a useful performance indicator?

PURPOSE

In 2003 the Dutch Health Care Inspectorate introduced performance indicators to monitor and compare quality of care in Dutch hospitals. In 2007, the new performance indicator 'one-year mortality after a first visit to a cardiology outpatient clinic' was introduced. We set out to evaluate this new indicator in three Dutch teaching hospitals.

METHODS

Using electronic medical records, information was collected retrospectively of patients aged >/=70 years who visited the cardiology outpatient clinic of Medical Centre Alkmaar, Meander Medical Centre Amersfoort and Deventer Hospital between 1 January 2006 and 31 January 2006. Diagnoses were based on the diagnosis treatment combination (DBC) coding system.

RESULTS

547 patients (mean age 78.0 years, 53% men) were included, 35 (6.4%) of whom had died after one year. Cardiovascular disease was the most frequent cause of death (22/35, 62.9%). The oneyear mortality among the three hospitals varied from 5.0 to 7.3% (NS).

CONCLUSION

One-year mortality after the first visit to a cardiology outpatient clinic amounted to 6.4% in patients aged >/=70 years and did not differ significantly between the three Dutch teaching hospitals. The administrative load to obtain the necessary information was considerable. One-year mortality should be regarded as an 'outcome' parameter rather than a 'performance' indicator. (Neth Heart J 2009;17:52-5.).

Building quality report cards for geriatric care in The Netherlands: using concept mapping to identify the appropriate "building blocks" from the consumer's perspective

PURPOSE

This article reports on a study to identify "building blocks" for quality report cards for geriatric care. Its aim is to present (a) the results of the study and (b) the innovative step-by-step approach that was developed to arrive at these results.

DESIGN AND METHODS

We used Concept Mapping/Structured Conceptualization to define the building blocks. Applied to this study, we carried out Concept Mapping using several data collection methods: (a) a Web search, (b) semistructured interviews, (c) document analysis, (d) questionnaires, and (e) focus groups.

RESULTS

The findings showed that, although home care and institutional care for elderly adults share many quality themes, experts need to develop separate quality report cards for the two types of geriatric care. Home care consumers attach more value to the availability, continuity, and reliability of care, whereas consumers of institutional care value privacy, respect, and autonomy most. This study also showed, unlike many other quality report card studies, that consumers want information on structure, process and outcome indicators, and rating outcome indicators such as effectiveness and safety of care both for home care and for institutional care. Concept Mapping proved to be a valuable method for developing quality report cards in health care.

IMPLICATIONS

Building blocks were delivered for two quality report cards for geriatric care and will be used when quality report cards are built in The Netherlands. For the U.S. context, this study shows that current national report cards for geriatric care should be supplemented with quality-of-life data.

Mortality in Dutch hospitals: trends in time, place and cause of death after admission for myocardial infarction and stroke. An observational study

Background

Patterns in time, place and cause of death can have an important impact on calculated hospital mortality rates. Objective is to quantify these patterns following myocardial infarction and stroke admissions in Dutch hospitals during the period 1996–2003, and to compare trends in the commonly used 30-day in-hospital mortality rates with other types of mortality rates which use more extensive follow-up in time and place of death.

Methods

Discharge data for all Dutch admissions for index conditions (1996–2003) were linked to the death certification registry. Then, mortality rates within the first 30, 90 and 365 days following admissions were analyzed for deaths occurring within and outside hospitals.

Results

Most deaths within a year after admission occurred within 30 days (60–70%). No significant trends in this distribution of deaths over time were observed. Significant trends in the distribution over place of death were observed for both conditions. For myocardial infarction, the proportion of deaths after transfer to another hospital has doubled from 1996–2003. For stroke a significant rise of the proportion of deaths outside hospital was found. For MI the proportion of deaths attributed to a circulatory disease has significantly fallen ovtime. Seven types of hospital mortality indicators, different in scope and observation period, all show a drop of hospital mortality for both MI and stroke over the period 1996–2003. For stroke the observed absolute reduction in death rate increases for the first year after admission, for MI the observed drop in 365-day overall mortality almost equals the observed drop in 30-day in hospital mortality over 1996–2003.

Conclusion

Changes in the timing, place and causes of death following admissions for myocardial infarction and stroke have important implications for the definitions of in-hospital and post-admission mortality rates as measures of hospital performance. Although necessary for understanding mortality patterns over time, including within mortality rates deaths which occur outside hospitals and after longer periods following index admissions remain debatable and may not reflect actual hospital performance but probably mirrors transfer, efficiency, and other health care policies.

Quality measurement at intensive care units: which indicators should we use?

OBJECTIVE

This study was conducted to develop a set of indicators that measure the quality of care in intensive care units (ICU) in Dutch hospitals and to evaluate the feasibility of the registration of these indicators.

METHODS

To define potential indicators for measuring quality, 3 steps were made. First, a literature search was carried out to obtain peer-reviewed articles from 2000 to 2005, describing process or structure indicators in intensive care, which are associated with patient outcome. Additional indicators were suggested by a panel of experts. Second, a selection of indicators was made by a panel of experts using a questionnaire and ranking in a consensus procedure. Third, a study was done for 6 months in 18 ICUs to evaluate the feasibility of using the identified quality indicators. Site visits, interviews, and written questionnaires were used to evaluate the use of indicators.

RESULTS

Sixty-two indicators were initially found, either in the literature or suggested by the members of the expert panel. From these, 12 indicators were selected by the expert panel by consensus. After the feasibility study, 11 indicators were eventually selected. "Interclinical transport," referring to a change of hospital, was dropped because of lack of reliability and support for further implementation by the participating hospitals in the study. The following structure indicators were selected: availability of intensivist (hours per day), patient-to-nurse ratio, strategy to prevent medication errors, measurement of patient/family satisfaction. Four process indicators were selected: length of ICU stay, duration of mechanical ventilation, proportion of days with all ICU beds occupied, and proportion of glucose measurement exceeding 8.0 mmol/L or lower than 2.2 mmol/L. The selected outcome indicators were as follows: standardized mortality (APACHE II), incidence of decubitus, number of unplanned extubations. The time for registration varied from less than 30 minutes to more than 1 hour per day to collect the items. Among other factors, this variation in workload was related to the availability of computerized systems to collect the data.

CONCLUSION

In this study, a set of 11 quality indicators for intensive care was defined based on literature research, expert opinion, and testing. The set gives a quick view of the quality of care in individual ICUs. The availability of a computerized data collection system is important for an acceptable workload.

Comparing health system performance assessment and management approaches in the Netherlands and Ontario, Canada

Background

Given the proliferation and the growing complexity of performance measurement initiatives in many health systems, the Netherlands and Ontario, Canada expressed interests in cross-national comparisons in an effort to promote knowledge transfer and best practise. To support this cross-national learning, a study was undertaken to compare health system performance approaches in The Netherlands with Ontario, Canada.

Methods

We explored the performance assessment framework and system of each constituency, the embeddedness of performance data in management and policy processes, and the interrelationships between the frameworks. Methods used included analysing governmental strategic planning and policy documents, literature and internet searches, comparative descriptive tables, and schematics. Data collection and analysis took place in Ontario and The Netherlands. A workshop to validate and discuss the findings was conducted in Toronto, adding important insights to the study.

Results

Both Ontario and The Netherlands conceive health system performance within supportive frameworks. However they differ in their assessment approaches. Ontario's Scorecard links performance measurement with strategy, aimed at health system integration. The Dutch Health Care Performance Report (Zorgbalans) does not explicitly link performance with strategy, and focuses on the technical quality of healthcare by measuring dimensions of quality, access, and cost against healthcare needs. A backbone 'five diamond' framework maps both frameworks and articulates the interrelations and overlap between their goals, themes, dimensions and indicators. The workshop yielded more contextual insights and further validated the comparative values of each constituency's performance assessment system.

Conclusion

To compare the health system performance approaches between The Netherlands and Ontario, Canada, several important conceptual and contextual issues must be addressed, before even attempting any future content comparisons and benchmarking. Such issues would lend relevant interpretational credibility to international comparative assessments of the two health systems.