How much confidence should we place in efficiency estimates?

Health systems efficiency in Eastern Mediterranean Region: a data envelopment analysis

BACKGROUND

One of the most important issues in public policy and welfare state is health care. Poor management leads to the waste of resources, including money, human resources, facilities, and equipment.

AIMS

This paper seeks to answer the question of which eastern Mediterranean countries are more effective in allocating their health resources, and does Iran, in relation to those countries, have an effective health system.

METHODS

This study examined technical efficiency among eastern Mediterranean countries in 2018. Data were extracted from Global Health Observatory data World Health Organization. We applied input-oriented Data Envelopment Analysis (DEA) models to estimate efficiency scores. Inputs are Physicians density per 10,000 populations, Total hospital beds per 10,000 populations, Current expenditure on health, % of gross domestic product and outputs are infant survival rate and Life expectancy.

RESULTS

The most efficient health systems in the eastern Mediterranean were Bahrain, Egypt, Iran, Lebanon, Morocco, Oman, Pakistan, Qatar, Tunisia and the United Arab Emirates. The inefficient countries are Iraq, Jordan, Kuwait, Libya, Palestine and Saudi Arabia.

CONCLUSIONS

Among the efficient countries, one category of high-entry countries such as Bahrain and Qatar with high input especially in health expenditure had higher output. The second group of countries with lower inputs such as Iran and Morocco has been able to produce similar output with other countries. Also, inefficiency in countries such as Saudi Arabia can be attributed to this with higher input such as health expenditure has lower output such life expectancy and infant survival rate.

Measuring and optimising the efficiency of community hospital inpatient care for older people: the MoCHA mixed-methods study

Background

Community hospitals are small hospitals providing local inpatient and outpatient services. National surveys report that inpatient rehabilitation for older people is a core function but there are large differences in key performance measures. We have investigated these variations in community hospital ward performance.

Objectives

(1) To measure the relative performance of community hospital wards (studies 1 and 2); (2) to identify characteristics of community hospital wards that optimise performance (studies 1 and 3); (3) to develop a web-based interactive toolkit that supports operational changes to optimise ward performance (study 4); (4) to investigate the impact of community hospital wards on secondary care use (study 5); and (5) to investigate associations between short-term community (intermediate care) services and secondary care utilisation (study 5).

Methods

Study 1 – we used national data to conduct econometric estimations using stochastic frontier analysis in which a cost function was modelled using significant predictors of community hospital ward costs. Study 2 – a national postal survey was developed to collect data from a larger sample of community hospitals. Study 3 – three ethnographic case studies were performed to provide insight into less tangible aspects of community hospital ward care. Study 4 – a web-based interactive toolkit was developed by integrating the econometrics (study 1) and case study (study 3) findings. Study 5 – regression analyses were conducted using data from the Atlas of Variation Map 61 (rate of emergency admissions to hospital for people aged ≥ 75 years with a length of stay of < 24 hours) and the National Audit of Intermediate Care.

Results

Community hospital ward efficiency is comparable with the NHS acute hospital sector (mean cost efficiency 0.83, range 0.72–0.92). The rank order of community hospital ward efficiencies was distinguished to facilitate learning across the sector. On average, if all community hospital wards were operating in line with the highest cost efficiency, savings of 17% (or £47M per year) could be achieved (price year 2013/14) for our sample of 101 wards. Significant economies of scale were found: a 1% rise in output was associated with an average 0.85% increase in costs. We were unable to obtain a larger community hospital sample because of the low response rate to our national survey. The case studies identified how rehabilitation was delivered through collaborative, interdisciplinary working; interprofessional communication; and meaningful patient and family engagement. We also developed insight into patients’ recovery trajectories and care transitions. The web-based interactive toolkit was established [http://mocha.nhsbenchmarking.nhs.uk/ (accessed 9 September 2019)]. The crisis response team type of intermediate care, but not community hospitals, had a statistically significant negative association with emergency admissions.

Limitations

The econometric analyses were based on cross-sectional data and were also limited by missing data. The low response rate to our national survey means that we cannot extrapolate reliably from our community hospital sample.

Conclusions

The results suggest that significant community hospital ward savings may be realised by improving modifiable performance factors that might be augmented further by economies of scale.

Future work

How less efficient hospitals might reduce costs and sustain quality requires further research.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 1. See the NIHR Journals Library website for further project information.

Determinants of transient and persistent hospital efficiency: The case of Italy

In this paper, we extend the 4-random-component closed skew-normal stochastic frontier model by including exogenous determinants of hospital persistent (long-run) and transient (short-run) inefficiency, separated from unobserved heterogeneity. We apply this new model to a dataset composed by 133 Italian hospitals during the period 2008-2013. We show that average total inefficiency is about 23%, higher than previous estimates; hence, a model where the different types of inefficiency and hospital unobserved characteristics are not confounded allows us to get less biased estimates of hospital inefficiency. Moreover, we find that transient efficiency is more important than persistent efficiency, as it accounts for 60% of the total one. Last, we find that ownership (for-profit hospitals are more transiently inefficient and less persistently inefficient than not-for-profit ones, whereas public hospitals are less transiently inefficient than not-for-profit ones), specialization (specialized hospitals are more transiently inefficient than general ones; i.e., there is evidence of scope economies in short-run efficiency), and size (large-sized hospitals are better than medium and small ones in terms of transient inefficiency) are determinants of both types of inefficiency, although we do not find any statistically significant effect of multihospital systems and teaching hospitals.

The efficiency of the local health systems: investigating the roles of health administrations and health care providers

The analysis of efficiency in health care has largely focussed either on individual health care providers, or on sub-national health systems conceived as a unique decision-making unit. However, in hierarchically organized national health services, two separate entities are responsible for turning financial resources into services at the local level: health administrations and health care providers. Their separate roles and the one of health administrations in particular have not been explicitly considered in efficiency analysis. We applied stochastic frontier analysis to district-level panel data from Mozambique (2008-2011) to assess district efficiency in delivering outpatient care. We first assessed the efficiency of the whole district considered as an individual decision-making unit, and then we assessed separately the efficiency of health administrations and health care providers within the same district. We found that on average only 73% of the outpatient consultations deliverable using available inputs were realized, with large differences in performance across districts. Individual districts performed differently in administrative or health care delivery functions. On average, a reduction of administrative inefficiency by 10 percentage points, for a given expenditure would increase by 0.2% the volume of services delivered per thousand population per year. Identifying and targeting the specific drivers of administrative inefficiencies can contribute to increase service.

An international comparison of efficiency of inpatient mental health care systems

There is a fundamental gap in the evidence base on quantitative cross-country comparison of mental healthcare systems due to the challenges of comparative analysis in mental health including a paucity of good quality data. We explore whether existing limited data sources can potentially be exploited to examine technical efficiency of inpatient mental healthcare systems in 32 OECD countries in 2010. We use two analytical approaches: Data Envelopment Analysis (DEA) with bootstrapping to produce confidence intervals of efficiency scores and country rankings, and Cluster Analysis to group countries according to two broad efficiency groupings. We incorporate environmental variables using a two-stage truncated regression. We find slightly tighter confidence intervals for the less efficient countries which loosely corresponds with the 'inefficient' cluster grouping in the Cluster Analysis. However there is little stability in country rankings making it difficult with current data to draw any policy inferences. Environmental factors do not appear to significantly impact on efficiency scores. The most pressing pursuit remains the search for better national data in mental healthcare to underpin future analyses. Otherwise the use of any sophisticated analytic techniques will prove futile for establishing robust conclusions regarding international comparisons of the performance of mental healthcare systems.

Applying risk adjusted cost-effectiveness (RAC-E) analysis to hospitals: estimating the costs and consequences of variation in clinical practice

Cost-effectiveness analysis is well established for pharmaceuticals and medical technologies but not for evaluating variations in clinical practice. This paper describes a novel methodology--risk adjusted cost-effectiveness (RAC-E)--that facilitates the comparative evaluation of applied clinical practice processes. In this application, risk adjustment is undertaken with a multivariate matching algorithm that balances the baseline characteristics of patients attending different settings (e.g., hospitals). Linked, routinely collected data are used to analyse patient-level costs and outcomes over a 2-year period, as well as to extrapolate costs and survival over patient lifetimes. The study reports the relative cost-effectiveness of alternative forms of clinical practice, including a full representation of the statistical uncertainty around the mean estimates. The methodology is illustrated by a case study that evaluates the relative cost-effectiveness of services for patients presenting with acute chest pain across the four main public hospitals in South Australia. The evaluation finds that services provided at two hospitals were dominated, and of the remaining services, the more effective hospital gained life years at a low mean additional cost and had an 80% probability of being the most cost-effective hospital at realistic cost-effectiveness thresholds. Potential determinants of the estimated variation in costs and effects were identified, although more detailed analyses to identify specific areas of variation in clinical practice are required to inform improvements at the less cost-effective institutions.

Federal state differentials in the efficiency of health production in Germany: an artifact of spatial dependence?

Due to regional competition and patient migration, the efficiency of healthcare provision at the regional level is subject to spatial dependence. We address this issue by applying a spatial autoregressive model to longitudinal data from Germany at the district ('Kreis') level. The empirical model is specified to explain efficiency scores, which we derive through non-parametric order-m efficiency analysis of regional health production. The focus is on the role of health policy of federal states ('Bundesländer') for district efficiency. Regression results reveal significant spatial spillover effects. Notably, accounting for spatial dependence does not decrease but increases the estimated effect of federal states on district efficiency. It appears that genuinely more efficient states are less affected by positive efficiency spillovers, so that taking into account spatial dependence clarifies the importance of health policy at the state level.

Healthcare policy tools as determinants of health-system efficiency: evidence from the OECD

This paper assesses which policy-relevant characteristics of a healthcare system contribute to health-system efficiency. Health-system efficiency is measured using the stochastic frontier approach. Characteristics of the health system are included as determinants of efficiency. Data from 21 OECD countries from 1970 to 2008 are analysed. Results indicate that broader health-system structures, such as Beveridgian or Bismarckian financing arrangements or gatekeeping, are not significant determinants of efficiency. Significant contributors to efficiency are policy instruments that directly target patient behaviours, such as insurance coverage and cost sharing, and those that directly target physician behaviours, such as physician payment methods. From the perspective of the policymaker, changes in cost-sharing arrangements or physician remuneration are politically easier to implement than changes to the foundational financing structure of the system.

What have we learned from the application of stochastic frontier analysis to U.S. hospitals?

This article focuses on the lessons learned from stochastic frontier analysis studies of U.S. hospitals, of which at least 27 have been published. A brief discussion of frontier techniques is provided, but a technical review of the literature is not included because overviews of estimation issues have been published recently. The primary focus is on the correlates of hospital inefficiency. In addition to examining the association of market pressures and hospital inefficiency, the authors also examined the relationship between inefficiency and hospital behavior (e.g., hospital exits) and inefficiency and other measures of hospital performance (e.g., outcome measures of quality). The authors found that consensus is emerging on the relationship of some factors to hospital efficiency; however, further research is needed to better understand others. The application of stochastic frontier analysis to specific policy issues is in its infancy; however, the methodology holds promise for being useful in certain contexts.

Separating managerial inefficiency from influences of the operating environment: an application in dialysis

In any production unit, the ability to achieve technical efficiency is influenced by characteristics of the external operating environment. This study uses the Greek dialysis sector to employ a previously reported frontier procedure to obtain a measure of managerial inefficiency that controls for exogenous features. The sample consisted of 124 dialysis facilities. Two inputs -nursing staff and dialysis machines- and one output -dialysis sessions--were used in an input-oriented, variable-returns-to-scale DEA model. Input slacks were regressed against environmental characteristics such as ownership, location, operating years and facility size, and parameter estimates were used to adjust primary input data. New efficiency scores were generated to measure managerial inefficiency. Older, public, regional facilities were operating under unfavorable circumstances, whereas newer, private, Athens-based facilities under favorable conditions. This respectively generated lower and higher efficiency scores than would have been attained on a level "playing field". After adjustment, scores reflected only management inefficiency and could be compared fairly. This study emphasizes the importance of efficiency comparisons, which take into account external conditions beyond the influence ofmanagement, as these have been shown to under- or overstate true management inefficiency.

The measurement of efficiency and productivity of health care delivery

The measurement of efficiency and productivity of health service delivery has become a small industry. This is a review of 317 published papers on frontier efficiency measurement. The techniques used are mainly based on non-parametric data envelopment analysis, but there is increasing use of parametric techniques, such as stochastic frontier analysis. Applications to hospitals and other health care organizations and areas are reviewed and summarised, and some meta-type analysis undertaken. Cautious conclusions are that public provision may be potentially more efficient than private, in certain settings. The paper also considers conceptualizations of efficiency, and points to dangers and opportunities in generating such information. Finally, some criteria for assessing the use and usefulness of efficiency studies are established, with a view to helping both researchers and those assessing whether or not to act upon published results.

Cost inefficiency and hospital health outcomes

This study explores the association between cost inefficiency and health outcomes in a national sample of acute-care hospitals in the US over the period 1999-2001, with health outcomes being measured by both mortality and complications rates. The empirical analysis examines health outcomes as a function of cost inefficiency and other determinants of outcomes, using stochastic frontier analysis to obtain hospital cost inefficiency scores. The results showed no systematic pattern of association between cost inefficiency and hospital health outcomes; the basic results were unchanged regardless of whether cost inefficiency was measured with or without using instrumental variables. The analysis also indicated, however, that the association between cost inefficiency and health outcomes may vary substantially across geographical regions. The study highlights the importance of distinguishing between 'good' costs that reflect the efficient use of resources and 'bad' costs that stem from waste and other forms of inefficiency. In particular, the study's results suggest that hospital programs focused on reducing cost inefficiency are unlikely to be associated with worsened hospital-level mortality or complications rates, while, on the other hand, across-the-board reductions in cost could well have adverse consequences on health outcomes by reducing efficient as well as inefficient costs.

Universal access to HIV treatment in developing countries: going beyond the misinterpretations of the 'cost-effectiveness' algorithm

BACKGROUND

Economic cost-effectiveness analysis (CEA) has been proposed as the appropriate tool to set priorities for resource allocation among available health interventions. Controversy remains about the way CEA should be used in the field of HIV/AIDS.

METHODS AND OBJECTIVES

This paper reviews the general literature in health economics and public economics about the use of CEA for priority setting in public health, in order better to inform current debates about resource allocation in the fight against HIV/AIDS.

RESULTS

Theoretical and practical limitations of CEA do not raise major problems when it is applied to compare alternatives for treating the same medical condition or public health problem. Using CEA to set priorities among different health interventions by ranking them from the lowest to the highest values of their cost per life-year saved is appropriate only under the very restrictive and unrealistic assumptions that all interventions compared are discrete and finite alternatives that cannot vary in terms of size and scale. In order for CEA to inform resource allocation compared across programmes to fight the AIDS epidemic, a pragmatic interpretation of this economic approach, like that proposed by the Commission on Macroeconomics and Health, is better suited. Interventions, like a number of preventive strategies and first-line antiretroviral treatments for HIV, whose marginal costs per additional life-year saved are less than three times the gross domestic product per capita, should be considered cost-effective.

CONCLUSION

Because of their empirical and theoretical limitations, results of CEA should only be one element in priority setting among interventions for HIV/AIDS, which should also be informed by explicit debates about societal and ethical preferences.

Effects of ownership, subsidization and teaching activities on hospital costs in Switzerland

This paper explores the cost structure of Swiss hospitals, focusing on differences due to teaching activities and those related to ownership and subsidization types. A stochastic total cost frontier with a Cobb-Douglas functional form has been estimated for a panel of 148 general hospitals over the six-year period from 1998 to 2003. Inpatient cases adjusted by DRG cost weights and ambulatory revenues are considered as two separate outputs. The adopted econometric specification allows for unobserved heterogeneity across hospitals. The results suggest that teaching activities are an important cost-driving factor and hospitals that have a broader range of specialization are relatively more costly. The excess costs of university hospitals can be explained by more extensive teaching activities as well as the relative complexity of the offered medical treatments from a teaching point of view. However, even after controlling for such differences university hospitals have shown a relatively low cost-efficiency especially in the first two or three years of the sample period. The analysis does not provide any evidence of significant efficiency differences across ownership/subsidy categories.

Hospital inefficiency: what is the impact of membership in different types of systems?

The primary objective of this study is to assess whether systematic differences in inefficiency are associated with hospital membership in different types of systems. We employed the Battese/Coelli simultaneous stochastic frontier analysis (SFA) technique to estimate hospital cost inefficiency. Mean estimated inefficiency was 8.42%. Membership in different types of systems was related to estimated cost inefficiency (p < .05). Compared to hospitals that were members of centralized health systems, membership in centralized physician/insurance or decentralized systems was associated with decreased inefficiency; membership in independent systems was associated with increased inefficiency.

Stochastic frontier analysis of hospital inefficiency: a review of empirical issues and an assessment of robustness

Twenty stochastic frontier analysis (SFA) studies of hospital inefficiency in the United States were analyzed. Results from best-practice methods were compared against previously used methods in hospital studies to ascertain the robustness of SFA in estimating cost inefficiency. To compare past studies and analyze new data, SFA methods were varied by (a) the assumptions of the structure of costs and distribution of the error term, (b) inclusion of quality and product descriptor measures, and (c) use of simultaneous and two-stage estimation techniques. SFA results were relatively insensitive to several model variations.

The cost efficiency and clinical quality of institutional long-term care for the elderly

This study applied the stochastic frontier cost function with inefficiency effects to estimate the association between quality of care and cost efficiency in institutional long-term care wards for the elderly in Finland. We used several clinical quality indicators for indicating adverse care processes and outcomes, based on the Resident Assessment Instrument (RAI)/Minimum Data Set (MDS). Average cost inefficiency among the wards was 22%. We found an association between the clinical quality indicators and cost inefficiency. Higher prevalence of pressure ulcers was associated with higher costs, whereas the higher prevalence of use of depressants and hypnotics increased inefficiency.

Measuring the productive efficiency and clinical quality of institutional long-term care for the elderly

The authors consider the association between productive efficiency and clinical quality in institutional long-term care for the elderly. Cross-sectional data were collected from 122 wards in health-centre hospitals and residential homes in Finland in 2001. Productive efficiency was measured in terms of technical efficiency, which was defined as the unit's distance from the (best practice) production frontier. The analysis employed stochastic production frontier estimation, where technical inefficiency in the production function was specified to be a function of ward characteristics and clinical quality of care. Several quality indicators based on the Resident Assessment Instrument, such as prevalence of pressure ulcers and depression with no treatment, were used in the analysis. The results did not reveal systematic association between technical efficiency and clinical quality of care. However, the prevalence of pressure ulcers, indicating poor quality of care was associated with technical efficiency, a fact which highlights the importance of including quality measures in the assessment of efficiency in long-term care.

Frontier efficiency measurement in health care: a review of empirical techniques and selected applications

Health care institutions worldwide are increasingly the subject of analyses aimed at defining, measuring, and improving organizational efficiency. However, despite the importance of efficiency measurement in health care services, it is only relatively recently that the more advanced econometric and mathematical programming frontier techniques have been applied to hospitals, nursing homes, health management organizations, and physician practices, among others. This article provides a synoptic survey of the comparatively few empirical analyses of frontier efficiency measurement in health care services. Both the measurement of efficiency in a range of health care services and the posited determinants of health care efficiency are examined.