Estimating technical efficiency in the hospital sector with panel data: a comparison of parametric and non-parametric techniques

Modelling efficiency in primary healthcare using the DEA methodology: an empirical analysis in a healthcare district

BACKGROUND

Primary healthcare management efficiency conditions the functioning of specialized care and has a direct impact on the outcomes of the health system and its sustainability. The objective of this research is to develop models to evaluate the efficiency, including health outcomes, of the primary healthcare centres (PHC) of the Clínico - La Malvarrosa Health District in Valencia.

METHODS

To evaluate efficiency, Data Envelopment Analysis (DEA) was used with output orientation and variable returns to scale, with panel data from the years 2015 to 2019. In rates per 10,000 inhabitants, the inputs are: medical and nursing staff and pharmacy cost. The outputs are: number of consultations, hospital emergencies, referrals, avoidable hospitalisations, avoidable mortality and pharmaceutical prescription efficiency. As exogenous variables: the percentage of population over 65 years old, over 80 and case-mix. Three models were developed, all of them with the same inputs and different combinations of outputs related to: healthcare activity, outcomes, and both, in order to study the influence of the different approaches on efficiency. Each model is analysed both without exogenous variables and with each of them.

RESULTS

The efficiency results vary depending on the model used, although certain PHCs are always on, or very close to, the efficient frontier, while others are always inefficient. When healthcare activity outputs are considered, efficiency scores improve and the number of efficient PHCs increases. However, in general, the PHC score decreases throughout the evaluated period. This decrease is more pronounced when only activity outputs are included.

CONCLUSIONS

DEA allows the inefficiencies of PHCs to be analysed and the efficient ones are clearly distinguished from the inefficient, although different efficiency scores are obtained depending on the model used. Evaluation can be according to healthcare activity, health outcomes or both, making it necessary to identify the expected objectives of the PHCs, as the perspective of the analysis influences the results.

Measuring the efficiency of public hospitals: A multistage data envelopment analysis in Fujian Province, China

Objective

The present study aimed to evaluate the operational efficiency of public hospitals in Fujian Province and the factors responsible for the inefficiency of these hospitals and provide relevant suggestions for health policymakers in allocating service resources.

Method

In the first stage of the research, the variables affecting the efficiency of hospitals were extracted by qualitative and quantitative methods, including literature optimization, gray related analysis and gray clustering evaluation. In the second stage, the data envelopment analysis (DEA) method was used to evaluate the operational efficiency of 49 hospitals of different levels and types selected by sampling in 2020. Finally, a Tobit regression model with introduced institutional factors and background factors was established to study the main influencing factors of hospital inefficiency.

Results

In the first stage, 10 input variables and 10 output variables necessary from the mangers' point of view were identified to test efficiency. In the second stage, the average comprehensive TE, PTE, and SE of 49 sample hospitals was 0.802, 0.888, and 0.902, respectively. 22.45% of these hospitals met the effective criteria, i.e., the overall effective rate was 22.45%. The low SE value of the hospital was the main reason hindering the improvement of the comprehensive efficiency value. The overall effective rate of secondary public hospitals (30.77%) was higher than that of tertiary public hospitals (19.44%), and the overall effective rate of public specialized hospitals (30%) was higher than that of general public hospitals (18.92%). Based on the third stage results, the bed occupancy rate (BOR) and the proportion of beds (POB) were major factors affecting the operation efficiency of grade III hospitals (p < 0.01). However, the operating efficiency of grade II hospitals was significantly affected by POB and regional per capita GDP(GDPPC) (p < 0.05). Moreover, the impact of BOR and GDPPC was positive, and POB was negatively correlated with hospital operation efficiency.

Conclusions

The study results indicated that the overall operation efficiency of public hospitals in Fujian Province is low. This study revealed that intervention should be strengthened from a policy and management perspective to improve the operation efficiency of public hospitals.

Inefficiency of public hospitals: a multistage data envelopment analysis in an Italian region

Background

The objective of this study was to assess public hospital efficiency, including quality outputs, inefficiency determinants, and changes to efficiency over time, in an Italian region. To achieve this aim, the study used secondary data from the Veneto region for the years 2018 and 2019.

Methods

A nonparametric approach—that is, multistage data envelopment analysis (DEA)—was applied to a sample of 43 hospitals. We identified three categories of input: capital investments (Beds), labor (FTE), operating expenses. We selected five efficiency outputs (outpatient visits, inpatients, outpatient visit revenue, inpatient revenue, bed occupancy rate) and two quality outputs (mortality rate and inappropriate admission rate). Efficiency scores were estimated and decomposed into two components. Slack analysis was then conducted. Further, DEA efficiency scores were regressed on internal and external variables using a Tobit model. Finally, the Malmquist Productivity Index was applied.

Results

On average, the hospitals in the Veneto region operated at more than 95% efficiency. Technical and scale inefficiencies often occurred jointly, with 77% of inefficient hospitals needing a downsizing strategy to gain efficiency. The inputs identified as needing significant reductions were full-time employee (FTE) administrative staff and technicians. The size of the hospital in relation to the size of the population served and the length of patient stay were important factors for the efficiency score. The major cause of decreased efficiency over time was technical change (0.908) rather than efficiency change (0.974).

Conclusions

The study reveals improvements that should be made from both the policy and managerial perspectives. Hospital size is an important feature of inefficiency. On average, the results show that it is advisable for hospitals to reorganize nonmedical staff to enhance efficiency. Further, increasing technology investment could enable higher efficiency levels.

Efficiency Comparison of Public Hospitals under Different Administrative Affiliations in China: A Pilot City Case

This study seeks to measure the efficiency disparity and productivity change of tertiary general public hospitals in Wuhan city, central China from the perspective of administrative affiliations by using panel data from 2013 to 2017. Sample hospitals were divided into three categories, namely provincial hospitals, municipal hospitals, and other levels of hospitals. Data envelopment analysis with bootstrapping technique was used to estimate efficiency scores, and a sensitive analysis was performed by varying the specification of model by considering undesirable outputs to test robustness of estimation, and efficiency evolution analysis was carried out by using the Malmquist index. The results indicated that the average values of provincial hospitals and municipal hospitals have experienced efficiency improvement over the period, especially after the initiation of Pilot Public Hospital Reform, but hospitals under other affiliations showed an opposite trend. Meanwhile, differences of administrative subordination in technical efficiency of public hospitals emerged, and the disparity was likely to grow over time. The higher efficiency of hospitals affiliated with municipality, as compared with those governed by province and under other administrative affiliations, may be attributed to better governance and organization structure.

Data Envelopment Analysis Applications on Primary Health Care Using Exogenous Variables and Health Outcomes

A data envelopment analysis was used to evaluate the efficiency of 18 primary healthcare centres in a health district of the Valencian Community, Spain. Factor analysis was used as a first step in order to identify the most explanatory variables to be incorporated in the models. Included as variable inputs were the ratios of general practitioners, nurses, and costs; as output variables, those included were consultations, emergencies, avoidable hospitalisations, and prescription efficiency; as exogenous variables, those included were the percentage of population over 65 and a multimorbidity index. Confidence intervals were calculated using bootstrapping to correct possible biases. Efficient organisations within the set were identified, although the results depend on the models used and the introduction of exogenous variables. Pharmaceutical expenditure showed the greatest slack and room for improvement in its management. Data envelopment analysis allows an evaluation of efficiency that is focussed on achieving better results and a proper distribution and use of healthcare resources, although it needs the desired goals of the healthcare managers to be clearly identified, as the perspective of the analysis influences the results, as does including variables that measure the achievements and outcomes of the healthcare services.

Nonparametric estimation of a primary care production function in urban Brazil

BACKGROUND

The Brazilian public health system is one of the largest health systems in the world, with a mandate to deliver medical care to more than 200 million Brazilians. The objective of this study is to estimate a production function for primary care in urban Brazil. Our goal is to use flexible estimates to identify heterogeneous returns and complementarities between medical capital and labor.

METHODS

We use a large dataset from 2012 to 2016 (with more than 400 million consultations, 270 thousand physicians, and 11 thousand clinics) to nonparametrically estimate a primary care production function and calculate the elasticity of doctors' visits (output) to two inputs: capital stock (number of clinics) and labor (number of physicians). We benchmark our nonparametric estimates against estimates of a Cobb-Douglas (CD) production function. The CD model was chosen as a baseline because it is arguably the most popular parametric production function model. By comparing our nonparametric results with those from the CD model, our paper shed some light on the limitations of the parametric approach, and on the novelty of nonparametric insights.

RESULTS

The nonparametric results show significantly heterogeneity of returns to both capital and labor, depending on the scale of operation. We find that diseconomies of scale, diminishing returns to scale, and increasing returns to scale are possible, depending on the input range.

CONCLUSIONS

The nonparametric model identifies complementarities between capital and labor, which is essential in designing efficient policy interventions. For example, we find that the response of primary care consultations to labor is steeper when capital level is high. This means that, if the goal is to allocate labor to maximize increases in consultations, adding physicians in cities with a high number of clinics is preferred to allocating physicians to low medical infrastructure municipalities. The results highlight how the CD model hides useful policy information by not accounting for the heterogeneity in the data.

Measuring individual physician clinical productivity in an era of consolidated group practices

BACKGROUND

As physician groups consolidate and value-based payment replaces traditional fee-for-service systems, physician practices have greater need to accurately measure individual physician clinical productivity within team-based systems. We compared methodologies to measure individual physician outpatient clinical productivity after adjustment for shared practice resources.

METHODS

For cardiologists at our hospital between January 2015 and June 2016, we assessed productivity by examining completed patient visits per clinical session per week. Using mixed-effects models, we sequentially accounted for shared practice resources and underlying baseline characteristics. We compared mixed-effects and Generalized Estimating Equations (GEE) models using K-fold cross validation, and compared mixed-effect, GEE, and Data Envelopment Analysis (DEA) models based on ranking of physicians by productivity.

RESULTS

A mixed-effects model adjusting for shared practice resources reduced variation in productivity among providers by 63% compared to an unadjusted model. Mixed-effects productivity rankings correlated strongly with GEE rankings (Spearman 0.99), but outperformed GEE on K-fold cross validation (root mean squared error 2.66 vs 3.02; mean absolute error 1.89 vs 2.20, respectively). Mixed-effects model rankings had moderate correlation with DEA model rankings (Spearman 0.692), though this improved upon exclusion of outliers (Spearman 0.755).

CONCLUSIONS

Mixed-effects modeling accounts for significant variation in productivity secondary to shared practice resources, outperforms GEE in predictive power, and is less vulnerable to outliers than DEA.

IMPLICATIONS

With mixed-effects regression analysis using otherwise easily accessible administrative data, practices can evaluate physician clinical productivity more fairly and make more informed management decisions on physician compensation and resource allocation.

[Analysis of the technical efficiency of hospitals in the Spanish National Health Service]

OBJECTIVE

To analyse the technical efficiency and productivity of general hospitals in the Spanish National Health Service (NHS) (2010-2012) and identify explanatory hospital and regional variables.

METHODS

230 NHS hospitals were analysed by data envelopment analysis for overall, technical and scale efficiency, and Malmquist index. The robustness of the analysis is contrasted with alternative input-output models. A fixed effects multilevel cross-sectional linear model was used to analyse the explanatory efficiency variables.

RESULTS

The average rate of overall technical efficiency (OTE) was 0.736 in 2012; there was considerable variability by region. Malmquist index (2010-2012) is 1.013. A 23% variability in OTE is attributable to the region in question. Statistically significant exogenous variables (residents per 100 physicians, aging index, average annual income per household, essential public service expenditure and public health expenditure per capita) explain 42% of the OTE variability between hospitals and 64% between regions. The number of residents showed a statistically significant relationship. As regards regions, there is a statistically significant direct linear association between OTE and annual income per capita and essential public service expenditure, and an indirect association with the aging index and annual public health expenditure per capita.

DISCUSSION

The significant room for improvement in the efficiency of hospitals is conditioned by region-specific characteristics, specifically aging, wealth and the public expenditure policies of each one.

Care Appropriateness and Health Productivity Evolution: A Non-Parametric Analysis of the Italian Regional Health Systems

BACKGROUND

There has been increasing interest in measuring the productive performance of healthcare services since the mid-1980s.

OBJECTIVE

By applying bootstrapped data envelopment analysis across the 20 Italian Regional Health Systems (RHSs) for the period 2008-2012, we employed a two-stage procedure to investigate the relationship between care appropriateness and productivity evolution in public hospital services.

METHODS

In the first stage, we estimated the Malmquist index and decomposed this overall measure of productivity into efficiency and technological change. In the second stage, the two components of the Malmquist index were regressed on a set of variables measuring per capita health expenditure, care appropriateness, and clinical appropriateness.

RESULTS

Malmquist analysis shows that no gains in productivity in the health industry have been achieved in Italy despite the sequence of reforms that took place during the 1990s, which were devoted to increasing efficiency and reducing costs. Analysis of the efficiency change index clearly indicates that the source of productivity gain relies on a rationalization of the employed inputs in the Italian RHSs. At the same time, the trend of the technological change index reveals that the health systems in the three macro-areas (North, Central, and South) are characterized by technological regress.

CONCLUSION

Overall, our results suggest that productivity increases could be achieved in the Italian health system by reducing the level of inputs, improving care and clinical appropriateness, and by counteracting the 'DRG (diagnosis-related group) creep' phenomenon.

An analysis of hospital efficiency and productivity growth using the Luenberger indicator

We analyze hospital efficiency and productivity growth using an innovative approach which employs the directional distance function and the Luenberger productivity indicator. The primary advantage of our approach is that both input contractions and output expansions are considered. Our model generates a productivity indicator that is decomposed into the usual constituents of productivity growth: technological change and efficiency change. For the sake of comparison, we also use the Malmquist productivity index. The empirical results based on a sample of Portuguese hospitals from 1997 to 2004 show that, on average, those hospitals experienced very weak productivity growth over that period. In addition, the incidence of technological change was remarkably low.

The measurement of efficiency and productivity of health care delivery

The measurement of efficiency and productivity of health service delivery has become a small industry. This is a review of 317 published papers on frontier efficiency measurement. The techniques used are mainly based on non-parametric data envelopment analysis, but there is increasing use of parametric techniques, such as stochastic frontier analysis. Applications to hospitals and other health care organizations and areas are reviewed and summarised, and some meta-type analysis undertaken. Cautious conclusions are that public provision may be potentially more efficient than private, in certain settings. The paper also considers conceptualizations of efficiency, and points to dangers and opportunities in generating such information. Finally, some criteria for assessing the use and usefulness of efficiency studies are established, with a view to helping both researchers and those assessing whether or not to act upon published results.