1
|
Peng Q, Lai X, Liu L, Chen J. A new standardized mortality ratio method for hospital quality evaluation. COMMUN STAT-THEOR M 2021. [DOI: 10.1080/03610926.2021.1955381] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Affiliation(s)
- Qing Peng
- School of Mathematics and V.C. & V.R. Key Lab, Sichuan Normal University, Chengdu, China
| | - Xin Lai
- School of Computer Science and Technology, Xi’an Jiaotong University, Xi’an, China
| | - Liu Liu
- School of Mathematics and V.C. & V.R. Key Lab, Sichuan Normal University, Chengdu, China
| | - Jing Chen
- School of Mathematical Sciences, University of Electronic Science and Technology of China, Chengdu, China
| |
Collapse
|
2
|
Thompson MP, Luo Z, Gardiner J, Burke JF, Nickles A, Reeves MJ. Impact of Missing Stroke Severity Data on the Accuracy of Hospital Ischemic Stroke Mortality Profiling. Circ Cardiovasc Qual Outcomes 2019; 11:e004951. [PMID: 30354572 DOI: 10.1161/circoutcomes.118.004951] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
BACKGROUND The Centers for Medicare and Medicaid Services have proposed 30-day ischemic stroke risk-standardized mortality rates that include adjustment for stroke severity using the National Institute of Health Stroke Scale (NIHSS), which is often undocumented. We used simulations to quantify the effect of missing NIHSS data on the accuracy of hospital-level ischemic stroke risk-standardized mortality rate profiling for 100 hypothetical hospitals with different case volumes. METHODS AND RESULTS We generated simulated data sets of patients with NIHSS scores and other predictors of 30-day mortality based on empirical analysis of data from 7654 patients with ischemic stroke in the Michigan Stroke Registry. We assigned and rank-ordered a true (known) 30-day mortality rate to each hospital in the simulated data sets of N=100 hospitals with either low (100 patients with stroke), medium (300), or high (500) case volumes. We then estimated and rank-ordered 30-day risk-standardized mortality rates for the N=100 hospitals in each simulated data set using hierarchical logistic regression models. In each data set, we systematically varied the rate of missing NIHSS data and whether missing NIHSS data was independent (missing completely at random) or dependent (missing not at random) on the NIHSS score. With no missing NIHSS data, the Spearman correlation between the true hospital performance rank order assigned during data set generation and the estimated 30-day risk-standardized mortality rate rank order was 0.72, 0.88, and 0.91 in low, medium, and high volume hospitals, respectively. However, this fell to as low as 0.50, 0.71, and 0.79 as missing NIHSS data reached 70%. CONCLUSIONS Missing NIHSS data had substantial detrimental effects on the accuracy of profiling of ischemic stroke mortality, especially in lower volume hospitals. Even with complete NIHSS documentation, significant limitations in ischemic stroke mortality profiling remain.
Collapse
Affiliation(s)
- Michael P Thompson
- Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI (M.P.T., Z.L., J.G., M.J.R.).,Department of Cardiac Surgery, University of Michigan Medical School, Ann Arbor, MI (M.P.T.)
| | - Zhehui Luo
- Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI (M.P.T., Z.L., J.G., M.J.R.)
| | - Joseph Gardiner
- Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI (M.P.T., Z.L., J.G., M.J.R.)
| | - James F Burke
- Department of Neurology, University of Michigan Medical School, Ann Arbor, MI (J.F.B.)
| | | | - Mathew J Reeves
- Department of Epidemiology and Biostatistics, Michigan State University, East Lansing, MI (M.P.T., Z.L., J.G., M.J.R.)
| |
Collapse
|
3
|
Mao J, Resnic FS, Girardi LN, Gaudino MF, Sedrakyan A. Challenges in outlier surgeon assessment in the era of public reporting. Heart 2018; 105:721-727. [PMID: 30415207 DOI: 10.1136/heartjnl-2018-313650] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/26/2018] [Revised: 09/26/2018] [Accepted: 10/04/2018] [Indexed: 11/04/2022] Open
Abstract
OBJECTIVE To assess the effect of various evaluation and reporting strategies in determining outlier surgeons, defined by having worse-than-expected mortality after cardiac surgery. METHODS Our study included 33 394 isolated coronary artery bypass graft (CABG) procedures performed by 136 surgeons and 12 172 surgical aortic valve replacement (SAVR) procedures performed by 113 surgeons between 2010 and 2014. Three current methodologies based on the framework of comparing observed and expected (O/E ratio) mortality, with different distributional assumptions, were examined. We further assessed the consistency of outliers detected by these three methods and the impact of using different time windows and aggregating data of CABG and SAVR procedures. RESULTS The three methods were consistent and detected same outliers, with the least conservative method detecting additional outliers (outliers detected for methods 1, 2 and 3: CABG 3 (2.2%), 2 (1.5%) and 8 (5.9%); SAVR 1 (0.9%), 0 (0.0%) and 11 (9.7%)). When numbers of cases recorded were low and events were rare, the two more conservative methods were unlikely to detect outliers unless the O/E ratios were extremely high. However, these two methods were more consistent in detecting the same surgeons as outliers across different time windows for assessment. Of the surgeons who performed both CABG and SAVR, none was an outlier for both procedures when assessed separately. Aggregating data from CABG and SAVR may lead to results to be dominated by the procedure that had a higher caseload. CONCLUSIONS The choices of outlier assessment method, time window for assessment and data aggregation have an intertwined impact on detecting outlier surgeons, often representing different value assumptions toward patient protection and provider penalty. It is desirable to use different methods as sensitivity analyses, avoid aggregating procedures and avoid rare-event endpoints if possible.
Collapse
Affiliation(s)
- Jialin Mao
- Department of Healthcare Policy and Research, Weill Cornell Medical College, New York, USA
| | - Frederic Scott Resnic
- Division of Cardiovascular Medicine, Tufts University School of Medicine, Lahey Hospital and Medical Center, Burlington, Massachusetts, USA
| | - Leonard N Girardi
- Department of Cardiothoracic Surgery, New York-Presbyterian Hospital-Weill Cornell Medical College, New York, USA
| | - Mario Fl Gaudino
- Department of Cardiothoracic Surgery, New York-Presbyterian Hospital-Weill Cornell Medical College, New York, USA
| | - Art Sedrakyan
- Department of Healthcare Policy and Research, Weill Cornell Medical College, New York, USA
| |
Collapse
|
4
|
Damman OC, De Jong A, Hibbard JH, Timmermans DRM. Making comparative performance information more comprehensible: an experimental evaluation of the impact of formats on consumer understanding. BMJ Qual Saf 2015; 25:860-869. [PMID: 26543066 PMCID: PMC5136725 DOI: 10.1136/bmjqs-2015-004120] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2015] [Revised: 09/22/2015] [Accepted: 10/13/2015] [Indexed: 11/07/2022]
Abstract
Study objectives We aimed to investigate how different presentation formats influence comprehension and use of comparative performance information (CPI) among consumers. Methods An experimental between-subjects and within-subjects design with manipulations of CPI presentation formats. We enrolled both consumers with lower socioeconomic status (SES)/cognitive skills and consumers with higher SES/cognitive skills, recruited through an online access panel. Respondents received fictitious CPI and completed questions about interpretation and information use. Between subjects, we tested (1) displaying an overall performance score (yes/no); (2) displaying a small number of quality indicators (5 vs 9); and (3) displaying different types of evaluative symbols (star ratings, coloured dots and word icons vs numbers and bar graphs). Within subjects, we tested the effect of a reduced number of healthcare providers (5 vs 20). Data were analysed using descriptive analysis, analyses of variance and paired-sampled t tests. Results A total of 902 (43%) respondents participated. Displaying an overall performance score and the use of coloured dots and word icons particularly enhanced consumer understanding. Importantly, respondents provided with coloured dots most often correctly selected the top three healthcare providers (84.3%), compared with word icons (76.6% correct), star ratings (70.6% correct), numbers (62.0%) and bars (54.2%) when viewing performance scores of 20 providers. Furthermore, a reduced number of healthcare providers appeared to support consumers, for example, when provided with 20 providers, 69.5% correctly selected the top three, compared with 80.2% with five providers. Discussion Particular presentation formats enhanced consumer understanding of CPI, most importantly the use of overall performance scores, word icons and coloured dots, and a reduced number of providers displayed. Public report efforts should use these formats to maximise impact on consumers.
Collapse
Affiliation(s)
- Olga C Damman
- Department of Public and Occupational Health and the EMGO+ Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands
| | - Anco De Jong
- Dutch National Health Care Institute, Diemen, The Netherlands
| | - Judith H Hibbard
- Department of Planning, Public Policy & Management, University of Oregon, Eugene, OR, USA
| | - Danielle R M Timmermans
- Department of Public and Occupational Health and the EMGO+ Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands
| |
Collapse
|
5
|
Staggs VS, Gajewski BJ. Bayesian and frequentist approaches to assessing reliability and precision of health-care provider quality measures. Stat Methods Med Res 2015; 26:1341-1349. [DOI: 10.1177/0962280215577410] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Our purpose was to compare frequentist, empirical Bayes, and Bayesian hierarchical model approaches to estimating reliability of health care quality measures, including construction of credible intervals to quantify uncertainty in reliability estimates, using data on inpatient fall rates on hospital nursing units. Precision of reliability estimates and Bayesian approaches to estimating reliability are not well studied. We analyzed falls data from 2372 medical units; the rate of unassisted falls per 1000 inpatient days was the measure of interest. The Bayesian methods “shrunk” the observed fall rates and frequentist reliability estimates toward their posterior means. We examined the association between reliability and precision in fall rate rankings by plotting the length of a 90% credible interval for each unit’s percentile rank against the unit’s estimated reliability. Precision of rank estimates tended to increase as reliability increased but was limited even at higher reliability levels: Among units with reliability >0.8, only 5.5% had credible interval length <20; among units with reliability >0.9, only 31.9% had credible interval length <20. Thus, a high reliability estimate may not be sufficient to ensure precise differentiation among providers. Bayesian approaches allow for assessment of this precision.
Collapse
Affiliation(s)
- Vincent S Staggs
- Health Services and Outcomes Research, Children's Mercy Hospitals and Clinics, Kansas City, MO, USA
| | - Byron J Gajewski
- Department of Biostatistics, University of Kansas Medical Center, Kansas City, KS, USA
| |
Collapse
|
6
|
|
7
|
Burnside ES, Lin Y, Munoz del Rio A, Pickhardt PJ, Wu Y, Strigel RM, Elezaby MA, Kerr EA, Miglioretti DL. Addressing the challenge of assessing physician-level screening performance: mammography as an example. PLoS One 2014; 9:e89418. [PMID: 24586763 PMCID: PMC3931752 DOI: 10.1371/journal.pone.0089418] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2013] [Accepted: 01/20/2014] [Indexed: 11/30/2022] Open
Abstract
Background Motivated by the challenges in assessing physician-level cancer screening performance and the negative impact of misclassification, we propose a method (using mammography as an example) that enables confident assertion of adequate or inadequate performance or alternatively recognizes when more data is required. Methods Using established metrics for mammography screening performance–cancer detection rate (CDR) and recall rate (RR)–and observed benchmarks from the Breast Cancer Surveillance Consortium (BCSC), we calculate the minimum volume required to be 95% confident that a physician is performing at or above benchmark thresholds. We graphically display the minimum observed CDR and RR values required to confidently assert adequate performance over a range of interpretive volumes. We use a prospectively collected database of consecutive mammograms from a clinical screening program outside the BCSC to illustrate how this method classifies individual physician performance as volume accrues. Results Our analysis reveals that an annual interpretive volume of 2770 screening mammograms, above the United States’ (US) mandatory (480) and average (1777) annual volumes but below England’s mandatory (5000) annual volume is necessary to confidently assert that a physician performed adequately. In our analyzed US practice, a single year of data uniformly allowed confident assertion of adequate performance in terms of RR but not CDR, which required aggregation of data across more than one year. Conclusion For individual physician quality assessment in cancer screening programs that target low incidence populations, considering imprecision in observed performance metrics due to small numbers of patients with cancer is important.
Collapse
Affiliation(s)
- Elizabeth S. Burnside
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, E3/311 Clinical Science Center, Madison, Wisconsin, United States of America
- Department of Biostatistics and Medical Informatics, University of Wisconsin, Madison Wisconsin, United States of America
- * E-mail:
| | - Yunzhi Lin
- Department of Biostatistics and Medical Informatics, University of Wisconsin, Madison Wisconsin, United States of America
| | - Alejandro Munoz del Rio
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, E3/311 Clinical Science Center, Madison, Wisconsin, United States of America
- Department of Medical Physics, University of Wisconsin, Madison, Wisconsin, United States of America
| | - Perry J. Pickhardt
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, E3/311 Clinical Science Center, Madison, Wisconsin, United States of America
| | - Yirong Wu
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, E3/311 Clinical Science Center, Madison, Wisconsin, United States of America
| | - Roberta M. Strigel
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, E3/311 Clinical Science Center, Madison, Wisconsin, United States of America
| | - Mai A. Elezaby
- Department of Radiology, University of Wisconsin School of Medicine and Public Health, E3/311 Clinical Science Center, Madison, Wisconsin, United States of America
| | - Eve A. Kerr
- Veterans Affairs Center for Clinical Management Research, VA Ann Arbor Healthcare System, Ann Arbor, Michigan, United States of America
- Department of Internal Medicine, University of Michigan Medical School, Ann Arbor, Michigan, United States of America
| | - Diana L. Miglioretti
- Group Health Research Institute, Group Health Cooperative, Seattle, Washington, United States of America
| |
Collapse
|
8
|
Katzan IL, Spertus J, Bettger JP, Bravata DM, Reeves MJ, Smith EE, Bushnell C, Higashida RT, Hinchey JA, Holloway RG, Howard G, King RB, Krumholz HM, Lutz BJ, Yeh RW. Risk adjustment of ischemic stroke outcomes for comparing hospital performance: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 2014; 45:918-44. [PMID: 24457296 DOI: 10.1161/01.str.0000441948.35804.77] [Citation(s) in RCA: 84] [Impact Index Per Article: 8.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
BACKGROUND AND PURPOSE Stroke is the fourth-leading cause of death and a leading cause of long-term major disability in the United States. Measuring outcomes after stroke has important policy implications. The primary goals of this consensus statement are to (1) review statistical considerations when evaluating models that define hospital performance in providing stroke care; (2) discuss the benefits, limitations, and potential unintended consequences of using various outcome measures when evaluating the quality of ischemic stroke care at the hospital level; (3) summarize the evidence on the role of specific clinical and administrative variables, including patient preferences, in risk-adjusted models of ischemic stroke outcomes; (4) provide recommendations on the minimum list of variables that should be included in risk adjustment of ischemic stroke outcomes for comparisons of quality at the hospital level; and (5) provide recommendations for further research. METHODS AND RESULTS This statement gives an overview of statistical considerations for the evaluation of hospital-level outcomes after stroke and provides a systematic review of the literature for the following outcome measures for ischemic stroke at 30 days: functional outcomes, mortality, and readmissions. Data on outcomes after stroke have primarily involved studies conducted at an individual patient level rather than a hospital level. On the basis of the available information, the following factors should be included in all hospital-level risk-adjustment models: age, sex, stroke severity, comorbid conditions, and vascular risk factors. Because stroke severity is the most important prognostic factor for individual patients and appears to be a significant predictor of hospital-level performance for 30-day mortality, inclusion of a stroke severity measure in risk-adjustment models for 30-day outcome measures is recommended. Risk-adjustment models that do not include stroke severity or other recommended variables must provide comparable classification of hospital performance as models that include these variables. Stroke severity and other variables that are included in risk-adjustment models should be standardized across sites, so that their reliability and accuracy are equivalent. There is a pressing need for research in multiple areas to better identify methods and metrics to evaluate outcomes of stroke care. CONCLUSIONS There are a number of important methodological challenges in undertaking risk-adjusted outcome comparisons to assess the quality of stroke care in different hospitals. It is important for stakeholders to recognize these challenges and for there to be a concerted approach to improving the methods for quality assessment and improvement.
Collapse
|
9
|
Issues in Quality Measurement: Target Population, Risk Adjustment, and Ratings. Ann Thorac Surg 2013; 96:718-26. [DOI: 10.1016/j.athoracsur.2013.03.029] [Citation(s) in RCA: 24] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/04/2012] [Revised: 03/13/2013] [Accepted: 03/18/2013] [Indexed: 11/23/2022]
|
10
|
Shahian DM, Iezzoni LI, Meyer GS, Kirle L, Normand SLT. Hospital-wide mortality as a quality metric: conceptual and methodological challenges. Am J Med Qual 2011; 27:112-23. [PMID: 21918014 DOI: 10.1177/1062860611412358] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
Hospital-wide mortality rates are used as a measure of overall hospital quality. However, their parsimony and apparent simplicity belie significant conceptual and methodological concerns. For many diagnoses included in hospital-wide mortality, the association between short-term mortality and quality of care is not well established. Furthermore, compared with condition-specific or procedure-specific mortality, hospital-wide mortality rates pose greater methodological challenges (ie, eligibility and exclusion criteria, risk adjustment, statistical techniques for aggregating across diagnoses, usability). Many of these result from substantial interprovider heterogeneity in diagnosis frequency, sample sizes, and patient severity. Hospital-wide mortality is problematic as a quality metric for public reporting, although hospitals may elect to use such measures for other purposes. Potential alternative approaches include multidimensional composite metrics or mortality measurement limited to selected conditions and procedures for which the link between hospital mortality and quality is clear, legitimate exclusions are uncommon, and sample sizes, end points, and risk adjustment are adequate.
Collapse
Affiliation(s)
- David M Shahian
- Center for Quality and Safety and Department of Surgery, Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA.
| | | | | | | | | |
Collapse
|
11
|
Shahian DM, Edwards FH, Jacobs JP, Prager RL, Normand SLT, Shewan CM, O'Brien SM, Peterson ED, Grover FL. Public Reporting of Cardiac Surgery Performance: Part 2—Implementation. Ann Thorac Surg 2011; 92:S12-23. [DOI: 10.1016/j.athoracsur.2011.06.101] [Citation(s) in RCA: 82] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/03/2010] [Revised: 06/07/2011] [Accepted: 06/09/2011] [Indexed: 01/18/2023]
|
12
|
Austin PC. Bayes rules for optimally using Bayesian hierarchical regression models in provider profiling to identify high-mortality hospitals. BMC Med Res Methodol 2008; 8:30. [PMID: 18474094 PMCID: PMC2415179 DOI: 10.1186/1471-2288-8-30] [Citation(s) in RCA: 15] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2007] [Accepted: 05/12/2008] [Indexed: 11/28/2022] Open
Abstract
Background There is a growing trend towards the production of "hospital report-cards" in which hospitals with higher than acceptable mortality rates are identified. Several commentators have advocated for the use of Bayesian hierarchical models in provider profiling. Several researchers have shown that some degree of misclassification will result when hospital report cards are produced. The impact of misclassifying hospital performance can be quantified using different loss functions. Methods We propose several families of loss functions for hospital report cards and then develop Bayes rules for these families of loss functions. The resultant Bayes rules minimize the expected loss arising from misclassifying hospital performance. We develop Bayes rules for generalized 1-0 loss functions, generalized absolute error loss functions, and for generalized squared error loss functions. We then illustrate the application of these decision rules on a sample of 19,757 patients hospitalized with an acute myocardial infarction at 163 hospitals. Results We found that the number of hospitals classified as having higher than acceptable mortality is affected by the relative penalty assigned to false negatives compared to false positives. However, the choice of loss function family had a lesser impact upon which hospitals were identified as having higher than acceptable mortality. Conclusion The design of hospital report cards can be placed in a decision-theoretic framework. This allows researchers to minimize costs arising from the misclassification of hospitals. The choice of loss function can affect the classification of a small number of hospitals.
Collapse
Affiliation(s)
- Peter C Austin
- Institute for Clinical Evaluative Sciences, Toronto, Ontario.
| |
Collapse
|
13
|
Austin PC, Brunner LJ. Optimal Bayesian probability levels for hospital report cards. HEALTH SERVICES AND OUTCOMES RESEARCH METHODOLOGY 2007. [DOI: 10.1007/s10742-007-0025-4] [Citation(s) in RCA: 14] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
14
|
Fedeli U, Brocco S, Alba N, Rosato R, Spolaore P. The choice between different statistical approaches to risk-adjustment influenced the identification of outliers. J Clin Epidemiol 2007; 60:858-62. [PMID: 17606184 DOI: 10.1016/j.jclinepi.2006.11.017] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2006] [Revised: 10/25/2006] [Accepted: 11/02/2006] [Indexed: 10/23/2022]
Abstract
OBJECTIVE Many statistical approaches have been applied to compare health care providers' performance, but few studies have examined how the outlier status of providers depends on the choice between risk-adjustment techniques. STUDY DESIGN AND SETTING We analyzed the recourse to breast-conserving surgery (BCS) for breast carcinoma across 31 hospitals of the Veneto Region (Italy). The following methods were compared: the ratio of observed to expected events (O/E), regression models with provider effects introduced as dummy variables obtained by standard or weighted effect coding, and multilevel analysis. RESULTS The O/E method classified seven hospitals (one with high and six with low BCS rates) as outliers. The regression model with the weighted parameterization gave similar results, whereas through standard effect coding all odds ratios shifted and different outliers were identified. Multilevel analysis was quite conservative in identifying small hospitals with BCS rates lower than the regional mean. CONCLUSION Whenever feasible, results obtained through different statistical methodologies should be compared. If providers are modeled as dummy variables obtained by effect coding, departures of the model intercept from the regional mean should be checked. The increasing use of multilevel models could entail a lower sensitivity in identifying low-quality outliers if a volume-outcome relationship exists.
Collapse
Affiliation(s)
- Ugo Fedeli
- SER-Epidemiological Department, Veneto Region, Via Ospedale 18, 31033 Castelfranco Veneto (TV), Italy.
| | | | | | | | | |
Collapse
|