Feasibility of using administrative data to compare hospital performance in the EU

Impact of early death recording on international comparison of acute myocardial infarction mortality - administrative hospital data study using the example of Germany and the United States

BACKGROUND

In-hospital mortality from acute myocardial infarction (AMI) is widely used in international comparisons as an indicator of health system performance. Because of the high risk of early death after AMI, international comparisons may be biased by differences in the recording of early death cases in hospital inpatient data. This study examined whether differences in the recording of early deaths affect international comparisons of AMI in-hospital mortality by using the example of Germany and the United States, and explored approaches to address this issue.

METHODS

The German Diagnosis-Related Groups Statistics (DRG Statistics), the U.S. National Inpatient Sample (NIS) and the U.S. Nationwide Emergency Department Sample (NEDS) were analysed from 2014 to 2019. Cases with treatment for AMI were identified in German and U.S. inpatient data. AMI deaths occurring in the emergency department (ED) without inpatient admission were extracted from NEDS data. 30-day in-hospital mortality figures were calculated according to the OECD indicator definition (unlinked data) and modified by including ED deaths, or excluding all same-day cases.

RESULTS

German age-and-sex standardized 30-day in-hospital mortality was substantially higher compared to the U.S. (in 2019, 7.3% vs. 4.6%). The ratio of German vs. U.S. mortality was 1.6. After inclusion of ED deaths in U.S. data this ratio declined to 1.4. Exclusion of same-day cases in German and U.S. data led to a similar ratio.

CONCLUSIONS

While short-duration treatments due to early death are generally recorded in German inpatient data, in U.S. inpatient data those cases are partially missing. Excluding cases with short-duration treatment from the calculation of mortality indicators could be a feasible approach to account for differences in the recording of early deaths, that might be existent in other countries as well.

Handling missing values in the analysis of between-hospital differences in ordinal and dichotomous outcomes: a simulation study

Missing data are frequently encountered in registries that are used to compare performance across hospitals. The most appropriate method for handling missing data when analysing differences in outcomes between hospitals with a generalised linear mixed model is unclear. We aimed to compare methods for handling missing data when comparing hospitals on ordinal and dichotomous outcomes. We performed a simulation study using data from the Multicentre Randomised Controlled Trial of Endovascular Treatment for Acute Ischaemic Stroke in the Netherlands (MR CLEAN) Registry, a prospective cohort study in 17 hospitals performing endovascular therapy for ischaemic stroke in the Netherlands. The investigated methods for handling missing data, both case-mix adjustment variables and outcomes, were complete case analysis, single imputation, multiple imputation, single imputation with deletion of imputed outcomes and multiple imputation with deletion of imputed outcomes. Data were generated as missing completely at random (MCAR), missing at random and missing not at random (MNAR) in three scenarios: (1) 10% missing data in case-mix and outcome; (2) 40% missing data in case-mix and outcome; and (3) 40% missing data in case-mix and outcome with varying degree of missing data among hospitals. Bias and reliability of the methods were compared on the mean squared error (MSE, a summary measure combining bias and reliability) relative to the hospital effect estimates from the complete reference data set. For both the ordinal outcome (ie, the modified Rankin Scale) and a common dichotomised version thereof, all methods of handling missing data were biased, likely due to shrinkage of the random effects. The MSE of all methods was on average lowest under MCAR and with fewer missing data, and highest with more missing data and under MNAR. The 'multiple imputation, then deletion' method had the lowest MSE for both outcomes under all simulated patterns of missing data. Thus, when estimating hospital effects on ordinal and dichotomous outcomes in the presence of missing data, the least biased and most reliable method to handle these missing data is 'multiple imputation, then deletion'.

Development of a set of quality indicators in paediatric and perinatal care in Japan with a modified Delphi method

BACKGROUNDS

Few paediatric and perinatal quality indicators (QIs) have been developed in the Japanese setting, and the quality of care is not assured or validated. The aim of this study was to develop QIs in paediatric and perinatal care in Japan using an administrative database and confirm the feasibility and applicability of the indicators using a single-site practice test.

METHODS

We used a RAND-modified Delphi method that integrates evidence review with expert consensus development. QI candidates were generated from clinical practice guidelines (CPGs) available in English or Japanese and existing QIs in nine selected paediatric or perinatal conditions. Consensus building was based on independent panel ratings. The performance of QIs was retrospectively assessed using data from an administrative database at the National Children's Hospital. Data between April 2018 and March 2019 were used, while data between April 2019 and March 2021 were also used for selected condition, considering the small number of patients. Each QI was calculated as follows: number of times the indicator was met/number of participants×100.

RESULTS

From the literature review conducted between 2010 and 2020, 124 CPGs and 193 existing indicators were identified to generate QI candidates. Through the consensus-building process, 133 QI candidates were assessed and 79 QIs were accepted. The practice test revealed wide variations in the process-level performance of QIs in four categories: patient safety: median 43.9% (IQR 16.7%-85.6%), general paediatrics: median 98.8% (IQR 84.2%-100%), advanced paediatrics: median 94.4% (IQR 46.0%-100%) and advanced obstetrics: median 80.3% (IQR 59.6%-100%).

CONCLUSIONS

We established 79 QIs for paediatric and perinatal care in Japan using an administrative database that can be applied to hospitals nationwide. The practice test confirmed the measurability of the developed QIs. Benchmarking these QIs will be an attractive approach to improving the quality of care.

Excess Cardiovascular Mortality in Latvia: A Novel Approach Based on Patient-Level Data to Estimate the Separate Contributions of Primary Prevention, Accessibility and Quality of Hospital Care

BACKGROUND

Because quantifying the relative contributions of prevention and medical care to the decline in cardiovascular mortality is controversial, at present mortality indicators use a fifty-fifty allocation to fraction avoidable cardiovascular deaths as being partly preventable and partly amenable. The aim of this study was to develop a dynamic approach to estimate the contributions of preventable versus amenable mortality, and to estimate the proportion of amenable mortality due to non-utilisation of care versus suboptimal quality of care.

METHODS

We calculated the contribution of primary prevention, healthcare utilisation and healthcare quality in Latvia by using Emilia-Romagna (ER) (Italy) as the best performer reference standard. In particular, we considered preventable mortality as the number of cardiovascular deaths that could be avoided if Latvia had the same incidence as ER, and then apportioned non-preventable mortality into the two components of non-utilisation versus suboptimal quality of hospital care based on the presence of hospital admissions in the days before death. This calculation was possible thanks to the availability of the unique patient identifier in the administrative databases of Latvia and ER.

RESULTS

41.5 people per 100 000 population died in Latvia in 2016 from cardiovascular causes amenable to healthcare; about half of these (21.4 per 100 000) had had no contact with acute care settings, while the other half (20.1 per 100 000) had accessed the hospital but received suboptimal-quality healthcare. Another estimated 26.8 deaths per 100 000 population were due to lack of primary prevention. Deaths attributable to suboptimal quality or non-utilisation of hospital care constituted 60.7% of all avoidable cardiovascular mortality.

CONCLUSION

If research is undertaken to understand the reasons for differences between territories and their possible relevance to lower performing countries, the dynamic assessment of country-specific contributions to avoidable mortality has considerable potential to stimulate cross-national learning and continuous improvement in population health outcomes.

Stability over time of the "hospital effect" on 30-day unplanned readmissions: Evidence from administrative data

Past studies showed that hospital characteristics affect hospital performance in terms of 30-day unplanned readmissions, proving the existence of a "hospital effect". However, the stability over time of this effect has been under-investigated. This study offers new evidence about the stability over time of the hospital effect on 30-day unplanned readmissions. Using 78,907 heart failure (HF) records collected from 116 hospitals in the Lombardy Region (Northern Italy) over three years (2010-2012), this study analysed hospital performance in terms of 30-day unplanned readmissions. Hospitals with unusually high and low readmission rates were identified through multi-level regression that combined both patient and hospital covariates in each year. Our results confirm that although hospital covariates - and the connected managerial choices - affect the 30-day unplanned readmissions of a specific year, their effect is not stable in the short-term (3 years). This has important implications for pay-for-performance schemes and quality improvement initiatives.

Can benchmarking Australian hospitals for quality identify and improve high and low performers? Disseminating research findings for hospitals

This paper examines the principles of benchmarking in healthcare and how benchmarking can contribute to practice improvement and improved health outcomes for patients. It uses the Deepening our Understanding of Quality in Australia (DUQuA) study published in this Supplement and DUQuA’s predecessor in Europe, the Deepening our Understanding of Quality improvement in Europe (DUQuE) study, as models. Benchmarking is where the performances of institutions or individuals are compared using agreed indicators or standards. The rationale for benchmarking is that institutions will respond positively to being identified as a low outlier or desire to be or stay as a high performer, or both, and patients will be empowered to make choices to seek care at institutions that are high performers. Benchmarking often begins with a conceptual framework that is based on a logic model. Such a framework can drive the selection of indicators to measure performance, rather than their selection being based on what is easy to measure. A Donabedian range of indicators can be chosen, including structure, process and outcomes, created around multiple domains or specialties. Indicators based on continuous variables allow organizations to understand where their performance is within a population, and their interdependencies and associations can be understood. Benchmarking should optimally target providers, in order to drive them towards improvement. The DUQuA and DUQuE studies both incorporated some of these principles into their design, thereby creating a model of how to incorporate robust benchmarking into large-scale health services research.

Potentials, challenges, and limitations of the analysis of administrative data on vascular limb amputations in health care

Although more and more data on lower limb amputations are becoming available by leveraging the widening access to health care administrative databases, the applicability of these data for public health decisions is still limited. Problems can be traced back to methodological issues, how data are generated and to conceptual issues, namely, how data are interpreted in a multidimensional environment. The present review summarised all of the steps from converting the claims data of administrative databases into the analytical data and reviewed the wide array of sources of potential biases in the analysis of such data. The origins of uncertainty of administrative data analysis include uncontrolled confounding due to a lack of clinical data, the left- and right-censored nature of data collection, the non-standardized diagnosis/procedure-based data extraction methods (i.e., numerator/denominator problems) and additional methodological problems associated with temporal and spatial analyses. The existence of these methodological challenges in the administrative data-based analysis should not deter the analysts from using these data as a powerful tool in the armamentarium of clinical research. However, it must be done with caution and a thorough understanding and respect of the methodological limitations. In addition to this requirement, there is a profound need for pursuing further research on methodology and widening the search for other indicators (structural, process or outcome) that allow a deeper insight how the quality of vascular care may be assessed. Effective research using administrative data is based on strong collaboration in three domains, namely expertise in claims data handling and processing, the clinical field, and statistical analysis. The final interpretations of results and the countermeasures on the level of vascular care ought to be grounded on the integrity of research, open discussions and institutionalized mechanisms of science arbitration and honest brokering.

Statewide Longitudinal Progression of the Whole-Patient Measure of Safety in South Carolina

Meaningful improvement in patient safety encompasses a vast number of quality metrics, but a single measure to represent the overall level of safety is challenging to produce. Recently, Perla et al. established the Whole-Person Measure of Safety (WPMoS) to reflect the concept of global risk assessment at the patient level. We evaluated the WPMoS across an entire state to understand the impact of urban/rural setting, academic status, and hospital size on patient safety outcomes. The population included all South Carolina (SC) inpatient discharges from January 1, 2008, through to December 31, 2013, and was evaluated using established definitions of highly undesirable events (HUEs). Over the study period, the proportion of hospital discharges with at least one HUE significantly decreased from 9.7% to 8.8%, including significant reductions in nine of the 14 HUEs. Academic, large, and urban hospitals had a significantly lower proportion of hospital discharges with at least one HUE in 2008, but only urban hospitals remained significantly lower by 2013. Results indicate that there has been a decrease in harm events captured through administrative coded data over this 6-year period. A composite measure, such as the WPMoS, is necessary for hospitals to evaluate their progress toward reducing preventable harm.

Adherence to quality breast cancer survivorship care in four Canadian provinces: a CanIMPACT retrospective cohort study

BACKGROUND

In order to maximize later health, there are established components and guidelines for quality follow-up care of breast cancer survivors. However, adherence to quality follow-up in Canada may not be optimal, and may vary by province. We determined and compared the proportion of patients in each province who received adherent and non-adherent surveillance for recurrence, new cancers and late effects, recommended preventive care, and recommended physician visits for comorbidities.

METHODS

Cohorts consisted of all adult women diagnosed with incident invasive breast cancer between 2007 and 2010/2012 in four Canadian provinces (British Columbia (BC) N = 9338; Manitoba N = 2688; Ontario N = 23,700; Nova Scotia (NS) N = 2735), identified from provincial cancer registries, alive and cancer-free at 30 months post-diagnosis. Their healthcare utilization was determined from one to 5 years post-treatment, using linked administrative databases. Adherence, underuse, and overuse of recommended services were evaluated yearly and compared using descriptive statistics.

RESULTS

In all provinces and follow-up years, the majority of survivors had more than the recommended number of visits to either an oncologist or primary care physician (range 53.8% NS Year 3; 85.8% Ontario Year 4). The proportion of patients with the guideline-recommended number of oncologist visits varied by province (range 29.8% BC Year 5; 74.8% Ontario Year 5), and the proportion of patients with less than the recommended number of specified breast cancer-related visits with either an oncologist or primary care physician ranged from 32.6% (Ontario Year 2) to 84.4% (NS Year 3). Underuse of surveillance breast imaging was identified in NS and BC. The proportion of patients receiving imaging for metastatic disease (not recommended in the guidelines) in BC, Manitoba, and Ontario (not reported in NS) ranged from 20.3% (BC Year 5) to 53.3% (Ontario Year 2). Compliance with recommended physician visits for patients with several chronic conditions was high in Ontario and NS. Preventive care was less than optimal in all provinces with available data.

CONCLUSIONS

Quality of breast cancer survivor follow-up care varies among provinces. Results point to exploration of factors affecting differences, province-specific opportunities for care improvement, and the value of administrative datasets for health system assessment.

Do perceived working conditions and patient safety culture correlate with objective workload and patient outcomes: A cross-sectional explorative study from a German university hospital

BACKGROUND

Workload and demands on hospital staff have been growing over recent years. To ensure patient and occupational safety, hospitals increasingly survey staff about perceived working conditions and safety culture. At the same time, routine data are used to manage resources and performance. This study aims to understand the relation between survey-derived measures of how staff perceive their work-related stress and strain and patient safety on the one hand, and routine data measures of workload and quality of care (patient safety) on the other.

METHODS

We administered a written questionnaire to all physicians and nurses in the inpatient units at a German university hospital. The questionnaire was an amalgam of the Copenhagen Psychosocial Questionnaire (COPSOQ), the Copenhagen Burnout Inventory (CBI) scale to assess patient-related burnout of and portions of the Hospital Survey on Patient Safety Culture (HSPSC). Indicators from administrative data used to assess workload and patient-related work-strain were: amount of overtime worked, work intensity recording of nurses, cost weight, occupancy rate and DRG-related length of stay. Quality of care was assessed using readmission rates and disease-related length of stay. Univariate associations were tested with Pearson correlations.

RESULTS

Response rate were 37% (224) for physicians and 39% (351) for nurses. Physicians' overtime correlated strongly with perceived quantitative demands (.706, 95% CI: 0.634 to 0.766), emotional demands (.765; 95% CI: 0.705 to 0.814), and perceived role conflicts (.655, 95% CI: 0.573 to 0.724). Nurses' work-intensity measures were associated with decreasing physician job satisfaction and with less favorable perceptions of the appropriateness of staffing (-.527, 95% CI:-0.856 to 0.107). Both professional groups showed medium to strong associations between the morbidity measure (cost weight) and role conflicts; between occupancy rates and role clarity (-.482, 95% CI: -0.782 to -0.02) and predictability of work (-.62, 95% CI: -0.848 to -0.199); and between length of stay and internal team functioning (-.555, 95% CI: -0.818 to -0.101). Higher readmission rates were associated with lower perceived patient safety (-.476, 95% CI: -0.779 to 0.006), inadequate staffing (-.702, 95% CI: -0.884 to -0.334), and worse team functioning (-.520, 95% CI: -0.801 to -0.052). Shorter disease-related length of stay was associated with better teamwork within units (-.555, 95% CI: -0.818 to -0.101) and a lower risk of physician burnout (-.588, 95% CI: -0.846 to -0.108).

CONCLUSION

Perceptions of hospital personnel regarding sub-optimal workplace safety and teamwork issues correlated with worse patient outcome measures. Furthermore, objective measures of overtime work as well as objective measures of workload correlated clearly with subjective work-related stress and strain. This suggests that objective workload measures (such as overtime worked) could be used to indirectly monitor job-related psychosocial strain on employees and, thus, improve not only staff wellbeing but also patient outcomes. On the other hand, listening to their personnel could help hospitals to improve patient (and employee) safety.

Comparing regional models of congenital bleeding disorders: preliminary steps in the Italian context

Background

Among these diseases, congenital bleeding disorders (CBD) represent a significant societal burden in terms of high morbidity costs and health outcomes. In Italy, the organization and provision of health care is a regional responsibility and regions must assure equity and quality to all their residents. This is also true for CBD care which is provided by 54 multidisciplinary Hemophilia Treatment Centers (HTCs) distributed among the regions. With the present study, we intend to stimulate a debate on the effect that the decentralization process have in the delivery of services to CBD patients across Italy.

Methods

The available comparable measures of caseloads per center and interregional patient mobility, as proxies of quality and responsiveness of the regional network of HTCs, were first analyzed for the using data from the Italian Hemophilia Centers Association for the year 2012.

Results

Nine thousand one hundred and thirty four Italian residents with CBD received care in at least one of the Italian HTC in 2012. Preliminary findings suggested room for improvement in health care delivery for CBD patients. In 2012, 16 HTCs out of 51 (31.4%) treated a number of patients under the minimum requirement for treatment center accreditation (10 severe patients). Moreover, data on interregional patient mobility highlighted differences in the ability of each region to retain its own residents or to attract residents from other regions.

Conclusions

Preliminary study results showed significant disparities among regions in terms of volumes and mobility of residents with CBDs that cannot be completely explained by the different geographical characteristics. Therefore, the central government should consider taking concrete measures to bridge the gap between regions to assure access to quality care for all individuals with CBD independently from where they live and therefore to move toward a more integrated and homogeneous national network of care centers. Typology of disease, patients’ needs, and cost for outcomes, should have high priority on the political agenda. For CBD patients, even in a federal healthcare system, the national government should have the global responsibility to guaranteeing uniform levels of quality care over the country and overcome local institutions when necessary.

Accuracy and generalizability of using automated methods for identifying adverse events from electronic health record data: a validation study protocol

Background

Adverse events (AEs) in acute care hospitals are frequent and associated with significant morbidity, mortality, and costs. Measuring AEs is necessary for quality improvement and benchmarking purposes, but current detection methods lack in accuracy, efficiency, and generalizability. The growing availability of electronic health records (EHR) and the development of natural language processing techniques for encoding narrative data offer an opportunity to develop potentially better methods. The purpose of this study is to determine the accuracy and generalizability of using automated methods for detecting three high-incidence and high-impact AEs from EHR data: a) hospital-acquired pneumonia, b) ventilator-associated event and, c) central line-associated bloodstream infection.

Methods

This validation study will be conducted among medical, surgical and ICU patients admitted between 2013 and 2016 to the Centre hospitalier universitaire de Sherbrooke (CHUS) and the McGill University Health Centre (MUHC), which has both French and English sites. A random 60% sample of CHUS patients will be used for model development purposes (cohort 1, development set). Using a random sample of these patients, a reference standard assessment of their medical chart will be performed. Multivariate logistic regression and the area under the curve (AUC) will be employed to iteratively develop and optimize three automated AE detection models (i.e., one per AE of interest) using EHR data from the CHUS. These models will then be validated on a random sample of the remaining 40% of CHUS patients (cohort 1, internal validation set) using chart review to assess accuracy. The most accurate models developed and validated at the CHUS will then be applied to EHR data from a random sample of patients admitted to the MUHC French site (cohort 2) and English site (cohort 3)—a critical requirement given the use of narrative data –, and accuracy will be assessed using chart review. Generalizability will be determined by comparing AUCs from cohorts 2 and 3 to those from cohort 1.

Discussion

This study will likely produce more accurate and efficient measures of AEs. These measures could be used to assess the incidence rates of AEs, evaluate the success of preventive interventions, or benchmark performance across hospitals.

Systematic Review of Hospital Readmissions in Stroke Patients

Background. Previous evidence on factors and causes of readmissions associated with high-impact users of stroke is scanty. The aim of the study was to investigate common causes and pattern of short- and long-term readmissions stroke patients by conducting a systematic review of studies using hospital administrative data. Common risk factors associated with the change of readmission rate were also examined. Methods. The literature search was conducted from 15 February to 15 March 2016 using various databases, such as Medline, Embase, and Web of Science. Results. There were a total of 24 studies (n = 2,126,617) included in the review. Only 4 studies assessed causes of readmissions in stroke patients with the follow-up duration from 30 days to 5 years. Common causes of readmissions in majority of the studies were recurrent stroke, infections, and cardiac conditions. Common patient-related risk factors associated with increased readmission rate were age and history of coronary heart disease, heart failure, renal disease, respiratory disease, peripheral arterial disease, and diabetes. Among stroke-related factors, length of stay of index stroke admission was associated with increased readmission rate, followed by bowel incontinence, feeding tube, and urinary catheter. Conclusion. Although risk factors and common causes of readmission were identified, none of the previous studies investigated causes and their sequence of readmissions among high-impact stroke users.

Evaluating patient safety indicators in orthopedic surgery between Italy and the USA

OBJECTIVE

To compare patient safety in major orthopedic procedures between an orthopedic hospital in Italy, and 26 US hospitals of similar size.

DESIGN

Retrospective analysis of administrative data from hospital discharge records in Italy and Florida, USA, 2011-13. Patient Safety Indicators (PSIs) developed by the Agency for Healthcare Quality and Research were used to identify inpatient adverse events (AEs). We examined the factors associated with the development of each different PSI, taking into account known confounders, using logistic regression.

SETTING

One Italian orthopedic hospital and 26 hospitals in Florida with ≥ 1000 major orthopedic procedures per year.

PARTICIPANTS

Patients ≥ 18 years who underwent 1 of the 17 major orthopedic procedures, and with a length of stay (LOS) > 1 day.

INTERVENTION

Patient Safety management between Italy and the USA.

MAIN OUTCOME MEASURE

Patient Safety Indicators.

RESULTS

A total of 14 393 patients in Italy (mean age = 59.8 years) and 131 371 in the USA (mean age = 65.4 years) were included. US patients had lower adjusted odds of developing a PSI compared to Italy for pressure ulcers (odds ratio [OR]: 0.21; 95% confidence interval [CI]: 0.10-0.45), hemorrhage or hematoma (OR: 0.42; CI 0.23-0.78), physiologic and metabolic derangement (OR: 0.08; CI 0.02-0.37). Italian patients had lower odds of pulmonary embolism/deep vein thrombosis (OR: 3.17; CI 2.16-4.67) compared to US patients.

CONCLUSIONS

Important differences in patient safety events were identified across countries using US developed PSIs. Though caution about potential coding differences is wise when comparing PSIs internationally, other differences may explain AEs, and offer opportunities for cross-country learning about safe practices.

Variation between Hospitals with Regard to Diagnostic Practice, Coding Accuracy, and Case-Mix. A Retrospective Validation Study of Administrative Data versus Medical Records for Estimating 30-Day Mortality after Hip Fracture

BACKGROUND

The purpose of this study was to assess the validity of patient administrative data (PAS) for calculating 30-day mortality after hip fracture as a quality indicator, by a retrospective study of medical records.

METHODS

We used PAS data from all Norwegian hospitals (2005-2009), merged with vital status from the National Registry, to calculate 30-day case-mix adjusted mortality for each hospital (n = 51). We used stratified sampling to establish a representative sample of both hospitals and cases. The hospitals were stratified according to high, low and medium mortality of which 4, 3, and 5 hospitals were sampled, respectively. Within hospitals, cases were sampled stratified according to year of admission, age, length of stay, and vital 30-day status (alive/dead). The final study sample included 1043 cases from 11 hospitals. Clinical information was abstracted from the medical records. Diagnostic and clinical information from the medical records and PAS were used to define definite and probable hip fracture. We used logistic regression analysis in order to estimate systematic between-hospital variation in unmeasured confounding. Finally, to study the consequences of unmeasured confounding for identifying mortality outlier hospitals, a sensitivity analysis was performed.

RESULTS

The estimated overall positive predictive value was 95.9% for definite and 99.7% for definite or probable hip fracture, with no statistically significant differences between hospitals. The standard deviation of the additional, systematic hospital bias in mortality estimates was 0.044 on the logistic scale. The effect of unmeasured confounding on outlier detection was small to moderate, noticeable only for large hospital volumes.

CONCLUSIONS

This study showed that PAS data are adequate for identifying cases of hip fracture, and the effect of unmeasured case mix variation was small. In conclusion, PAS data are adequate for calculating 30-day mortality after hip-fracture as a quality indicator in Norway.

Addressing Missing Data in Patient-Reported Outcome Measures (PROMS): Implications for the Use of PROMS for Comparing Provider Performance

Patient-reported outcome measures (PROMs) are now routinely collected in the English National Health Service and used to compare and reward hospital performance within a high-powered pay-for-performance scheme. However, PROMs are prone to missing data. For example, hospitals often fail to administer the pre-operative questionnaire at hospital admission, or patients may refuse to participate or fail to return their post-operative questionnaire. A key concern with missing PROMs is that the individuals with complete information tend to be an unrepresentative sample of patients within each provider and inferences based on the complete cases will be misleading. This study proposes a strategy for addressing missing data in the English PROM survey using multiple imputation techniques and investigates its impact on assessing provider performance. We find that inferences about relative provider performance are sensitive to the assumptions made about the reasons for the missing data.

Accuracy of reported service use in a cohort of people who are chronically homeless and seriously mentally ill

BACKGROUND

Self-reported service use is an integral feature of interventional research with people who are homeless and mentally ill. The objective of this study was to investigate the accuracy of self-reported involvement with major categories of publicly funded services (health, justice, social welfare) within this sub-population.

METHODS

Measures were administered pre-randomization in two randomized controlled trials, using timeline follow back with calendar aids for Health, Social, and Justice Service Use, compared to linked administrative data. Variables examined were: psychiatric admissions (both extended stays of more than 6 months and two or more stays within 5 years); emergency department visits, general hospitalization and jail in the past 6 months; and income assistance in the past 1 month. Participants (n = 433) met criteria for homelessness and a least one mental illness.

RESULTS

Prevalence adjusted and bias adjusted kappa (PABAK) values ranged between moderate and almost perfect for extended psychiatric hospital separations (PABAK: 0.77; 95 % confidence interval (CI) = 0.71, 0.83), multiple psychiatric hospitalizations (PABAK = 0.50, 95 % CI = 0.41, 0.59), emergency department visits (PABAK: 0.77; 95 % CI = 0.71, 0.83), jail (PABAK: 0.74; 95 % CI = 0.68, 0.81), and income assistance (PABAK: 0.82; 95 % CI = 0.76, 0.87). Significant differences in under versus over reporting were also found.

CONCLUSIONS

People who are homeless and mentally ill reliably reported their overall use of health, justice, and income assistance services. Evidence of under-reporting and over-reporting of certain variables has implications for specific research questions. ISRCTN registry: 57595077 (Vancouver at Home Study: Housing First plus Assertive Community Treatment versus congregate housing plus supports versus treatment as usual); and 66721740 (Vancouver at Home study: Housing First plus Intensive Case management versus treatment as usual).

Enhanced capture of healthcare-related harms and injuries in the 11th revision of the International Classification of Diseases (ICD-11)

The World Health Organization (WHO) plans to submit the 11th revision of the International Classification of Diseases (ICD) to the World Health Assembly in 2018. The WHO is working toward a revised classification system that has an enhanced ability to capture health concepts in a manner that reflects current scientific evidence and that is compatible with contemporary information systems. In this paper, we present recommendations made to the WHO by the ICD revision's Quality and Safety Topic Advisory Group (Q&S TAG) for a new conceptual approach to capturing healthcare-related harms and injuries in ICD-coded data. The Q&S TAG has grouped causes of healthcare-related harm and injuries into four categories that relate to the source of the event: (a) medications and substances, (b) procedures, (c) devices and (d) other aspects of care. Under the proposed multiple coding approach, one of these sources of harm must be coded as part of a cluster of three codes to depict, respectively, a healthcare activity as a 'source' of harm, a 'mode or mechanism' of harm and a consequence of the event summarized by these codes (i.e. injury or harm). Use of this framework depends on the implementation of a new and potentially powerful code-clustering mechanism in ICD-11. This new framework for coding healthcare-related harm has great potential to improve the clinical detail of adverse event descriptions, and the overall quality of coded health data.

Opportunities and challenges for quality and safety applications in ICD-11: an international survey of users of coded health data

OBJECTIVE

In 2018, the World Health Organization (WHO) plans to release the 11th revision of the International Classification of Diseases (ICD). The overall goal of the WHO is to produce a new disease classification that has an enhanced ability to capture health concepts in a manner that is compatible with contemporary information systems. Accordingly, our objective was to identify opportunities and challenges in improving the utility of ICD-11 for quality and safety applications.

DESIGN

A survey study of international stakeholders with expertise in either the production or use of coded health data.

SETTING

International producers or users of ICD-coded health care data.

STUDY PARTICIPANTS

We used a snowball sampling approach to identify individuals with relevant expertise in 12 countries, mostly from North America, Europe, and Australasia. An 8-item online survey included questions on demographic characteristics, familiarity with ICD, experience using ICD-coded data on healthcare quality and safety, opinions regarding the use of ICD classification systems for quality and safety measurement, and current limitations and potential future improvements that would permit better coding of quality and safety concepts in ICD-11.

RESULTS

Two-hundred fifty-eight unique individuals accessed the online survey; 246 provided complete responses. The respondents identified specific desires for the ICD revision: more code content for adverse events/complications; a desire for code clustering mechanisms; the need for diagnosis timing information; and the addition of better code definitions to reference materials.

CONCLUSION

These findings reinforce the vision and existing work plan of the WHO's ICD revision process, because each of these desires is being addressed.

Data quality issues impede comparability of hospital treatment delay performance indicators

Aim

To assess the comparability of five performance indicator scores for treatment delay among patients diagnosed with ST-segment elevation myocardial infarction (STEMI) undergoing primary percutaneous coronary intervention in relation to the quality of the underlying data.

Methods

Secondary analyses were performed on data from 1017 patients in seven Dutch hospitals. Data were collected using standardised forms for patients discharged in 2012. Comparability was assessed as the number of occasions the indicator threshold was reached for each hospital.

Results

Hospitals recorded different time points based on different interpretations of the definitions. This led to substantial differences in indicator scores, ranging from 57 to 100 % of the indictor threshold being reached. Some hospitals recorded all the required data elements for calculating the performance indicators but none of the data elements could be retrieved in a fully automated way. Moreover, recording accessibility and completeness of time points varied widely within and between hospitals.

Conclusion

Hospitals use different definitions for treatment delay and vary greatly in the extent to which the necessary data are available, accessible and complete, impeding comparability between hospitals. Indicator developers, users and hospitals providing data should be aware of these issues and aim to improve data quality in order to facilitate comparability of performance indicators.

A review of accessibility of administrative healthcare databases in the Asia-Pacific region

OBJECTIVE

We describe and compare the availability and accessibility of administrative healthcare databases (AHDB) in several Asia-Pacific countries: Australia, Japan, South Korea, Taiwan, Singapore, China, Thailand, and Malaysia.

METHODS

The study included hospital records, reimbursement databases, prescription databases, and data linkages. Databases were first identified through PubMed, Google Scholar, and the ISPOR database register. Database custodians were contacted. Six criteria were used to assess the databases and provided the basis for a tool to categorise databases into seven levels ranging from least accessible (Level 1) to most accessible (Level 7). We also categorised overall data accessibility for each country as high, medium, or low based on accessibility of databases as well as the number of academic articles published using the databases.

RESULTS

Fifty-four administrative databases were identified. Only a limited number of databases allowed access to raw data and were at Level 7 [Medical Data Vision EBM Provider, Japan Medical Data Centre (JMDC) Claims database and Nihon-Chouzai Pharmacy Claims database in Japan, and Medicare, Pharmaceutical Benefits Scheme (PBS), Centre for Health Record Linkage (CHeReL), HealthLinQ, Victorian Data Linkages (VDL), SA-NT DataLink in Australia]. At Levels 3-6 were several databases from Japan [Hamamatsu Medical University Database, Medi-Trend, Nihon University School of Medicine Clinical Data Warehouse (NUSM)], Australia [Western Australia Data Linkage (WADL)], Taiwan [National Health Insurance Research Database (NHIRD)], South Korea [Health Insurance Review and Assessment Service (HIRA)], and Malaysia [United Nations University (UNU)-Casemix]. Countries were categorised as having a high level of data accessibility (Australia, Taiwan, and Japan), medium level of accessibility (South Korea), or a low level of accessibility (Thailand, China, Malaysia, and Singapore). In some countries, data may be available but accessibility was restricted based on requirements by data custodians.

CONCLUSIONS

Compared with previous research, this study describes the landscape of databases in the selected countries with more granularity using an assessment tool developed for this purpose. A high number of databases were identified but most had restricted access, preventing their potential use to support research. We hope that this study helps to improve the understanding of the AHDB landscape, increase data sharing and database research in Asia-Pacific countries.

Novelties in the ongoing 11th revision of the International Classification of Diseases

The development of the 11th revision of the International Classification of Diseases has been underway since 2007. The World Health Organisation, its coordinator, currently schedules the completion to 2017. The novel classification system has a similar high-level structure than the 10th revision of the International Classification of Diseases, while the entities are significantly more detailed and contain more parameters. The development process aims at the harmonisation of the 11th revision definitions with the codes of other information technology systems, potentially allowing the statistical use of far more clinical information than at present. Meanwhile, this complex development process is confronted at several difficulties. The developers are awaiting the comments of all professionals concerning the Beta-version of the system, available on the Internet. Orv. Hetil., 2014, 155(39), 1535–1537.

Deepening our understanding of quality improvement in Europe (DUQuE): overview of a study of hospital quality management in seven countries

Introduction and Objective

This paper provides an overview of the DUQuE (Deepening our Understanding of Quality Improvement in Europe) project, the first study across multiple countries of the European Union (EU) to assess relationships between quality management and patient outcomes at EU level. The paper describes the conceptual framework and methods applied, highlighting the novel features of this study.

Design

DUQuE was designed as a multi-level cross-sectional study with data collection at hospital, pathway, professional and patient level in eight countries.

Setting and Participants

We aimed to collect data for the assessment of hospital-wide constructs from up to 30 randomly selected hospitals in each country, and additional data at pathway and patient level in 12 of these 30.

Main outcome measures

A comprehensive conceptual framework was developed to account for the multiple levels that influence hospital performance and patient outcomes. We assessed hospital-specific constructs (organizational culture and professional involvement), clinical pathway constructs (the organization of care processes for acute myocardial infarction, stroke, hip fracture and deliveries), patient-specific processes and outcomes (clinical effectiveness, patient safety and patient experience) and external constructs that could modify hospital quality (external assessment and perceived external pressure).

Results

Data was gathered from 188 hospitals in 7 participating countries. The overall participation and response rate were between 75% and 100% for the assessed measures.

Conclusions

This is the first study assessing relation between quality management and patient outcomes at EU level. The study involved a large number of respondents and achieved high response rates. This work will serve to develop guidance in how to assess quality management and makes recommendations on the best ways to improve quality in healthcare for hospital stakeholders, payers, researchers, and policy makers throughout the EU.