[Assessing the balance of quality indicator sets of external quality assurance according to SGB V section 136]

Introduction of quality indicators in German hospital capacity planning - Do results show an improvement in quality?

In Germany, the use of quality data to support hospital capacity planning was introduced in 2017. On behalf of the Federal Joint Committee, IQTIG suggested 11 quality indicators and developed a program on how to collect, evaluate and report data for the clinical areas gynaecological surgery, obstetrics and breast surgery. By analysing data from 2015 to 2021, effects of the introduction of the program on indicator results, statistical discrepancies and impact on care quality are examined. Effects on capacity planning are discussed. Since the program started, indicator results improved in all clinical areas, and statistical discrepancies and the number of assessments with insufficient quality decreased due to enhanced adherence to quality standards and data validity. Effects on capacity planning or the allocation of hospitals have not occurred. Thus, a change of the legal basis to allow a better link between quality and hospital planning is recommended. The approach to use quality data on hospital regulation in Germany is evolving. The current hospital reform in Germany also addresses other approaches to quality-based regulation. Already now, there have been clear improvements in specific indicators as well as lessons for quality assurance and its link to capacity planning provided by the program, which are also applicable to other countries.

[The potentials of patient surveys for quality assurance in medical care]

What constitutes "quality" in medical care and the means by which it is controlled, secured, or (re-)established has largely been assigned to the joint self-administration of service providers and health insurers in Germany. The current implementation uses an understanding of quality that does not sufficiently take into account important patient-centered quality dimensions. One of the reasons for this might be that patient surveys are not yet recognized as an equivalent data source for quality assessment in the German healthcare context and are not yet established nationwide. Where patient survey instruments are used, they are often reduced to the survey of so-called patient satisfaction. In the future, survey instruments must be viewed and used in a much more differentiated manner.Indication-specific patient surveys on treatment results (patient-reported outcome measures; PROMs) can provide information on the success of treatment in certain diseases. With the targeted collection of patient experiences (patient-reported experience measures; PREMs), the process quality can also be determined. The routine use of corresponding patient surveys is provided according to §§ 136 ff. and § 299 Social Security Code Book V (SGB V) for the purposes of legally binding external quality assurance. Their use is intended to support the alignment of the health system towards value and benefit for patients. To this extent, there is an urgent need for implementation and considerable further development.

What makes a good quality indicator set? A systematic review of criteria

BACKGROUND

While single indicators measure a specific aspect of quality (e.g. timely support during labour), users of these indicators, such as patients, providers and policy-makers, are typically interested in some broader construct (e.g. quality of maternity care) whose measurement requires a set of indicators. However, guidance on desirable properties of indicator sets is lacking.

OBJECTIVE

Based on the premise that a set of valid indicators does not guarantee a valid set of indicators, the aim of this review is 2-fold: First, we introduce content validity as a desirable property of indicator sets and review the extent to which studies in the peer-reviewed health care quality literature address this criterion. Second, to obtain a complete inventory of criteria, we examine what additional criteria of quality indicator sets were used so far.

METHODS

We searched the databases Web of Science, Medline, Cinahl and PsycInfo from inception to May 2021 and the reference lists of included studies. English- or German-language, peer-reviewed studies concerned with desirable characteristics of quality indicator sets were included. Applying qualitative content analysis, two authors independently coded the articles using a structured coding scheme and discussed conflicting codes until consensus was reached.

RESULTS

Of 366 studies screened, 62 were included in the review. Eighty-five per cent (53/62) of studies addressed at least one of the component criteria of content validity (content coverage, proportional representation and contamination) and 15% (9/62) addressed all component criteria. Studies used various content domains to structure the targeted construct (e.g. quality dimensions, elements of the care pathway and policy priorities), providing a framework to assess content validity. The review revealed four additional substantive criteria for indicator sets: cost of measurement (21% [13/62] of the included studies), prioritization of 'essential' indicators (21% [13/62]), avoidance of redundancy (13% [8/62]) and size of the set (15% [9/62]). Additionally, four procedural criteria were identified: stakeholder involvement (69% [43/62]), using a conceptual framework (44% [27/62]), defining the purpose of measurement (26% [16/62]) and transparency of the development process (8% [5/62]).

CONCLUSION

The concept of content validity and its component criteria help assessing whether conclusions based on a set of indicators are valid conclusions about the targeted construct. To develop a valid indicator set, careful definition of the targeted construct including its (sub-)domains is paramount. Developers of quality indicators should specify the purpose of measurement and consider trade-offs with other criteria for indicator sets whose application may reduce content validity (e.g. costs of measurement) in light thereof.

[Association of Structural Variables with Quality of Care in German Hospitals]

BACKGROUND

The Hospital sector in Germany is undergoing uncoordinated structural changes for financial reasons. At the same time, quality-oriented control measures are used to ensure and improve the quality of treatment. It is unclear whether the right structures, namely those that guarantee positive results, will be maintained.

OBJECTIVE

Investigation of the association between different structural hospital characteristics with quality of care.

METHODS

In a secondary data analysis, the association of quality of care and the structural characteristics of hospital size, type of ownership, region, teaching status and case mix index (CMI) were compared based on hospital quality reports. Quality indices were calculated for each hospital using selected quality indicators of external quality assurance (eQA). First, the associations were examined univariately using the Mann-Whitney U test or Kruskal-Wallis test and then using multiple linear regression analysis, taking into account the interaction effects of the independent variables.

FINDINGS

About 90% of the indices examined showed significant associations between structural hospital characteristics and the quality of care. Positive associations with the quality of care were found in the structural characteristics of hospital size of less than 100 beds, private ownership, no teaching hospital and a low CMI. Negative associations were observed in the structural variables hospital size more than 500 beds, public ownership, teaching-status and higher CMI.

CONCLUSION

According to the results of this study, small hospitals seem to provide the best quality of care. This counterintuitive result suggests that the examined eQA quality indicators used in the index calculation may not be sufficiently risk-adjusted or scientifically evaluated, and their use as quality management tools is not recommended.

Public reporting of hospital quality data: What do referring physicians want to know?

OBJECTIVE

To identify ambulatory care physicians' priorities for hospital quality criteria to support them in counselling patients what hospital to choose.

METHODS

Three hundred non-hospital-based stratified randomly sampled physicians, representing the five main referring specialties in Germany participated in a cross-sectional survey. Physicians rated the importance of 80 hospital quality criteria to be used in their counselling of patients in need of hospital care. Criteria selection was based on a literature analysis and the content of Germany's mandatory hospital quality reports. We calculated the most important criteria and performed an ordinal regression analysis to examine whether the physicians' characteristics 'age', 'sex', 'specialty', 'practice type' and 'region' affected physicians' importance ratings.

RESULTS

To counsel patients in need of a hospital referral, physicians preferred hospital quality criteria that reflect their own and their patients' experiences with a hospital. Additionally, hospitals' expertise and results of treatment were rated highly important. In contrast, hospitals' structural characteristics and compliance with external requirements were rated less important. Physicians' characteristics affected importance ratings only negligibly.

CONCLUSIONS

To support referring physicians' counselling of patients regarding what hospital to choose in order to achieve optimal patient outcomes eventually, hospital report cards must be enriched by information on physicians' and their patients' experiences with hospitals. Hospitals' structural characteristics play a minor role in counselling of patients needing hospital care.