Does current reporting of lung function by the UK cystic fibrosis registry allow a fair comparison of adult centres?

Prospectively predicting Pseudomonas aeruginosa infection/s using routine data from the UK cystic fibrosis register

RATIONALE AND AIMS

Lung health of people with cystic fibrosis (PwCF) can be preserved by daily use of inhaled therapy. Adherence to inhaled therapy, therefore, provides an important process measure to understand the success of care and can be used as a quality indicator. Defining adherence is problematic, however, since the number of prescribed treatments varies considerably between PwCF. The problem is less pronounced among those with Pseudomonas aeruginosa (PA), for whom at least three daily doses of nebulized therapy should be prescribed and who thus constitute a more homogeneous group. The UK CF Registry provides routine data on PA status, but data are only available 12 months after collection. In this study, we aim to prospectively identify contemporary PA status from historic registry data.

METHOD

UK CF Registry data from 2011 to 2015 for PwCF aged ≥16 was used to determine a pragmatic prediction rule for identifying contemporary PA status using historic registry data. Accuracy of three different prediction rules was assessed using the positive predictive value (PPV). The number and proportion of adults predicted to have PA infection were determined overall and per center for the selected prediction rule. Known characteristics linked to PA status were explored to ensure the robustness of the prediction rule.

RESULTS

Having CF Registry defined chronic PA status in the two previous years is the selected definition to predict a patient will have PA infection within the current year (population-level PPV = 96%-97%, centre level PPV = 85%-100%). This approach provides a subset of data between 1852 and 1872 patients overall and a range of 8 to 279 patients per center.

CONCLUSION

Historic registry data can be used to contemporaneously identify a subgroup of patients with chronic PA. Since this patient group has a narrower treatment schedule, this can facilitate a better benchmarking of adherence across centers.

Understanding FEV1 for the purpose of cystic fibrosis registry comparisons: Does bias in annual review FEV1 affect between-centre comparison within the UK? An analysis of registry data

RATIONALE, AIMS, AND OBJECTIVE

We previously demonstrated that annual review %FEV₁ underestimates lung health of adults with CF compared with %FEV₁ captured during periods of clinical stability. This has implications in the comparisons against registries with encounter-based FEV₁ , such as the United States. It is uncertain whether this bias affects between-centre comparison within the United Kingdom. Previous funnel plot analyses have identified variation in annual review %FEV₁ according to centre size; hence, we investigated whether paired differences between annual review and best %FEV₁ also vary according to centre size.

METHODS

This registry analysis included 18 adult CF centres in the United Kingdom with ≥80% completeness for best FEV₁ data in 2014. Mean discrepancy between annual review and best %FEV₁ is a surrogate for the extent by which annual review %FEV₁ underestimates lung health, and was plotted against centre size. A local polynomial regression (LOESS) curve was used to explore the relationship between the two variables. An appropriate model is fitted based on the LOESS curve to determine the strength of relationship between discrepancies in %FEV₁ and centre size.

RESULTS

There is an inverted U-shaped relationship between mean discrepancies in %FEV₁ and centre size. A regression of the paired mean difference in %FEV₁ against centre size showed a significant improvement in the goodness of fit for a quadratic model (R²  = 23.8% for a quadratic model compared with 0.4% for a linear one; P = 0.048 for the quadratic term).

CONCLUSIONS

Annual review %FEV₁ underestimated lung health of adults from small and large centres in the United Kingdom to a greater extent compared with medium-sized centres. A plot of %FEV₁ against centre size (eg, funnel plot comparison) would be affected by systematic bias in annual review %FEV₁ . Therefore, annual review %FEV₁ is an unreliable metric to compare health outcomes of adult CF centres within the United Kingdom.

Identifying exceptional cystic fibrosis care services: combining statistical process control with focus groups

Background

The Cystic Fibrosis (CF) Registry collects clinical data on all patients attending specialist CF centres in the UK. These data have been used to make comparisons between centres on key outcomes such as forced expiratory volume in 1 second (FEV1) using simple rankings, which promote the assumption that those with the highest measures provide ‘better’ care.

Objectives

To explore whether or not using statistical ‘process control’ charts that move away from league tables and adjusting for case mix (age, where appropriate; sex; CF genotype; pancreatic sufficiency; and socioeconomic status) could identify exceptional CF care services in terms of clinically meaningful outcomes. Then, using insight from patients and clinicians on what structures, processes and policies are necessary for delivering good CF care, to explore whether or not care is associated with observed differences in outcomes.

Design

Cross-sectional analyses.

Setting

Specialist CF centres in the UK.

Participants

Patients aged ≥ 6 years attending specialist CF centres and clinicians at these centres.

Main outcome measures

FEV1% predicted.

Data sources

Annual reviews taken from the UK CF Registry (2007–15).

Results

We studied FEV1 in many different ways and in different periods. In our analyses of both adult and paediatric centres, we observed that some centres showed repeated evidence of ‘special-cause variation’, with mean FEV1 being greater than the mean in some cases and lower than the mean in others. Some of these differences were explained by statistical adjustment for different measures of case mix, such as age, socioeconomic status, genotype and pancreatic sufficiency. After adjustment, there was some remaining evidence of special-cause variation for some centres. Our data at these centres suggest that there may be an association with the use of intravenous antibiotics. Workshops and focus groups with clinicians at paediatric and adult centres identified a number of structures, processes and policies that were felt to be associated with good care. From these, questionnaires for CF centre directors were developed and disseminated. However, the response rate was low, limiting the questionnaires’ use. Focus groups with patients to gain their insights into what is necessary for the delivery of good care identified themes similar to those identified by clinicians, and a patient questionnaire was developed based on these insights.

Limitations

Our data analyses suggest that differences in intravenous antibiotic usage may be associated with centre-level outcomes; this needs to be explored further in partnership with the centres. Our survey of centre directors yielded a low response, making it difficult to gain useful knowledge to inform further discussions with sites.

Conclusions

Our findings confirm that the CF Registry can be used to identify differences in clinical outcomes between centres and that case mix might explain some of these differences. As such, adjustment for case mix is essential when trying to understand how and why centres differ from the mean.

Future work

Future work will involve exploring with clinicians how care is delivered so that we can understand associations between care and outcomes. Patients will also be asked for their perspectives on the care they receive.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Using different methods to process forced expiratory volume in one second (FEV 1) data can impact on the interpretation of FEV 1 as an outcome measure to understand the performance of an adult cystic fibrosis centre: A retrospective chart review

Background: Forced expiratory volume in one second (FEV 1) is an important cystic fibrosis (CF) prognostic marker and an established endpoint for CF clinical trials. FEV 1 is also used in observation studies, e.g. to compare different centre's outcomes. We wished to evaluate whether different methods of processing FEV 1 data can impact on a centre's outcome. Methods: This is a single-centre retrospective analysis of routinely collected data from 2013-2016 which included 208 adults with CF. Year-to-year %FEV 1 change was calculated by subtracting best %FEV 1 at Year 1 from Year 2 (i.e. negative values indicate %FEV 1 decline), and compared using Friedman test. Three methods were used to process %FEV 1 data. First, %FEV 1 calculated with Knudson equation was extracted directly from spirometer machines. Second, FEV 1 volume were extracted then converted to %FEV 1 using clean height data and Knudson equation. Third, FEV 1 volume were extracted then converted to %FEV 1 using clean height data and GLI equation. In addition, %FEV 1 decline calculated using GLI equation was adjusted for baseline %FEV 1 to understand the impact of case-mix adjustment. Results: There was a trend of reduction in %FEV 1 decline with all three data processing methods but the magnitude of %FEV 1 decline differed. Median change in %FEV 1 for 2013-2014, 2014-2015 and 2015-2016 was -2.0, -1.0 and 0.0 respectively using %FEV 1 in Knudson equation whereas the median change was -1.1, -0.9 and -0.3 respectively using %FEV 1 in the GLI equation. A statistically significant p-value (0.016) was only obtained when using %FEV 1 in Knudson equation extracted directly from spirometer machines. Conclusions: Although the trend of reduction in %FEV 1 decline was robust, different data processing methods yielded varying results when %FEV 1 decline was compared using a standard related group non-parametric statistical test. Observational studies with %FEV 1 decline as an outcome measure should carefully consider and clearly specify the data processing methods used.

Using different methods to process forced expiratory volume in one second (FEV 1) data can impact on the interpretation of FEV 1 as an outcome measure to understand the performance of an adult cystic fibrosis centre: A retrospective chart review

Background: Forced expiratory volume in one second (FEV 1) is an important cystic fibrosis (CF) prognostic marker and an established endpoint for CF clinical trials. FEV 1 is also used in observation studies, e.g. to compare different centre's outcomes. We wished to evaluate whether different methods of processing FEV 1 data can impact on centre outcome. Methods: This is a single-centre retrospective analysis of routinely collected data from 2013-2016 among 208 adults. Year-to-year %FEV 1 change was calculated by subtracting best %FEV 1 at Year 1 from Year 2 (i.e. negative values indicate fall in %FEV 1), and compared using Friedman test. Three methods were used to process %FEV 1 data. First, %FEV 1 calculated with Knudson equation was extracted directly from spirometer machines. Second, FEV 1 volume were extracted then converted to %FEV 1 using clean height data and Knudson equation. Third, FEV 1 volume were extracted then converted to %FEV 1 using clean height data and GLI equation. In addition, year-to-year variation in %FEV 1 calculated using GLI equation was adjusted for baseline %FEV 1 to understand the impact of case-mix adjustment. Results: Year-to-year fall in %FEV 1 reduced with all three data processing methods but the magnitude of this change differed. Median change in %FEV 1 for 2013-2014, 2014-2015 and 2015-2016 was -2.0, -1.0 and 0.0 respectively using %FEV 1 in Knudson equation whereas the median change was -1.1, -0.9 and -0.3 respectively using %FEV 1 in the GLI equation. A statistically significant p-value (0.016) was only obtained when using %FEV 1 in Knudson equation extracted directly from spirometer machines. Conclusions: Although the trend of reduced year-to-year fall in %FEV 1 was robust, different data processing methods yielded varying results when year-to-year variation in %FEV 1 was compared using a standard related group non-parametric statistical test. Observational studies with year-to-year variation in %FEV 1 as an outcome measure should carefully consider and clearly specify the data processing methods used.

Cystic fibrosis patient registries: A valuable source for clinical research

Cystic Fibrosis (CF) patient registries are valuable data sources for researchers studying the natural history, treatment paradigms, and long-term health outcomes of individuals with CF. In this review, we discuss the role of CF patient registries in facilitating comparative effectiveness research, particularly evaluating therapies and variation in health care delivery. We also discuss the limitations of registry-based research, particularly indication bias, as well as statistical methods that can be used to address these issues.

Do cystic fibrosis centres with the lowest FEV1 still use the least amount of intravenous antibiotics? A registry-based comparison of intravenous antibiotic use among adult CF centres in the UK

BACKGROUND

The Epidemiologic Study of Cystic Fibrosis using 1995-1996 and 2003-2005 data found that CF centres with lowest FEV₁ tended to use fewer intravenous antibiotics. We repeated the analyses using 2013-2014 UK CF registry data to determine if this was still the case.

METHODS

Analysing data for 2013 and 2014 separately, 28 adult CF centres were ranked according to median % age-adjusted FEV₁. The top 7 centres were placed in the 'upper quarter' (best FEV₁), the bottom 7 centres in 'lower quarter' (lowest FEV₁), and the rest in 'middle half'. IV use was stratified according to %FEV₁, then compared between the three groups.

RESULTS

Centres in the 'upper quarter' and 'middle half' used significantly more IV antibiotics compared to centres in the 'lower quarter' (van Elteren test P-value<0.001). Regression analyses showed that people with CF attending centres in the 'upper quarter' or 'middle half' are 30-50% more likely to receive at least one IV course per year compared to people attending centres in the 'lower quarter'.

CONCLUSIONS

CF centres with lowest FEV₁ are still distinguished by lower use of intravenous antibiotics.