Detecting and Visualizing Outliers in Provider Profiling Using Funnel Plots and Mixed Effects Models-An Example from Prescription Claims Data

Contribution of medico-administrative databases to health vigilance: example of post-oocyte retrieval infections

STUDY QUESTION

Can we monitor post-oocyte retrieval infections in the French national health data system to complement the French ART vigilance system?

SUMMARY ANSWER

Medico-administrative databases provide a more comprehensive view of post-oocyte retrieval infections and can be used to detect abnormal increases in frequency and outlier ART centers as a complementary tool to the ART vigilance system.

WHAT IS KNOWN ALREADY

The various studies of ART complications are reassuring, showing relatively low overall complication rates. Nonetheless, the European Union has set up a vigilance system to monitor these complications. However, this system is not an exhaustive source of information and does not provide a complete overview of post-ART complications.

STUDY DESIGN, SIZE, DURATION

The study population was identified from the comprehensive French national hospital discharge database. It included women under 46 years of age undergoing an oocyte retrieval in 2019, classified into three population subgroups according to the indication of oocyte retrieval: infertility (IF) , fertility preservation (FP), and oocyte donation (OD) . The study population included 52 098 women who had undergone 65 948 oocyte retrievals in 2019.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Hospital stays and delivery of antibiotics within 31 days after oocyte retrieval were analyzed. Women and infections were characterized according to various characteristics (age, comorbidities, indication of oocyte retrieval, type of hospital stay, length of hospital stay, type of antibiotherapy, etc.). Multivariate analysis was performed to determine the relation between the occurrence of infection and women's characteristics, and results are expressed as odds ratios (ORs) and 95% CI. A funnel plot and a box plot were used to compare the infection rate per center with the national average and to detect outliers.

MAIN RESULTS AND THE ROLE OF CHANCE

Infections in the month following the oocyte retrieval represented 6.9% of the procedures in 2019 (n = 4522). Of these infections, 112 were hospitalized (0.2% of oocyte retrievals), and 4410 were non-hospitalized (6.7% of oocyte retrievals). The hospitalized infections were essentially gynecological infections (40.9%) and urinary tract infections (23.5%). In 87.9% of non-hospitalized infections, a single antibiotic therapy was prescribed. Mixed-effect model analysis showed that the risk of infection was significantly higher in women under 30 years of age, in the FP population, in supplementary universal health coverage (CMU-C) beneficiaries, and women with endometriosis. Funnel plot and box plot analysis showed that three ART centers have an infection rate significantly higher than the national average. In the three centers that stand out from all the others, the objective is to return to these centers to understand the possible reasons for this observed rate and to implement corrective measures.

LIMITATIONS, REASONS FOR CAUTION

Despite all its advantages, the French national health data system presents some limitations, such as the risk of inappropriate coding. Another limitation of this study is that we cannot confirm an attributable relation between the infection and the ART procedure, even if the delay of 31 days after oocyte retrieval is consistent with the occurrence of a post-retrieval complication. In addition, antibiotics may be prescribed as a 'precautionary' measure in certain situations (women with a susceptibility to infection, complicated procedures), or as antibiotic prophylaxis for embryo transfer.

WIDER IMPLICATIONS OF THE FINDINGS

Despite the limits in identifying post-ART infections in medico-administrative databases, this approach is a promising way to complement the ART vigilance reporting system. This concept developed for infections will also be generalized to other complications with regular feedback to professionals.

STUDY FUNDING/COMPETING INTEREST(S)

No specific funding was sought for the study. The study was supported by the Agence de la biomédecine, France. The authors declare that they have no conflict of interest.

TRIAL REGISTRATION NUMBER

N/A.

Identifying Patterns of Clinical Interest in Clinicians' Treatment Preferences: Hypothesis-free Data Science Approach to Prioritizing Prescribing Outliers for Clinical Review

BACKGROUND

Data analysis is used to identify signals suggestive of variation in treatment choice or clinical outcome. Analyses to date have generally focused on a hypothesis-driven approach.

OBJECTIVE

This study aimed to develop a hypothesis-free approach to identify unusual prescribing behavior in primary care data. We aimed to apply this methodology to a national data set in a cross-sectional study to identify chemicals with significant variation in use across Clinical Commissioning Groups (CCGs) for further clinical review, thereby demonstrating proof of concept for prioritization approaches.

METHODS

Here we report a new data-driven approach to identify unusual prescribing behaviour in primary care data. This approach first applies a set of filtering steps to identify chemicals with prescribing rate distributions likely to contain outliers, then applies two ranking approaches to identify the most extreme outliers amongst those candidates. This methodology has been applied to three months of national prescribing data (June-August 2017).

RESULTS

Our methodology provides rankings for all chemicals by administrative region. We provide illustrative results for 2 antipsychotic drugs of particular clinical interest: promazine hydrochloride and pericyazine, which rank highly by outlier metrics. Specifically, our method identifies that, while promazine hydrochloride and pericyazine are barely used by most clinicians (with national prescribing rates of 11.1 and 6.2 per 1000 antipsychotic prescriptions, respectively), they make up a substantial proportion of antipsychotic prescribing in 2 small geographic regions in England during the study period (with maximum regional prescribing rates of 298.7 and 241.1 per 1000 antipsychotic prescriptions, respectively).

CONCLUSIONS

Our hypothesis-free approach is able to identify candidates for audit and review in clinical practice. To illustrate this, we provide 2 examples of 2 very unusual antipsychotics used disproportionately in 2 small geographic areas of England.