Evaluation of the quality of clinical data collection for a pan-Canadian cohort of children affected by inherited metabolic diseases: lessons learned from the Canadian Inherited Metabolic Diseases Research Network

Data accuracy, consistency and completeness of the national Swiss cystic fibrosis patient registry: Lessons from an ECFSPR data quality project

BACKGROUND

Good data quality is essential when rare disease registries are used as a data source for pharmacovigilance studies. This study investigated data quality of the Swiss cystic fibrosis (CF) registry in the frame of a European Cystic Fibrosis Society Patient Registry (ECFSPR) project aiming to implement measures to increase data reliability for registry-based research.

METHODS

All 20 pediatric and adult Swiss CF centers participated in a data quality audit between 2018 and 2020, and in a re-audit in 2022. Accuracy, consistency and completeness of variables and definitions were evaluated, and missing source data and informed consents (ICs) were assessed.

RESULTS

The first audit included 601 out of 997 Swiss people with CF (60.3 %). Data quality, as defined by data correctness ≥95 %, was high for most of the variables. Inconsistencies of specific variables were observed because of an incorrect application of the variable definition. The proportion of missing data was low with <5 % for almost all variables. A considerable number of missing source data occurred for CFTR variants. Availability of ICs varied largely between centers (10 centers had >5 % of missing documents). After providing feedback to the centers, availability of genetic source data and ICs improved.

CONCLUSIONS

Data audits demonstrated an overall good data quality in the Swiss CF registry. Specific measures such as support of the participating sites, training of data managers and centralized data collection should be implemented in rare disease registries to optimize data quality and provide robust data for registry-based scientific research.

Assessing the quality and value of metabolic chart data for capturing core outcomes for pediatric medium-chain acyl-CoA dehydrogenase (MCAD) deficiency

BACKGROUND

Generating rigorous evidence to inform care for rare diseases requires reliable, sustainable, and longitudinal measurement of priority outcomes. Having developed a core outcome set for pediatric medium-chain acyl-CoA dehydrogenase (MCAD) deficiency, we aimed to assess the feasibility of prospective measurement of these core outcomes during routine metabolic clinic visits.

METHODS

We used existing cohort data abstracted from charts of 124 children diagnosed with MCAD deficiency who participated in a Canadian study which collected data from birth to a maximum of 11 years of age to investigate the frequency of clinic visits and quality of metabolic chart data for selected outcomes. We recorded all opportunities to collect outcomes from the medical chart as a function of visit rate to the metabolic clinic, by treatment centre and by child age. We applied a data quality framework to evaluate data based on completeness, conformance, and plausibility for four core MCAD outcomes: emergency department use, fasting time, metabolic decompensation, and death.

RESULTS

The frequency of metabolic clinic visits decreased with increasing age, from a rate of 2.8 visits per child per year (95% confidence interval, 2.3-3.3) among infants 2 to 6 months, to 1.0 visit per child per year (95% confidence interval, 0.9-1.2) among those ≥ 5 years of age. Rates of emergency department visits followed anticipated trends by child age. Supplemental findings suggested that some emergency visits occur outside of the metabolic care treatment centre but are not captured. Recommended fasting times were updated relatively infrequently in patients' metabolic charts. Episodes of metabolic decompensation were identifiable but required an operational definition based on acute manifestations most commonly recorded in the metabolic chart. Deaths occurred rarely in these patients and quality of mortality data was not evaluated.

CONCLUSIONS

Opportunities to record core outcomes at the metabolic clinic occur at least annually for children with MCAD deficiency. Methods to comprehensively capture emergency care received at outside institutions are needed. To reduce substantial heterogeneous recording of core outcome across treatment centres, improved documentation standards are required for recording of recommended fasting times and a consensus definition for metabolic decompensations needs to be developed and implemented.

The Minimum Data Set for Rare Diseases: Systematic Review

BACKGROUND

The minimum data set (MDS) is a collection of data elements to be grouped using a standard approach to allow the use of data for clinical and research purposes. Health data are typically voluminous, complex, and sometimes too ambiguous to generate indicators that can provide knowledge and information on health. This complexity extends further to the rare disease (RD) domain. MDSs are essential for health surveillance as they help provide services and generate recommended population indicators. There is a bottleneck in international literature that reveals a global problem with data collection, recording, and structuring in RD.

OBJECTIVE

This study aimed to identify and analyze the MDSs used for RD in health care networks worldwide and compare them with World Health Organization (WHO) guidelines.

METHODS

The population, concept, and context methodology proposed by the Joanna Briggs Institute was used to define the research question of this systematic review. A total of 4 databases were reviewed, and all the processes were reported using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. The data elements were analyzed, extracted, and organized into 10 categories according to WHO digital health guidelines. The quality assessment used the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) checklist.

RESULTS

We included 20 studies in our review, 70% (n=14) of which focused on a specific health domain and 30% (n=6) of which referred to RD in general. WHO recommends that health systems and networks use standard terminology to exchange data, information, knowledge, and intelligence in health. However, there was a lack of terminological standardization of the concepts in MDSs. Moreover, the selected studies did not follow the same standard structure for classifying the data from their MDSs. All studies presented MDSs with limitations or restrictions because they covered only a specific RD, or their scope of application was restricted to a specific context or geographic region. Data science methods and clinical experience were used to design, structure, and recommend a fundamental global MDS for RD patient records in health care networks.

CONCLUSIONS

Our study highlights the difficulties in standardizing and categorizing findings from MDSs for RD because of the varying structures used in different studies. The fundamental RD MDS designed in this study comprehensively covers the data needs in the clinical and management sectors. These results can help public policy makers support other aspects of their policies. We highlight the potential of our results to help strategic decisions related to RD.

TRIAL REGISTRATION

PROSPERO CRD42021221593; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=221593.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1016/j.procs.2021.12.034.

A systematic overview of rare disease patient registries: challenges in design, quality management, and maintenance

Patient registries serve to overcome the research limitations inherent in the study of rare diseases, where patient numbers are typically small. Despite the value of real-world data collected through registries, adequate design and maintenance are integral to data quality. We aimed to describe an overview of the challenges in design, quality management, and maintenance of rare disease registries.A systematic search of English articles was conducted in PubMed, Ovid Medline/Embase, and Cochrane Library. Search terms included "rare diseases, patient registries, common data elements, quality, hospital information systems, and datasets". Inclusion criteria were any manuscript type focused upon rare disease patient registries describing design, quality monitoring or maintenance. Biobanks and drug surveillances were excluded.A total of 37 articles, published between 2001 and 2021, met the inclusion criteria. Patient registries covered a wide range of disease areas and covered multiple geographical locations, with a predisposition for Europe. Most articles were methodological reports and described the design and setup of a registry. Most registries recruited clinical patients (92%) with informed consent (81%) and protected the collected data (76%). Whilst the majority (57%) collected patient-reported outcome measures, only few (38%) consulted PAGs during the registry design process. Few reports described details regarding quality management (51%) and maintenance (46%).Rare disease patient registries are valuable for research and evaluation of clinical care, and an increasing number have emerged. However, registries need to be continuously evaluated for data quality and long-term sustainability to remain relevant for future use.

Extended Risk of Mortality in Children with Inborn Errors of Metabolism: A Longitudinal Cohort Study

OBJECTIVES

To determine the long-term risk of mortality among children with inborn errors of metabolism.

STUDY DESIGN

We conducted a retrospective cohort study of 1750 children with inborn errors of metabolism (excluding mitochondrial disorders) and 1 036 668 children without errors of metabolism who were born in Quebec, Canada, between 2006 and 2019. Main outcome measures included all-cause and cause-specific mortality between birth and 14 years of age. We used adjusted survival regression models to estimate HRs and 95% CIs for the association between inborn errors of metabolism and mortality over time.

RESULTS

Mortality rates were greater for children with errors of metabolism than for unaffected children (69.1 vs 3.2 deaths per 10 000 person-years). During 7 702 179 person-years of follow-up, inborn errors of metabolism were associated with 21.2 times the risk of mortality compared with no error of metabolism (95% CI 17.23-26.11). Disorders of mineral metabolism were associated with greater mortality the first 28 days of life (HR 60.62, 95% CI 10.04-365.98), and disorders of sphingolipid metabolism were associated with greater mortality by 1 year (HR 284.73, 95% CI 139.20-582.44) and 14 years (HR 1066.00, 95% CI 298.91-3801.63). Errors of metabolism were disproportionately associated with death from hepatic/digestive (HR 208.21, 95% CI 90.28-480.22), respiratory (HR 116.57, 95% CI 71.06-191.23), and infectious causes (HR 119.83, 95% CI 40.56-354.04).

CONCLUSIONS

Children with errors of metabolism have a considerably elevated risk of mortality before 14 years, including death from hepatic/digestive, respiratory, and infectious causes. Targeting these causes of death may help improve long-term survival.

Observational study of birth outcomes in children with inborn errors of metabolism

BACKGROUND

We examined the birth outcomes of children with inborn errors of metabolism detected at birth or later in life.

METHODS

We carried out a retrospective cohort study of 1733 children with inborn errors of metabolism and 1,033,693 unaffected children born in Canada between 2006 and 2019. Primary outcomes included preterm birth, low birth weight, congenital anomalies, and other neonatal complications. We estimated adjusted risk ratios (RR) and 95% confidence intervals (CI) for the association of inborn errors of metabolism with each outcome.

RESULTS

Children with inborn errors of metabolism had 2.51 times the risk of preterm birth (95% CI 2.27-2.77) and 3.08 times the risk of low birth weight (95% CI 2.77-3.42) compared with unaffected children. Disorders of mineral and lipoprotein metabolism were more strongly associated with adverse birth outcomes. Inborn errors of metabolism were associated with congenital anomalies (RR 2.62; 95% CI 2.36-2.90), particularly abdominal wall defects (RR 8.35; 95% CI 5.18-13.44). Associations were present for errors of metabolism diagnosed both at birth and later in life.

CONCLUSIONS

Children with inborn errors of metabolism, whether detected at birth or later, are at high risk of adverse birth outcomes and congenital anomalies.

IMPACT

Inborn errors of metabolism may affect fetal development, but the association with adverse birth outcomes is not well characterized. This study indicates that children with inborn errors of metabolism are at risk of preterm birth, neonatal jaundice, congenital anomalies, and a range of other adverse birth outcomes. Mothers of children with inborn errors of metabolism are at risk of preeclampsia and cesarean delivery. Adverse birth outcomes may be a first sign of inborn errors of metabolism that merit increased screening.

Families' healthcare experiences for children with inherited metabolic diseases: protocol for a mixed methods cohort study

INTRODUCTION

Children with inherited metabolic diseases (IMDs) often have complex and intensive healthcare needs and their families face challenges in receiving high-quality, family centred health services. Improvement in care requires complex interventions involving multiple components and stakeholders, customised to specific care contexts. This study aims to comprehensively understand the healthcare experiences of children with IMDs and their families across Canada.

METHODS AND ANALYSIS

A two-stage explanatory sequential mixed methods design will be used. Stage 1: quantitative data on healthcare networks and encounter experiences will be collected from 100 parent/guardians through a care map, 2 baseline questionnaires and 17 weekly diaries over 5-7 months. Care networks will be analysed using social network analysis. Relationships between demographic or clinical variables and ratings of healthcare experiences across a range of family centred care dimensions will be analysed using generalised linear regression. Other quantitative data related to family experiences and healthcare experiences will be summarised descriptively. Ongoing analysis of quantitative data and purposive, maximum variation sampling will inform sample selection for stage 2: a subset of stage 1 participants will participate in one-on-one videoconference interviews to elaborate on the quantitative data regarding care networks and healthcare experiences. Interview data will be analysed thematically. Qualitative and quantitative data will be merged during analysis to arrive at an enhanced understanding of care experiences. Quantitative and qualitative data will be combined and presented narratively using a weaving approach (jointly on a theme-by-theme basis) and visually in a side-by-side joint display.

ETHICS AND DISSEMINATION

The study protocol and procedures were approved by the Children's Hospital of Eastern Ontario's Research Ethics Board, the University of Ottawa Research Ethics Board and the research ethics boards of each participating study centre. Findings will be published in peer-reviewed journals and presented at scientific conferences.

Core Outcome Sets for Medium-Chain Acyl-CoA Dehydrogenase Deficiency and Phenylketonuria

BACKGROUND

Evidence to guide treatment of pediatric medium-chain acyl-coenzyme A dehydrogenase (MCAD) deficiency and phenylketonuria (PKU) is fragmented because of large variability in outcome selection and measurement. Our goal was to develop core outcome sets (COSs) for these diseases to facilitate meaningful future evidence generation and enhance the capacity to compare and synthesize findings across studies.

METHODS

Parents and/or caregivers, health professionals, and health policy advisors completed a Delphi survey and participated in a consensus workshop to select core outcomes from candidate lists of outcomes for MCAD deficiency and PKU. Delphi participants rated the importance of outcomes on a nine-point scale (1-3: not important, 4-6: important but not critical, 7-9: critical). Candidate outcomes were progressively narrowed down over 3 survey rounds. At the workshop, participants evaluated the remaining candidate outcomes using an adapted nominal technique, open discussion, and voting. After the workshop, we finalized the COSs and recommended measurement instruments for each outcome.

RESULTS

There were 85, 61, and 53 participants across 3 Delphi rounds, respectively. The candidate core outcome lists were narrowed down to 20 outcomes per disease to be discussed at the consensus workshop. Voting by 18 workshop participants led to COSs composed of 8 and 9 outcomes for MCAD deficiency and PKU, respectively, with measurement recommendations.

CONCLUSIONS

These are the first known pediatric COSs for MCAD deficiency and PKU. Adoption in future studies will help to ensure best use of limited research resources to ultimately improve care for children with these rare diseases.

Independent Registries Are Cost-Effective Tools to Provide Mandatory Postauthorization Surveillance for Orphan Medicinal Products

OBJECTIVES

Orphan medicinal products (OMPs) often receive market authorization under conditions imposed by regulators for ongoing postauthorization surveillance (PAS) to answer questions that remain at the time of market entry. This surveillance may be provided through industry-funded registries (IFRs). Nevertheless, data in these registries may not be of sufficient quality to answer these questions and may not always be accessible for regulatory review. We propose that a mandatory independent registry is an efficient and cost-effective tool for PAS for OMPs.

METHODS

Using data from the Canadian Fabry Disease Initiative, we reviewed costs per unique patient from sites participating in both the independent national registry and IFRs for Fabry disease and compared data completeness from the Canadian Fabry Disease Initiative to that in published documents from IFRs.

RESULTS

The costs of data collection through the independent registry were 17% to 36% (depending on site) lower than costs to collect data in the IFRs, and completeness of data collected through the independent registry was higher than that through the IFRs. Data from the independent registry were reviewed annually to guide indications for publicly funded Fabry disease therapy. Even when enrollment ceased to be a requirement to receive therapy, 77% of patients continued to enroll in the registry, suggesting the structure was acceptable to patients.

CONCLUSIONS

Independent registries are cost-effective and efficient tools and should be mandated by regulatory agencies as the preferred tool for PAS for OMPs. Countries with publicly funded health systems should consider investment in registry infrastructure for OMPs.

Developments in evidence creation for treatments of inborn errors of metabolism

Inborn errors of metabolism (IEM) represent the first group of genetic disorders, amenable to causal therapies. In addition to traditional medical diet and cofactor treatments, new treatment strategies such as enzyme replacement and small molecule therapies, solid organ transplantation, and cell-and gene-based therapies have become available. Inherent to the rare nature of the single conditions, generating high-quality evidence for these treatments in clinical trials and under real-world conditions has been challenging. Guidelines developed with standardized methodologies have contributed to improve the practice of care and long-term clinical outcomes. Adaptive trial designs allow for changes in sample size, group allocation and trial duration as the trial proceeds. n-of-1 studies may be used in small sample sized when participants are clinically heterogeneous. Multicenter observational and registry-based clinical trials are promoted via international research networks. Core outcome and standard data element sets will enhance comparative analysis of clinical trials and observational studies. Patient-centered outcome-research as well as patient-led research initiatives will further accelerate the development of therapies for IEM.