1
|
Rippin G, Sanz H, Hoogendoorn WE, Ballarini NM, Largent JA, Demas E, Postmus D, Framke T, Dávila LMA, Quinten C, Pignatti F. Examining the Effect of Missing Data and Unmeasured Confounding on External Comparator Studies: Case Studies and Simulations. Drug Saf 2024:10.1007/s40264-024-01467-9. [PMID: 39102176 DOI: 10.1007/s40264-024-01467-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/04/2024] [Indexed: 08/06/2024]
Abstract
BACKGROUND AND OBJECTIVE Missing data and unmeasured confounding are key challenges for external comparator studies. This work evaluates bias and other performance characteristics depending on missingness and unmeasured confounding by means of two case studies and simulations. METHODS Two case studies were constructed by taking the treatment arms from two randomised controlled trials and an external real-world data source that exhibited substantial missingness. The indications of the randomised controlled trials were multiple myeloma and metastatic hormone-sensitive prostate cancer. Overall survival was taken as the main endpoint. The effects of missing data and unmeasured confounding were assessed for the case studies by reporting estimated external comparator versus randomised controlled trial treatment effects. Based on the two case studies, simulations were performed broadening the settings by varying the underlying hazard ratio, the sample size, the sample size ratio between the experimental arm and the external comparator, the number of missing covariates and the percentage of missingness. Thereby, bias and other performance metrics could be quantified dependent on these factors. RESULTS For the multiple myeloma external comparator study, results were in line with the randomised controlled trial, despite missingness and potential unmeasured confounding, while for the metastatic hormone-sensitive prostate cancer case study missing data led to a low sample size, leading overall to inconclusive results. Furthermore, for the metastatic hormone-sensitive prostate cancer study, missing data in important eligibility criteria led to further limitations. Simulations were successfully applied to gain a quantitative understanding of the effects of missing data and unmeasured confounding. CONCLUSIONS This exploratory study confirmed external comparator strengths and limitations by quantifying the impact of missing data and unmeasured confounding using case studies and simulations. In particular, missing data in key eligibility criteria were seen to limit the ability to derive the external comparator target analysis population accurately, while simulations demonstrated the magnitude of bias to expect for various settings.
Collapse
Affiliation(s)
- Gerd Rippin
- IQVIA, Unterschweinstiege 2-14, 60549, Frankfurt, Germany.
| | - Héctor Sanz
- IQVIA, Unterschweinstiege 2-14, 60549, Frankfurt, Germany
| | | | | | - Joan A Largent
- IQVIA, Unterschweinstiege 2-14, 60549, Frankfurt, Germany
| | - Eleni Demas
- IQVIA, Unterschweinstiege 2-14, 60549, Frankfurt, Germany
| | - Douwe Postmus
- European Medicines Agency, Domenico Scarlattilaan 6, Amsterdam, 1083 HS, The Netherlands
| | - Theodor Framke
- European Medicines Agency, Domenico Scarlattilaan 6, Amsterdam, 1083 HS, The Netherlands
- Institute for Biostatistics, Hannover Medical School, Carl-Neuberg-Str. 1, 30625, Hannover, Germany
| | | | - Chantal Quinten
- European Medicines Agency, Domenico Scarlattilaan 6, Amsterdam, 1083 HS, The Netherlands
| | - Francesco Pignatti
- European Medicines Agency, Domenico Scarlattilaan 6, Amsterdam, 1083 HS, The Netherlands
| |
Collapse
|
2
|
Mahon P, Chatzitheofilou I, Dekker A, Fernández X, Hall G, Helland A, Traverso A, Van Marcke C, Vehreschild J, Ciliberto G, Tonon G. A federated learning system for precision oncology in Europe: DigiONE. Nat Med 2024; 30:334-337. [PMID: 38195748 DOI: 10.1038/s41591-023-02715-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/11/2024]
Affiliation(s)
- Piers Mahon
- Digital Institute for Cancer Outcomes Research E.E.I.G, Brussels, Belgium.
- IQVIA Cancer Research BV, Zaventem, Belgium.
| | | | - Andre Dekker
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre, Maastricht, the Netherlands
| | | | - Geoff Hall
- Leeds Teaching Hospital NHS Trust, Leeds, UK
- DATA-CAN, the Health Data Research UK Hub for Cancer, Leeds, UK
| | - Aslaug Helland
- Division for Cancer Medicine, Oslo University Hospital, Oslo, Norway
| | - Alberto Traverso
- Digital Institute for Cancer Outcomes Research E.E.I.G, Brussels, Belgium
- Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre, Maastricht, the Netherlands
- IRCCS Ospedale San Raffaele, Milan, Italy
| | - Cedric Van Marcke
- Department of Medical Oncology, Institut Roi Albert II, Cliniques Universitaires Saint-Luc, Brussels, Belgium
- Pôle Oncologie, Institut de Recherche Clinique et Expérimentale, UCLouvain, Brussels, Belgium
| | - Janne Vehreschild
- Goethe University Frankfurt, University Hospital, Center for Internal Medicine, Medical Department II, Frankfurt, Germany
| | | | - Giovanni Tonon
- IRCCS Ospedale San Raffaele, Milan, Italy
- Università Vita-Salute San Raffaele, Milan, Italy
| |
Collapse
|
3
|
Mehtälä J, Ali M, Miettinen T, Partanen L, Laapas K, Niemelä PT, Khorlo I, Ström S, Kurki S, Vapalahti J, Abdelgawwad K, Leinonen JV. Utilization of anonymization techniques to create an external control arm for clinical trial data. BMC Med Res Methodol 2023; 23:258. [PMID: 37925415 PMCID: PMC10625188 DOI: 10.1186/s12874-023-02082-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 10/26/2023] [Indexed: 11/06/2023] Open
Abstract
BACKGROUND Subject-level real-world data (RWD) collected during daily healthcare practices are increasingly used in medical research to assess questions that cannot be addressed in the context of a randomized controlled trial (RCT). A novel application of RWD arises from the need to create external control arms (ECAs) for single-arm RCTs. In the analysis of ECAs against RCT data, there is an evident need to manage and analyze RCT data and RWD in the same technical environment. In the Nordic countries, legal requirements may require that the original subject-level data be anonymized, i.e., modified so that the risk to identify any individual is minimal. The aim of this study was to conduct initial exploration on how well pseudonymized and anonymized RWD perform in the creation of an ECA for an RCT. METHODS This was a hybrid observational cohort study using clinical data from the control arm of the completed randomized phase II clinical trial (PACIFIC-AF) and RWD cohort from Finnish healthcare data sources. The initial pseudonymized RWD were anonymized within the (k, ε)-anonymity framework (a model for protecting individuals against identification). Propensity score matching and weighting methods were applied to the anonymized and pseudonymized RWD, to balance potential confounders against the RCT data. Descriptive statistics for the potential confounders and overall survival analyses were conducted prior to and after matching and weighting, using both the pseudonymized and anonymized RWD sets. RESULTS Anonymization affected the baseline characteristics of potential confounders only marginally. The greatest difference was in the prevalence of chronic obstructive pulmonary disease (4.6% vs. 5.4% in the pseudonymized compared to the anonymized data, respectively). Moreover, the overall survival changed in anonymization by only 8% (95% CI 4-22%). Both the pseudonymized and anonymized RWD were able to produce matched ECAs for the RCT data. Anonymization after matching impacted overall survival analysis by 22% (95% CI -21-87%). CONCLUSIONS Anonymization may be a viable technique for cases where flexible data transfer and sharing are required. As anonymization necessarily affects some aspects of the original data, further research and careful consideration of anonymization strategies are needed.
Collapse
Affiliation(s)
| | - Mehreen Ali
- Veil.ai Oy, Helsinki, Finland
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, Finland
| | - Timo Miettinen
- Veil.ai Oy, Helsinki, Finland
- Institute for Molecular Medicine Finland (FIMM), HiLIFE, University of Helsinki, Helsinki, Finland
| | | | | | | | | | | | | | | | | | | |
Collapse
|
4
|
Le Tourneau C, André F, Helland Å, Mileshkin L, Minnaard W, Schiel A, Taskén K, Thomas DM, Veronese ML, Durán-Pacheco G, Leyens L, Rufibach K, Thomas M, Krämer A. Modified study designs to expand treatment options in personalised oncology: a multistakeholder view. Eur J Cancer 2023; 194:113278. [PMID: 37820553 DOI: 10.1016/j.ejca.2023.113278] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 07/31/2023] [Accepted: 08/01/2023] [Indexed: 10/13/2023]
Abstract
Personalised oncology, whereby patients are given therapies based on their molecular tumour profile, is rapidly becoming an essential part of optimal clinical care, at least partly facilitated by recent advances in next-generation sequencing-based technology using liquid- and tissue-based biopsies. Consequently, clinical trials have shifted in approach, from traditional studies evaluating cytotoxic chemotherapy in largely histology-based populations to modified, biomarker-driven studies (e.g. basket, umbrella, platform) of molecularly guided therapies and cancer immunotherapies in selected patient subsets. Such modified study designs may assess, within the same trial structure, multiple cancer types and treatments, and should incorporate a multistakeholder perspective. This is key to generating complementary, fit-for-purpose and timely evidence for molecularly guided therapies that can be used as proof-of-concept to inform further study designs, lead to approval by regulatory authorities and be used as confirmation of clinical benefit for health technology assessment bodies. In general, the future of cancer clinical trials requires a framework for the application of innovative technologies and dynamic design methodologies, in order to efficiently transform scientific discoveries into clinical utility. Next-generation, modified studies that involve the joint efforts of all key stakeholders will offer individualised strategies that ultimately contribute to globalised knowledge and collective learning. In this review, we outline the background and purpose of such modified study designs and detail key aspects from a multistakeholder perspective. We also provide methodological considerations for designing the studies and highlight how insights from already-ongoing studies may address current challenges and opportunities in the era of personalised oncology.
Collapse
Affiliation(s)
- Christophe Le Tourneau
- Department of Drug Development and Innovation (D3i), Institut Curie, INSERM U900 Research Unit, Paris-Saclay University, Paris, France
| | | | - Åslaug Helland
- Division of Cancer Medicine, Oslo University Hospital, Oslo, Norway; Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Linda Mileshkin
- Peter MacCallum Cancer Centre and Sir Peter MacCallum Department of Oncology, The University of Melbourne, Melbourne, Australia
| | | | | | - Kjetil Taskén
- Division of Cancer Medicine, Oslo University Hospital, Oslo, Norway; Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - David M Thomas
- Garvan Institute of Medical Research, Darlinghurst, Australia
| | | | | | - Lada Leyens
- F. Hoffmann-La Roche Ltd, Basel, Switzerland
| | | | | | - Alwin Krämer
- Clinical Cooperation Unit Molecular Hematology/Oncology, German Cancer Research Center (DKFZ) and Department of Internal Medicine V, University of Heidelberg, Heidelberg, Germany.
| |
Collapse
|
5
|
Curtis LH, Sola-Morales O, Heidt J, Saunders-Hastings P, Walsh L, Casso D, Oliveria S, Mercado T, Zusterzeel R, Sobel RE, Jalbert JJ, Mastey V, Harnett J, Quek RGW. Regulatory and HTA Considerations for Development of Real-World Data Derived External Controls. Clin Pharmacol Ther 2023; 114:303-315. [PMID: 37078264 DOI: 10.1002/cpt.2913] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2023] [Accepted: 04/12/2023] [Indexed: 04/21/2023]
Abstract
Regulators and Health Technology Assessment (HTA) bodies are increasingly familiar with, and publishing guidance on, external controls derived from real-world data (RWD) to generate real-world evidence (RWE). We recently conducted a systematic literature review (SLR) evaluating publicly available information on the use of RWD-derived external controls to contextualize outcomes from uncontrolled trials submitted to the European Medicines Agency (EMA), the US Food and Drug Administration (FDA), and/or select HTA bodies. The review identified several key operational and methodological aspects for which more detailed guidance and alignment within and between regulatory agencies and HTA bodies is necessary. This paper builds on the SLR findings by delineating a set of key takeaways for the responsible generation of fit-for-purpose RWE. Practical methodological and operational guidelines for designing, conducting, and reporting RWD-derived external control studies are explored and discussed. These considerations include: (i) early engagement with regulators and HTA bodies during the study planning phase; (ii) consideration of the appropriateness and comparability of external controls across multiple dimensions, including eligibility criteria, temporality, population representation, and clinical evaluation; (iii) ensuring adequate sample sizes, including hypothesis testing considerations; (iv) implementation of a clear and transparent strategy for assessing and addressing data quality, including data missingness across trials and RWD; (v) selection of comparable and meaningful endpoints that are operationalized and analyzed using appropriate analytic methods; and (vi) conduct of sensitivity analyses to assess the robustness of findings in the context of uncertainty and sources of potential bias.
Collapse
Affiliation(s)
- Lesley H Curtis
- Duke Department of Population Health Sciences and Duke Clinical Research Institute, Durham, North Carolina, USA
| | - Oriol Sola-Morales
- Fundació HiTT and Universitat Internacional de Catalunya (UIC), Barcelona, Spain
| | - Julien Heidt
- IQVIA, Regulatory Science and Strategy, Falls Church, Virginia, USA
| | | | - Laura Walsh
- IQVIA, Epidemiology and Drug Safety Practice, Boston, Massachusetts, USA
| | - Deborah Casso
- IQVIA, Epidemiology and Drug Safety Practice, Seattle, Washington, USA
| | - Susan Oliveria
- IQVIA, Epidemiology and Drug Safety Practice, New York, New York, USA
| | - Tiffany Mercado
- IQVIA, Regulatory Science and Strategy, Falls Church, Virginia, USA
| | | | - Rachel E Sobel
- Regeneron Pharmaceuticals Inc., Pharmacoepidemiology, Tarrytown, New York, USA
| | - Jessica J Jalbert
- Regeneron Pharmaceuticals Inc., Health Economics & Outcomes Research, Tarrytown, New York, USA
| | - Vera Mastey
- Regeneron Pharmaceuticals Inc., Health Economics & Outcomes Research, Tarrytown, New York, USA
| | - James Harnett
- Regeneron Pharmaceuticals Inc., Health Economics & Outcomes Research, Tarrytown, New York, USA
| | - Ruben G W Quek
- Regeneron Pharmaceuticals Inc., Health Economics & Outcomes Research, Tarrytown, New York, USA
| |
Collapse
|
6
|
Moon RJ, Reginster JY, Al-Daghri NM, Thiyagarajan JA, Beaudart C, Bruyère O, Burlet N, Chandran M, da Silva MC, Conaghan PG, Dere WH, Diez-Perez A, Hadji P, Halbout P, Hiligsmann M, Kanis JA, McCloskey EV, Ormarsdottir S, Prieto-Alhambra D, Radermecker RP, Rizzoli R, Al-Saleh Y, Silverman SL, Simon LS, Thomasius F, van Staa T, Laslop A, Cooper C, Harvey NC. Real-world evidence: new opportunities for osteoporosis research. Recommendations from a Working Group from the European Society for Clinical and Economic Aspects of Osteoporosis, Osteoarthritis and Musculoskeletal Diseases (ESCEO). Osteoporos Int 2023; 34:1283-1299. [PMID: 37351614 PMCID: PMC10382414 DOI: 10.1007/s00198-023-06827-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 04/28/2023] [Indexed: 06/24/2023]
Abstract
This narrative review summarises the recommendations of a Working Group of the European Society for Clinical and Economic Aspects of Osteoporosis, Osteoarthritis and Musculoskeletal Diseases (ESCEO) for the conduct and reporting of real-world evidence studies with a focus on osteoporosis research. PURPOSE Vast amounts of data are routinely generated at every healthcare contact and activity, and there is increasing recognition that these real-world data can be analysed to generate scientific evidence. Real-world evidence (RWE) is increasingly used to delineate the natural history of disease, assess real-life drug effectiveness, understand adverse events and in health economic analysis. The aim of this work was to understand the benefits and limitations of this type of data and outline approaches to ensure that transparent and high-quality evidence is generated. METHODS A ESCEO Working Group was convened in December 2022 to discuss the applicability of RWE to osteoporosis research and approaches to best practice. RESULTS This narrative review summarises the agreed recommendations for the conduct and reporting of RWE studies with a focus on osteoporosis research. CONCLUSIONS It is imperative that research using real-world data is conducted to the highest standards with close attention to limitations and biases of these data, and with transparency at all stages of study design, data acquisition and curation, analysis and reporting to increase the trustworthiness of RWE study findings.
Collapse
Affiliation(s)
- Rebecca J Moon
- MRC Lifecourse Epidemiology Centre, University of Southampton, Southampton, SO16 6YD, UK
- Paediatric Endocrinology, University Hospital Southampton NHS Foundation Trust, Southampton, UK
| | - Jean-Yves Reginster
- WHO Collaborating Center for Epidemiology of Musculoskeletal Health and Ageing, Liège, Belgium
- Division of Epidemiology, Public Health and Health Economics, University of Liège, Liège, Belgium
| | - Nasser M Al-Daghri
- Biochemistry Department, College of Science, King Saud University, Riyadh, Saudi Arabia
| | | | - Charlotte Beaudart
- WHO Collaborating Center for Epidemiology of Musculoskeletal Health and Ageing, Liège, Belgium
- Division of Epidemiology, Public Health and Health Economics, University of Liège, Liège, Belgium
| | - Olivier Bruyère
- WHO Collaborating Center for Epidemiology of Musculoskeletal Health and Ageing, Liège, Belgium
- Division of Epidemiology, Public Health and Health Economics, University of Liège, Liège, Belgium
| | - Nansa Burlet
- Division of Epidemiology, Public Health and Health Economics, University of Liège, Liège, Belgium
| | - Manju Chandran
- Osteoporosis and Bone Metabolism Unit, Department of Endocrinology, Singapore General Hospital, Singapore, Singapore
| | | | - Philip G Conaghan
- Leeds Institute of Rheumatic and Musculoskeletal Medicine, University of Leeds, Leeds, UK
- NIHR Leeds Musculoskeletal Biomedical Research Unit, Leeds, UK
| | - Willard H Dere
- Department of Internal Medicine, Utah Center for Clinical and Translational Science, University of Utah School of Medicine, Salt Lake City, UT, USA
| | - Adolfo Diez-Perez
- Department of Internal Medicine, Hospital del Mar-IMIM, Autonomous University of Barcelona and CIBERFES, Instituto Carlos III, Barcelona, Spain
| | - Peyman Hadji
- Frankfurt Centre for Bone Health, Frankfurt, Germany
- Philipps University of Marburg, Hesse, Germany
| | | | - Mickaël Hiligsmann
- Department of Health Services Research, CAPHRI Care and Public Health Research Institute, Maastricht University, Maastricht, the Netherlands
| | - John A Kanis
- Mary McKillop Institute for Health Research, Australian Catholic University, Melbourne, Australia
- Centre for Metabolic Bone Diseases, University of Sheffield, Sheffield, UK
| | - Eugene V McCloskey
- MRC Versus Arthritis Centre for Integrated Research in Musculoskeletal Ageing, University of Sheffield, Sheffield, UK
- Mellanby Centre for Musculoskeletal Research, Department of Oncology & Metabolism, University of Sheffield, Sheffield, UK
| | | | - Daniel Prieto-Alhambra
- Pharmaco- and Device Epidemiology, Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford, Oxford, UK
| | - Régis P Radermecker
- Department of Clinical Pharmacology, Diabetes, Nutrition and Metabolic Disorders, CHU Liege, Liege, Belgium
| | - René Rizzoli
- Geneva University Hospitals and Faculty of Medicine, Geneva, Switzerland
| | - Yousef Al-Saleh
- College of Medicine, King Saud Bin Abdulaziz University for Health Sciences, Riyadh, Saudi Arabia
- King Abdullah International Medical Research Center, Riyadh, Saudi Arabia
- Department of Medicine, King Abdulaziz Medical City, Riyadh, Ministry of National Guard-Health Affairs, Riyadh, Saudi Arabia
| | | | | | | | - Tjeerd van Staa
- Centre for Health Informatics, University of Manchester, Manchester, UK
| | - Andrea Laslop
- Scientific Office, Austrian Medicines and Medical Devices Agency, Vienna, Austria
| | - Cyrus Cooper
- MRC Lifecourse Epidemiology Centre, University of Southampton, Southampton, SO16 6YD, UK
- NIHR Southampton Biomedical Research Centre, University of Southampton and University Hospital Southampton NHS Foundation Trust, Southampton, UK
- National Institute for Health Research (NIHR) Musculoskeletal Biomedical Research Centre, University of Oxford, Oxford, UK
| | - Nicholas C Harvey
- MRC Lifecourse Epidemiology Centre, University of Southampton, Southampton, SO16 6YD, UK.
- NIHR Southampton Biomedical Research Centre, University of Southampton and University Hospital Southampton NHS Foundation Trust, Southampton, UK.
| |
Collapse
|
7
|
Moingeon P, Chenel M, Rousseau C, Voisin E, Guedj M. Virtual patients, digital twins and causal disease models: paving the ground for in silico clinical trials. Drug Discov Today 2023; 28:103605. [PMID: 37146963 DOI: 10.1016/j.drudis.2023.103605] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2022] [Revised: 03/22/2023] [Accepted: 04/27/2023] [Indexed: 05/07/2023]
Abstract
Computational models are being explored to simulate in silico the efficacy and safety of drug candidates and medical devices. Disease models that are based on patients' profiling data are being produced to represent interactomes of genes or proteins and to infer causality in the pathophysiology {AuQ: Edit OK?}, which makes it possible to mimic the impact of drugs on relevant targets. Virtual patients designed from medical records as well as digital twins were generated to simulate specific organs and to predict treatment efficacy at the individual patient level {AuQ: Edit OK?}. As the acceptance of digital evidence by regulators grows, predictive artificial intelligence (AI)-based models will support the design of confirmatory trials in humans and will accelerate the development of efficient drugs and medical devices.
Collapse
|