CODE-EHR best-practice framework for the use of structured electronic health-care records in clinical research

Creating a next-generation phenotype library: the health data research UK Phenotype Library

Objective

To enable reproducible research at scale by creating a platform that enables health data users to find, access, curate, and re-use electronic health record phenotyping algorithms.

Materials and Methods

We undertook a structured approach to identifying requirements for a phenotype algorithm platform by engaging with key stakeholders. User experience analysis was used to inform the design, which we implemented as a web application featuring a novel metadata standard for defining phenotyping algorithms, access via Application Programming Interface (API), support for computable data flows, and version control. The application has creation and editing functionality, enabling researchers to submit phenotypes directly.

Results

We created and launched the Phenotype Library in October 2021. The platform currently hosts 1049 phenotype definitions defined against 40 health data sources and >200K terms across 16 medical ontologies. We present several case studies demonstrating its utility for supporting and enabling research: the library hosts curated phenotype collections for the BREATHE respiratory health research hub and the Adolescent Mental Health Data Platform, and it is supporting the development of an informatics tool to generate clinical evidence for clinical guideline development groups.

Discussion

This platform makes an impact by being open to all health data users and accepting all appropriate content, as well as implementing key features that have not been widely available, including managing structured metadata, access via an API, and support for computable phenotypes.

Conclusions

We have created the first openly available, programmatically accessible resource enabling the global health research community to store and manage phenotyping algorithms. Removing barriers to describing, sharing, and computing phenotypes will help unleash the potential benefit of health data for patients and the public.

Thromboembolic events and vascular dementia in patients with atrial fibrillation and low apparent stroke risk

The prevention of thromboembolism in atrial fibrillation (AF) is typically restricted to patients with specific risk factors and ignores outcomes such as vascular dementia. This population-based cohort study used electronic healthcare records from 5,199,994 primary care patients (UK; 2005-2020). A total of 290,525 (5.6%) had a diagnosis of AF and were aged 40-75 years, of which 36,340 had no history of stroke, a low perceived risk of stroke based on clinical risk factors and no oral anticoagulant prescription. Matching was performed for age, sex and region to 117,298 controls without AF. During 5 years median follow-up (831,005 person-years), incident stroke occurred in 3.8% with AF versus 1.5% control (adjusted hazard ratio (HR) 2.06, 95% confidence interval (CI) 1.91-2.21; P < 0.001), arterial thromboembolism 0.3% versus 0.1% (HR 2.39, 95% CI 1.83-3.11; P < 0.001), and all-cause mortality 8.9% versus 5.0% (HR 1.44, 95% CI 1.38-1.50; P < 0.001). AF was associated with all-cause dementia (HR 1.17, 95% CI 1.04-1.32; P = 0.010), driven by vascular dementia (HR 1.68, 95% CI 1.33-2.12; P < 0.001) rather than Alzheimer's disease (HR 0.85, 95% CI 0.70-1.03; P = 0.09). Death and thromboembolic outcomes, including vascular dementia, are substantially increased in patients with AF despite a lack of conventional stroke risk factors.

Clinical coding of long COVID in primary care 2020-2023 in a cohort of 19 million adults: an OpenSAFELY analysis

Background

Long COVID is the patient-coined term for the persistent symptoms of COVID-19 illness for weeks, months or years following the acute infection. There is a large burden of long COVID globally from self-reported data, but the epidemiology, causes and treatments remain poorly understood. Primary care is used to help identify and treat patients with long COVID and therefore Electronic Health Records (EHRs) of past COVID-19 patients could be used to help fill these knowledge gaps. We aimed to describe the incidence and differences in demographic and clinical characteristics in recorded long COVID in primary care records in England.

Methods

With the approval of NHS England we used routine clinical data from over 19 million adults in England linked to SARS-COV-2 test result, hospitalisation and vaccination data to describe trends in the recording of 16 clinical codes related to long COVID between November 2020 and January 2023. Using OpenSAFELY, we calculated rates per 100,000 person-years and plotted how these changed over time. We compared crude and adjusted (for age, sex, 9 NHS regions of England, and the dominant variant circulating) rates of recorded long COVID in patient records between different key demographic and vaccination characteristics using negative binomial models.

Findings

We identified a total of 55,465 people recorded to have long COVID over the study period, which included 20,025 diagnoses codes and 35,440 codes for further assessment. The incidence of new long COVID records increased steadily over 2021, and declined over 2022. The overall rate per 100,000 person-years was 177.5 cases in women (95% CI: 175.5-179) and 100.5 in men (99.5-102). The majority of those with a long COVID record did not have a recorded positive SARS-COV-2 test 12 or more weeks before the long COVID record.

Interpretation

In this descriptive study, EHR recorded long COVID was very low between 2020 and 2023, and incident records of long COVID declined over 2022. Using EHR diagnostic or referral codes unfortunately has major limitations in identifying and ascertaining true cases and timing of long COVID.

Funding

This research was supported by the National Institute for Health and Care Research (NIHR) (OpenPROMPT: COV-LT2-0073).

Real-world evaluation of an algorithmic machine-learning-guided testing approach in stable chest pain: a multinational, multicohort study

Aims

An algorithmic strategy for anatomical vs. functional testing in suspected coronary artery disease (CAD) (Anatomical vs. Stress teSting decIsion Support Tool; ASSIST) is associated with better outcomes than random selection. However, in the real world, this decision is rarely random. We explored the agreement between a provider-driven vs. simulated algorithmic approach to cardiac testing and its association with outcomes across multinational cohorts.

Methods and results

In two cohorts of functional vs. anatomical testing in a US hospital health system [Yale; 2013-2023; n = 130 196 (97.0%) vs. n = 4020 (3.0%), respectively], and the UK Biobank [n = 3320 (85.1%) vs. n = 581 (14.9%), respectively], we examined outcomes stratified by agreement between the real-world and ASSIST-recommended strategies. Younger age, female sex, Black race, and diabetes history were independently associated with lower odds of ASSIST-aligned testing. Over a median of 4.9 (interquartile range [IQR]: 2.4-7.1) and 5.4 (IQR: 2.6-8.8) years, referral to the ASSIST-recommended strategy was associated with a lower risk of acute myocardial infarction or death (hazard ratioadjusted: 0.81, 95% confidence interval [CI] 0.77-0.85, P < 0.001 and 0.74 [95% CI 0.60-0.90], P = 0.003, respectively), an effect that remained significant across years, test types, and risk profiles. In post hoc analyses of anatomical-first testing in the Prospective Multicentre Imaging Study for Evaluation of Chest Pain (PROMISE) trial, alignment with ASSIST was independently associated with a 17% and 30% higher risk of detecting CAD in any vessel or the left main artery/proximal left anterior descending coronary artery, respectively.

Conclusion

In cohorts where historical practices largely favour functional testing, alignment with an algorithmic approach to cardiac testing defined by ASSIST was associated with a lower risk of adverse outcomes. This highlights the potential utility of a data-driven approach in the diagnostic management of CAD.

Using routinely collected clinical data for circadian medicine: A review of opportunities and challenges

A wealth of data is available from electronic health records (EHR) that are collected as part of routine clinical care in hospitals worldwide. These rich, longitudinal data offer an attractive object of study for the field of circadian medicine, which aims to translate knowledge of circadian rhythms to improve patient health. This narrative review aims to discuss opportunities for EHR in studies of circadian medicine, highlight the methodological challenges, and provide recommendations for using these data to advance the field. In the existing literature, we find that data collected in real-world clinical settings have the potential to shed light on key questions in circadian medicine, including how 24-hour rhythms in clinical features are associated with-or even predictive of-health outcomes, whether the effect of medication or other clinical activities depend on time of day, and how circadian rhythms in physiology may influence clinical reference ranges or sampling protocols. However, optimal use of EHR to advance circadian medicine requires careful consideration of the limitations and sources of bias that are inherent to these data sources. In particular, time of day influences almost every interaction between a patient and the healthcare system, creating operational 24-hour patterns in the data that have little or nothing to do with biology. Addressing these challenges could help to expand the evidence base for the use of EHR in the field of circadian medicine.

Review of codelists used to define hypertension in electronic health records and development of a codelist for research

BACKGROUND AND AIMS

Hypertension is a leading risk factor for cardiovascular disease. Electronic health records (EHRs) are routinely collected throughout a person's care, recording all aspects of health status, including current and past conditions, prescriptions and test results. EHRs can be used for epidemiological research. However, there are nuances in the way conditions are recorded using clinical coding; it is important to understand the methods which have been applied to define exposures, covariates and outcomes to enable interpretation of study findings. This study aimed to identify codelists used to define hypertension in studies that use EHRs and generate recommended codelists to support reproducibility and consistency.

ELIGIBILITY CRITERIA

Studies included populations with hypertension defined within an EHR between January 2010 and August 2023 and were systematically identified using MEDLINE and Embase. A summary of the most frequently used sources and codes is described. Due to an absence of Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) codelists in the literature, a recommended SNOMED CT codelist was developed to aid consistency and standardisation of hypertension research using EHRs.

FINDINGS

375 manuscripts met the study criteria and were eligible for inclusion, and 112 (29.9%) reported codelists. The International Classification of Diseases (ICD) was the most frequently used clinical terminology, 59 manuscripts provided ICD 9 codelists (53%) and 58 included ICD 10 codelists (52%). Informed by commonly used ICD and Read codes, usage recommendations were made. We derived SNOMED CT codelists informed by National Institute for Health and Care Excellence guidelines for hypertension management. It is recommended that these codelists be used to identify hypertension in EHRs using SNOMED CT codes.

CONCLUSIONS

Less than one-third of hypertension studies using EHRs included their codelists. Transparent methodology for codelist creation is essential for replication and will aid interpretation of study findings. We created SNOMED CT codelists to support and standardise hypertension definitions in EHR studies.

Quality of Reporting Electronic Health Record Data in Glaucoma: A Systematic Literature Review

TOPIC

Assessing reporting standards in glaucoma studies utilizing electronic health records (EHR).

CLINICAL RELEVANCE

Glaucoma's significance, underscored by its status as a leading cause of irreversible blindness worldwide, necessitates reliable research findings. This study evaluates adherence to the CODE-EHR best-practice framework in glaucoma studies using EHR, aiming to improve clinical care and patient outcomes.

METHODS

A systematic review, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (PROSPERO CRD42023430025), identified relevant studies (January 2022-May 2023) in MEDLINE, EMBASE, CINAHL, and Web of Science. Eligible studies, using EHR data from clinical institutions for glaucoma research, were assessed for study design, participant characteristics, EHR data, and sources. Quality appraisal used the CODE-EHR best-practice framework, focusing on data construction, linkage, fitness for purpose, disease and outcome definitions, analysis, and ethics and governance.

RESULTS

Of 31 identified studies, predominant EHR sources were hospitals and clinical warehouses. Commonly reported elements included age, gender, glaucoma diagnosis, and intraocular pressure. Only 16% fully adhered to CODE-EHR best-practice framework's minimum standards, with none meeting preferred standards. While statistical analysis and ethical considerations were relatively well-addressed, areas such as EHR data management and study design showed room for improvement. Patient and public involvement, and acknowledgment of data linkage processes, data security, and storage reporting were often missed.

CONCLUSION

Adherence to CODE-EHR best-practice framework's standards in EHR-based studies of glaucoma can be improved upon. Standardized reporting of EHR data are essential to ensure the reliability of research, facilitating its translation into clinical practice and improving healthcare decision-making for better patient outcomes.

FINANCIAL DISCLOSURE(S)

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Use of Artificial Intelligence in Improving Outcomes in Heart Disease: A Scientific Statement From the American Heart Association

A major focus of academia, industry, and global governmental agencies is to develop and apply artificial intelligence and other advanced analytical tools to transform health care delivery. The American Heart Association supports the creation of tools and services that would further the science and practice of precision medicine by enabling more precise approaches to cardiovascular and stroke research, prevention, and care of individuals and populations. Nevertheless, several challenges exist, and few artificial intelligence tools have been shown to improve cardiovascular and stroke care sufficiently to be widely adopted. This scientific statement outlines the current state of the art on the use of artificial intelligence algorithms and data science in the diagnosis, classification, and treatment of cardiovascular disease. It also sets out to advance this mission, focusing on how digital tools and, in particular, artificial intelligence may provide clinical and mechanistic insights, address bias in clinical studies, and facilitate education and implementation science to improve cardiovascular and stroke outcomes. Last, a key objective of this scientific statement is to further the field by identifying best practices, gaps, and challenges for interested stakeholders.

Healthcare utilization and clinical characteristics of genetic epilepsy in electronic health records

Understanding the clinical characteristics and medical treatment of individuals affected by genetic epilepsies is instrumental in guiding selection for genetic testing, defining the phenotype range of these rare disorders, optimizing patient care pathways and pinpointing unaddressed medical need by quantifying healthcare resource utilization. To date, a matched longitudinal cohort study encompassing the entire spectrum of clinical characteristics and medical treatment from childhood through adolescence has not been performed. We identified individuals with genetic and non-genetic epilepsies and onset at ages 0-5 years by linkage across the Cleveland Clinic Health System. We used natural language processing to extract medical terms and procedures from longitudinal electronic health records and tested for cross-sectional and temporal associations with genetic epilepsy. We implemented a two-stage design: in the discovery cohort, individuals were stratified as being 'likely genetic' or 'non-genetic' by a natural language processing algorithm, and controls did not receive genetic testing. The validation cohort consisted of cases with genetic epilepsy confirmed by manual chart review and an independent set of controls who received negative genetic testing. The discovery and validation cohorts consisted of 503 and 344 individuals with genetic epilepsy and matched controls, respectively. The median age at the first encounter was 0.1 years and 7.9 years at the last encounter, and the mean duration of follow-up was 8.2 years. We extracted 188,295 Unified Medical Language System annotations for statistical analysis across 9659 encounters. Individuals with genetic epilepsy received an earlier epilepsy diagnosis and had more frequent and complex encounters with the healthcare system. Notably, the highest enrichment of encounters compared with the non-genetic groups was found during the transition from paediatric to adult care. Our computational approach could validate established comorbidities of genetic epilepsies, such as behavioural abnormality and intellectual disability. We also revealed novel associations for genitourinary abnormalities (odds ratio 1.91, 95% confidence interval: 1.66-2.20, P = 6.16 × 10-19) linked to a spectrum of underrecognized epilepsy-associated genetic disorders. This case-control study leveraged real-world data to identify novel features associated with the likelihood of a genetic aetiology and quantified the healthcare utilization of genetic epilepsies compared with matched controls. Our results strongly recommend early genetic testing to stratify individuals into specialized care paths, thus improving the clinical management of people with genetic epilepsies.

A prototype protocol for evaluating the real-world data set using a structured electronic health record in glaucoma

PURPOSE

As the first step in monitoring and evaluating day-to-day glaucoma care, this study reports all real-world data recorded during the first full year after the implementation of a prototype for glaucoma-specific structured electronic healthcare record (EHR).

METHODS

In 2019, 4618 patients visited Tays Medical Glaucoma Clinic at Tays Eye Centre, Tampere University Hospital, Finland, that serves a population of 0.53 M. Patient data were entered into a glaucoma-specific EHR by trained nurses to be checked by glaucoma specialists. Tays Eye Centre follows the Finnish Current Care Guideline for glaucoma in which glaucoma is defined using a '2 out of 3' rule, that is, ≥2 findings evaluated as glaucomatous in optic nerve head (ONH), retinal nerve fibre layer (RNFL) and visual field (VF).

RESULTS

The clinical evaluations of ONH, RNFL and VF were recorded in 95%-100% of all eyes. ONH was evaluated as glaucomatous more often (44%) than RNFL (33%) and VF tests (30%). Progressive changes in any of the three tests were recorded in 35% of the '≥2/3 glaucoma group' compared to 2%-9% in the other groups. The mean IOP at visit was 15 mmHg. The mean target IOP was 17 mmHg, and it was recorded in 94% of eyes.

CONCLUSION

The developed structured data presentation enables comparisons between different population-based real-world glaucoma data sets and glaucoma clinics. Compared to a data set from the UK, the proportion of glaucoma suspicion-related visits was smaller in Tays Eye Centre and test intervals were longer.

Predicting incident heart failure from population-based nationwide electronic health records: protocol for a model development and validation study

INTRODUCTION

Heart failure (HF) is increasingly common and associated with excess morbidity, mortality, and healthcare costs. Treatment of HF can alter the disease trajectory and reduce clinical events in HF. However, many cases of HF remain undetected until presentation with more advanced symptoms, often requiring hospitalisation. Predicting incident HF is challenging and statistical models are limited by performance and scalability in routine clinical practice. An HF prediction model implementable in nationwide electronic health records (EHRs) could enable targeted diagnostics to enable earlier identification of HF.

METHODS AND ANALYSIS

We will investigate a range of development techniques (including logistic regression and supervised machine learning methods) on routinely collected primary care EHRs to predict risk of new-onset HF over 1, 5 and 10 years prediction horizons. The Clinical Practice Research Datalink (CPRD)-GOLD dataset will be used for derivation (training and testing) and the CPRD-AURUM dataset for external validation. Both comprise large cohorts of patients, representative of the population of England in terms of age, sex and ethnicity. Primary care records are linked at patient level to secondary care and mortality data. The performance of the prediction model will be assessed by discrimination, calibration and clinical utility. We will only use variables routinely accessible in primary care.

ETHICS AND DISSEMINATION

Permissions for CPRD-GOLD and CPRD-AURUM datasets were obtained from CPRD (ref no: 21_000324). The CPRD ethical approval committee approved the study. The results will be submitted as a research paper for publication to a peer-reviewed journal and presented at peer-reviewed conferences.

TRIAL REGISTRATION DETAILS

The study was registered on Clinical Trials.gov (NCT05756127). A systematic review for the project was registered on PROSPERO (registration number: CRD42022380892).

The Impact of Documentation Workflow on the Accuracy of the Coded Diagnoses in the Electronic Health Record

Objective

To determine the impact of documentation workflow on the accuracy of coded diagnoses in electronic health records (EHRs).

Design

Cross-sectional study.

Participants

All patients who completed visits at the Casey Eye Institute Retina Division faculty clinic between April 7, 2022 and April 13, 2022.

Main Outcome Measures

Agreement between coded diagnoses and clinical notes.

Methods

We assessed the rate of agreement between the diagnoses in the clinical notes and the coded diagnosis in the EHR using manual review and examined the impact of the documentation workflow on the rate of agreement in an academic retina practice.

Results

In 202 visits by 8 physicians, 78% (range, 22%-100%) had an agreement between the coded diagnoses and the clinical notes. When physicians integrated the diagnosis code entry and note composition, the rate of agreement was 87.9% (range, 62%-100%). For those who entered the diagnosis codes separately from writing notes, the agreement was 44.4% (22%-50%, P < 0.0001).

Conclusion

The visit-specific agreement between the coded diagnosis and the progress note can vary widely by workflow. The workflow and EHR design may be an important part of understanding and improving the quality of EHR data.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Artificial intelligence-based risk stratification, accurate diagnosis and treatment prediction in gynecologic oncology

As data-driven science, artificial intelligence (AI) has paved a promising path toward an evolving health system teeming with thrilling opportunities for precision oncology. Notwithstanding the tremendous success of oncological AI in such fields as lung carcinoma, breast tumor and brain malignancy, less attention has been devoted to investigating the influence of AI on gynecologic oncology. Hereby, this review sheds light on the ever-increasing contribution of state-of-the-art AI techniques to the refined risk stratification and whole-course management of patients with gynecologic tumors, in particular, cervical, ovarian and endometrial cancer, centering on information and features extracted from clinical data (electronic health records), cancer imaging including radiological imaging, colposcopic images, cytological and histopathological digital images, and molecular profiling (genomics, transcriptomics, metabolomics and so forth). However, there are still noteworthy challenges beyond performance validation. Thus, this work further describes the limitations and challenges faced in the real-word implementation of AI models, as well as potential solutions to address these issues.

Cardiovascular Care Innovation through Data-Driven Discoveries in the Electronic Health Record

The electronic health record (EHR) represents a rich source of patient information, increasingly being leveraged for cardiovascular research. Although its primary use remains the seamless delivery of health care, the various longitudinally aggregated structured and unstructured data elements for each patient within the EHR can define the computational phenotypes of disease and care signatures and their association with outcomes. Although structured data elements, such as demographic characteristics, laboratory measurements, problem lists, and medications, are easily extracted, unstructured data are underused. The latter include free text in clinical narratives, documentation of procedures, and reports of imaging and pathology. Rapid scaling up of data storage and rapid innovation in natural language processing and computer vision can power insights from unstructured data streams. However, despite an array of opportunities for research using the EHR, specific expertise is necessary to adequately address confidentiality, accuracy, completeness, and heterogeneity challenges in EHR-based research. These often require methodological innovation and best practices to design and conduct successful research studies. Our review discusses these challenges and their proposed solutions. In addition, we highlight the ongoing innovations in federated learning in the EHR through a greater focus on common data models and discuss ongoing work that defines such an approach to large-scale, multicenter, federated studies. Such parallel improvements in technology and research methods enable innovative care and optimization of patient outcomes.

Prognosis, characteristics, and provision of care for patients with the unspecified heart failure electronic health record phenotype: a population-based linked cohort study of 95262 individuals

Background

Whether the accuracy of the phenotype ascribed to patients in electronic health records (EHRs) is associated with variation in prognosis and care provision is unknown. We investigated this for heart failure (HF, characterised as HF with preserved ejection fraction [HFpEF], HF with reduced ejection fraction [HFrEF] and unspecified HF).

Methods

We included individuals aged 16 years and older with a new diagnosis of HF between January 2, 1998 and February 28, 2022 from linked primary and secondary care records in the Clinical Practice Research Datalink in England. We investigated the provision of guideline-recommended diagnostic investigations and pharmacological treatments. The primary outcome was a composite of HF hospitalisation or all-cause death, and secondary outcomes were time to HF hospitalisation, all-cause death and death from cardiovascular causes. We used Kaplan-Meier curves and log rank tests to compare survival across HF phenotypes and adjusted for potential confounders in Cox proportional hazards regression analyses.

Findings

Of a cohort of 95,262 individuals, 1271 (1.3%) were recorded as having HFpEF, 10,793 (11.3%) as HFrEF and 83,198 (87.3%) as unspecified HF. Individuals recorded as unspecified HF were older with a higher prevalence of dementia. Unspecified HF, compared to patients with a recorded HF phenotype, were less likely to receive specialist assessment, echocardiography or natriuretic peptide testing in the peri-diagnostic period, or receive angiotensin-converting enzyme inhibitors, beta blockers or mineralocorticoid receptor antagonists up to 12 months after diagnosis (risk ratios compared to HFrEF, 0.64, 95% CI 0.63-0.64; 0.59, 0.58-0.60; 0.57, 0.55-0.59; respectively) and had significantly worse outcomes (adjusted hazard ratios compared to HFrEF, HF hospitalisation and death 1.66, 95% CI 1.59-1.74; all-cause mortality 2.00, 1.90-2.10; cardiovascular death 1.77, 1.65-1.90).

Interpretation

Our findings suggested that absence of specification of HF phenotype in routine EHRs is inversely associated with clinical investigations, treatments and survival, representing an actionable target to mitigate prognostic and health resource burden.

Funding

Japan Research Foundation for Healthy Aging and British Heart Foundation.

Consolidated Reporting Guidelines for Prognostic and Diagnostic Machine Learning Modeling Studies: Development and Validation

BACKGROUND

The reporting of machine learning (ML) prognostic and diagnostic modeling studies is often inadequate, making it difficult to understand and replicate such studies. To address this issue, multiple consensus and expert reporting guidelines for ML studies have been published. However, these guidelines cover different parts of the analytics lifecycle, and individually, none of them provide a complete set of reporting requirements.

OBJECTIVE

We aimed to consolidate the ML reporting guidelines and checklists in the literature to provide reporting items for prognostic and diagnostic ML in in-silico and shadow mode studies.

METHODS

We conducted a literature search that identified 192 unique peer-reviewed English articles that provide guidance and checklists for reporting ML studies. The articles were screened by their title and abstract against a set of 9 inclusion and exclusion criteria. Articles that were filtered through had their quality evaluated by 2 raters using a 9-point checklist constructed from guideline development good practices. The average κ was 0.71 across all quality criteria. The resulting 17 high-quality source papers were defined as having a quality score equal to or higher than the median. The reporting items in these 17 articles were consolidated and screened against a set of 6 inclusion and exclusion criteria. The resulting reporting items were sent to an external group of 11 ML experts for review and updated accordingly. The updated checklist was used to assess the reporting in 6 recent modeling papers in JMIR AI. Feedback from the external review and initial validation efforts was used to improve the reporting items.

RESULTS

In total, 37 reporting items were identified and grouped into 5 categories based on the stage of the ML project: defining the study details, defining and collecting the data, modeling methodology, model evaluation, and explainability. None of the 17 source articles covered all the reporting items. The study details and data description reporting items were the most common in the source literature, with explainability and methodology guidance (ie, data preparation and model training) having the least coverage. For instance, a median of 75% of the data description reporting items appeared in each of the 17 high-quality source guidelines, but only a median of 33% of the data explainability reporting items appeared. The highest-quality source articles tended to have more items on reporting study details. Other categories of reporting items were not related to the source article quality. We converted the reporting items into a checklist to support more complete reporting.

CONCLUSIONS

Our findings supported the need for a set of consolidated reporting items, given that existing high-quality guidelines and checklists do not individually provide complete coverage. The consolidated set of reporting items is expected to improve the quality and reproducibility of ML modeling studies.

Ten-year real-world outcomes of antivascular endothelial growth factor therapy in neovascular age-related macular degeneration using pro re nata regimen

BACKGROUND/AIMS

To analyse long-term outcomes of antivascular endothelial growth factor (anti-VEGF) therapy for the treatment of neovascular age-related macular degeneration (nAMD) using pro re nata (PRN) regimen in a single-centre clinical practice.

METHODS

All patients receiving intravitreal injection (IVI) for nAMD between 1 January 2008 and 31 December 2020 were searched from electronic medical records. All 3844 treatment-naïve eyes of 3008 patients were included with a total of 50 146 IVIs (87% bevacizumab) administered. Main outcome measures were mean change in visual acuity (VA) from baseline, proportion of eyes within 15 letters of baseline, proportion of eyes with VA ≥20/40 Snellen and ≤20/200 Snellen, number of annual visits and number of annual IVIs.

RESULTS

The mean baseline VA was 55 Early Treatment Diabetic Retinopathy Study (ETDRS) letters and the mean change in VA from baseline was +2, +2, ±0, -2, -2 and -4 ETDRS letters at year 1, 2, 3, 5, 7 and 10, respectively. Proportions of eyes within 15 letters of baseline were 88%, 87%, 82%, 80%, 76% and 72% at the end of years 1, 2, 3, 5, 7 and 10, respectively. The median number of annual IVI was 6 at years 1-7 and 5 at year 10. The median number of annual total visits was 10 at year 1, 9 at years 2-7 and 8 at year 10, respectively.

CONCLUSIONS

VA was maintained short-term and long-term with anti-VEGF therapy using PRN treatment regimen.

A pipeline to further enhance quality, integrity and reusability of the NCCID clinical data

The National COVID-19 Chest Imaging Database (NCCID) is a centralized UK database of thoracic imaging and corresponding clinical data. It is made available by the National Health Service Artificial Intelligence (NHS AI) Lab to support the development of machine learning tools focused on Coronavirus Disease 2019 (COVID-19). A bespoke cleaning pipeline for NCCID, developed by the NHSx, was introduced in 2021. We present an extension to the original cleaning pipeline for the clinical data of the database. It has been adjusted to correct additional systematic inconsistencies in the raw data such as patient sex, oxygen levels and date values. The most important changes will be discussed in this paper, whilst the code and further explanations are made publicly available on GitLab. The suggested cleaning will allow global users to work with more consistent data for the development of machine learning tools without being an expert. In addition, it highlights some of the challenges when working with clinical multi-center data and includes recommendations for similar future initiatives.