Adaptive adversarial neural networks for the analysis of lossy and domain-shifted datasets of medical images

Generative artificial intelligence to produce high-fidelity blastocyst-stage embryo images

STUDY QUESTION

Can generative artificial intelligence (AI) models produce high-fidelity images of human blastocysts?

SUMMARY ANSWER

Generative AI models exhibit the capability to generate high-fidelity human blastocyst images, thereby providing substantial training datasets crucial for the development of robust AI models.

WHAT IS KNOWN ALREADY

The integration of AI into IVF procedures holds the potential to enhance objectivity and automate embryo selection for transfer. However, the effectiveness of AI is limited by data scarcity and ethical concerns related to patient data privacy. Generative adversarial networks (GAN) have emerged as a promising approach to alleviate data limitations by generating synthetic data that closely approximate real images.

STUDY DESIGN, SIZE, DURATION

Blastocyst images were included as training data from a public dataset of time-lapse microscopy (TLM) videos (n = 136). A style-based GAN was fine-tuned as the generative model.

PARTICIPANTS/MATERIALS, SETTING, METHODS

We curated a total of 972 blastocyst images as training data, where frames were captured within the time window of 110-120 h post-insemination at 1-h intervals from TLM videos. We configured the style-based GAN model with data augmentation (AUG) and pretrained weights (Pretrained-T: with translation equivariance; Pretrained-R: with translation and rotation equivariance) to compare their optimization on image synthesis. We then applied quantitative metrics including Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) to assess the quality and fidelity of the generated images. Subsequently, we evaluated qualitative performance by measuring the intelligence behavior of the model through the visual Turing test. To this end, 60 individuals with diverse backgrounds and expertise in clinical embryology and IVF evaluated the quality of synthetic embryo images.

MAIN RESULTS AND THE ROLE OF CHANCE

During the training process, we observed consistent improvement of image quality that was measured by FID and KID scores. Pretrained and AUG + Pretrained initiated with remarkably lower FID and KID values compared to both Baseline and AUG + Baseline models. Following 5000 training iterations, the AUG + Pretrained-R model showed the highest performance of the evaluated five configurations with FID and KID scores of 15.2 and 0.004, respectively. Subsequently, we carried out the visual Turing test, such that IVF embryologists, IVF laboratory technicians, and non-experts evaluated the synthetic blastocyst-stage embryo images and obtained similar performance in specificity with marginal differences in accuracy and sensitivity.

LIMITATIONS, REASONS FOR CAUTION

In this study, we primarily focused the training data on blastocyst images as IVF embryos are primarily assessed in blastocyst stage. However, generation of an array of images in different preimplantation stages offers further insights into the development of preimplantation embryos and IVF success. In addition, we resized training images to a resolution of 256 × 256 pixels to moderate the computational costs of training the style-based GAN models. Further research is needed to involve a more extensive and diverse dataset from the formation of the zygote to the blastocyst stage, e.g. video generation, and the use of improved image resolution to facilitate the development of comprehensive AI algorithms and to produce higher-quality images.

WIDER IMPLICATIONS OF THE FINDINGS

Generative AI models hold promising potential in generating high-fidelity human blastocyst images, which allows the development of robust AI models as it can provide sufficient training datasets while safeguarding patient data privacy. Additionally, this may help to produce sufficient embryo imaging training data with different (rare) abnormal features, such as embryonic arrest, tripolar cell division to avoid class imbalances and reach to even datasets. Thus, generative models may offer a compelling opportunity to transform embryo selection procedures and substantially enhance IVF outcomes.

STUDY FUNDING/COMPETING INTEREST(S)

This study was supported by a Horizon 2020 innovation grant (ERIN, grant no. EU952516) and a Horizon Europe grant (NESTOR, grant no. 101120075) of the European Commission to A.S. and M.Z.E., the Estonian Research Council (grant no. PRG1076) to A.S., and the EVA (Erfelijkheid Voortplanting & Aanleg) specialty program (grant no. KP111513) of Maastricht University Medical Centre (MUMC+) to M.Z.E.

TRIAL REGISTRATION NUMBER

Not applicable.

The prospect of artificial intelligence to personalize assisted reproductive technology

Infertility affects 1-in-6 couples, with repeated intensive cycles of assisted reproductive technology (ART) required by many to achieve a desired live birth. In ART, typically, clinicians and laboratory staff consider patient characteristics, previous treatment responses, and ongoing monitoring to determine treatment decisions. However, the reproducibility, weighting, and interpretation of these characteristics are contentious, and highly operator-dependent, resulting in considerable reliance on clinical experience. Artificial intelligence (AI) is ideally suited to handle, process, and analyze large, dynamic, temporal datasets with multiple intermediary outcomes that are generated during an ART cycle. Here, we review how AI has demonstrated potential for optimization and personalization of key steps in a reproducible manner, including: drug selection and dosing, cycle monitoring, induction of oocyte maturation, and selection of the most competent gametes and embryos, to improve the overall efficacy and safety of ART.

Rapidly adaptable automated interpretation of point-of-care COVID-19 diagnostics

BACKGROUND

Point-of-care diagnostic devices, such as lateral-flow assays, are becoming widely used by the public. However, efforts to ensure correct assay operation and result interpretation rely on hardware that cannot be easily scaled or image processing approaches requiring large training datasets, necessitating large numbers of tests and expert labeling with validated specimens for every new test kit format.

METHODS

We developed a software architecture called AutoAdapt POC that integrates automated membrane extraction, self-supervised learning, and few-shot learning to automate the interpretation of POC diagnostic tests using smartphone cameras in a scalable manner. A base model pre-trained on a single LFA kit is adapted to five different COVID-19 tests (three antigen, two antibody) using just 20 labeled images.

RESULTS

Here we show AutoAdapt POC to yield 99% to 100% accuracy over 726 tests (350 positive, 376 negative). In a COVID-19 drive-through study with 74 untrained users self-testing, 98% found image collection easy, and the rapidly adapted models achieved classification accuracies of 100% on both COVID-19 antigen and antibody test kits. Compared with traditional visual interpretation on 105 test kit results, the algorithm correctly identified 100% of images; without a false negative as interpreted by experts. Finally, compared to a traditional convolutional neural network trained on an HIV test kit, the algorithm showed high accuracy while requiring only 1/50th of the training images.

CONCLUSIONS

The study demonstrates how rapid domain adaptation in machine learning can provide quality assurance, linkage to care, and public health tracking for untrained users across diverse POC diagnostic tests.

Automation in ART: Paving the Way for the Future of Infertility Treatment

In vitro fertilisation (IVF) is estimated to account for the birth of more than nine million babies worldwide, perhaps making it one of the most intriguing as well as commoditised and industrialised modern medical interventions. Nevertheless, most IVF procedures are currently limited by accessibility, affordability and most importantly multistep, labour-intensive, technically challenging processes undertaken by skilled professionals. Therefore, in order to sustain the exponential demand for IVF on one hand, and streamline existing processes on the other, innovation is essential. This may not only effectively manage clinical time but also reduce cost, thereby increasing accessibility, affordability and efficiency. Recent years have seen a diverse range of technologies, some integrated with artificial intelligence, throughout the IVF pathway, which promise personalisation and, at least, partial automation in the not-so-distant future. This review aims to summarise the rapidly evolving state of these innovations in automation, with or without the integration of artificial intelligence, encompassing the patient treatment pathway, gamete/embryo selection, endometrial evaluation and cryopreservation of gametes/embryos. Additionally, it shall highlight the resulting prospective change in the role of IVF professionals and challenges of implementation of some of these technologies, thereby aiming to motivate continued research in this field.

Smartphone-based platforms implementing microfluidic detection with image-based artificial intelligence

The frequent outbreak of global infectious diseases has prompted the development of rapid and effective diagnostic tools for the early screening of potential patients in point-of-care testing scenarios. With advances in mobile computing power and microfluidic technology, the smartphone-based mobile health platform has drawn significant attention from researchers developing point-of-care testing devices that integrate microfluidic optical detection with artificial intelligence analysis. In this article, we summarize recent progress in these mobile health platforms, including the aspects of microfluidic chips, imaging modalities, supporting components, and the development of software algorithms. We document the application of mobile health platforms in terms of the detection objects, including molecules, viruses, cells, and parasites. Finally, we discuss the prospects for future development of mobile health platforms.

Advancements in the future of automating micromanipulation techniques in the IVF laboratory using deep convolutional neural networks

PURPOSE

To determine if deep learning artificial intelligence algorithms can be used to accurately identify key morphologic landmarks on oocytes and cleavage stage embryo images for micromanipulation procedures such as intracytoplasmic sperm injection (ICSI) or assisted hatching (AH).

METHODS

Two convolutional neural network (CNN) models were trained, validated, and tested over three replicates to identify key morphologic landmarks used to guide embryologists when performing micromanipulation procedures. The first model (CNN-ICSI) was trained (n = 13,992), validated (n = 1920), and tested (n = 3900) to identify the optimal location for ICSI through polar body identification. The second model (CNN-AH) was trained (n = 13,908), validated (n = 1908), and tested (n = 3888) to identify the optimal location for AH on the zona pellucida that maximizes distance from healthy blastomeres.

RESULTS

The CNN-ICSI model accurately identified the polar body and corresponding optimal ICSI location with 98.9% accuracy (95% CI 98.5-99.2%) with a receiver operator characteristic (ROC) with micro and macro area under the curves (AUC) of 1. The CNN-AH model accurately identified the optimal AH location with 99.41% accuracy (95% CI 99.11-99.62%) with a ROC with micro and macro AUCs of 1.

CONCLUSION

Deep CNN models demonstrate powerful potential in accurately identifying key landmarks on oocytes and cleavage stage embryos for micromanipulation. These findings are novel, essential stepping stones in the automation of micromanipulation procedures.

Shifting machine learning for healthcare from development to deployment and from models to data

In the past decade, the application of machine learning (ML) to healthcare has helped drive the automation of physician tasks as well as enhancements in clinical capabilities and access to care. This progress has emphasized that, from model development to model deployment, data play central roles. In this Review, we provide a data-centric view of the innovations and challenges that are defining ML for healthcare. We discuss deep generative models and federated learning as strategies to augment datasets for improved model performance, as well as the use of the more recent transformer models for handling larger datasets and enhancing the modelling of clinical text. We also discuss data-focused problems in the deployment of ML, emphasizing the need to efficiently deliver data to ML models for timely clinical predictions and to account for natural data shifts that can deteriorate model performance.

Could artificial intelligence revolutionize the development of nanovectors for gene therapy and mRNA vaccines?

Gene therapy enables the introduction of nucleic acids like DNA and RNA into host cells, and is expected to revolutionize the treatment of a wide range of diseases. This growth has been further accelerated by the discovery of CRISPR/Cas technology, which allows accurate genomic editing in a broad range of cells and organisms in vitro and in vivo. Despite many advances in gene delivery and the development of various viral and non-viral gene delivery vectors, the lack of highly efficient non-viral systems with low cellular toxicity remains a challenge. The application of cutting-edge technologies such as artificial intelligence (AI) has great potential to find new paradigms to solve this issue. Herein, we review AI and its major subfields including machine learning (ML), neural networks (NNs), expert systems, deep learning (DL), computer vision and robotics. We discuss the potential of AI-based models and algorithms in the design of targeted gene delivery vehicles capable of crossing extracellular and intracellular barriers by viral mimicry strategies. We finally discuss the role of AI in improving the function of CRISPR/Cas systems, developing novel nanobots, and mRNA vaccine carriers.

Deep learning-assisted sensitive detection of fentanyl using a bubbling-microchip

Deep learning-enabled smartphone-based image processing has significant advantages in the development of point-of-care diagnostics. Conventionally, most deep-learning applications require task specific large scale expertly annotated datasets. Therefore, these algorithms are oftentimes limited only to applications that have large retrospective datasets available for network development. Here, we report the possibility of utilizing adversarial neural networks to overcome this challenge by expanding the utility of non-specific data for the development of deep learning models. As a clinical model, we report the detection of fentanyl, a small molecular weight drug that is a type of opioid, at the point-of-care using a deep-learning empowered smartphone assay. We used the catalytic property of platinum nanoparticles (PtNPs) in a smartphone-enabled microchip bubbling assay to achieve high analytical sensitivity (detecting fentanyl at concentrations as low as 0.23 ng mL-1 in phosphate buffered saline (PBS), 0.43 ng mL-1 in human serum and 0.64 ng mL-1 in artificial human urine). Image-based inferences were made by our adversarial-based SPyDERMAN network that was developed using a limited dataset of 104 smartphone images of microchips with bubble signals from tests performed with known fentanyl concentrations and using our retrospective library of 17 573 non-specific bubbling-microchip images. The accuracy (± standard error of mean) of the developed system in determining the presence of fentanyl, when using a cutoff concentration of 1 ng mL-1, was 93 ± 0% in human serum (n = 100) and 95.3 ± 1.5% in artificial human urine (n = 100).

Lab on a chip devices for fertility: from proof-of-concept to clinical impact

Microfluidics offers tremendous opportunities to understand the underlying biology of fertilization at the single-cell level and improve infertility management, however, its true clinical impact is yet to be realized. Lab-on-a-chip devices have generally failed to diffuse into clinical practice due to issues associated with their translation or their practicality and performance in clinical settings. In this perspective, I reflect on how the full potential of microfluidic technologies for fertility can be realized by considering regulatory and manufacturing considerations at the development stage and by redefining our developmental goals to directly target the ultimate clinical needs. I also challenge the common rationale around developing technologies for infertility treatment based on reducing cost and complexity in operation as the ultimate outcome is invaluable, human life.

Artificial intelligence in the embryology laboratory: a review

The goal of an IVF cycle is a healthy live-born baby. Despite the many advances in the field of assisted reproductive technologies, accurately predicting the outcome of an IVF cycle has yet to be achieved. One reason for this is the method of selecting an embryo for transfer. Morphological assessment of embryos is the traditional method of evaluating embryo quality and selecting which embryo to transfer. However, this subjective method of assessing embryos leads to inter- and intra-observer variability, resulting in less than optimal IVF success rates. To overcome this, it is common practice to transfer more than one embryo, potentially resulting in high-risk multiple pregnancies. Although time-lapse incubators and preimplantation genetic testing for aneuploidy have been introduced to help increase the chances of live birth, the outcomes remain less than ideal. Utilization of artificial intelligence (AI) has become increasingly popular in the medical field and is increasingly being leveraged in the embryology laboratory to help improve IVF outcomes. Many studies have been published investigating the use of AI as an unbiased, automated approach to embryo assessment. This review summarizes recent AI advancements in the embryology laboratory.