Contactless facial video recording with deep learning models for the detection of atrial fibrillation

Challenges and prospects of visual contactless physiological monitoring in clinical study

The monitoring of physiological parameters is a crucial topic in promoting human health and an indispensable approach for assessing physiological status and diagnosing diseases. Particularly, it holds significant value for patients who require long-term monitoring or with underlying cardiovascular disease. To this end, Visual Contactless Physiological Monitoring (VCPM) is capable of using videos recorded by a consumer camera to monitor blood volume pulse (BVP) signal, heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2) and blood pressure (BP). Recently, deep learning-based pipelines have attracted numerous scholars and achieved unprecedented development. Although VCPM is still an emerging digital medical technology and presents many challenges and opportunities, it has the potential to revolutionize clinical medicine, digital health, telemedicine as well as other areas. The VCPM technology presents a viable solution that can be integrated into these systems for measuring vital parameters during video consultation, owing to its merits of contactless measurement, cost-effectiveness, user-friendly passive monitoring and the sole requirement of an off-the-shelf camera. In fact, the studies of VCPM technologies have been rocketing recently, particularly AI-based approaches, but few are employed in clinical settings. Here we provide a comprehensive overview of the applications, challenges, and prospects of VCPM from the perspective of clinical settings and AI technologies for the first time. The thorough exploration and analysis of clinical scenarios will provide profound guidance for the research and development of VCPM technologies in clinical settings.

Accuracy of Artificial Intelligence-Based Technologies for the Diagnosis of Atrial Fibrillation: A Systematic Review and Meta-Analysis

Atrial fibrillation (AF) is the most common arrhythmia with a high burden of morbidity including impaired quality of life and increased risk of thromboembolism. Early detection and management of AF could prevent thromboembolic events. Artificial intelligence (AI)--based methods in healthcare are developing quickly and can be proved as valuable for the detection of atrial fibrillation. In this metanalysis, we aim to review the diagnostic accuracy of AI-based methods for the diagnosis of atrial fibrillation. A predetermined search strategy was applied on four databases, the PubMed on 31 August 2022, the Google Scholar and Cochrane Library on 3 September 2022, and the Embase on 15 October 2022. The identified studies were screened by two independent investigators. Studies assessing the diagnostic accuracy of AI-based devices for the detection of AF in adults against a gold standard were selected. Qualitative and quantitative synthesis to calculate the pooled sensitivity and specificity was performed, and the QUADAS-2 tool was used for the risk of bias and applicability assessment. We screened 14,770 studies, from which 31 were eligible and included. All were diagnostic accuracy studies with case-control or cohort design. The main technologies used were: (a) photoplethysmography (PPG) with pooled sensitivity 95.1% and specificity 96.2%, and (b) single-lead ECG with pooled sensitivity 92.3% and specificity 96.2%. In the PPG group, 0% to 43.2% of the tracings could not be classified using the AI algorithm as AF or not, and in the single-lead ECG group, this figure fluctuated between 0% and 38%. Our analysis showed that AI-based methods for the diagnosis of atrial fibrillation have high sensitivity and specificity for the detection of AF. Further studies should examine whether utilization of these methods could improve clinical outcomes.

Measurements of pulse rate using facial video cameras from smart devices in patients diagnosed with atrial fibrillation

Clinical applications of passive long-term heart rate (HR) monitoring in patients with cardiac arrhythmias include adequate drug titration of atrioventricular (AV) nodal drugs and assessment of medical compliance with treatment. A majority of patients treated with beta-blockers, especially patients with atrial fibrillation (AF), require some degree of drug titration during the first 6 months of treatment to ensure that adequate HR control and medicine compliance has been achieved. Failing to achieve adequate rate control in patients with AF can lead to worsening symptoms, heart failure exacerbations, and potentially tachycardia-induced cardiomyopathy. Enabling video-based monitoring during telehealth patient visits could facilitate providers to measure heart rate (HR) without the need for a dedicated home device (smartwatch, SPO2 device, or others). Videoplethysmography (VPG) is a monitoring technology that measures pulse rate by utilizing front-facing cameras embedded in smart devices. VPG provides a remote and contactless cardiac monitoring solution. We conducted a clinical experiment to evaluate the accuracy of VPG in measuring HR while running on two portable devices: Samsung S10 smartphones and S3 tablets. We used a single‑lead ECG to measure the heart rate at the time of the VPG recordings in AF patients. We employed the Bland-Altman method to measure the level of agreement between videoplethysmography and ECG-based measurements of HR. The findings reveal that the mean difference in videoplethysmography and ECG-based heart rate was inferior to 1 bpm across the 2 devices with confidence intervals ranging from 3 to 12 BPM. Our facial video-based HR monitoring solution could assist providers in measuring heart rates in their patients with AF during remote telehealth visits.

Measurement of Vital Signs by Lifelight Software in Comparison to Standard of Care Multisite Development (VISION-MD): Protocol for an Observational Study

BACKGROUND

Measuring vital signs (VS) is an important aspect of clinical care but is time-consuming and requires multiple pieces of equipment and trained staff. Interest in the contactless measurement of VS has grown since the COVID-19 pandemic, including in nonclinical situations. Lifelight is an app being developed as a medical device for the contactless measurement of VS using remote photoplethysmography (rPPG) via the camera on smart devices. The VISION-D (Measurement of Vital Signs by Lifelight Software in Comparison to the Standard of Care-Development) and VISION-V (Validation) studies demonstrated the accuracy of Lifelight compared with standard-of-care measurement of blood pressure, pulse rate, and respiratory rate, supporting the certification of Lifelight as a class I Conformité Européenne (CE) medical device.

OBJECTIVE

To support further development of the Lifelight app, the observational VISION Multisite Development (VISION-MD) study is collecting high-quality data from a broad range of patients, including those with VS measurements outside the normal healthy range and patients who are critically ill.

METHODS

The study is recruiting adults (aged ≥16 years) who are inpatients (some critically ill), outpatients, and healthy volunteers, aiming to cover a broad range of normal and clinically relevant VS values; there are no exclusion criteria. High-resolution 60-second videos of the face are recorded by the Lifelight app while simultaneously measuring VS using standard-of-care methods (automated sphygmomanometer for blood pressure; finger clip sensor for pulse rate and oxygen saturation; manual counting of respiratory rate). Feedback from patients and nurses who use Lifelight is collected via a questionnaire. Data to estimate the cost-effectiveness of Lifelight compared with standard-of-care VS measurement are also being collected. A new method for rPPG signal processing is currently being developed, based on the identification of small areas of high-quality signals in each individual. Anticipated recruitment is 1950 participants, with the expectation that data from approximately 1700 will be used for software development. Data from 250 participants will be retained to test the performance of Lifelight against predefined performance targets.

RESULTS

Recruitment began in May 2021 but was hindered by the restrictions instigated during the COVID-19 pandemic. The development of data processing methodology is in progress. The data for analysis will become available from September 2022, and the algorithms will be refined continuously to improve clinical accuracy. The performance of Lifelight compared with that of the standard-of-care measurement of VS will then be tested. Recruitment will resume if further data are required. The analyses are expected to be completed in early 2023.

CONCLUSIONS

This study will support the refinement of data collection and processing toward the development of a robust app that is suitable for routine clinical use.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04763746; https://clinicaltrials.gov/ct2/show/NCT04763746.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/41533.

Dynamic region of interest selection in remote photoplethysmography: proof of principle (Preprint)

BACKGROUND

Remote photoplethysmography (rPPG) can record vital signs (VSs) by detecting subtle changes in the light reflected from the skin. Lifelight (Xim Ltd) is a novel software being developed as a medical device for the contactless measurement of VSs using rPPG via integral cameras on smart devices. Research to date has focused on extracting the pulsatile VS from the raw signal, which can be influenced by factors such as ambient light, skin thickness, facial movements, and skin tone.

OBJECTIVE

This preliminary proof-of-concept study outlines a dynamic approach to rPPG signal processing wherein green channel signals from the most relevant areas of the face (the midface, comprising the cheeks, nose, and top of the lip) are optimized for each subject using tiling and aggregation (T&A) algorithms.

METHODS

High-resolution 60-second videos were recorded during the VISION-MD study. The midface was divided into 62 tiles of 20×20 pixels, and the signals from multiple tiles were evaluated using bespoke algorithms through weighting according to signal-to-noise ratio in the frequency domain (SNR-F) score or segmentation. Midface signals before and after T&A were categorized by a trained observer blinded to the data processing as 0 (high quality, suitable for algorithm training), 1 (suitable for algorithm testing), or 2 (inadequate quality). On secondary analysis, observer categories were compared for signals predicted to improve categories following T&A based on the SNR-F score. Observer ratings and SNR-F scores were also compared before and after T&A for Fitzpatrick skin tones 5 and 6, wherein rPPG is hampered by light absorption by melanin.

RESULTS

The analysis used 4310 videos recorded from 1315 participants. Category 2 and 1 signals had lower mean SNR-F scores than category 0 signals. T&A improved the mean SNR-F score using all algorithms. Depending on the algorithm, 18% (763/4212) to 31% (1306/4212) of signals improved by at least one category, with up to 10% (438/4212) improving into category 0, and 67% (2834/4212) to 79% (3337/4212) remaining in the same category. Importantly, 9% (396/4212) to 21% (875/4212) improved from category 2 (not usable) into category 1. All algorithms showed improvements. No more than 3% (137/4212) of signals were assigned to a lower-quality category following T&A. On secondary analysis, 62% of signals (32/52) were recategorized, as predicted from the SNR-F score. T&A improved SNR-F scores in darker skin tones; 41% of signals (151/369) improved from category 2 to 1 and 12% (44/369) from category 1 to 0.

CONCLUSIONS

The T&A approach to dynamic region of interest selection improved signal quality, including in dark skin tones. The method was verified by comparison with a trained observer's rating. T&A could overcome factors that compromise whole-face rPPG. This method's performance in estimating VS is currently being assessed.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04763746; https://clinicaltrials.gov/ct2/show/NCT04763746.

Artificial intelligence in cardiology: Hope for the future and power for the present

Cardiovascular disease (CVD) is the principal cause of mortality and morbidity globally. With the pressures for improved care and translation of the latest medical advances and knowledge to an actionable plan, clinical decision-making for cardiologists is challenging. Artificial Intelligence (AI) is a field in computer science that studies the design of intelligent agents which take the best feasible action in a situation. It incorporates the use of computational algorithms which simulate and perform tasks that traditionally require human intelligence such as problem solving and learning. Whilst medicine is arguably the last to apply AI in its everyday routine, cardiology is at the forefront of AI revolution in the medical field. The development of AI methods for accurate prediction of CVD outcomes, non-invasive diagnosis of coronary artery disease (CAD), detection of malignant arrythmias through wearables, and diagnosis, treatment strategies and prediction of outcomes for heart failure (HF) patients, demonstrates the potential of AI in future cardiology. With the advancements of AI, Internet of Things (IoT) and the promotion of precision medicine, the future of cardiology will be heavily based on these innovative digital technologies. Despite this, ethical dilemmas regarding the implementation of AI technologies in real-world are still unaddressed.

Assessment of facial video-based detection of atrial fibrillation across human complexion

Background

Early self-detection of atrial fibrillation (AF) can help delay and/or prevent significant associated complications, including embolic stroke and heart failure. We developed a facial video technology, videoplethysmography (VPG), to detect AF based on the analysis of facial pulsatile signals.

Objective

The purpose of this study was to evaluate the accuracy of a video-based technology to detect AF on a smartphone and to test the performance of the technology in AF patients across the whole spectrum of skin complexion and under various recording conditions.

Methods

The performance of video-based monitoring depends on a set of factors such as the angle and the distance between the camera and the patient's face, the strength of illumination, and the patient's skin tone. We conducted a clinical study involving 60 subjects with a confirmed diagnosis of AF. A continuous electrocardiogram was used as the gold standard for cardiac rhythm annotation. The VPG technology was fine-tuned on a smartphone for the first 15 subjects. Validation recordings were then done using 7053 measurements collected from the remaining 45 subjects.

Results

The VPG technology detected the presence of AF using the video camera from a common smartphone with sensitivity and specificity ≥90%. The ambient level of illumination needs to be ≥100 lux for the technology to deliver consistent performance across all skin tones.

Conclusion

We demonstrated that facial video-based detection of AF provides accurate outpatient cardiac monitoring including high pulse rate accuracy and medical-grade performance for AF detection.