Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare

A bias evaluation checklist for predictive models and its pilot application for 30-day hospital readmission models

OBJECTIVE

Health care providers increasingly rely upon predictive algorithms when making important treatment decisions, however, evidence indicates that these tools can lead to inequitable outcomes across racial and socio-economic groups. In this study, we introduce a bias evaluation checklist that allows model developers and health care providers a means to systematically appraise a model's potential to introduce bias.

MATERIALS AND METHODS

Our methods include developing a bias evaluation checklist, a scoping literature review to identify 30-day hospital readmission prediction models, and assessing the selected models using the checklist.

RESULTS

We selected 4 models for evaluation: LACE, HOSPITAL, Johns Hopkins ACG, and HATRIX. Our assessment identified critical ways in which these algorithms can perpetuate health care inequalities. We found that LACE and HOSPITAL have the greatest potential for introducing bias, Johns Hopkins ACG has the most areas of uncertainty, and HATRIX has the fewest causes for concern.

DISCUSSION

Our approach gives model developers and health care providers a practical and systematic method for evaluating bias in predictive models. Traditional bias identification methods do not elucidate sources of bias and are thus insufficient for mitigation efforts. With our checklist, bias can be addressed and eliminated before a model is fully developed or deployed.

CONCLUSION

The potential for algorithms to perpetuate biased outcomes is not isolated to readmission prediction models; rather, we believe our results have implications for predictive models across health care. We offer a systematic method for evaluating potential bias with sufficient flexibility to be utilized across models and applications.

The Potential and Emerging Role of Quantitative Imaging Biomarkers for Cancer Characterization

Simple Summary

Modern, personalized therapy approaches are increasingly changing advanced cancer into a chronic disease. Compared to imaging, novel omics methodologies in molecular biology have already achieved an individual characterization of cancerous lesions. With quantitative imaging biomarkers, analyzed by radiomics or deep learning, an imaging-based assessment of tumoral biology can be brought into clinical practice. Combining these with other non-invasive methods, e.g., liquid profiling, could allow for more individual decision making regarding therapies and applications.

Abstract

Similar to the transformation towards personalized oncology treatment, emerging techniques for evaluating oncologic imaging are fostering a transition from traditional response assessment towards more comprehensive cancer characterization via imaging. This development can be seen as key to the achievement of truly personalized and optimized cancer diagnosis and treatment. This review gives a methodological introduction for clinicians interested in the potential of quantitative imaging biomarkers, treating of radiomics models, texture visualization, convolutional neural networks and automated segmentation, in particular. Based on an introduction to these methods, clinical evidence for the corresponding imaging biomarkers—(i) dignity and etiology assessment; (ii) tumoral heterogeneity; (iii) aggressiveness and response; and (iv) targeting for biopsy and therapy—is summarized. Further requirements for the clinical implementation of these imaging biomarkers and the synergistic potential of personalized molecular cancer diagnostics and liquid profiling are discussed.

A Review of Artificial Intelligence and Robotics in Transformed Health Ecosystems

Health care is shifting toward become proactive according to the concept of P5 medicine-a predictive, personalized, preventive, participatory and precision discipline. This patient-centered care heavily leverages the latest technologies of artificial intelligence (AI) and robotics that support diagnosis, decision making and treatment. In this paper, we present the role of AI and robotic systems in this evolution, including example use cases. We categorize systems along multiple dimensions such as the type of system, the degree of autonomy, the care setting where the systems are applied, and the application area. These technologies have already achieved notable results in the prediction of sepsis or cardiovascular risk, the monitoring of vital parameters in intensive care units, or in the form of home care robots. Still, while much research is conducted around AI and robotics in health care, adoption in real world care settings is still limited. To remove adoption barriers, we need to address issues such as safety, security, privacy and ethical principles; detect and eliminate bias that could result in harmful or unfair clinical decisions; and build trust in and societal acceptance of AI.

Explanations of Machine Learning Models in Repeated Nested Cross-Validation: An Application in Age Prediction Using Brain Complexity Features

SHAP (Shapley additive explanations) is a framework for explainable AI that makes explanations locally and globally. In this work, we propose a general method to obtain representative SHAP values within a repeated nested cross-validation procedure and separately for the training and test sets of the different cross-validation rounds to assess the real generalization abilities of the explanations. We applied this method to predict individual age using brain complexity features extracted from MRI scans of 159 healthy subjects. In particular, we used four implementations of the fractal dimension (FD) of the cerebral cortex—a measurement of brain complexity. Representative SHAP values highlighted that the most recent implementation of the FD had the highest impact over the others and was among the top-ranking features for predicting age. SHAP rankings were not the same in the training and test sets, but the top-ranking features were consistent. In conclusion, we propose a method—and share all the source code—that allows a rigorous assessment of the SHAP explanations of a trained model in a repeated nested cross-validation setting.

Digital biomarkers and sex impacts in Alzheimer's disease management - potential utility for innovative 3P medicine approach

Digital biomarkers are defined as objective, quantifiable physiological and behavioral data that are collected and measured by means of digital devices. Their use has revolutionized clinical research by enabling high-frequency, longitudinal, and sensitive measurements. In the field of neurodegenerative diseases, an example of a digital biomarker-based technology is instrumental activities of daily living (iADL) digital medical application, a predictive biomarker of conversion from mild cognitive impairment (MCI) due to Alzheimer's disease (AD) to dementia due to AD in individuals aged 55 + . Digital biomarkers show promise to transform clinical practice. Nevertheless, their use may be affected by variables such as demographics, genetics, and phenotype. Among these factors, sex is particularly important in Alzheimer's, where men and women present with different symptoms and progression patterns that impact diagnosis. In this study, we explore sex differences in Altoida's digital medical application in a sample of 568 subjects consisting of a clinical dataset (MCI and dementia due to AD) and a healthy population. We found that a biological sex-classifier, built on digital biomarker features captured using Altoida's application, achieved a 75% ROC-AUC (receiver operating characteristic - area under curve) performance in predicting biological sex in healthy individuals, indicating significant differences in neurocognitive performance signatures between males and females. The performance dropped when we applied this classifier to more advanced stages on the AD continuum, including MCI and dementia, suggesting that sex differences might be disease-stage dependent. Our results indicate that neurocognitive performance signatures built on data from digital biomarker features are different between men and women. These results stress the need to integrate traditional approaches to dementia research with digital biomarker technologies and personalized medicine perspectives to achieve more precise predictive diagnostics, targeted prevention, and customized treatment of cognitive decline.

Supplementary Information

The online version contains supplementary material available at 10.1007/s13167-022-00284-3.

Diagnostic classification of Parkinson’s disease based on non-motor manifestations and machine learning strategies

Non-motor manifestations of Parkinson’s disease (PD) appear early and have a significant impact on the quality of life of patients, but few studies have evaluated their predictive potential with machine learning algorithms. We evaluated 9 algorithms for discriminating PD patients from controls using a wide collection of non-motor clinical PD features from two databases: Biocruces (96 subjects) and PPMI (687 subjects). In addition, we evaluated whether the combination of both databases could improve the individual results. For each database 2 versions with different granularity were created and a feature selection process was performed. We observed that most of the algorithms were able to detect PD patients with high accuracy (>80%). Support Vector Machine and Multi-Layer Perceptron obtained the best performance, with an accuracy of 86.3% and 84.7%, respectively. Likewise, feature selection led to a significant reduction in the number of variables and to better performance. Besides, the enrichment of Biocruces database with data from PPMI moderately benefited the performance of the classification algorithms, especially the recall and to a lesser extent the accuracy, while the precision worsened slightly. The use of interpretable rules obtained by the RIPPER algorithm showed that simply using two variables (autonomic manifestations and olfactory dysfunction), it was possible to achieve an accuracy of 84.4%. Our study demonstrates that the analysis of non-motor parameters of PD through machine learning techniques can detect PD patients with high accuracy and recall, and allows us to select the most discriminative non-motor variables to create potential tools for PD screening.

Regulating the Safety of Health-Related Artificial Intelligence

This article analyzes whether Canada's present approach to regulating health-related artificial intelligence (AI) can address relevant safety-related challenges. Focusing primarily on Health Canada's regulation of medical devices with AI, it examines whether the existing regulatory approach can adequately address general safety concerns, as well as those related to algorithmic bias and challenges posed by the intersections of these concerns with privacy and security interests. It identifies several issues and proposes reforms that aim to ensure that Canadians can access beneficial AI while keeping unsafe products off Canadian markets and motivating safe, effective use of AI products for appropriate purposes and populations.

Does imbalance in chest X-ray datasets produce biased deep learning approaches for COVID-19 screening?

Background

The health crisis resulting from the global COVID-19 pandemic highlighted more than ever the need for rapid, reliable and safe methods of diagnosis and monitoring of respiratory diseases. To study pulmonary involvement in detail, one of the most common resources is the use of different lung imaging modalities (like chest radiography) to explore the possible affected areas.

Methods

The study of patient characteristics like sex and age in pathologies of this type is crucial for gaining knowledge of the disease and for avoiding biases due to the clear scarcity of data when developing representative systems. In this work, we performed an analysis of these factors in chest X-ray images to identify biases. Specifically, 11 imbalance scenarios were defined with female and male COVID-19 patients present in different proportions for the sex analysis, and 6 scenarios where only one specific age range was used for training for the age factor. In each study, 3 different approaches for automatic COVID-19 screening were used: Normal vs COVID-19, Pneumonia vs COVID-19 and Non-COVID-19 vs COVID-19. The study was validated using two public chest X-ray datasets, allowing a reliable analysis to support the clinical decision-making process.

Results

The results for the sex-related analysis indicate this factor slightly affects the system in the Normal VS COVID-19 and Pneumonia VS COVID-19 approaches, although the identified differences are not relevant enough to worsen considerably the system. Regarding the age-related analysis, this factor was observed to be influencing the system in a more consistent way than the sex factor, as it was present in all considered scenarios. However, this worsening does not represent a major factor, as it is not of great magnitude.

Conclusions

Multiple studies have been conducted in other fields in order to determine if certain patient characteristics such as sex or age influenced these deep learning systems. However, to the best of our knowledge, this study has not been done for COVID-19 despite the urgency and lack of COVID-19 chest x-ray images. The presented results evidenced that the proposed methodology and tested approaches allow a robust and reliable analysis to support the clinical decision-making process in this pandemic scenario.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-022-01578-w).

Investigating for bias in healthcare algorithms: a sex-stratified analysis of supervised machine learning models in liver disease prediction

OBJECTIVES

The Indian Liver Patient Dataset (ILPD) is used extensively to create algorithms that predict liver disease. Given the existing research describing demographic inequities in liver disease diagnosis and management, these algorithms require scrutiny for potential biases. We address this overlooked issue by investigating ILPD models for sex bias.

METHODS

Following our literature review of ILPD papers, the models reported in existing studies are recreated and then interrogated for bias. We define four experiments, training on sex-unbalanced/balanced data, with and without feature selection. We build random forests (RFs), support vector machines (SVMs), Gaussian Naïve Bayes and logistic regression (LR) classifiers, running experiments 100 times, reporting average results with SD.

RESULTS

We reproduce published models achieving accuracies of >70% (LR 71.31% (2.37 SD) - SVM 79.40% (2.50 SD)) and demonstrate a previously unobserved performance disparity. Across all classifiers females suffer from a higher false negative rate (FNR). Presently, RF and LR classifiers are reported as the most effective models, yet in our experiments they demonstrate the greatest FNR disparity (RF; -21.02%; LR; -24.07%).

DISCUSSION

We demonstrate a sex disparity that exists in published ILPD classifiers. In practice, the higher FNR for females would manifest as increased rates of missed diagnosis for female patients and a consequent lack of appropriate care. Our study demonstrates that evaluating biases in the initial stages of machine learning can provide insights into inequalities in current clinical practice, reveal pathophysiological differences between the male and females, and can mitigate the digitisation of inequalities into algorithmic systems.

CONCLUSION

Our findings are important to medical data scientists, clinicians and policy-makers involved in the implementation medical artificial intelligence systems. An awareness of the potential biases of these systems is essential in preventing the digital exacerbation of healthcare inequalities.

Annual Research Review: Translational machine learning for child and adolescent psychiatry

Children and adolescents could benefit from the use of predictive tools that facilitate personalized diagnoses, prognoses, and treatment selection. Such tools have not yet been deployed using traditional statistical methods, potentially due to the limitations of the paradigm and the need to leverage large amounts of digital data. This review will suggest that a machine learning approach could address these challenges and is designed to introduce new readers to the background, methods, and results in the field. A rationale is first introduced followed by an outline of fundamental elements of machine learning approaches. To provide an overview of the use of the techniques in child and adolescent literature, a scoping review of broad trends is then presented. Selected studies are also highlighted in order to draw attention to research areas that are closest to translation and studies that exhibit a high degree of experimental innovation. Limitations to the research, and machine learning approaches generally, are outlined in the penultimate section highlighting issues related to sample sizes, validation, clinical utility, and ethical challenges. Finally, future directions are discussed that could enhance the possibility of clinical implementation and address specific questions relevant to the child and adolescent psychiatry. The review gives a broad overview of the machine learning paradigm in order to highlight the benefits of a shift in perspective towards practically oriented statistical solutions that aim to improve clinical care of children and adolescents.

On the reproducibility of enzyme reactions and kinetic modelling

Enzyme reactions are highly dependent on reaction conditions. To ensure reproducibility of enzyme reaction parameters, experiments need to be carefully designed and kinetic modeling meticulously executed. Furthermore, to enable quality control of enzyme reaction parameters, the experimental conditions, the modeling process as well as the raw data need to be reported comprehensively. By taking these steps, enzyme reaction parameters can be open and FAIR (findable, accessible, interoperable, re-usable) as well as repeatable, replicable and reproducible. This review discusses these requirements and provides a practical guide to designing initial rate experiments for the determination of enzyme reaction parameters and gives an open, FAIR and re-editable example of the kinetic modeling of an enzyme reaction. Both the guide and example are scripted with Python in Jupyter Notebooks and are publicly available (https://fairdomhub.org/investigations/483/snapshots/1). Finally, the prerequisites of automated data analysis and machine learning algorithms are briefly discussed to provide further motivation for the comprehensive, open and FAIR reporting of enzyme reaction parameters.

The Role of Artificial Intelligence in Early Cancer Diagnosis

Improving the proportion of patients diagnosed with early-stage cancer is a key priority of the World Health Organisation. In many tumour groups, screening programmes have led to improvements in survival, but patient selection and risk stratification are key challenges. In addition, there are concerns about limited diagnostic workforces, particularly in light of the COVID-19 pandemic, placing a strain on pathology and radiology services. In this review, we discuss how artificial intelligence algorithms could assist clinicians in (1) screening asymptomatic patients at risk of cancer, (2) investigating and triaging symptomatic patients, and (3) more effectively diagnosing cancer recurrence. We provide an overview of the main artificial intelligence approaches, including historical models such as logistic regression, as well as deep learning and neural networks, and highlight their early diagnosis applications. Many data types are suitable for computational analysis, including electronic healthcare records, diagnostic images, pathology slides and peripheral blood, and we provide examples of how these data can be utilised to diagnose cancer. We also discuss the potential clinical implications for artificial intelligence algorithms, including an overview of models currently used in clinical practice. Finally, we discuss the potential limitations and pitfalls, including ethical concerns, resource demands, data security and reporting standards.

Machine learning to predict progression of non-alcoholic fatty liver to non-alcoholic steatohepatitis or fibrosis

Background

Non-alcoholic fatty liver (NAFL) can progress to the severe subtype non-alcoholic steatohepatitis (NASH) and/or fibrosis, which are associated with increased morbidity, mortality, and healthcare costs. Current machine learning studies detect NASH; however, this study is unique in predicting the progression of NAFL patients to NASH or fibrosis.

Aim

To utilize clinical information from NAFL-diagnosed patients to predict the likelihood of progression to NASH or fibrosis.

Methods

Data were collected from electronic health records of patients receiving a first-time NAFL diagnosis. A gradient boosted machine learning algorithm (XGBoost) as well as logistic regression (LR) and multi-layer perceptron (MLP) models were developed. A five-fold cross-validation grid search was utilized for hyperparameter optimization of variables, including maximum tree depth, learning rate, and number of estimators. Predictions of patients likely to progress to NASH or fibrosis within 4 years of initial NAFL diagnosis were made using demographic features, vital signs, and laboratory measurements.

Results

The XGBoost algorithm achieved area under the receiver operating characteristic (AUROC) values of 0.79 for prediction of progression to NASH and 0.87 for fibrosis on both hold-out and external validation test sets. The XGBoost algorithm outperformed the LR and MLP models for both NASH and fibrosis prediction on all metrics.

Conclusion

It is possible to accurately identify newly diagnosed NAFL patients at high risk of progression to NASH or fibrosis. Early identification of these patients may allow for increased clinical monitoring, more aggressive preventative measures to slow the progression of NAFL and fibrosis, and efficient clinical trial enrollment.

Sources of bias in artificial intelligence that perpetuate healthcare disparities-A global review

BACKGROUND

While artificial intelligence (AI) offers possibilities of advanced clinical prediction and decision-making in healthcare, models trained on relatively homogeneous datasets, and populations poorly-representative of underlying diversity, limits generalisability and risks biased AI-based decisions. Here, we describe the landscape of AI in clinical medicine to delineate population and data-source disparities.

METHODS

We performed a scoping review of clinical papers published in PubMed in 2019 using AI techniques. We assessed differences in dataset country source, clinical specialty, and author nationality, sex, and expertise. A manually tagged subsample of PubMed articles was used to train a model, leveraging transfer-learning techniques (building upon an existing BioBERT model) to predict eligibility for inclusion (original, human, clinical AI literature). Of all eligible articles, database country source and clinical specialty were manually labelled. A BioBERT-based model predicted first/last author expertise. Author nationality was determined using corresponding affiliated institution information using Entrez Direct. And first/last author sex was evaluated using the Gendarize.io API.

RESULTS

Our search yielded 30,576 articles, of which 7,314 (23.9%) were eligible for further analysis. Most databases came from the US (40.8%) and China (13.7%). Radiology was the most represented clinical specialty (40.4%), followed by pathology (9.1%). Authors were primarily from either China (24.0%) or the US (18.4%). First and last authors were predominately data experts (i.e., statisticians) (59.6% and 53.9% respectively) rather than clinicians. And the majority of first/last authors were male (74.1%).

INTERPRETATION

U.S. and Chinese datasets and authors were disproportionately overrepresented in clinical AI, and almost all of the top 10 databases and author nationalities were from high income countries (HICs). AI techniques were most commonly employed for image-rich specialties, and authors were predominantly male, with non-clinical backgrounds. Development of technological infrastructure in data-poor regions, and diligence in external validation and model re-calibration prior to clinical implementation in the short-term, are crucial in ensuring clinical AI is meaningful for broader populations, and to avoid perpetuating global health inequity.

Cardiovascular Disease Screening in Women: Leveraging Artificial Intelligence and Digital Tools

Cardiovascular disease remains the leading cause of death in women. Given accumulating evidence on sex- and gender-based differences in cardiovascular disease development and outcomes, the need for more effective approaches to screening for risk factors and phenotypes in women is ever urgent. Public health surveillance and health care delivery systems now continuously generate massive amounts of data that could be leveraged to enable both screening of cardiovascular risk and implementation of tailored preventive interventions across a woman's life span. However, health care providers, clinical guidelines committees, and health policy experts are not yet sufficiently equipped to optimize the collection of data on women, use or interpret these data, or develop approaches to targeting interventions. Therefore, we provide a broad overview of the key opportunities for cardiovascular screening in women while highlighting the potential applications of artificial intelligence along with digital technologies and tools.

Does sex matter? Analysis of sex-related differences in the diagnostic performance of a market-approved convolutional neural network for skin cancer detection

BACKGROUND

Advances in biomedical artificial intelligence may introduce or perpetuate sex and gender discriminations. Convolutional neural networks (CNN) have proven a dermatologist-level performance in image classification tasks but have not been assessed for sex and gender biases that may affect training data and diagnostic performance. In this study, we investigated sex-related imbalances in training data and diagnostic performance of a market-approved CNN for skin cancer classification (Moleanalyzer Pro®, Fotofinder Systems GmbH, Bad Birnbach, Germany).

METHODS

We screened open-access dermoscopic image repositories widely used for CNN training for distribution of sex. Moreover, the sex-related diagnostic performance of the market-approved CNN was tested in 1549 dermoscopic images stratified by sex (female n = 773; male n = 776).

RESULTS

Most open-access repositories showed a marked under-representation of images originating from female (40%) versus male (60%) patients. Despite these imbalances and well-known sex-related differences in skin anatomy or skin-directed behaviour, the tested CNN achieved a comparable sensitivity of 87.0% [80.9%-91.3%] versus 87.1% [81.1%-91.4%], specificity of 98.7% [97.4%-99.3%] versus 96.9% [95.2%-98.0%] and ROC-AUC of 0.984 [0.975-0.993] versus 0.979 [0.969-0.988] in dermoscopic images of female versus male origin, respectively. In the sample at hand, sex-related differences in ROC-AUCs were not statistically significant in the per-image analysis nor in an additional per-individual analysis (p ≥ 0.59).

CONCLUSION

Design and training of artificial intelligence algorithms for medical applications should generally acknowledge sex and gender dimensions. Despite sex-related imbalances in open-access training data, the diagnostic performance of the tested CNN showed no sex-related bias in the classification of skin lesions.

Artificial intelligence for mechanical ventilation: systematic review of design, reporting standards, and bias

BACKGROUND

Artificial intelligence (AI) has the potential to personalise mechanical ventilation strategies for patients with respiratory failure. However, current methodological deficiencies could limit clinical impact. We identified common limitations and propose potential solutions to facilitate translation of AI to mechanical ventilation of patients.

METHODS

A systematic review was conducted in MEDLINE, Embase, and PubMed Central to February 2021. Studies investigating the application of AI to patients undergoing mechanical ventilation were included. Algorithm design and adherence to reporting standards were assessed with a rubric combining published guidelines, satisfying the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis [TRIPOD] statement. Risk of bias was assessed by using the Prediction model Risk Of Bias ASsessment Tool (PROBAST), and correspondence with authors to assess data and code availability.

RESULTS

Our search identified 1,342 studies, of which 95 were included: 84 had single-centre, retrospective study design, with only one randomised controlled trial. Access to data sets and code was severely limited (unavailable in 85% and 87% of studies, respectively). On request, data and code were made available from 12 and 10 authors, respectively, from a list of 54 studies published in the last 5 yr. Ethnicity was frequently under-reported 18/95 (19%), as was model calibration 17/95 (18%). The risk of bias was high in 89% (85/95) of the studies, especially because of analysis bias.

CONCLUSIONS

Development of algorithms should involve prospective and external validation, with greater code and data availability to improve confidence in and translation of this promising approach.

TRIAL REGISTRATION NUMBER

PROSPERO - CRD42021225918.

AI in health and medicine

Artificial intelligence (AI) is poised to broadly reshape medicine, potentially improving the experiences of both clinicians and patients. We discuss key findings from a 2-year weekly effort to track and share key developments in medical AI. We cover prospective studies and advances in medical image analysis, which have reduced the gap between research and deployment. We also address several promising avenues for novel medical AI research, including non-image data sources, unconventional problem formulations and human-AI collaboration. Finally, we consider serious technical and ethical challenges in issues spanning from data scarcity to racial bias. As these challenges are addressed, AI's potential may be realized, making healthcare more accurate, efficient and accessible for patients worldwide.

Racism and Cardiology: A Global Call to Action

Racism and racial bias influence the lives and cardiovascular health of minority individuals. The fact that minority groups tend to have a higher burden of cardiovascular disease risk factors is often a result of racist policies that restrict opportunities to live in healthy neighbourhoods and have access to high-quality education and healthcare. The fact that minorities tend to have the worst outcomes when cardiovascular disease develops is often a result of institutional or individual racial bias encountered when they interact with the healthcare system. In this review, we discuss bias, discrimination, and structural racism from the viewpoints of cardiologists in Canada, the United Kingdom, and the US, and how racial bias impacts cardiovascular care. Finally, we discuss proposals to mitigate the impact of racism in our specialty.

Searching for New Technology Acceptance Model under Social Context: Analyzing the Determinants of Acceptance of Intelligent Information Technology in Digital Transformation and Implications for the Requisites of Digital Sustainability

Intelligent information technology (IIT) based on AI and intelligent network communication technology is rapidly changing the social structure and the personal lives. However, IIT acceptancefrom various perspectives still requires extensive research. The research question in this paper examines how five factors—psychological, technological, resource, risk perception, and value factors—influence IIT acceptance. Based on an analysis of survey data, it was first found that the acceptance rate of IIT itself was generally very high. Second, in terms of IIT acceptance, among twenty-five predictors, voluntariness (+), positive image of technology (+), performance expectancy (+), relative advantage (+), radical innovation (+), and experience of use (+) were found to have significant effects on the IIT acceptance. Third, in addition to technological factors, psychological factors and risk perception factors also played an important role in individuals’ decisions regarding IIT acceptance.

The Impact of Delayed Symptomatic Treatment Implementation in the Intensive Care Unit

We estimated the harm related to medication delivery delays across 12,474 medication administration instances in an intensive care unit using retrospective data in a large urban academic medical center between 2012 and 2015. We leveraged an instrumental variables (IV) approach that addresses unobserved confounds in this setting. We focused on nurse shift changes as disruptors of timely medication (vasodilators, antipyretics, and bronchodilators) delivery to estimate the impact of delay. The average delay around a nurse shift change was 60.8 min (p < 0.001) for antipyretics, 39.5 min (p < 0.001) for bronchodilators, and 57.1 min (p < 0.001) for vasodilators. This delay can increase the odds of developing a fever by 32.94%, tachypnea by 79.5%, and hypertension by 134%, respectively. Compared to estimates generated by a naïve regression approach, our IV estimates tend to be higher, suggesting the existence of a bias from providers prioritizing more critical patients.

Aligning Patient's Ideas of a Good Life with Medically Indicated Therapies in Geriatric Rehabilitation Using Smart Sensors

New technologies such as smart sensors improve rehabilitation processes and thereby increase older adults’ capabilities to participate in social life, leading to direct physical and mental health benefits. Wearable smart sensors for home use have the additional advantage of monitoring day-to-day activities and thereby identifying rehabilitation progress and needs. However, identifying and selecting rehabilitation priorities is ethically challenging because physicians, therapists, and caregivers may impose their own personal values leading to paternalism. Therefore, we develop a discussion template consisting of a series of adaptable questions for the patient–physician encounter based on the capability approach. The goal is to improve geriatric rehabilitation and thereby increase participation in social life and well-being. To achieve this goal, we first analyzed what is considered important for participation on basis of the capability approach, human rights, and ethics of care. Second, we conducted an ethical analysis of each of the four identified dimensions of participation: political, economic, socio-cultural, and care. To improve compliance with rehabilitation measures, health professionals must align rehabilitation measures in an open dialogue with the patient’s aspiration for participation in each dimension. A discussion template based on the capability approach allows for a proactive approach in patient information and stimulates a critical assessment of treatment alternatives while reducing the risk of imposing personal values.

Artificial Intelligence: A Shifting Paradigm in Cardio-Cerebrovascular Medicine

The future of healthcare is an organic blend of technology, innovation, and human connection. As artificial intelligence (AI) is gradually becoming a go-to technology in healthcare to improve efficiency and outcomes, we must understand our limitations. We should realize that our goal is not only to provide faster and more efficient care, but also to deliver an integrated solution to ensure that the care is fair and not biased to a group of sub-population. In this context, the field of cardio-cerebrovascular diseases, which encompasses a wide range of conditions-from heart failure to stroke-has made some advances to provide assistive tools to care providers. This article aimed to provide an overall thematic review of recent development focusing on various AI applications in cardio-cerebrovascular diseases to identify gaps and potential areas of improvement. If well designed, technological engines have the potential to improve healthcare access and equitability while reducing overall costs, diagnostic errors, and disparity in a system that affects patients and providers and strives for efficiency.

Regulating artificial-intelligence applications to achieve the sustainable development goals

Artificial intelligence is producing a revolution with increasing impacts on the people, planet, and prosperity. This perspective illustrates some of the AI applications that can accelerate the achievement of the United Nations Sustainable Development Goals (SDGs) and highlights some of the considerations that could hinder the efforts towards them. In this context, we strongly support the development of an 18thSDG on digital technologies. This emphasizes the importance of establishing standard AI guidelines and regulations for the beneficial applications of AI. Such regulations should focus on concrete applications of AI, rather than generally on AI technology, to facilitate both AI development and enforceability of legal implications.

A healthy debate: Exploring the views of medical doctors on the ethics of artificial intelligence

Artificial Intelligence (AI) is moving towards the health space. It is generally acknowledged that, while there is great promise in the implementation of AI technologies in healthcare, it also raises important ethical issues. In this study we surveyed medical doctors based in The Netherlands, Portugal, and the U.S. from a diverse mix of medical specializations about the ethics surrounding Health AI. Four main perspectives have emerged from the data representing different views about this matter. The first perspective (AI is a helpful tool: Let physicians do what they were trained for) highlights the efficiency associated with automation, which will allow doctors to have the time to focus on expanding their medical knowledge and skills. The second perspective (Rules & Regulations are crucial: Private companies only think about money) shows strong distrust in private tech companies and emphasizes the need for regulatory oversight. The third perspective (Ethics is enough: Private companies can be trusted) puts more trust in private tech companies and maintains that ethics is sufficient to ground these corporations. And finally the fourth perspective (Explainable AI tools: Learning is necessary and inevitable) emphasizes the importance of explainability of AI tools in order to ensure that doctors are engaged in the technological progress. Each perspective provides valuable and often contrasting insights about ethical issues that should be operationalized and accounted for in the design and development of AI Health.

Preventing dataset shift from breaking machine-learning biomarkers

Machine learning brings the hope of finding new biomarkers extracted from cohorts with rich biomedical measurements. A good biomarker is one that gives reliable detection of the corresponding condition. However, biomarkers are often extracted from a cohort that differs from the target population. Such a mismatch, known as a dataset shift, can undermine the application of the biomarker to new individuals. Dataset shifts are frequent in biomedical research, e.g.,  because of recruitment biases. When a dataset shift occurs, standard machine-learning techniques do not suffice to extract and validate biomarkers. This article provides an overview of when and how dataset shifts break machine-learning-extracted biomarkers, as well as detection and correction strategies.

How to Design a Relevant Corpus for Sleepiness Detection Through Voice?

This article presents research on the detection of pathologies affecting speech through automatic analysis. Voice processing has indeed been used for evaluating several diseases such as Parkinson, Alzheimer, or depression. If some studies present results that seem sufficient for clinical applications, this is not the case for the detection of sleepiness. Even two international challenges and the recent advent of deep learning techniques have still not managed to change this situation. This article explores the hypothesis that the observed average performances of automatic processing find their cause in the design of the corpora. To this aim, we first discuss and refine the concept of sleepiness related to the ground-truth labels. Second, we present an in-depth study of four corpora, bringing to light the methodological choices that have been made and the underlying biases they may have induced. Finally, in light of this information, we propose guidelines for the design of new corpora.

Training data distribution significantly impacts the estimation of tissue microstructure with machine learning

PURPOSE

Supervised machine learning (ML) provides a compelling alternative to traditional model fitting for parameter mapping in quantitative MRI. The aim of this work is to demonstrate and quantify the effect of different training data distributions on the accuracy and precision of parameter estimates when supervised ML is used for fitting.

METHODS

We fit a two- and three-compartment biophysical model to diffusion measurements from in-vivo human brain, as well as simulated diffusion data, using both traditional model fitting and supervised ML. For supervised ML, we train several artificial neural networks, as well as random forest regressors, on different distributions of ground truth parameters. We compare the accuracy and precision of parameter estimates obtained from the different estimation approaches using synthetic test data.

RESULTS

When the distribution of parameter combinations in the training set matches those observed in healthy human data sets, we observe high precision, but inaccurate estimates for atypical parameter combinations. In contrast, when training data is sampled uniformly from the entire plausible parameter space, estimates tend to be more accurate for atypical parameter combinations but may have lower precision for typical parameter combinations.

CONCLUSION

This work highlights that estimation of model parameters using supervised ML depends strongly on the training-set distribution. We show that high precision obtained using ML may mask strong bias, and visual assessment of the parameter maps is not sufficient for evaluating the quality of the estimates.

Predicting the gender of individuals with tinnitus based on daily life data of the TrackYourTinnitus mHealth platform

Tinnitus is an auditory phantom perception in the absence of an external sound stimulation. People with tinnitus often report severe constraints in their daily life. Interestingly, indications exist on gender differences between women and men both in the symptom profile as well as in the response to specific tinnitus treatments. In this paper, data of the TrackYourTinnitus platform (TYT) were analyzed to investigate whether the gender of users can be predicted. In general, the TYT mobile Health crowdsensing platform was developed to demystify the daily and momentary variations of tinnitus symptoms over time. The goal of the presented investigation is a better understanding of gender-related differences in the symptom profiles of users from TYT. Based on two questionnaires of TYT, four machine learning based classifiers were trained and analyzed. With respect to the provided daily answers, the gender of TYT users can be predicted with an accuracy of 81.7%. In this context, worries, difficulties in concentration, and irritability towards the family are the three most important characteristics for predicting the gender. Note that in contrast to existing studies on TYT, daily answers to the worst symptom question were firstly investigated in more detail. It was found that results of this question significantly contribute to the prediction of the gender of TYT users. Overall, our findings indicate gender-related differences in tinnitus and tinnitus-related symptoms. Based on evidence that gender impacts the development of tinnitus, the gathered insights can be considered relevant and justify further investigations in this direction.

Application of Artificial Intelligence in Community-Based Primary Health Care: Systematic Scoping Review and Critical Appraisal

BACKGROUND

Research on the integration of artificial intelligence (AI) into community-based primary health care (CBPHC) has highlighted several advantages and disadvantages in practice regarding, for example, facilitating diagnosis and disease management, as well as doubts concerning the unintended harmful effects of this integration. However, there is a lack of evidence about a comprehensive knowledge synthesis that could shed light on AI systems tested or implemented in CBPHC.

OBJECTIVE

We intended to identify and evaluate published studies that have tested or implemented AI in CBPHC settings.

METHODS

We conducted a systematic scoping review informed by an earlier study and the Joanna Briggs Institute (JBI) scoping review framework and reported the findings according to PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analysis-Scoping Reviews) reporting guidelines. An information specialist performed a comprehensive search from the date of inception until February 2020, in seven bibliographic databases: Cochrane Library, MEDLINE, EMBASE, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), ScienceDirect, and IEEE Xplore. The selected studies considered all populations who provide and receive care in CBPHC settings, AI interventions that had been implemented, tested, or both, and assessed outcomes related to patients, health care providers, or CBPHC systems. Risk of bias was assessed using the Prediction Model Risk of Bias Assessment Tool (PROBAST). Two authors independently screened the titles and abstracts of the identified records, read the selected full texts, and extracted data from the included studies using a validated extraction form. Disagreements were resolved by consensus, and if this was not possible, the opinion of a third reviewer was sought. A third reviewer also validated all the extracted data.

RESULTS

We retrieved 22,113 documents. After the removal of duplicates, 16,870 documents were screened, and 90 peer-reviewed publications met our inclusion criteria. Machine learning (ML) (41/90, 45%), natural language processing (NLP) (24/90, 27%), and expert systems (17/90, 19%) were the most commonly studied AI interventions. These were primarily implemented for diagnosis, detection, or surveillance purposes. Neural networks (ie, convolutional neural networks and abductive networks) demonstrated the highest accuracy, considering the given database for the given clinical task. The risk of bias in diagnosis or prognosis studies was the lowest in the participant category (4/49, 4%) and the highest in the outcome category (22/49, 45%).

CONCLUSIONS

We observed variabilities in reporting the participants, types of AI methods, analyses, and outcomes, and highlighted the large gap in the effective development and implementation of AI in CBPHC. Further studies are needed to efficiently guide the development and implementation of AI interventions in CBPHC settings.

Artificial Intelligence for Mental Health Care: Clinical Applications, Barriers, Facilitators, and Artificial Wisdom

Artificial intelligence (AI) is increasingly employed in health care fields such as oncology, radiology, and dermatology. However, the use of AI in mental health care and neurobiological research has been modest. Given the high morbidity and mortality in people with psychiatric disorders, coupled with a worsening shortage of mental health care providers, there is an urgent need for AI to help identify high-risk individuals and provide interventions to prevent and treat mental illnesses. While published research on AI in neuropsychiatry is rather limited, there is a growing number of successful examples of AI's use with electronic health records, brain imaging, sensor-based monitoring systems, and social media platforms to predict, classify, or subgroup mental illnesses as well as problems such as suicidality. This article is the product of a study group held at the American College of Neuropsychopharmacology conference in 2019. It provides an overview of AI approaches in mental health care, seeking to help with clinical diagnosis, prognosis, and treatment, as well as clinical and technological challenges, focusing on multiple illustrative publications. Although AI could help redefine mental illnesses more objectively, identify them at a prodromal stage, personalize treatments, and empower patients in their own care, it must address issues of bias, privacy, transparency, and other ethical concerns. These aspirations reflect human wisdom, which is more strongly associated than intelligence with individual and societal well-being. Thus, the future AI or artificial wisdom could provide technology that enables more compassionate and ethically sound care to diverse groups of people.

Sex and Gender Aspects in Vascular Ageing - Focus on Epidemiology, Pathophysiology, and Outcomes

Sex and gender are important modifiers of cardiovascular system physiology, pathophysiology, and disease development. The atherosclerosis process, together with the progressive loss of arterial elasticity with age, is a major factor influencing the development of overt cardiovascular, renal, and cerebrovascular disease. While differences between women and men in epidemiology and pathophysiology of vascular ageing are increasingly reported, sex-disaggregated data are still scarcely available for prospective studies. A better knowledge of sex differences in physiological ageing as well as in disease-related changes in vascular ageing trajectories is crucial to avoid misdiagnosis and mistreatment. This review presents key concepts and knowledge gaps identified in vascular ageing due to gonadal function, vascular physiology, pathophysiology, psychosocial factors, pregnancy, and prognostic relevance. Gender roles determine the effectiveness of any cardiovascular preventive strategy and acceptance for non-invasive or invasive diagnostics and therapeutics. Gender differences in health behaviour, also due to sociocultural norms conditioned by society, contribute to behaviours that may lead to premature arterial vascular ageing. These include differences in risk behaviours like smoking, diet, exercise, and in stress, but also conditions such as housing, noise pollution, poverty, disability, and any kind of stigmatisation. The VascAgeNet Gender Expert Group aims to advance the use of non-invasive vascular ageing measures in routine clinical settings by providing facts to fill in the gaps concerning sex and gender differences at each step of this process, and to search for solutions.

Let's talk about sex: Differences in drug therapy in males and females

Professor Henry Higgins in My Fair Lady said, 'Why can't a woman be more like a man?' Perhaps unintended, such narration extends to the reality of current drug development. A clear sex-gap exists in pharmaceutical research spanning from preclinical studies, clinical trials to post-marketing surveillance with a bias towards males. Consequently, women experience adverse drug reactions from approved drug products more often than men. Distinct differences in pharmaceutical response across drug classes and the lack of understanding of disease pathophysiology also exists between the sexes, often leading to suboptimal drug therapy in women. This review explores the influence of sex as a biological variable in drug delivery, pharmacokinetic response and overall efficacy in the context of pharmaceutical research and practice in the clinic. Prospective recommendations are provided to guide researchers towards the consideration of sex differences in methodologies and analyses. The promotion of disaggregating data according to sex to strengthen scientific rigour, encouraging innovation through the personalisation of medicines and adopting machine learning algorithms is vital for optimised drug development in the sexes and population health equity.

The future of bone regeneration: integrating AI into tissue engineering

Tissue engineering is a branch of regenerative medicine that harnesses biomaterial and stem cell research to utilise the body's natural healing responses to regenerate tissue and organs. There remain many unanswered questions in tissue engineering, with optimal biomaterial designs still to be developed and a lack of adequate stem cell knowledge limiting successful application. Advances in artificial intelligence (AI), and deep learning specifically, offer the potential to improve both scientific understanding and clinical outcomes in regenerative medicine. With enhanced perception of how to integrate artificial intelligence into current research and clinical practice, AI offers an invaluable tool to improve patient outcome.

Learning from machine learning: prediction of age-related athletic performance decline trajectories

Factors that determine individual age-related decline rates in physical performance are poorly understood and prediction poses a challenge. Linear and quadratic regression models are usually applied, but often show high prediction errors for individual athletes. Machine learning approaches may deliver more accurate predictions and help to identify factors that determine performance decline rates. We hypothesized that it is possible to predict the performance development of a master athlete from a single measurement, that prediction by a machine learning approach is superior to prediction by the average decline curve or an individually shifted decline curve, and that athletes with a higher starting performance show a slower performance decline than those with a lower performance. The machine learning approach was implemented using a multilayer neuronal network. Results showed that performance prediction from a single measurement is possible and that the prediction by a machine learning approach was superior to the other models. The estimated performance decline rate was highest in athletes with a high starting performance and a low starting age, as well as in those with a low starting performance and high starting age, while the lowest decline rate was found for athletes with a high starting performance and a high starting age. Machine learning was superior and predicted trajectories with significantly lower prediction errors compared to conventional approaches. New insights into factors determining decline trajectories were identified by visualization of the model outputs. Machine learning models may be useful in revealing unknown factors that determine the age-related performance decline.

Towards gender equity in artificial intelligence and machine learning applications in dermatology

There has been increased excitement around the use of machine learning (ML) and artificial intelligence (AI) in dermatology for the diagnosis of skin cancers and assessment of other dermatologic conditions. As these technologies continue to expand, it is essential to ensure they do not create or widen sex- and gender-based disparities in care. While desirable bias may result from the explicit inclusion of sex or gender in diagnostic criteria of diseases with gender-based differences, undesirable biases can result from usage of datasets with an underrepresentation of certain groups. We believe that sex and gender differences should be taken into consideration in ML/AI algorithms in dermatology because there are important differences in the epidemiology and clinical presentation of dermatologic conditions including skin cancers, sex-specific cancers, and autoimmune conditions. We present recommendations for ensuring sex and gender equity in the development of ML/AI tools in dermatology to increase desirable bias and avoid undesirable bias.

Ensuring patient and public involvement in the transition to AI-assisted mental health care: A systematic scoping review and agenda for design justice

Background

Machine‐learning algorithms and big data analytics, popularly known as ‘artificial intelligence’ (AI), are being developed and taken up globally. Patient and public involvement (PPI) in the transition to AI‐assisted health care is essential for design justice based on diverse patient needs.

Objective

To inform the future development of PPI in AI‐assisted health care by exploring public engagement in the conceptualization, design, development, testing, implementation, use and evaluation of AI technologies for mental health.

Methods

Systematic scoping review drawing on design justice principles, and (i) structured searches of Web of Science (all databases) and Ovid (MEDLINE, PsycINFO, Global Health and Embase); (ii) handsearching (reference and citation tracking); (iii) grey literature; and (iv) inductive thematic analysis, tested at a workshop with health researchers.

Results

The review identified 144 articles that met inclusion criteria. Three main themes reflect the challenges and opportunities associated with PPI in AI‐assisted mental health care: (a) applications of AI technologies in mental health care; (b) ethics of public engagement in AI‐assisted care; and (c) public engagement in the planning, development, implementation, evaluation and diffusion of AI technologies.

Conclusion

The new data‐rich health landscape creates multiple ethical issues and opportunities for the development of PPI in relation to AI technologies. Further research is needed to understand effective modes of public engagement in the context of AI technologies, to examine pressing ethical and safety issues and to develop new methods of PPI at every stage, from concept design to the final review of technology in practice. Principles of design justice can guide this agenda.

Will Big Data and personalized medicine do the gender dimension justice?

Over the last decade, humans have produced each year as much data as were produced throughout the entire history of humankind. These data, in quantities that exceed current analytical capabilities, have been described as “the new oil,” an incomparable source of value. This is true for healthcare, as well. Conducting analyses of large, diverse, medical datasets promises the detection of previously unnoticed clinical correlations and new diagnostic or even therapeutic possibilities. However, using Big Data poses several problems, especially in terms of representing the uniqueness of each patient and expressing the differences between individuals, primarily gender and sex differences. The first two sections of the paper provide a definition of “Big Data” and illustrate the uses of Big Data in medicine. Subsequently, the paper explores the struggle to represent exhaustively the uniqueness of the patient through Big Data is highlighted prior to a deeper investigation of the digital representation of gender in personalized medicine. The final part of the paper put forward a series of recommendations for better approaching the complexity of gender in medical and clinical research involving Big Data for the creation or enhancement of personalized medicine services.

Using Unsupervised Machine Learning to Identify Age- and Sex-Independent Severity Subgroups Among Patients with COVID-19: Observational Longitudinal Study

Background

Early detection and intervention are the key factors for improving outcomes in patients with COVID-19.

Objective

The objective of this observational longitudinal study was to identify nonoverlapping severity subgroups (ie, clusters) among patients with COVID-19, based exclusively on clinical data and standard laboratory tests obtained during patient assessment in the emergency department.

Methods

We applied unsupervised machine learning to a data set of 853 patients with COVID-19 from the HM group of hospitals (HM Hospitales) in Madrid, Spain. Age and sex were not considered while building the clusters, as these variables could introduce biases in machine learning algorithms and raise ethical implications or enable discrimination in triage protocols.

Results

From 850 clinical and laboratory variables, four tests—the serum levels of aspartate transaminase (AST), lactate dehydrogenase (LDH), C-reactive protein (CRP), and the number of neutrophils—were enough to segregate the entire patient pool into three separate clusters. Further, the percentage of monocytes and lymphocytes and the levels of alanine transaminase (ALT) distinguished cluster 3 patients from the other two clusters. The highest proportion of deceased patients; the highest levels of AST, ALT, LDH, and CRP; the highest number of neutrophils; and the lowest percentages of monocytes and lymphocytes characterized cluster 1. Cluster 2 included a lower proportion of deceased patients and intermediate levels of the previous laboratory tests. The lowest proportion of deceased patients; the lowest levels of AST, ALT, LDH, and CRP; the lowest number of neutrophils; and the highest percentages of monocytes and lymphocytes characterized cluster 3.

Conclusions

A few standard laboratory tests, deemed available in all emergency departments, have shown good discriminative power for the characterization of severity subgroups among patients with COVID-19.

Gender blindness: On health and welfare technology, AI and gender equality in community care

Digital health and welfare technologies and artificial intelligence are proposed to revolutionise healthcare systems around the world by enabling new models of care. Digital health and welfare technologies enable remote monitoring and treatments, and artificial intelligence is proposed as a means of prediction instead of reaction to individuals' health and as an enabler of proactive care and rehabilitation. The digital transformation not only affects hospital and primary care but also how the community meets older people's needs. Community care is often provided by informal and formal care-givers, most of whom are women. Gender equality is at the heart of many national strategies, but do all genders have equal rights, responsibilities and opportunities when it comes to community care and its digital transformation? The digital transformation of community care is entangled with how care is provided to older people and the working conditions of community-care professionals. Current and, even more so, future community-care systems are and will be partly constituted by networks of technological artefacts. These health and welfare technological artefacts and the discourse surrounding them mediate and constitute social relations and community care. This article looks into how health and welfare technology and artificial intelligence-based devices and systems mediate and constitute gender relations in community care and presents an argument for reflexivity, embodiment, pluralism, participation and ecology as an alternative strategy to treating community care as one-size-fit-all and being blind to gender-related issues.