1
|
Adler DA, Stamatis CA, Meyerhoff J, Mohr DC, Wang F, Aranovich GJ, Sen S, Choudhury T. Measuring algorithmic bias to analyze the reliability of AI tools that predict depression risk using smartphone sensed-behavioral data. NPJ MENTAL HEALTH RESEARCH 2024; 3:17. [PMID: 38649446 PMCID: PMC11035598 DOI: 10.1038/s44184-024-00057-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 02/07/2024] [Indexed: 04/25/2024]
Abstract
AI tools intend to transform mental healthcare by providing remote estimates of depression risk using behavioral data collected by sensors embedded in smartphones. While these tools accurately predict elevated depression symptoms in small, homogenous populations, recent studies show that these tools are less accurate in larger, more diverse populations. In this work, we show that accuracy is reduced because sensed-behaviors are unreliable predictors of depression across individuals: sensed-behaviors that predict depression risk are inconsistent across demographic and socioeconomic subgroups. We first identified subgroups where a developed AI tool underperformed by measuring algorithmic bias, where subgroups with depression were incorrectly predicted to be at lower risk than healthier subgroups. We then found inconsistencies between sensed-behaviors predictive of depression across these subgroups. Our findings suggest that researchers developing AI tools predicting mental health from sensed-behaviors should think critically about the generalizability of these tools, and consider tailored solutions for targeted populations.
Collapse
Affiliation(s)
- Daniel A Adler
- Cornell Tech, Information Science, 2 W Loop Rd, New York, NY, 10044, USA.
| | - Caitlin A Stamatis
- Northwestern University Feinberg School of Medicine, Center for Behavioral Intervention Technologies, Chicago, IL, 60611, USA
| | - Jonah Meyerhoff
- Northwestern University Feinberg School of Medicine, Center for Behavioral Intervention Technologies, Chicago, IL, 60611, USA
| | - David C Mohr
- Northwestern University Feinberg School of Medicine, Center for Behavioral Intervention Technologies, Chicago, IL, 60611, USA
| | - Fei Wang
- Weill Cornell Medicine, Population Health Sciences, New York, NY, 10065, USA
| | | | - Srijan Sen
- Michigan Medicine, Department of Psychiatry, Ann Arbor, MI, 48109, USA
| | - Tanzeem Choudhury
- Cornell Tech, Information Science, 2 W Loop Rd, New York, NY, 10044, USA
| |
Collapse
|
2
|
Adler DA, Stamatis CA, Meyerhoff J, Mohr DC, Wang F, Aranovich GJ, Sen S, Choudhury T. Measuring algorithmic bias to analyze the reliability of AI tools that predict depression risk using smartphone sensed-behavioral data. RESEARCH SQUARE 2024:rs.3.rs-3044613. [PMID: 38746448 PMCID: PMC11092819 DOI: 10.21203/rs.3.rs-3044613/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
AI tools intend to transform mental healthcare by providing remote estimates of depression risk using behavioral data collected by sensors embedded in smartphones. While these tools accurately predict elevated symptoms in small, homogenous populations, recent studies show that these tools are less accurate in larger, more diverse populations. In this work, we show that accuracy is reduced because sensed-behaviors are unreliable predictors of depression across individuals; specifically the sensed-behaviors that predict depression risk are inconsistent across demographic and socioeconomic subgroups. We first identified subgroups where a developed AI tool underperformed by measuring algorithmic bias, where subgroups with depression were incorrectly predicted to be at lower risk than healthier subgroups. We then found inconsistencies between sensed-behaviors predictive of depression across these subgroups. Our findings suggest that researchers developing AI tools predicting mental health from behavior should think critically about the generalizability of these tools, and consider tailored solutions for targeted populations.
Collapse
|
3
|
Chan J, Ng DWL, Liao Q, Fielding R, Soong I, Chan KKL, Lee C, Ng AWY, Sze WK, Chan WL, Lee VHF, Lam WWT. Trajectories of sleep disturbance in cancer survivors during the first 2 years post-treatment. Sleep 2023; 46:zsad052. [PMID: 36861253 DOI: 10.1093/sleep/zsad052] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 01/13/2023] [Indexed: 03/03/2023] Open
Abstract
STUDY OBJECTIVES To examine the trajectories of sleep disturbance in cancer survivors during the first 2 years post-treatment and to investigate whether psychological, cognitive, and physical factors differentiate trajectories. METHODS A total of 623 Chinese cancer survivors of diverse cancer types participated in a 2-year-long prospective study after the completion of cancer treatment. Sleep disturbance was measured using Pittsburgh Sleep Quality Index at 3 (T2), 6 (T3), 12 (T4), 18 (T5), and 24 (T6) months after baseline (within 6-months post-treatment; T1). Latent growth mixture modeling identified distinctive sleep disturbance trajectories and tested if these longitudinal patterns were predicted by baseline psychological distress, attentional control, attentional bias and physical symptom distress and T2 cancer-related distress. Fully adjusted multinomial logistic regression then identified whether these factors differentiated trajectories. RESULTS Two distinct sleep disturbance trajectories were identified, namely stable good sleepers (69.7%) and persistent high sleep disturbance (30.3%). Compared to those in the stable good sleep group, patients in the persistent high sleep disturbance group were less likely to report avoidant (OR=0.49, 95% CI = 0.26-0.90), while more likely to report intrusive thoughts (OR = 1.76, 95% CI = 1.06-2.92) and cancer-related hyperarousal (OR = 3.37, 95% CI = 1.78-6.38). Higher depression scores also predicted persistent high sleep disturbance group membership (OR = 1.13, 95% CI = 1.03-1.25). Attentional bias, attentional control, anxiety, and physical symptom distress did not predict sleep trajectory membership. CONCLUSIONS One in three cancer survivors experienced persistent high sleep disturbance. Screening and managing depressive symptoms and cancer-related distress in early cancer rehabilitation may reduce risk of persistent sleep disturbance among cancer survivors.
Collapse
Affiliation(s)
- Julia Chan
- School of Public Health, Centre for Psycho-Oncology Research and Training, University of Hong Kong, Hong Kong, Hong Kong SAR, China
- Li Ka Shing Faculty of Medicine, Jockey Club Institute of Cancer Care, University of Hong Kong, Hong Kong, Hong Kong SAR, China
| | - Danielle Wing Lam Ng
- School of Public Health, Centre for Psycho-Oncology Research and Training, University of Hong Kong, Hong Kong, Hong Kong SAR, China
- Li Ka Shing Faculty of Medicine, Jockey Club Institute of Cancer Care, University of Hong Kong, Hong Kong, Hong Kong SAR, China
| | - Qiuyan Liao
- School of Public Health, Centre for Psycho-Oncology Research and Training, University of Hong Kong, Hong Kong, Hong Kong SAR, China
| | - Richard Fielding
- School of Public Health, Centre for Psycho-Oncology Research and Training, University of Hong Kong, Hong Kong, Hong Kong SAR, China
- Li Ka Shing Faculty of Medicine, Jockey Club Institute of Cancer Care, University of Hong Kong, Hong Kong, Hong Kong SAR, China
| | - Inda Soong
- Department of Clinical Oncology, Pamela Youde Nethersole Eastern Hospital, Hong Kong, Hong Kong SAR, China
| | - Karen Kar Loen Chan
- Department of Obstetrics and Gynaecology, University of Hong Kong, Hong Kong, Hong Kong SAR, China
| | - Conrad Lee
- Department of Clinical Oncology, Princess Margaret Hospital, Hong Kong, Hong Kong SAR, China
| | - Alice Wan Ying Ng
- Department of Clinical Oncology, Tuen Mun Hospital, Hong Kong, Hong Kong SAR, China
| | - Wing Kin Sze
- Department of Clinical Oncology, Tuen Mun Hospital, Hong Kong, Hong Kong SAR, China
| | - Wing Lok Chan
- Li Ka Shing Faculty of Medicine, Department of Clinical Oncology, University of Hong Kong, Hong Kong SAR, China
| | - Victor Ho Fun Lee
- Li Ka Shing Faculty of Medicine, Department of Clinical Oncology, University of Hong Kong, Hong Kong SAR, China
| | - Wendy Wing Tak Lam
- School of Public Health, Centre for Psycho-Oncology Research and Training, University of Hong Kong, Hong Kong, Hong Kong SAR, China
- Li Ka Shing Faculty of Medicine, Jockey Club Institute of Cancer Care, University of Hong Kong, Hong Kong, Hong Kong SAR, China
| |
Collapse
|
4
|
Riemann D. Sleep health. J Sleep Res 2022; 31:e13586. [PMID: 35506276 DOI: 10.1111/jsr.13586] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Affiliation(s)
- Dieter Riemann
- Department of Psychiatry and Psychotherapy, Medical Center, Faculty of Medicine, University of Freiburg, Freiburg, Germany
| |
Collapse
|