1
|
Aaronson A, Ashford MT, Jin C, Bride J, Decker J, DeNicola A, Turner RW, Conti C, Tank R, Truran D, Camacho MR, Fockler J, Flenniken D, Ulbricht A, Grill JD, Rabinovici G, Carrillo MC, Mackin RS, Weiner MW, Nosheny RL. Brain Health Registry Study Partner Portal: Novel infrastructure for digital, dyadic data collection. Alzheimers Dement 2024; 20:846-857. [PMID: 37797205 PMCID: PMC10916998 DOI: 10.1002/alz.13492] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Revised: 09/05/2023] [Accepted: 09/07/2023] [Indexed: 10/07/2023]
Abstract
BACKGROUND In Alzheimer's disease (AD) research, subjective reports of cognitive and functional decline from participant-study partner dyads is an efficient method of assessing cognitive impairment and clinical progression. METHODS Demographics and subjective cognitive/functional decline (Everyday Cognition Scale [ECog]) scores from dyads enrolled in the Brain Health Registry (BHR) Study Partner Portal were analyzed. Associations between dyad characteristics and both ECog scores and study engagement were investigated. RESULTS A total of 10,494 BHR participants (mean age = 66.9 ± 12.16 standard deviations, 67.4% female) have enrolled study partners (mean age = 64.3 ± 14.3 standard deviations, 49.3% female), including 8987 dyads with a participant 55 years of age or older. Older and more educated study partners were more likely to complete tasks and return for follow-up. Twenty-five percent to 27% of older adult participants had self and study partner-report ECog scores indicating a possible cognitive impairment. DISCUSSION The BHR Study Partner Portal is a unique digital tool for capturing dyadic data, with high impact applications in the clinical neuroscience and AD fields. Highlights The Brain Health Registry (BHR) Study Partner Portal is a novel, digital platform of >10,000 dyads. Collection of dyadic online subjective cognitive and functional data is feasible. The portal has good usability as evidenced by positive study partner feedback. The portal is a potential scalable strategy for cognitive impairment screening in older adults.
Collapse
Affiliation(s)
- Anna Aaronson
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Miriam T. Ashford
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Northern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Chengshi Jin
- Department of Epidemiology and BiostatisticsUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Jessica Bride
- Department of Clinical Research and LeadershipSchool of Medicine and Health SciencesThe George Washington UniversityWashingtonDCUSA
| | - Josephine Decker
- Department of Clinical Research and LeadershipSchool of Medicine and Health SciencesThe George Washington UniversityWashingtonDCUSA
| | - Aaron DeNicola
- Department of Clinical Research and LeadershipSchool of Medicine and Health SciencesThe George Washington UniversityWashingtonDCUSA
| | - Robert W. Turner
- Department of Clinical Research and LeadershipSchool of Medicine and Health SciencesThe George Washington UniversityWashingtonDCUSA
| | - Catherine Conti
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Northern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Rachana Tank
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of California San FranciscoSan FranciscoCaliforniaUSA
- Dementia Research CentreUCL Institute of NeurologyUniversity College LondonLondonUK
| | - Diana Truran
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Northern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Monica R. Camacho
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Northern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Juliet Fockler
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Derek Flenniken
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Northern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Aaron Ulbricht
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Joshua D. Grill
- Departments of Psychiatry & Human Behavior and Neurobiology & BehaviorInstitute for Memory Impairments and Neurological DisordersUniversity of California IrvineIrvineCaliforniaUSA
| | - Gil Rabinovici
- Department of Radiology and Biomedical ImagingUniversity of California San FranciscoSan FranciscoCaliforniaUSA
- Department of NeurologyUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | | | - R. Scott Mackin
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Psychiatry and Behavioral SciencesUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Michael W. Weiner
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of California San FranciscoSan FranciscoCaliforniaUSA
- Department of NeurologyUniversity of California San FranciscoSan FranciscoCaliforniaUSA
- Department of Psychiatry and Behavioral SciencesUniversity of California San FranciscoSan FranciscoCaliforniaUSA
- Department of MedicineUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Rachel L. Nosheny
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of California San FranciscoSan FranciscoCaliforniaUSA
- Department of Psychiatry and Behavioral SciencesUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| |
Collapse
|
2
|
Ashford MT, Aaronson A, Kwang W, Eichenbaum J, Gummadi S, Jin C, Cashdollar N, Thorp E, Wragg E, Zavitz KH, Cormack F, Banh T, Neuhaus JM, Ulbricht A, Camacho MR, Fockler J, Flenniken D, Truran D, Mackin RS, Weiner MW, Nosheny RL. Unsupervised Online Paired Associates Learning Task from the Cambridge Neuropsychological Test Automated Battery (CANTAB®) in the Brain Health Registry. J Prev Alzheimers Dis 2024; 11:514-524. [PMID: 38374758 PMCID: PMC10879687 DOI: 10.14283/jpad.2023.117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/21/2024]
Abstract
BACKGROUND Unsupervised online cognitive assessments have demonstrated promise as an efficient and scalable approach for evaluating cognition in aging, and Alzheimer's disease and related dementias. OBJECTIVES The aim of this study was to evaluate the feasibility, usability, and construct validity of the Paired Associates Learning task from the Cambridge Neuropsychological Test Automated Battery® in adults enrolled in the Brain Health Registry. DESIGN, SETTING, PARTICIPANTS, MEASUREMENTS The Paired Associates Learning task was administered to Brain Health Registry participants in a remote, unsupervised, online setting. In this cross-sectional analysis, we 1) evaluated construct validity by analyzing associations between Paired Associates Learning performance and additional participant registry data, including demographics, self- and study partner-reported subjective cognitive change (Everyday Cognition scale), self-reported memory concern, and depressive symptom severity (Patient Health Questionnaire-9) using multivariable linear regression models; 2) determined the predictive value of Paired Associates Learning and other registry variables for identifying participants who self-report Mild Cognitive Impairment by employing multivariable binomial logistic regressions and calculating the area under the receiver operator curve; 3) investigated feasibility by looking at task completion rates and statistically comparing characteristics of task completers and non-completers; and 4) evaluated usability in terms of participant requests for support from BHR related to the assessment. RESULTS In terms of construct validity, in participants who took the Paired Associates Learning for the first time (N=14,528), worse performance was associated with being older, being male, lower educational attainment, higher levels of self- and study partner-reported decline, more self-reported memory concerns, greater depressive symptom severity, and self-report of Mild Cognitive Impairment. Paired Associates Learning performance and Brain Health Registry variables together identified those with self-reported Mild Cognitive Impairment with moderate accuracy (areas under the curve: 0.66-0.68). In terms of feasibility, in a sub-sample of 29,176 participants who had the opportunity to complete Paired Associates Learning for the first time in the registry, 14,417 started the task. 11,647 (80.9% of those who started) completed the task. Compared to those who did not complete the task at their first opportunity, those who completed were older, had more years of education, more likely to self-identify as White, less likely to self-identify as Latino, less likely to have a subjective memory concern, and more likely to report a family history of Alzheimer's disease. In terms of usability, out of 8,395 received requests for support from BHR staff via email, 4.4% (n=374) were related to PAL. Of those, 82% were related to technical difficulties. CONCLUSIONS Our findings support moderate feasibility, good usability, and construct validity of cross-sectional Paired Associates Learning in an unsupervised online registry, but also highlight the need to make the assessment more inclusive and accessible to individuals from ethnoculturally and socioeconomically diverse communities. A future, improved version could be a scalable, efficient method to assess cognition in many different settings, including clinical trials, observational studies, healthcare, and public health.
Collapse
Affiliation(s)
- M T Ashford
- Miriam Ashford, 4150 Clement St, San Francisco, CA 94121, , Phone: +16502089267
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
3
|
Tank R, Diaz A, Ashford MT, Miller MJ, Eichenbaum J, Aaronson A, Landavazo B, Neuhaus J, Weiner MW, Mackin RS, Barnes J, Nosheny RL. Examining Demographic Factors, Psychosocial Wellbeing and Cardiovascular Health in Subjective Cognitive Decline in the Brain Health Registry Cohort. J Prev Alzheimers Dis 2024; 11:787-797. [PMID: 38706295 PMCID: PMC11061024 DOI: 10.14283/jpad.2024.39] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2023] [Accepted: 12/19/2023] [Indexed: 05/07/2024]
Abstract
BACKGROUND Subjective cognitive decline (SCD) is defined as an individual's perception of sustained cognitive decline compared to their normal state while still performing within boundaries for normal functioning. Demographic, psychosocial and medical factors have been linked to age-related cognitive decline, and Alzheimer's dementia (AD). However, their relation to risk for SCD remains unclear. This study aims to identify demographic factors, psychosocial and cardiovascular health associated with SCD within the Brain Health Registry (BHR) online cohort. METHODS Participants aged 55+ (N=27,596) in the BHR self-reported SCD measured using the Everyday Cognition Scale (ECog) and medical conditions, depressive symptoms, body mass index, quality of sleep, health, family history of AD, years of education, race, ethnicity and gender. Multivariable linear regression was used to examine whether SCD was associated with demographic, psychosocial, and medical conditions. RESULTS We found that advanced age, depressive symptoms, poorer sleep quality and poorer quality of health were positively associated with more self-reported SCD in all models. No race or ethnicity differences were found in association with SCD. Males who reported alcohol and tobacco use or underweight BMI had higher ECog scores compared with females. CONCLUSION In addition to well-established risk factors for cognitive decline, such as age, our study consistently and robustly identified a strong association between psychosocial factors and self-reported cognitive decline in an online cohort. These findings provide further evidence that psychosocial health plays a pivotal role in comprehending the risk of SCD and early-stage cognitive ageing. Our findings emphasise the significance of psychosocial factors within the broader context of cardiovascular and demographic risk factors.
Collapse
Affiliation(s)
- R Tank
- Rachana Tank, Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London, United Kingdom,
| | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
4
|
Ashford MT, Jin C, Neuhaus J, Diaz A, Aaronson A, Tank R, Eichenbaum J, Camacho MR, Fockler J, Ulbricht A, Flenniken D, Truran D, Mackin RS, Weiner MW, Mindt MR, Nosheny RL. Participant completion of longitudinal assessments in an online cognitive aging registry: The role of medical conditions. Alzheimers Dement (N Y) 2024; 10:e12438. [PMID: 38188606 PMCID: PMC10767283 DOI: 10.1002/trc2.12438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/21/2023] [Revised: 11/08/2023] [Accepted: 11/16/2023] [Indexed: 01/09/2024]
Abstract
INTRODUCTION This study aimed to understand whether older adults' longitudinal completion of assessments in an online Alzheimer's disease and related dementias (ADRD)-related registry is influenced by self-reported medical conditions. METHODS Brain Health Registry (BHR) is an online cognitive aging and ADRD-related research registry that includes longitudinal health and cognitive assessments. Using logistic regressions, we examined associations between longitudinal registry completion outcomes and self-reported (1) number of medical conditions and (2) eight defined medical condition groups (cardiovascular, metabolic, immune system, ADRD, current psychiatric, substance use/abuse, acquired, other specified conditions) in adults aged 55+ (N = 23,888). Longitudinal registry completion outcomes were assessed by the completion of the BHR initial questionnaire (first questionnaire participants see at each visit) at least twice and completion of a cognitive assessment (Cogstate Brief Battery) at least twice. Models included ethnocultural identity, education, age, and subjective memory concern as covariates. RESULTS We found that the likelihood of longitudinally completing the initial questionnaire was negatively associated with reporting a diagnosis of ADRD and current psychiatric conditions but was positively associated with reporting substance use/abuse and acquired medical conditions. The likelihood of longitudinally completing the cognitive assessment task was negatively associated with number of reported medical conditions, as well as with reporting cardiovascular conditions, ADRD, and current psychiatric conditions. Previously identified associations between ethnocultural identity and longitudinal assessment completion in BHR remained after accounting for the presence of medical conditions. DISCUSSION This post hoc analysis provides novel, initial evidence that older adults' completion of longitudinal assessments in an online registry is associated with the number and types of participant-reported medical conditions. Our findings can inform future efforts to make online studies with longitudinal health and cognitive assessments more usable for older adults with medical conditions. The results need to be interpreted with caution due to selection biases, and the under-inclusion of minoritized communities.
Collapse
Affiliation(s)
- Miriam T. Ashford
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Chengshi Jin
- University of California San Francisco Department of Epidemiology and Biostatistics San FranciscoSan FranciscoCaliforniaUSA
| | - John Neuhaus
- University of California San Francisco Department of Epidemiology and Biostatistics San FranciscoSan FranciscoCaliforniaUSA
| | - Adam Diaz
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Anna Aaronson
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of CaliforniaSan FranciscoCaliforniaUSA
| | - Rachana Tank
- Department of Radiology and Biomedical ImagingUniversity of CaliforniaSan FranciscoCaliforniaUSA
- Dementia Research CentreUCL Institute of NeurologyUniversity College LondonLondonUK
| | - Joseph Eichenbaum
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of CaliforniaSan FranciscoCaliforniaUSA
| | - Monica R. Camacho
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Juliet Fockler
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of CaliforniaSan FranciscoCaliforniaUSA
| | - Aaron Ulbricht
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of CaliforniaSan FranciscoCaliforniaUSA
| | - Derek Flenniken
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Diana Truran
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
| | - Robert Scott Mackin
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Psychiatry and Behavioral SciencesUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Michael W. Weiner
- VA Advanced Imaging Research CenterSan Francisco Veteran's Administration Medical CenterSan FranciscoCaliforniaUSA
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
- Department of Radiology and Biomedical ImagingUniversity of CaliforniaSan FranciscoCaliforniaUSA
- Department of MedicineUniversity of California San FranciscoSan FranciscoCaliforniaUSA
- Department of NeurologyUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| | - Monica Rivera Mindt
- Psychology, Latin American Latino Studies Institute& African and African American StudiesFordham UniversityJoint Appointment in NeurologyIcahn School of Medicine at Mount Sinai ‐ New YorkNew YorkNew YorkUSA
| | - Rachel L. Nosheny
- Department of Veterans Affairs Medical CenterNorthern California Institute for Research and Education (NCIRE)San FranciscoCaliforniaUSA
- Department of Psychiatry and Behavioral SciencesUniversity of California San FranciscoSan FranciscoCaliforniaUSA
| |
Collapse
|
5
|
Weiner MW, Aaronson A, Eichenbaum J, Kwang W, Ashford MT, Gummadi S, Santhakumar J, Camacho MR, Flenniken D, Fockler J, Truran-Sacrey D, Ulbricht A, Mackin RS, Nosheny RL. Brain health registry updates: An online longitudinal neuroscience platform. Alzheimers Dement 2023; 19:4935-4951. [PMID: 36965096 PMCID: PMC10518371 DOI: 10.1002/alz.13077] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Revised: 02/28/2023] [Accepted: 03/08/2023] [Indexed: 03/27/2023]
Abstract
INTRODUCTION Remote, internet-based methods for recruitment, screening, and longitudinally assessing older adults have the potential to facilitate Alzheimer's disease (AD) clinical trials and observational studies. METHODS The Brain Health Registry (BHR) is an online registry that includes longitudinal assessments including self- and study partner-report questionnaires and neuropsychological tests. New initiatives aim to increase inclusion and engagement of commonly underincluded communities using digital, community-engaged research strategies. New features include multilingual support and biofluid collection capabilities. RESULTS BHR includes > 100,000 participants. BHR has made over 259,000 referrals resulting in 25,997 participants enrolled in 30 aging and AD studies. In addition, 28,278 participants are coenrolled in BHR and other studies with data linkage among studies. Data have been shared with 28 investigators. Recent efforts have facilitated the enrollment and engagement of underincluded ethnocultural communities. DISCUSSION The major advantages of the BHR approach are scalability and accessibility. Challenges include compliance, retention, cohort diversity, and generalizability. HIGHLIGHTS Brain Health Registry (BHR) is an online, longitudinal platform of > 100,000 members. BHR made > 259,000 referrals, which enrolled 25,997 participants in 32 studies. New efforts increased enrollment and engagement of underincluded communities in BHR. The major advantages of the BHR approach are scalability and accessibility. BHR provides a unique adjunct for clinical neuroscience research.
Collapse
Affiliation(s)
- Michael W. Weiner
- Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
- University of California, San Francisco Department of Psychiatry and Behavioral Sciences, San Francisco, California, USA
- University of California, San Francisco Department of Medicine, San Francisco, California, USA
- University of California, San Francisco Department of Neurology, San Francisco, California, USA
| | - Anna Aaronson
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Joseph Eichenbaum
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Winnie Kwang
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Miriam T. Ashford
- Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Shilpa Gummadi
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Jessica Santhakumar
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Monica R. Camacho
- Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Derek Flenniken
- Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Juliet Fockler
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Diana Truran-Sacrey
- Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Aaron Ulbricht
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - R. Scott Mackin
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Psychiatry and Behavioral Sciences, San Francisco, California, USA
| | - Rachel L. Nosheny
- VA Advanced Research Center, San Francisco, California, USA
- University of California, San Francisco Department of Psychiatry and Behavioral Sciences, San Francisco, California, USA
| |
Collapse
|
6
|
Ashford MT, Zhu D, Bride J, McLean E, Aaronson A, Conti C, Cypress C, Griffin P, Ross R, Duncan T, Deng X, Ulbricht A, Fockler J, Camacho MR, Flenniken D, Truran D, Mackin SR, Hill C, Weiner MW, Byrd D, Turner Ii RW, Cham H, Rivera Mindt M, Nosheny RL. Understanding Online Registry Facilitators and Barriers Experienced by Black Brain Health Registry Participants: The Community Engaged Digital Alzheimer's Research (CEDAR) Study. J Prev Alzheimers Dis 2023; 10:551-561. [PMID: 37357297 PMCID: PMC10395260 DOI: 10.14283/jpad.2023.25] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/04/2023]
Abstract
BACKGROUND Failure of Alzheimer's disease and related diseases (ADRD) research studies to include and engage Black participants is a major issue, which limits the impact and generalizability of research findings. Little is known about participation of Black adults in online ADRD-related research registries. OBJECTIVES As part of the Community Engaged Digital Alzheimer's Research (CEDAR) Study, this study aims to increase our understanding of facilitators and barriers of Black adults to participating in ADRD-related online registries, as well as to understand their preferences for communication channels. DESIGN, SETTING, PARTICIPANTS, MEASUREMENTS We invited all Black participants enrolled in the Brain Health Registry (BHR) to complete a cross-sectional online survey. The survey consisted of rating scales and open-text questions asking about their attitudes towards brain health research, reasons for joining and continuing to participate in BHR, difficulties with participating, and preferences for modes of contact and website usage. RESULTS Of all invited Black BHR participants (N=3,636), 198 (5.5%) completed the survey. The mean age was 58.4 (SD=11.3), mean years of education were 16.3 (SD=2.4), and 85.5% identified as female. Reported facilitators for joining and continuing to participate in BHR were personal interest (e.g., learning more about own brain health) and altruism (e.g., helping research). Among additional registry features which could encourage return, receiving feedback or scores about BHR tasks was rated the highest. Of those who found BHR participation difficult (21%), the most frequent reason was time burden. The most preferred way of receiving study information was via email. Participants reported that the websites that they used the most were YouTube and Facebook. DISCUSSION The results of our study can inform the development of culturally-responsive registry features and engagement efforts to improve inclusion and participation of Black adults in online ADRD research. Providing participants with feedback about their registry performance and reducing the number of registry tasks are among the recommended strategies.
Collapse
Affiliation(s)
- M T Ashford
- Miriam Ashford, 4150 Clement St, San Francisco, CA 94121, , Phone: (415) 750-6954, Fax number: (415) 750-9358
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
7
|
Mindt MR, Ashford MT, Zhu D, Cham H, Aaronson A, Conti C, Deng X, Alaniz R, Sorce J, Cypress C, Griffin P, Flenniken D, Camacho M, Fockler J, Truran D, Mackin RS, Hill C, Weiner MW, Byrd D, Turner Ii RW, Nosheny RL. The Community Engaged Digital Alzheimer's Research (CEDAR) Study: A Digital Intervention to Increase Research Participation of Black American Participants in the Brain Health Registry. J Prev Alzheimers Dis 2023; 10:847-856. [PMID: 37874107 PMCID: PMC10598330 DOI: 10.14283/jpad.2023.32] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
BACKGROUND Although Black/African American older adults bear significant inequities in prevalence, incidence, and outcomes of Alzheimer's disease and related dementias, they are profoundly under-included in Alzheimer's Disease research. Community-Engaged Research (e.g., equitable community/science partnerships) is an evidence-based approach for improving engagement of underrepresented populations into Alzheimer's Disease research, but has lacked scalability to the national level. As internet use among older adults from underrepresented populations continues to grow, internet-based research shows promise as a feasible, valid approach to engagement and longitudinal assessment. The Community Engaged Digital Alzheimer's Research (CEDAR) study utilizes a community-engaged research approach to increase the engagement and research participation of Black/African American adults in the Brain Health Registry (BHR) and Alzheimer Disease clinical research. OBJECTIVES To describe the methods and evaluate the feasibility of the CEDAR culturally-informed digital platform within BHR. DESIGN All Black/African American participants in BHR were invited to enroll in CEDAR and to consider serving on a newly convened Community-Scientific Partnership Board to guide the study. The community board guided the development a culturally-informed cadre of engagement materials and strategies to increase research participation. Engagement strategies included incentives for study task completion, culturally-informed communications (e.g., landing page, emails and social media), resources about brain health, and video and written testimonials by CEDAR participants. SETTING BHR, an Internet-based registry and cohort. PARTICIPANTS BHR participants self-identifying as Black/African American were invited to enroll. All participants who signed an online informed consent document were enrolled. MEASUREMENTS We report the number of participants invited, enrolled, completed tasks, and volunteered to join the community board. We compared the demographics, cognitive profile, and baseline BHR task completion rates between CEDAR participants and all those invited to join the study. RESULTS Of 3738 invited, 349 (9.34%) enrolled in CEDAR. 134 (37% of CEDAR participants) volunteered to join the community board, of which 19 were selected for the community board. Compared to those invited, the CEDAR cohort had a higher percentage of female participants (84.5%) and a lower percentage of participants who identify as belonging to more than one ethnocultural group (21.8%). Compared to those did not enroll in CEDAR, those enrolled in CEDAR had a higher percentage of participants completing all BHR tasks (22%) and a higher percentage of participants completing at least one cognitive test (76%). Those enrolled in CEDAR also had a higher percentage of participants having an enrolled study partner (18%). CONCLUSIONS A culturally-informed Community-Engaged Research approach, including a remotely-convened community board, to engagement of Black/African American participants in an online research registry is feasible. This approach can be adapted for use in various clinical studies and other settings. Future studies will evaluate the effectiveness of the engagement strategies.
Collapse
Affiliation(s)
- M R Mindt
- Rachel Nosheny, 4150 Clement Street, 114M, San Francisco, CA. 94121, USA, Telephone: 415-221-4810, Email address: Fax number: 415-221-4810
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
8
|
Ashford MT, Eichenbaum J, Jin C, Neuhaus J, Aaronson A, Ulbricht A, Camacho MR, Fockler J, Flenniken D, Truran D, Mackin RS, Maruff P, Weiner MW, Nosheny RL. Associations between Participant Characteristics and Participant Feedback about an Unsupervised Online Cognitive Assessment in a Research Registry. J Prev Alzheimers Dis 2023; 10:607-614. [PMID: 37357303 PMCID: PMC10126538 DOI: 10.14283/jpad.2023.40] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 03/21/2023] [Indexed: 07/26/2023]
Abstract
BACKGROUND This study aims to understand whether and how participant characteristics (age, gender, education, ethnocultural identity) are related to their feedback about taking a remote, unsupervised, online cognitive assessment. METHODS The Brain Health Registry is a public online registry which includes cognitive assessments. Multivariable ordinal regressions assessed associations between participant characteristics and feedback responses of older (55+) participants (N=11,553) regarding their Cogstate Brief Battery assessment experience. RESULTS Higher age, secondary education or less, Latino identity, and female gender were associated with a poorer assessment experience; higher age and a non-White identity were associated with experiencing the assessment instructions as less clear; and higher age, non-White identity, and secondary education or less were associated with rating additional human support with the assessment as more useful. DISCUSSION Our findings highlight the importance of improving the design and instructions of unsupervised, remote, online cognitive assessments to better suit the needs of diverse communities.
Collapse
Affiliation(s)
- M T Ashford
- Miriam Ashford, NCIRE - Northern California Institute for Research and Education, 4150 Clement Street, San Francisco, CA 94121, USA, , Phone: 650-208-9267
| | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
9
|
Weiner MW, Truran‐Sacrey D, Flenniken D, Fockler J, Camacho MR, Ashford MT, Howell T, Kwang W, Aaronson A, Eichenbaum J, Nosheny RL. The Brain Health Registry: Lessons learned and next steps for remote, online AD and aging clinical research. Alzheimers Dement 2022. [DOI: 10.1002/alz.062186] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Affiliation(s)
- Mike W. Weiner
- University of California, San Francisco San Francisco CA USA
| | | | - Derek Flenniken
- San Francisco Veterans Affairs Medical Center San Francisco CA USA
| | - Juliet Fockler
- San Francisco Veterans Affairs Medical Center San Francisco CA USA
| | - Monica R Camacho
- VA Advanced Imaging Research Center, Veterans Affairs Medical Center, San Francisco San Francisco CA USA
| | | | - Taylor Howell
- VA Advanced Imaging Research Center, Veterans Affairs Medical Center, San Francisco San Francisco CA USA
| | - Winnie Kwang
- San Francisco Veterans Affairs Medical Center San Francisco CA USA
| | - Anna Aaronson
- San Francisco Veterans Affairs Medical Center San Francisco CA USA
| | | | - Rachel L Nosheny
- University of California, San Francisco San Francisco CA USA
- San Francisco Veterans Affairs Medical Center San Francisco CA USA
| | | |
Collapse
|
10
|
Nosheny RL, Amariglio R, Sikkes SA, Van Hulle C, Bicalho MAC, Dowling NM, Brucki SMD, Ismail Z, Kasuga K, Kuhn E, Numbers K, Aaronson A, Moretti DV, Pereiro AX, Sánchez‐Benavides G, Sellek Rodríguez AF, Urwyler P, Zawaly K. The role of dyadic cognitive report and subjective cognitive decline in early ADRD clinical research and trials: Current knowledge, gaps, and recommendations. Alzheimers Dement (N Y) 2022; 8:e12357. [PMID: 36226046 PMCID: PMC9530696 DOI: 10.1002/trc2.12357] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Revised: 08/05/2022] [Accepted: 08/22/2022] [Indexed: 11/05/2022]
Abstract
Efficient identification of cognitive decline and Alzheimer's disease (AD) risk in early stages of the AD disease continuum is a critical unmet need. Subjective cognitive decline is increasingly recognized as an early symptomatic stage of AD. Dyadic cognitive report, including subjective cognitive complaints (SCC) from a participant and an informant/study partner who knows the participant well, represents an accurate, reliable, and efficient source of data for assessing risk. However, the separate and combined contributions of self- and study partner report, and the dynamic relationship between the two, remains unclear. The Subjective Cognitive Decline Professional Interest Area within the Alzheimer's Association International Society to Advance Alzheimer's Research and Treatment convened a working group focused on dyadic patterns of subjective report. Group members identified aspects of dyadic-report information important to the AD research field, gaps in knowledge, and recommendations. By reviewing existing data on this topic, we found evidence that dyadic measures are associated with objective measures of cognition and provide unique information in preclinical and prodromal AD about disease stage and progression and AD biomarker status. External factors including dyad (participant-study partner pair) relationship and sociocultural factors contribute to these associations. We recommend greater dyad report use in research settings to identify AD risk. Priority areas for future research include (1) elucidation of the contributions of demographic and sociocultural factors, dyad type, and dyad relationship to dyad report; (2) exploration of agreement and discordance between self- and study partner report across the AD syndromic and disease continuum; (3) identification of domains (e.g., memory, executive function, neuropsychiatric) that predict AD risk outcomes and differentiate cognitive impairment due to AD from other impairment; (4) development of best practices for study partner engagement; (5) exploration of study partner report as AD clinical trial endpoints; (6) continued development, validation, and optimization, of study partner report instruments tailored to the goals of the research and population.
Collapse
Affiliation(s)
- Rachel L. Nosheny
- University of California San FranciscoDepartment of PsychiatrySan FranciscoCaliforniaUSA,Veteran's Administration Advanced Research CenterSan FranciscoCaliforniaUSA
| | - Rebecca Amariglio
- Center for Alzheimer Research and TreatmentDepartment of NeurologyBrigham and Women's HospitalDepartment of Neurology Massachusetts General HospitalHarvard Medical SchoolBostonMassachusettsUSA
| | - Sietske A.M. Sikkes
- Amsterdam University Medical CentersDepartment of NeurologyAlzheimer Center AmsterdamNorth Hollandthe Netherlands/VU UniversityDepartment of ClinicalNeuro & Development PsychologyNorth Hollandthe Netherlands
| | - Carol Van Hulle
- Wisconsin Alzheimer's Disease Research CenterUniversity of Wisconsin‐MadisonMadisonWisconsinUSA
| | - Maria Aparecida Camargos Bicalho
- UFMG: Federal University of Minas GeraisDepartment of Clinical MedicineJenny de Andrade Faria – Center for Geriatrics and Gerontology of UFMGBelo HorizonteBrazil
| | - N. Maritza Dowling
- George Washington UniversityDepartment of Acute & Chronic CareSchool of NursingDepartment of Epidemiology & BiostatisticsMilken Institute School of Public HealthWashingtonDistrict of ColumbiaUSA
| | | | - Zahinoor Ismail
- Hotchkiss Brain Institute and O'Brien Institute for Public HealthCumming School of MedicineUniversity of CalgaryCalgaryAlbertaCanada
| | - Kensaku Kasuga
- Department of Molecular GeneticsBrain Research InstituteNiigata UniversityNiigataJapan
| | - Elizabeth Kuhn
- UNICAEN, INSERM, PhIND “Physiopathology and Imaging of Neurological Disorders,”Institut Blood and Brain @ Caen‐NormandieNormandie UniversityCaenFrance
| | - Katya Numbers
- Centre for Healthy Brain Ageing (CHeBA)Department of PsychiatryUniversity of New South WalesSydneyNew South WalesAustralia
| | - Anna Aaronson
- Veteran's Administration Advanced Research CenterSan FranciscoCaliforniaUSA
| | - Davide Vito Moretti
- IRCCS Istituto Centro San Giovanni di Dio FatebenefratelliAlzheimer Rehabilitation Operative UnitBresciaItaly
| | - Arturo X. Pereiro
- Faculty of PsychologyDepartment of Developmental PsychologyUniversity of Santiago de CompostelaGaliciaSpain
| | | | - Allis F. Sellek Rodríguez
- Costa Rican Foundation for the Care of Older Adults with Alzheimer's and Other Dementias (FundAlzheimer Costa Rica)CartagoCosta Rica
| | - Prabitha Urwyler
- ARTORG Center for Biomedical EngineeringUniversity of BernUniversity Neurorehabilitation UnitDepartment of NeurologyInselspitalBernSwitzerland
| | - Kristina Zawaly
- University of AucklandDepartment of General Practice and Primary Health CareSchool of Population HealthFaculty of Medical and Health SciencesAucklandNew Zealand
| | | |
Collapse
|
11
|
Ashford MT, Camacho MR, Jin C, Eichenbaum J, Ulbricht A, Alaniz R, Van De Mortel L, Sorce J, Aaronson A, Parmar S, Flenniken D, Fockler J, Truran D, Mackin RS, Rivera Mindt M, Morlett-Paredes A, González HM, Mayeda ER, Weiner MW, Nosheny RL. Digital culturally tailored marketing for enrolling Latino participants in a web-based registry: Baseline metrics from the Brain Health Registry. Alzheimers Dement 2022; 19:1714-1728. [PMID: 36193827 PMCID: PMC10070578 DOI: 10.1002/alz.12805] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 06/20/2022] [Accepted: 07/22/2022] [Indexed: 11/09/2022]
Abstract
INTRODUCTION This culturally tailored enrollment effort aims to determine the feasibility of enrolling 5000 older Latino adults from California into the Brain Health Registries (BHR) over 2.25 years. METHODS This paper describes (1) the development and deployment of culturally tailored BHR websites and digital ads, in collaboration with a Latino community science partnership board and a marketing company; (2) an interim feasibility analysis of the enrollment efforts and numbers, and participant characteristics (primary aim); as well as (3) an exploration of module completion and a preliminary efficacy evaluation of the culturally tailored digital efforts compared to BHR's standard non-culturally tailored efforts (secondary aim). RESULTS In 12.5 months, 3603 older Latino adults were enrolled (71% of the total California Latino BHR initiative enrollment goal). Completion of all BHR modules was low (6%). DISCUSSION Targeted ad placement, culturally tailored enrollment messaging, and culturally tailored BHR websites increased enrollment of Latino participants in BHR, but did not translate to increased module completion. HIGHLIGHTS Culturally tailored social marketing and website improvements were implemented. The efforts enrolled 5662 Latino individuals in 12.5 months. The number of Latino Brain Health Registry (BHR) participants increased by 122.7%. We failed to adequately enroll female Latinos and Latinos with lower education. Future work will evaluate effects of a newly released Spanish-language BHR website.
Collapse
Affiliation(s)
- Miriam T Ashford
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
| | - Monica R Camacho
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
| | - Chengshi Jin
- University of California San Francisco, Department of Epidemiology and Biostatistics, San Francisco, California, USA
| | - Joseph Eichenbaum
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Aaron Ulbricht
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | | | | | | | - Anna Aaronson
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
| | - Shivam Parmar
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Derek Flenniken
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
| | - Juliet Fockler
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA
| | - Diana Truran
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA
| | - R Scott Mackin
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,Department of Psychiatry, University of California San Francisco, San Francisco, California, USA
| | - Monica Rivera Mindt
- Psychology & Latin American Latino Studies Institute, Fordham University, Joint Appointment in Neurology, Icahn School of Medicine at Mount Sinai, New York, New York, USA
| | - Alejandra Morlett-Paredes
- University of California, San Diego, Department of Neurosciences and Shiley-Marcos Alzheimer's Disease Research Center in the School of Medicine, San Diego, California, USA
| | - Hector M González
- University of California, San Diego, Department of Neurosciences and Shiley-Marcos Alzheimer's Disease Research Center in the School of Medicine, San Diego, California, USA
| | - Elizabeth Rose Mayeda
- University of California, Los Angeles, Department of Epidemiology, Los Angeles, California, USA
| | - Michael W Weiner
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,Northern California Institute for Research and Education (NCIRE), Department of Veterans Affairs Medical Center, San Francisco, California, USA.,University of California, San Francisco Department of Radiology and Biomedical Imaging, San Francisco, California, USA.,Department of Neurology, University of California San Francisco, San Francisco, California, USA.,Department of Medicine, University of California San Francisco, San Francisco, California, USA
| | - Rachel L Nosheny
- VA, Advanced Imaging Research Center, San Francisco Veteran's Administration Medical Center, San Francisco, California, USA.,Department of Psychiatry, University of California San Francisco, San Francisco, California, USA
| |
Collapse
|
12
|
|
13
|
Graham JR, Aaronson A, Clark SD. Holly: a contrast study in thorium. Headache 1979; 19:195-9. [PMID: 221440 DOI: 10.1111/j.1526-4610.1979.hed1903195.x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
|
14
|
|