1
|
Chou YY, Wang MS, Lin CF, Lee YS, Lee PH, Huang SM, Wu CL, Lin SY. The application of machine learning for identifying frailty in older patients during hospital admission. BMC Med Inform Decis Mak 2024; 24:270. [PMID: 39334179 PMCID: PMC11430101 DOI: 10.1186/s12911-024-02684-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2023] [Accepted: 09/16/2024] [Indexed: 09/30/2024] Open
Abstract
BACKGROUND Early identification of frail patients and early interventional treatment can minimize the frailty-related medical burden. This study investigated the use of machine learning (ML) to detect frailty in hospitalized older adults with acute illnesses. METHODS We enrolled inpatients of the geriatric medicine ward at Taichung veterans general hospital between 2012 and 2022. We compared four ML models including logistic regression, random forest (RF), extreme gradient boosting, and support vector machine (SVM) for the prediction of frailty. The feature window as well as the prediction window was set as half a year before admission. Furthermore, Shapley additive explanation plots and partial dependence plots were used to identify Fried's frailty phenotype for interpreting the model across various levels including domain, feature, and individual aspects. RESULTS We enrolled 3367 patients. Of these, 2843 were frail. We used 21 features to train the prediction model. Of the 4 tested algorithms, SVM yielded the highest AUROC, precision and F1-score (78.05%, 94.53% and 82.10%). Of the 21 features, age, gender, multimorbidity frailty index, triage, hemoglobin, neutrophil ratio, estimated glomerular filtration rate, blood urea nitrogen, and potassium were identified as more impactful due to their absolute values. CONCLUSIONS Our results demonstrated that some easily accessed parameters from the hospital clinical data system can be used to predict frailty in older hospitalized patients using supervised ML methods.
Collapse
Affiliation(s)
- Yin-Yi Chou
- Center for Geriatrics & Gerontology, Taichung Veterans General Hospital, Taichung, Taiwan
- Division of Allergy, Immunology and Rheumatology, Taichung Veterans General Hospital, Taichung, Taiwan
| | - Min-Shian Wang
- Smart Healthcare Committee, Taichung Veterans General Hospital, Taichung, Taiwan
- Department of Applied Mathematics, National Chung Hsing University, Taichung, Taiwan
| | - Cheng-Fu Lin
- Center for Geriatrics & Gerontology, Taichung Veterans General Hospital, Taichung, Taiwan
- Department of Family Medicine, Taichung Veterans General Hospital, Taichung, Taiwan
- Division of Occupational Medicine, Department of Emergency, Taichung Veterans General Hospital, Taichung, Taiwan
- Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung, Taiwan
| | - Yu-Shan Lee
- Center for Geriatrics & Gerontology, Taichung Veterans General Hospital, Taichung, Taiwan
- Division of Neurology, Taichung Veterans General Hospital, Taichung, Taiwan
| | - Pei-Hua Lee
- Center for Geriatrics & Gerontology, Taichung Veterans General Hospital, Taichung, Taiwan
| | - Shih-Ming Huang
- Department of Pharmacy, Taichung Veterans General Hospital, Taichung, Taiwan
| | - Chieh-Liang Wu
- Department of Critical Care Medicine, Taichung Veterans General Hospital, Taichung, Taiwan
- Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung, Taiwan
| | - Shih-Yi Lin
- Center for Geriatrics & Gerontology, Taichung Veterans General Hospital, Taichung, Taiwan.
- Department of Post-Baccalaureate Medicine, College of Medicine, National Chung Hsing University, Taichung, Taiwan.
- Division of Endocrinology and Metabolism, Department of Internal Medicine, Taichung Veterans General Hospital, Taichung, Taiwan.
| |
Collapse
|
2
|
Zhang W, Wang J, Xie F, Wang X, Dong S, Luo N, Li F, Li Y. Development and validation of machine learning models to predict frailty risk for elderly. J Adv Nurs 2024. [PMID: 38605460 DOI: 10.1111/jan.16192] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Revised: 03/16/2024] [Accepted: 03/28/2024] [Indexed: 04/13/2024]
Abstract
AIMS Early identification and intervention of the frailty of the elderly will help lighten the burden of social medical care and improve the quality of life of the elderly. Therefore, we used machine learning (ML) algorithm to develop models to predict frailty risk in the elderly. DESIGN A prospective cohort study. METHODS We collected data on 6997 elderly people from Chinese Longitudinal Healthy Longevity Study wave 6-7 surveys (2011-2012, 2014). After the baseline survey in 1998 (wave 1), the project conducted follow-up surveys (wave 2-8) in 2000-2018. The osteoporotic fractures index was used to assess frailty. Four ML algorithms (random forest [RF], support vector machine, XGBoost and logistic regression [LR]) were used to develop models to identify the risk factors of frailty and predict the risk of frailty. Different ML models were used for the prediction of frailty risk in the elderly and frailty risk was trained on a cohort of 4385 elderly people with frailty (split into a training cohort [75%] and internal validation cohort [25%]). The best-performing model for each study outcome was tested in an external validation cohort of 6997 elderly people with frailty pooled from the surveys (wave 6-7). Model performance was assessed by receiver operating curve and F2-score. RESULTS Among the four ML models, the F2-score values were similar (0.91 vs. 0.91 vs. 0.88 vs. 0.90), and the area under the curve (AUC) values of RF model was the highest (0.75), followed by LR model (0.74). In the final two models, the AUC values of RF and LR model were similar (0.77 vs. 0.76) and their accuracy was identical (87.4% vs. 87.4%). CONCLUSION Our study developed a preliminary prediction model based on two different ML approaches to help predict frailty risk in the elderly. IMPACT The presented models from this study can be used to inform healthcare providers to predict the frailty probability among older adults and maybe help guide the development of effective frailty risk management interventions. IMPLICATIONS FOR THE PROFESSION AND/OR PATIENT CARE Detecting frailty at an early stage and implementing timely targeted interventions may help to improve the allocation of health care resources and to reduce frailty-related burden. Identifying risk factors for frailty could be beneficial to provide tailored and personalized care intervention for older adults to more accurately prevent or improve their frail conditions so as to improve their quality of life. REPORTING METHOD The study has adhered to STROBE guidelines. PATIENT OR PUBLIC CONTRIBUTION No patient or public contribution.
Collapse
Affiliation(s)
- Wei Zhang
- First Affiliated Hospital of Kunming Medical University, Kunming, China
| | - Junchao Wang
- China-Japan Union Hospital of Jilin University, Changchun, China
| | - Fang Xie
- Zhejiang University School of Medicine, Hangzhou, China
| | - Xinghui Wang
- School of Nursing, Jilin University, Changchun, China
| | - Shanshan Dong
- Hepatopancreatobiliary Surgery Department, General External Center, First Hospital of Jilin University, Changchun, China
| | - Nan Luo
- The Second Hospital of Jilin University, Changchun, China
| | - Feng Li
- School of Nursing, Jilin University, Changchun, China
| | - Yuewei Li
- School of Nursing, Jilin University, Changchun, China
| |
Collapse
|
3
|
Pan C, Luo H, Cheung G, Zhou H, Cheng R, Cullum S, Wu C. Identifying Frailty in Older Adults Receiving Home Care Assessment Using Machine Learning: Longitudinal Observational Study on the Role of Classifier, Feature Selection, and Sample Size. JMIR AI 2024; 3:e44185. [PMID: 38875533 PMCID: PMC11041467 DOI: 10.2196/44185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Revised: 07/22/2023] [Accepted: 01/01/2024] [Indexed: 06/16/2024]
Abstract
BACKGROUND Machine learning techniques are starting to be used in various health care data sets to identify frail persons who may benefit from interventions. However, evidence about the performance of machine learning techniques compared to conventional regression is mixed. It is also unclear what methodological and database factors are associated with performance. OBJECTIVE This study aimed to compare the mortality prediction accuracy of various machine learning classifiers for identifying frail older adults in different scenarios. METHODS We used deidentified data collected from older adults (65 years of age and older) assessed with interRAI-Home Care instrument in New Zealand between January 1, 2012, and December 31, 2016. A total of 138 interRAI assessment items were used to predict 6-month and 12-month mortality, using 3 machine learning classifiers (random forest [RF], extreme gradient boosting [XGBoost], and multilayer perceptron [MLP]) and regularized logistic regression. We conducted a simulation study comparing the performance of machine learning models with logistic regression and interRAI Home Care Frailty Scale and examined the effects of sample sizes, the number of features, and train-test split ratios. RESULTS A total of 95,042 older adults (median age 82.66 years, IQR 77.92-88.76; n=37,462, 39.42% male) receiving home care were analyzed. The average area under the curve (AUC) and sensitivities of 6-month mortality prediction showed that machine learning classifiers did not outperform regularized logistic regressions. In terms of AUC, regularized logistic regression had better performance than XGBoost, MLP, and RF when the number of features was ≤80 and the sample size ≤16,000; MLP outperformed regularized logistic regression in terms of sensitivities when the number of features was ≥40 and the sample size ≥4000. Conversely, RF and XGBoost demonstrated higher specificities than regularized logistic regression in all scenarios. CONCLUSIONS The study revealed that machine learning models exhibited significant variation in prediction performance when evaluated using different metrics. Regularized logistic regression was an effective model for identifying frail older adults receiving home care, as indicated by the AUC, particularly when the number of features and sample sizes were not excessively large. Conversely, MLP displayed superior sensitivity, while RF exhibited superior specificity when the number of features and sample sizes were large.
Collapse
Affiliation(s)
- Cheng Pan
- Department of Computer Science, The University of Hong Kong, Hong Kong, China (Hong Kong)
| | - Hao Luo
- Department of Social Work and Social Administration, The University of Hong Kong, Hong Kong, China (Hong Kong)
| | - Gary Cheung
- Department of Psychological Medicine, School of Medicine, The University of Auckland, Auckland, New Zealand
| | - Huiquan Zhou
- Department of Social Work and Social Administration, The University of Hong Kong, Hong Kong, China (Hong Kong)
| | - Reynold Cheng
- Department of Computer Science, The University of Hong Kong, Hong Kong, China (Hong Kong)
| | - Sarah Cullum
- Department of Psychological Medicine, School of Medicine, The University of Auckland, Auckland, New Zealand
| | - Chuan Wu
- Department of Computer Science, The University of Hong Kong, Hong Kong, China (Hong Kong)
| |
Collapse
|
4
|
Thandi M, Wong ST, Price M, Baumbusch J. Perspectives on the representation of frailty in the electronic frailty index. BMC PRIMARY CARE 2024; 25:4. [PMID: 38166753 PMCID: PMC10759446 DOI: 10.1186/s12875-023-02225-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/05/2023] [Accepted: 11/27/2023] [Indexed: 01/05/2024]
Abstract
BACKGROUND Frailty is a state of increased vulnerability from physical, social, and cognitive factors resulting in greater risk of negative health-related outcomes and increased healthcare expenditure. A 36-factor electronic frailty index (eFI) developed in the United Kingdom calculates frailty scores using electronic medical record data. There is currently no standardization of frailty screening in Canadian primary care. In order to implement the eFI in a Canadian context, adaptation of the tool is necessary because frailty is represented by different clinical terminologies in the UK and Canada. In considering the promise of implementing an eFI in British Columbia, Canada, we first looked at the content validation of the 36-factor eFI. Our research question was: Does the eFI represent frailty from the perspectives of primary care clinicians and older adults in British Columbia? METHODS A modified Delphi using three rounds of questionnaires with a panel of 23 experts (five family physicians, five nurse practitioners, five nurses, four allied health professionals, four older adults) reviewed and provided feedback on the 36-factor eFI. These professional groups were chosen because they closely work as interprofessional teams within primary care settings with older adults. Older adults provide real life context and experiences. Questionnaires involved rating the importance of each frailty factor on a 0-10 scale and providing rationale for ratings. Panelists were also given the opportunity to suggest additional factors that ought to be included in the screening tool. Suggested factors were similarly rated in two Delphi rounds. RESULTS Thirty-three of the 36 eFI factors achieved consensus (> 80% of panelists provided a rating of ≥ 8). Factors that did not achieve consensus were hypertension, thyroid disorder and peptic ulcer. These factors were perceived as easily treatable or manageable and/or not considered reflective of frailty on their own. Additional factors suggested by panelists that achieved consensus included: cancer, challenges to healthcare access, chronic pain, communication challenges, fecal incontinence, food insecurity, liver failure/cirrhosis, mental health challenges, medication noncompliance, poverty/financial difficulties, race/ethnic disparity, sedentary/low activity levels, and substance use/misuse. There was a 100% retention rate in each of the three Delphi rounds. CONCLUSIONS AND NEXT STEPS Three key findings emerged from this study: the conceptualization of frailty varied across participants, identification of frailty in community/primary care remains challenging, and social determinants of health affect clinicians' assessments and perceptions of frailty status. This study will inform the next phase of a broader mixed-method sequential study to build a frailty screening tool that could ultimately become a standard of practice for frailty screening in Canadian primary care. Early detection of frailty can help tailor decision making, frame discussions about goals of care, prevent advancement on the frailty trajectory, and ultimately decrease health expenditures, leading to improved patient and system level outcomes.
Collapse
Affiliation(s)
- Manpreet Thandi
- School of Nursing, University of British Columbia, T201 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada.
| | - Sabrina T Wong
- School of Nursing, University of British Columbia, T201 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada
- Centre for Health Services and Policy Research, University of British Columbia, 201-2206 East Mall, Vancouver, BC, V6T 1Z3, Canada
| | - Morgan Price
- Department of Family Practice, University of British Columbia, David Strangway Building, Suite 300, 5950 University Boulevard, Vancouver, BC, V6T 1Z3, Canada
| | - Jennifer Baumbusch
- School of Nursing, University of British Columbia, T201 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada
| |
Collapse
|
5
|
Fan S, Ye J, Xu Q, Peng R, Hu B, Pei Z, Yang Z, Xu F. Digital health technology combining wearable gait sensors and machine learning improve the accuracy in prediction of frailty. Front Public Health 2023; 11:1169083. [PMID: 37546315 PMCID: PMC10402732 DOI: 10.3389/fpubh.2023.1169083] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2023] [Accepted: 06/30/2023] [Indexed: 08/08/2023] Open
Abstract
Background Frailty is a dynamic and complex geriatric condition characterized by multi-domain declines in physiological, gait and cognitive function. This study examined whether digital health technology can facilitate frailty identification and improve the efficiency of diagnosis by optimizing analytical and machine learning approaches using select factors from comprehensive geriatric assessment and gait characteristics. Methods As part of an ongoing study on observational study of Aging, we prospectively recruited 214 individuals living independently in the community of Southern China. Clinical information and fragility were assessed using comprehensive geriatric assessment (CGA). Digital tool box consisted of wearable sensor-enabled 6-min walk test (6MWT) and five machine learning algorithms allowing feature selections and frailty classifications. Results It was found that a model combining CGA and gait parameters was successful in predicting frailty. The combination of these features in a machine learning model performed better than using either CGA or gait parameters alone, with an area under the curve of 0.93. The performance of the machine learning models improved by 4.3-11.4% after further feature selection using a smaller subset of 16 variables. SHapley Additive exPlanation (SHAP) dependence plot analysis revealed that the most important features for predicting frailty were large-step walking speed, average step size, age, total step walking distance, and Mini Mental State Examination score. Conclusion This study provides evidence that digital health technology can be used for predicting frailty and identifying the key gait parameters in targeted health assessments.
Collapse
Affiliation(s)
- Shaoyi Fan
- The Second Clinical College of Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Jieshun Ye
- School of Civil Engineering and Transportation, South China University of Technology, Guangzhou, China
| | - Qing Xu
- The Second Clinical College of Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Runxin Peng
- The Second Clinical College of Guangzhou University of Chinese Medicine, Guangzhou, China
| | - Bin Hu
- Division of Translational Neuroscience, Department of Clinical Neurosciences, Hotchkiss Brain Institute, Alberta Children’s Hospital Research Institute, Cumming School of Medicine, University of Calgary, Calgary, AB, Canada
| | - Zhong Pei
- Department of Neurology, First Affiliated Hospital of Sun Yat-Sen University, Guangzhou, China
| | - Zhimin Yang
- The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangdong Provincial Hospital of Chinese Medicine, Guangzhou, China
| | - Fuping Xu
- The Second Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangdong Provincial Hospital of Chinese Medicine, Guangzhou, China
| |
Collapse
|
6
|
Hakimjavadi R, Karunananthan S, Fung C, Levi C, Helmer-Smith M, LaPlante J, Gazarin M, Rahgozar A, Afkham A, Keely E, Liddy C. Using electronic consultation (eConsult) to identify frailty in provider-to-provider communication: a feasibility and validation study. BMC Geriatr 2023; 23:136. [PMID: 36894892 PMCID: PMC9999527 DOI: 10.1186/s12877-023-03870-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/09/2022] [Accepted: 03/03/2023] [Indexed: 03/11/2023] Open
Abstract
BACKGROUND Frailty is a complex age-related clinical condition that increases vulnerability to stressors. Early recognition of frailty is challenging. While primary care providers (PCPs) serve as the first point of contact for most older adults, convenient tools for identifying frailty in primary care are lacking. Electronic consultation (eConsult), a platform connecting PCPs to specialists, is a rich source of provider-to-provider communication data. Text-based patient descriptions on eConsult may provide opportunities for earlier identification of frailty. We sought to explore the feasibility and validity of identifying frailty status using eConsult data. METHODS eConsult cases closed in 2019 and submitted on behalf of long-term care (LTC) residents or community-dwelling older adults were sampled. A list of frailty-related terms was compiled through a review of the literature and consultation with experts. To identify frailty, eConsult text was parsed to measure the frequency of frailty-related terms. Feasibility of this approach was assessed by examining the availability of frailty-related terms in eConsult communication logs, and by asking clinicians to indicate whether they can assess likelihood of frailty by reviewing the cases. Construct validity was assessed by comparing the number of frailty-related terms in cases about LTC residents with those about community-dwelling older adults. Criterion validity was assessed by comparing clinicians' ratings of frailty to the frequency of frailty-related terms. RESULTS One hundred thirteen LTC and 112 community cases were included. Frailty-related terms identified per case averaged 4.55 ± 3.95 in LTC and 1.96 ± 2.68 in the community (p < .001). Clinicians consistently rated cases with ≥ 5 frailty-related terms as highly likely of living with frailty. CONCLUSIONS The availability of frailty-related terms establishes the feasibility of using provider-to-provider communication on eConsult to identify patients with high likelihood of living with this condition. The higher average of frailty-related terms in LTC (versus community) cases, and agreement between clinician-provided frailty ratings and the frequency of frailty-related terms, support the validity of an eConsult-based approach to identifying frailty. There is potential for eConsult to be used as a case-finding tool in primary care for early recognition and proactive initiation of care processes for older patients living with frailty.
Collapse
Affiliation(s)
- Ramtin Hakimjavadi
- Faculty of Medicine, University of Ottawa, Ottawa, Canada.,C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, Canada
| | - Sathya Karunananthan
- C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, Canada.,Interdisciplinary School of Health Sciences, University of Ottawa, Ottawa, Canada
| | - Celeste Fung
- Department of Family Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada.,St. Patrick's Home of Ottawa, Ottawa, Canada
| | - Cheryl Levi
- Emergency Department Outreach Program, The Ottawa Hospital, Ottawa, Canada
| | - Mary Helmer-Smith
- School of Population and Public Health, Centre for Health Services and Policy Research, University of British Columbia, Vancouver, British Columbia, Canada
| | - James LaPlante
- C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, Canada
| | - Mohamed Gazarin
- Centre of Excellence for Rural Health and Education, Winchester District Memorial Hospital, Winchester, Ontario, Canada
| | - Arya Rahgozar
- Department of Family Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada
| | - Amir Afkham
- Ontario Health East, Ottawa, Canada.,Ontario eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Canada
| | - Erin Keely
- Ontario eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Canada.,Department of Medicine, University of Ottawa, Ottawa, Canada
| | - Clare Liddy
- C.T. Lamont Primary Health Care Research Centre, Bruyère Research Institute, Ottawa, Canada. .,Department of Family Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada. .,Ontario eConsult Centre of Excellence, The Ottawa Hospital, Ottawa, Canada.
| |
Collapse
|
7
|
Luo J, Liao X, Zou C, Zhao Q, Yao Y, Fang X, Spicer J. Identifying Frail Patients by Using Electronic Health Records in Primary Care: Current Status and Future Directions. Front Public Health 2022; 10:901068. [PMID: 35812471 PMCID: PMC9256951 DOI: 10.3389/fpubh.2022.901068] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/21/2022] [Accepted: 05/31/2022] [Indexed: 11/21/2022] Open
Abstract
With the rapidly aging population, frailty, characterized by an increased risk of adverse outcomes, has become a major public health problem globally. Several frailty guidelines or consensuses recommend screening for frailty, especially in primary care settings. However, most of the frailty assessment tools are based on questionnaires or physical examinations, adding to the clinical workload, which is the major obstacle to converting frailty research into clinical practice. Medical data naturally generated by routine clinical work containing frailty indicators are stored in electronic health records (EHRs) (also called electronic health record (EHR) data), which provide resources and possibilities for frailty assessment. We reviewed several frailty assessment tools based on primary care EHRs and summarized the features and novel usage of these tools, as well as challenges and trends. Further research is needed to develop and validate frailty assessment tools based on EHRs in primary care in other parts of the world.
Collapse
Affiliation(s)
- Jianzhao Luo
- International Medical Centre/Ward of General Practice and National Clinical Research Centre for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
| | - Xiaoyang Liao
- International Medical Centre/Ward of General Practice and National Clinical Research Centre for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
- *Correspondence: Xiaoyang Liao ; orcid.org/0000000344099674
| | - Chuan Zou
- Department of General Practice, Chengdu Fifth People's Hospital, Chengdu University of Traditional Chinese Medicine, Chengdu, China
| | - Qian Zhao
- International Medical Centre/Ward of General Practice and National Clinical Research Centre for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
- Qian Zhao ; orcid.org/0000000295405726
| | - Yi Yao
- International Medical Centre/Ward of General Practice and National Clinical Research Centre for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
| | - Xiang Fang
- International Medical Centre/Ward of General Practice and National Clinical Research Centre for Geriatrics, West China Hospital, Sichuan University, Chengdu, China
| | - John Spicer
- GP and Senior Lecturer in Medical Law and Clinical Ethics, Institute of Medical and Biomedical Education, St George's University of London, London, United Kingdom
| |
Collapse
|
8
|
Thandi M, Wong ST, Aponte-Hao S, Grandy M, Mangin D, Singer A, Williamson T. Strategies for working across Canadian practice-based research and learning networks (PBRLNs) in primary care: focus on frailty. BMC FAMILY PRACTICE 2021; 22:220. [PMID: 34772356 PMCID: PMC8590340 DOI: 10.1186/s12875-021-01573-y] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/19/2021] [Accepted: 10/29/2021] [Indexed: 01/17/2023]
Abstract
Background Practice based research and learning networks (PBRLNs) are groups of learning communities that focus on improving delivery and quality of care. Accurate data from primary care electronic medical records (EMRs) is crucial in forming the backbone for PBRLNs. The purpose of this work is to: (1) report on descriptive findings from recent frailty work, (2) describe strategies for working across PBRLNs in primary care, and (3) provide lessons learned for engaging PBRLNs. Methods We carried out a participatory based descriptive study that engaged five different PBRLNs. We collected Clinical Frailty Scale scores from a sample of participating physicians within each PBRLN. Descriptive statistics were used to analyze frailty scores and patients’ associated risk factors and demographics. We used the Consolidated Framework for Implementation Research to inform thematic analysis of qualitative data (meeting minutes, notes, and conversations with co-investigators of each network) in recognizing challenges of working across networks. Results One hundred nine physicians participated in collecting CFS scores across the five provinces (n = 5466). Percentages of frail (11-17%) and not frail (82-91%) patients were similar in all networks, except Ontario who had a higher percentage of frail patients (25%). The majority of frail patients were female (65%) and had a significantly higher prevalence of hypertension, dementia, and depression. Frail patients had more prescribed medications and numbers of healthcare encounters. There were several noteworthy challenges experienced throughout the research process related to differences across provinces in the areas of: numbers of stakeholders/staff involved and thus levels of burden, recruitment strategies, data collection strategies, enhancing engagement, and timelines. Discussion Lessons learned throughout this multi-jurisdictional work included: the need for continuity in ethics, regular team meetings, enhancing levels of engagement with stakeholders, the need for structural support and recognizing differences in data sharing across provinces. Conclusion The differences noted across CPCSSN networks in our frailty study highlight the challenges of multi-jurisdictional work across provinces and the need for consistent and collaborative healthcare planning efforts.
Collapse
Affiliation(s)
- Manpreet Thandi
- Centre for Health Services and Policy Research & School of Nursing, University of British Columbia, 201-2206 East Mall, Vancouver, BC, V6T IZ3, Canada.
| | - Sabrina T Wong
- Centre for Health Services and Policy Research & School of Nursing, University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5, Canada
| | - Sylvia Aponte-Hao
- Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, Alberta, T2N 2Z6, Canada
| | - Mathew Grandy
- Department of Family Medicine, Dalhousie University, 1465 Brenton Street, Suite 402, Halifax, Nova Scotia, B3J 3T4, Canada
| | - Dee Mangin
- Department of Family Medicine, McMaster University, 1280 Main St W, Hamilton, ON, L8S 4L8, Canada
| | - Alexander Singer
- Department of Family Medicine, University of Manitoba, D009-780 Bannatyne Ave, Winnipeg, MB, R3E 0W2, Canada
| | - Tyler Williamson
- Centre for Health Informatics & Department of Community Health Sciences, Cumming School of Medicine, University of Calgary, 3280 Hospital Drive NW, Calgary, Alberta, T2N 2Z6, Canada
| |
Collapse
|
9
|
Aponte-Hao S, Wong ST, Thandi M, Ronksley P, McBrien K, Lee J, Grandy M, Mangin D, Katz A, Singer A, Manca D, Williamson T. Machine learning for identification of frailty in Canadian primary care practices. Int J Popul Data Sci 2021; 6:1650. [PMID: 34541337 PMCID: PMC8431345 DOI: 10.23889/ijpds.v6i1.1650] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022] Open
Abstract
Introduction Frailty is a medical syndrome, commonly affecting people aged 65 years and over and is characterized by a greater risk of adverse outcomes following illness or injury. Electronic medical records contain a large amount of longitudinal data that can be used for primary care research. Machine learning can fully utilize this wide breadth of data for the detection of diseases and syndromes. The creation of a frailty case definition using machine learning may facilitate early intervention, inform advanced screening tests, and allow for surveillance. Objectives The objective of this study was to develop a validated case definition of frailty for the primary care context, using machine learning. Methods Physicians participating in the Canadian Primary Care Sentinel Surveillance Network across Canada were asked to retrospectively identify the level of frailty present in a sample of their own patients (total n = 5,466), collected from 2015–2019. Frailty levels were dichotomized using a cut-off of 5. Extracted features included previously prescribed medications, billing codes, and other routinely collected primary care data. We used eight supervised machine learning algorithms, with performance assessed using a hold-out test set. A balanced training dataset was also created by oversampling. Sensitivity analyses considered two alternative dichotomization cut-offs. Model performance was evaluated using area under the receiver-operating characteristic curve, F1, accuracy, sensitivity, specificity, negative predictive value and positive predictive value. Results The prevalence of frailty within our sample was 18.4%. Of the eight models developed to identify frail patients, an XGBoost model achieved the highest sensitivity (78.14%) and specificity (74.41%). The balanced training dataset did not improve classification performance. Sensitivity analyses did not show improved performance for cut-offs other than 5. Conclusion Supervised machine learning was able to create well performing classification models for frailty. Future research is needed to assess frailty inter-rater reliability, and link multiple data sources for frailty identification.
Collapse
Affiliation(s)
| | - Sabrina T Wong
- Centre for Health Services and Policy Research, University of British Columbia.,School of Nursing, University of British Columbia
| | - Manpreet Thandi
- Centre for Health Services and Policy Research, University of British Columbia.,School of Nursing, University of British Columbia
| | | | | | - Joon Lee
- Cumming School of Medicine, University of Calgary
| | | | - Dee Mangin
- Department of Family Medicine, McMaster University
| | - Alan Katz
- Manitoba Centre for Health Policy, University of Manitoba.,College of Medicine Faculty of Health Sciences, University of Manitoba
| | | | - Donna Manca
- Department of Family Medicine, University of Alberta
| | | |
Collapse
|
10
|
Wong ST, Katz A, Williamson T, Singer A, Peterson S, Taylor C, Price M, McCracken R, Thandi M. Can Linked Electronic Medical Record and Administrative Data Help Us Identify Those Living with Frailty? Int J Popul Data Sci 2020; 5:1343. [PMID: 33644409 PMCID: PMC7893852 DOI: 10.23889/ijpds.v5i1.1343] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023] Open
Abstract
Introduction Frailty is a complex condition that affects many aspects of patients’ wellbeing and health outcomes. Objectives We used available Electronic Medical Record (EMR) and administrative data to determine definitions of frailty. We also examined whether there were differences in demographics or health conditions among those identified as frail in either the EMR or administrative data. Methods EMR and administrative data were linked in British Columbia (BC) and Manitoba (MB) to identify those aged 65 years and older who were frail. The EMR data were obtained from the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) and the administrative data (e.g. billing, hospitalizations) was obtained from Population Data BC and the Manitoba Population Research Data Repository. Sociodemographic characteristics, risk factors, prescribed medications, use and costs of healthcare are described for those identified as frail. Results Sociodemographic and utilization differences were found among those identified as frail from the EMR compared to those in the administrative data. Among those who were >65 years, who had a record in both EMR and administrative data, 5%-8% (n=191 of 3,553, BC; n=2,396 of 29,382, MB) were identified as frail. There was a higher likelihood of being frail with increasing age and being a woman. In BC and MB, those identified as frail in both data sources have approximately twice the number of contacts with primary care (n=20 vs. n=10) and more days in hospital (n=7.2 vs. n=1.9 in BC; n=9.8 vs. n=2.8 in MB) compared to those who are not frail; 27% (BC) and 14% (MB) of those identified as frail in 2014 died in 2015. Conclusions Identifying frailty using EMR data is particularly challenging because many functional deficits are not routinely recorded in structured data fields. Our results suggest frailty can be captured along a continuum using both EMR and administrative data.
Collapse
Affiliation(s)
- S T Wong
- University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5
| | - A Katz
- University of Manitoba, 408-727 McDermot Ave, Winnipeg, Mb, R3E 3P5
| | - T Williamson
- University of Calgary, 3330 Hospital Drive NW, Calgary, Alberta, T2N 4N1
| | - A Singer
- University of Manitoba, 408-727 McDermot Ave, Winnipeg, Mb, R3E 3P5
| | - S Peterson
- University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5
| | - C Taylor
- University of Manitoba, 408-727 McDermot Ave, Winnipeg, Mb, R3E 3P5
| | - M Price
- University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5
| | - R McCracken
- University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5
| | - M Thandi
- University of British Columbia, 2211 Wesbrook Mall, Vancouver, BC, V6T 2B5
| |
Collapse
|