1
|
Wu CJ, Yeh TP, Chu G, Ho YF. Effectiveness of computerised decision aids for patients with chronic diseases in shared decision-making: A systematic review and meta-analysis. J Clin Nurs 2024; 33:2732-2754. [PMID: 38553843 DOI: 10.1111/jocn.17095] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2023] [Revised: 11/24/2023] [Accepted: 02/28/2024] [Indexed: 06/14/2024]
Abstract
AIMS To synthesise the composition and effectiveness of computer-based patient decision aid (PDAs) in interventions for patients with chronic diseases. DESIGN A systematic review with meta-analysis. METHODS Five databases were searched, and only randomised controlled trials (RCTs)were included. This review was conducted with the PRISMA guidelines. The JBI Appraisal Tools for randomised trials were used to assess the risk of bias. We used the random-effects model to conduct meta-analyses. Evidence from RCTs was synthesised using standardised mean differences or mean differences. The GRADE system was employed to assess the certainty of evidence and recommendations. This study was registered on PROSPERO (number: CRD42022369340). DATA SOURCES PubMed, Embase, Web of Science, CINAHL and Cochrane Library were searched for studies published before October 2022. RESULTS The review included 22 studies, and most computer-based PDAs reported information on the disease, treatment options, pros and cons and risk comparison and value clarification. The use of computer-based PDAs showed a significant effect on decision conflict and knowledge, but not on decision regret, satisfaction, self-efficacy, anxiety and quality of life. The overall GRADE certainty of evidence was low. CONCLUSION Although the quality of evidence was low, however, using computer-based PDAs could reduce decision conflict and enhance knowledge when making medical decisions. More research is needed to support the contention above. RELEVANCE TO CLINICAL PRACTICE Computer-based PDAs could assist health-care providers and patients in the shared decision-making process and improving the quality of decision-making. REPORTING METHOD This study adhered to PRISMA guidelines. NO PATIENT OR PUBLIC CONTRIBUTION.
Collapse
Affiliation(s)
- Chih-Jung Wu
- School of Nursing, China Medical University, Taichung, Taiwan
- Department of Nursing, China Medical University Hospital, Taichung, Taiwan
| | - Tzu-Pei Yeh
- School of Nursing, China Medical University, Taichung, Taiwan
- Department of Nursing, China Medical University Hospital, Taichung, Taiwan
| | - Ginger Chu
- School of Nursing & Midwifery, University of Newcastle, Callaghan, Australia
| | - Ya-Fang Ho
- School of Nursing, China Medical University, Taichung, Taiwan
- Department of Nursing, China Medical University Hospital, Taichung, Taiwan
| |
Collapse
|
2
|
Zhu M, Cui M, Nathan AG, Press VG, Wan W, Miles C, Ali R, Pusinelli M, Huisingh‐Scheetz M, Huang ES. A nurse driven care management program to engage older diabetes patients in personalized goal setting and disease management. Health Sci Rep 2024; 7:e2208. [PMID: 38915356 PMCID: PMC11194180 DOI: 10.1002/hsr2.2208] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2023] [Revised: 05/15/2024] [Accepted: 06/06/2024] [Indexed: 06/26/2024] Open
Abstract
Background and Aims Multiple diabetes care guidelines have called for the personalization of risk factor goals, medication management, and self-care plans among older patients. Study of the implementation of these recommendations is needed. This study aimed to test whether a patient survey embedded in the Electronic Healthcare Record (EHR), coupled with telephonic nurse care management, could engage patients in personalized goal setting and chronic disease management. Methods We conducted a single-center equal-randomization delayed comparator trial at the primary care clinics of the University of Chicago Medicine from 2018.6 to 2019.12. Patients over the age of 65 years with type 2 diabetes with an active patient portal account were recruited and randomized to receive an EHR embedded goal setting and preference survey immediately in the intervention arm or after 6 months in the delayed intervention control arm. In the intervention arm, nurses reviewed American Diabetes Association recommendations for A1C goals based on health status class, established personalized goals, and provided monthly telephonic care management phone calls for a maximum of 6 months. Our primary outcome was the documentation of a personalized A1C goal in the EHR. Results A total of 100 patients completed the trial (mean age, 72.51 [SD, 5.22] years; mean baseline A1C, 7.14% [SD, 1.06%]; 68% women). The majority were in the Healthy (59%) followed by Complex (30%) and Very Complex (11%) health status classes. Documentation of an A1C goal in the EHR increased from 42% to 90% (p < 0.001) at 6 months in the intervention group and from 54% to 56% in the control group. Across health status classes, patients set similar A1C goals. Conclusions Older patients can be engaged in personalized goal setting and disease management through an embedded EHR intervention. The clinical impact of the intervention may differ if deployed among older patients with more complex health needs and higher glucose levels. Trial Registration ClinicalTrials.gov Identifier: NCT03692208.
Collapse
Affiliation(s)
- Mengqi Zhu
- Department of MedicineUniversity of ChicagoChicagoIllinoisUSA
- Division of Epidemiology and BiostatisticsUniversity of Illinois at ChicagoChicagoIllinoisUSA
| | - Michael Cui
- Department of MedicineUniversity of ChicagoChicagoIllinoisUSA
- Department of Internal MedicineRush University System for HealthChicagoIllinoisUSA
| | - Aviva G. Nathan
- Department of MedicineUniversity of ChicagoChicagoIllinoisUSA
| | | | - Wen Wan
- Department of MedicineUniversity of ChicagoChicagoIllinoisUSA
| | - Cristy Miles
- Department of MedicineUniversity of ChicagoChicagoIllinoisUSA
- School of MedicineStanford UniversityChicagoIllinoisUSA
| | - Rabia Ali
- Department of MedicineUniversity of ChicagoChicagoIllinoisUSA
- Division of EndocrinologyUC San Diego HealthSan DiegoCaliforniaUSA
| | - Mariko Pusinelli
- Department of MedicineUniversity of ChicagoChicagoIllinoisUSA
- Affiliated OncologistsPark RidgeIllinoisUSA
| | | | - Elbert S. Huang
- Department of MedicineUniversity of ChicagoChicagoIllinoisUSA
| |
Collapse
|
3
|
Stacey D, Lewis KB, Smith M, Carley M, Volk R, Douglas EE, Pacheco-Brousseau L, Finderup J, Gunderson J, Barry MJ, Bennett CL, Bravo P, Steffensen K, Gogovor A, Graham ID, Kelly SE, Légaré F, Sondergaard H, Thomson R, Trenaman L, Trevena L. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2024; 1:CD001431. [PMID: 38284415 PMCID: PMC10823577 DOI: 10.1002/14651858.cd001431.pub6] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/30/2024]
Abstract
BACKGROUND Patient decision aids are interventions designed to support people making health decisions. At a minimum, patient decision aids make the decision explicit, provide evidence-based information about the options and associated benefits/harms, and help clarify personal values for features of options. This is an update of a Cochrane review that was first published in 2003 and last updated in 2017. OBJECTIVES To assess the effects of patient decision aids in adults considering treatment or screening decisions using an integrated knowledge translation approach. SEARCH METHODS We conducted the updated search for the period of 2015 (last search date) to March 2022 in CENTRAL, MEDLINE, Embase, PsycINFO, EBSCO, and grey literature. The cumulative search covers database origins to March 2022. SELECTION CRITERIA We included published randomized controlled trials comparing patient decision aids to usual care. Usual care was defined as general information, risk assessment, clinical practice guideline summaries for health consumers, placebo intervention (e.g. information on another topic), or no intervention. DATA COLLECTION AND ANALYSIS Two authors independently screened citations for inclusion, extracted intervention and outcome data, and assessed risk of bias using the Cochrane risk of bias tool. Primary outcomes, based on the International Patient Decision Aid Standards (IPDAS), were attributes related to the choice made (informed values-based choice congruence) and the decision-making process, such as knowledge, accurate risk perceptions, feeling informed, clear values, participation in decision-making, and adverse events. Secondary outcomes were choice, confidence in decision-making, adherence to the chosen option, preference-linked health outcomes, and impact on the healthcare system (e.g. consultation length). We pooled results using mean differences (MDs) and risk ratios (RRs) with 95% confidence intervals (CIs), applying a random-effects model. We conducted a subgroup analysis of 105 studies that were included in the previous review version compared to those published since that update (n = 104 studies). We used Grading of Recommendations Assessment, Development, and Evaluation (GRADE) to assess the certainty of the evidence. MAIN RESULTS This update added 104 new studies for a total of 209 studies involving 107,698 participants. The patient decision aids focused on 71 different decisions. The most common decisions were about cardiovascular treatments (n = 22 studies), cancer screening (n = 17 studies colorectal, 15 prostate, 12 breast), cancer treatments (e.g. 15 breast, 11 prostate), mental health treatments (n = 10 studies), and joint replacement surgery (n = 9 studies). When assessing risk of bias in the included studies, we rated two items as mostly unclear (selective reporting: 100 studies; blinding of participants/personnel: 161 studies), due to inadequate reporting. Of the 209 included studies, 34 had at least one item rated as high risk of bias. There was moderate-certainty evidence that patient decision aids probably increase the congruence between informed values and care choices compared to usual care (RR 1.75, 95% CI 1.44 to 2.13; 21 studies, 9377 participants). Regarding attributes related to the decision-making process and compared to usual care, there was high-certainty evidence that patient decision aids result in improved participants' knowledge (MD 11.90/100, 95% CI 10.60 to 13.19; 107 studies, 25,492 participants), accuracy of risk perceptions (RR 1.94, 95% CI 1.61 to 2.34; 25 studies, 7796 participants), and decreased decisional conflict related to feeling uninformed (MD -10.02, 95% CI -12.31 to -7.74; 58 studies, 12,104 participants), indecision about personal values (MD -7.86, 95% CI -9.69 to -6.02; 55 studies, 11,880 participants), and proportion of people who were passive in decision-making (clinician-controlled) (RR 0.72, 95% CI 0.59 to 0.88; 21 studies, 4348 participants). For adverse outcomes, there was high-certainty evidence that there was no difference in decision regret between the patient decision aid and usual care groups (MD -1.23, 95% CI -3.05 to 0.59; 22 studies, 3707 participants). Of note, there was no difference in the length of consultation when patient decision aids were used in preparation for the consultation (MD -2.97 minutes, 95% CI -7.84 to 1.90; 5 studies, 420 participants). When patient decision aids were used during the consultation with the clinician, the length of consultation was 1.5 minutes longer (MD 1.50 minutes, 95% CI 0.79 to 2.20; 8 studies, 2702 participants). We found the same direction of effect when we compared results for patient decision aid studies reported in the previous update compared to studies conducted since 2015. AUTHORS' CONCLUSIONS Compared to usual care, across a wide variety of decisions, patient decision aids probably helped more adults reach informed values-congruent choices. They led to large increases in knowledge, accurate risk perceptions, and an active role in decision-making. Our updated review also found that patient decision aids increased patients' feeling informed and clear about their personal values. There was no difference in decision regret between people using decision aids versus those receiving usual care. Further studies are needed to assess the impact of patient decision aids on adherence and downstream effects on cost and resource use.
Collapse
Affiliation(s)
- Dawn Stacey
- School of Nursing, University of Ottawa, Ottawa, Canada
- Centre for Implementation Research, Ottawa Hospital Research Institute, Ottawa, Canada
| | | | | | - Meg Carley
- Centre for Implementation Research, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Robert Volk
- The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | - Elisa E Douglas
- Health Services Research, The University of Texas MD Anderson Cancer Center, Houston, TX, USA
| | | | - Jeanette Finderup
- Department of Renal Medicine, Aarhus University Hospital, Aarhus, Denmark
| | | | - Michael J Barry
- Informed Medical Decisions Program, Massachusetts General Hospital, Boston, MA, USA
| | - Carol L Bennett
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Paulina Bravo
- Education and Cancer Prevention, Fundación Arturo López Pérez, Santiago, Chile
| | - Karina Steffensen
- Center for Shared Decision Making, IRS - Lillebælt Hospital, Vejle, Denmark
| | - Amédé Gogovor
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec, Canada
| | - Ian D Graham
- Centre for Implementation Research, Ottawa Hospital Research Institute, Ottawa, Canada
- School of Epidemiology, Public Health and Preventative Medicine, University of Ottawa, Ottawa, Canada
| | - Shannon E Kelly
- Cardiovascular Research Methods Centre, University of Ottawa Heart Institute, Ottawa, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada
| | - France Légaré
- Centre de recherche sur les soins et les services de première ligne de l'Université Laval (CERSSPL-UL), Université Laval, Quebec, Canada
| | | | - Richard Thomson
- Institute of Health and Society, Newcastle University, Newcastle upon Tyne, UK
| | - Logan Trenaman
- Department of Health Systems and Population Health, School of Public Health, University of Washington, Seattle, WA, USA
| | | |
Collapse
|
4
|
Bernstein S, Gilson S, Zhu M, Nathan AG, Cui M, Press VG, Shah S, Zarei P, Laiteerapong N, Huang ES. Diabetes Life Expectancy Prediction Model Inputs and Results From Patient Surveys Compared With Electronic Health Record Abstraction: Survey Study. JMIR Aging 2023; 6:e44037. [PMID: 37962566 PMCID: PMC10662674 DOI: 10.2196/44037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2022] [Revised: 06/13/2023] [Accepted: 09/19/2023] [Indexed: 11/15/2023] Open
Abstract
Background Prediction models are being increasingly used in clinical practice, with some requiring patient-reported outcomes (PROs). The optimal approach to collecting the needed inputs is unknown. Objective Our objective was to compare mortality prediction model inputs and scores based on electronic health record (EHR) abstraction versus patient survey. Methods Older patients aged ≥65 years with type 2 diabetes at an urban primary care practice in Chicago were recruited to participate in a care management trial. All participants completed a survey via an electronic portal that included items on the presence of comorbid conditions and functional status, which are needed to complete a mortality prediction model. We compared the individual data inputs and the overall model performance based on the data gathered from the survey compared to the chart review. Results For individual data inputs, we found the largest differences in questions regarding functional status such as pushing/pulling, where 41.4% (31/75) of participants reported difficulties that were not captured in the chart with smaller differences for comorbid conditions. For the overall mortality score, we saw nonsignificant differences (P=.82) when comparing survey and chart-abstracted data. When allocating participants to life expectancy subgroups (<5 years, 5-10 years, >10 years), differences in survey and chart review data resulted in 20% having different subgroup assignments and, therefore, discordant glucose control recommendations. Conclusions In this small exploratory study, we found that, despite differences in data inputs regarding functional status, the overall performance of a mortality prediction model was similar when using survey and chart-abstracted data. Larger studies comparing patient survey and chart data are needed to assess whether these findings are reproduceable and clinically important.
Collapse
Affiliation(s)
- Sean Bernstein
- Rush University Medical Center, ChicagoIL, United States
| | - Sarah Gilson
- Section of General Internal Medicine, Department of Medicine, University of Chicago, ChicagoIL, United States
| | - Mengqi Zhu
- Section of General Internal Medicine, Department of Medicine, University of Chicago, ChicagoIL, United States
| | - Aviva G Nathan
- Section of General Internal Medicine, Department of Medicine, University of Chicago, ChicagoIL, United States
| | - Michael Cui
- Rush University Medical Center, ChicagoIL, United States
| | - Valerie G Press
- Section of General Internal Medicine, Department of Medicine, University of Chicago, ChicagoIL, United States
| | - Sachin Shah
- Section of General Internal Medicine, Department of Medicine, University of Chicago, ChicagoIL, United States
| | - Parmida Zarei
- College of Medicine, University of Illinois Chicago, ChicagoIL, United States
| | - Neda Laiteerapong
- Section of General Internal Medicine, Department of Medicine, University of Chicago, ChicagoIL, United States
| | - Elbert S Huang
- Section of General Internal Medicine, Department of Medicine, University of Chicago, ChicagoIL, United States
| |
Collapse
|
5
|
Gans EA, van Mun LAM, de Groot JF, van Munster BC, Rake EA, van Weert JCM, Festen S, van den Bos F. Supporting older patients in making healthcare decisions: The effectiveness of decision aids; A systematic review and meta-analysis. PATIENT EDUCATION AND COUNSELING 2023; 116:107981. [PMID: 37716242 DOI: 10.1016/j.pec.2023.107981] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/17/2023] [Revised: 08/25/2023] [Accepted: 09/10/2023] [Indexed: 09/18/2023]
Abstract
OBJECTIVE To systematically review randomized controlled trials and clinical controlled trials evaluating the effectiveness of Decision Aids (DAs) compared to usual care or alternative interventions for older patients facing treatment, screening, or care decisions. METHODS A systematic search of several databases was conducted. Eligible studies included patients ≥ 65 years or reported a mean of ≥ 70 years. Primary outcomes were attributes of the choice made and decision making process, user experience and ways in which DAs were tailored to older patients. Meta-analysis was conducted, if possible, or outcomes were synthesized descriptively. RESULTS Overall, 15 studies were included. Using DAs were effective in increasing knowledge (SMD 0.90; 95% CI [0.48, 1.32]), decreasing decisional conflict (SMD -0.15; 95% CI [-0.29, -0.01]), improving patient-provider communication (RR 1.67; 95% CI [1.21, 2.29]), and preparing patients to make an individualized decision (MD 35.7%; 95% CI [26.8, 44.6]). Nine studies provided details on how the DA was tailored to older patients. CONCLUSION This review shows a number of favourable results for the effectiveness of DAs in decision making with older patients. PRACTICE IMPLICATIONS Current DAs can be used to support shared decision making with older patients when faced with treatment, screening or care decisions.
Collapse
Affiliation(s)
- Emma A Gans
- University Center of Geriatric Medicine, University Medical Center Groningen, Groningen, the Netherlands; Knowledge Institute of the Dutch Association of Medical Specialists, Utrecht, the Netherlands.
| | - Liza A M van Mun
- Knowledge Institute of the Dutch Association of Medical Specialists, Utrecht, the Netherlands
| | - Janke F de Groot
- Knowledge Institute of the Dutch Association of Medical Specialists, Utrecht, the Netherlands
| | - Barbara C van Munster
- University Center of Geriatric Medicine, University Medical Center Groningen, Groningen, the Netherlands
| | - Ester A Rake
- Knowledge Institute of the Dutch Association of Medical Specialists, Utrecht, the Netherlands
| | - Julia C M van Weert
- Amsterdam School of Communication Research/ASCoR, University of Amsterdam, Amsterdam, the Netherlands
| | - Suzanne Festen
- University Center of Geriatric Medicine, University Medical Center Groningen, Groningen, the Netherlands
| | - Frederiek van den Bos
- Department of Gerontology and Geriatrics, Leiden University Medical Center, Leiden, the Netherlands
| |
Collapse
|
6
|
Scheckel B, Schmidt K, Stock S, Redaèlli M. Patient Portals as Facilitators of Engagement in Patients With Diabetes and Chronic Heart Disease: Scoping Review of Usage and Usability. J Med Internet Res 2023; 25:e38447. [PMID: 37624629 PMCID: PMC10492174 DOI: 10.2196/38447] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2022] [Revised: 02/08/2023] [Accepted: 06/28/2023] [Indexed: 08/26/2023] Open
Abstract
BACKGROUND Patient portals have the potential to improve care for chronically ill patients by engaging them in their treatment. These platforms can work, for example, as a standalone self-management intervention or a tethered link to treatment providers in routine care. Many different types of portals are available for different patient groups, providing various features. OBJECTIVE This scoping review aims to summarize the current literature on patient portals for patients with diabetes mellitus and chronic heart disease regarding usage behavior and usability. METHODS We conducted this review according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) statement for scoping reviews. We performed database searches using PubMed, PsycInfo, and CINAHL, as well as additional searches in reviews and reference lists. We restricted our search to 2010. Qualitative and quantitative studies, and studies using both approaches that analyzed usage behavior or usability of patient portals were eligible. We mapped portal features according to broad thematic categories and summarized the results of the included studies separately according to outcome and research design. RESULTS After screening, we finally included 85 studies. Most studies were about patients with diabetes, included patients younger than 65 years, and were conducted in the United States. Portal features were categorized into educational/general information, reminder, monitoring, interactivity, personal health information, electronic/personal health record, and communication. Portals mostly provided educational, monitoring, and communication-related features. Studies reported on usage behavior including associated variables, usability dimensions, and suggestions for improvement. Various ways of reporting usage frequency were identified. A noticeable decline in portal usage over time was reported frequently. Age was most frequently studied in association with portal use, followed by gender, education, and eHealth literacy. Younger age and higher education were often associated with higher portal use. In two-thirds of studies reporting on portal usability, the portals were rated as user friendly and comprehensible, although measurement and reporting were heterogeneous. Portals were considered helpful for self-management through positive influences on motivation, health awareness, and behavioral changes. Helpful features for self-management were educational/general information and monitoring. Barriers to portal use were general (eg, aspects of design or general usability), related to specific situations during portal use (eg, login procedure), or not portal specific (eg, user skills and preferences). Frequent themes were aspects of design, usability, and technology. Suggestions for improvement were mainly related to technical issues and need for support. CONCLUSIONS The current state of research emphasizes the importance of involving patients in the development and evaluation of patient portals. The consideration of various research designs in a scoping review is helpful for a deeper understanding of usage behavior and usability. Future research should focus on the role of disease burden, and usage behavior and usability among older patients.
Collapse
Affiliation(s)
- Benjamin Scheckel
- Institute of Health Economics and Clinical Epidemiology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
| | - Katharina Schmidt
- Institute of Health Economics and Clinical Epidemiology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
| | - Stephanie Stock
- Institute of Health Economics and Clinical Epidemiology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
| | - Marcus Redaèlli
- Institute of Health Economics and Clinical Epidemiology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
| |
Collapse
|
7
|
Weiner M, Adeoye P, Boeh MJ, Bodke K, Broughton J, Butler AR, Dafferner ML, Dirlam LA, Ferguson D, Keegan AL, Keith NR, Lee JL, McCorkle CB, Pino DG, Shan M, Srinivas P, Tang Q, Teal E, Tu W, Savoy A, Callahan CM, Clark DO. Continuous Glucose Monitoring and Other Wearable Devices to Assess Hypoglycemia among Older Adult Outpatients with Diabetes Mellitus. Appl Clin Inform 2023; 14:37-44. [PMID: 36351548 PMCID: PMC9848893 DOI: 10.1055/a-1975-4136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022] Open
Abstract
BACKGROUND Hypoglycemia (HG) causes symptoms that can be fatal, and confers risk of dementia. Wearable devices can improve measurement and feedback to patients and clinicians about HG events and risk. OBJECTIVES The aim of the study is to determine whether vulnerable older adults could use wearables, and explore HG frequency over 2 weeks. METHODS First, 10 participants with diabetes mellitus piloted a continuous glucometer, physical activity monitor, electronic medication bottles, and smartphones facilitating prompts about medications, behaviors, and symptoms. They reviewed graphs of glucose values, and were asked about the monitoring experience. Next, a larger sample (N = 70) wore glucometers and activity monitors, and used the smartphone and bottles, for 2 weeks. Participants provided feedback about the devices. Descriptive statistics summarized demographics, baseline experiences, behaviors, and HG. RESULTS In the initial pilot, 10 patients aged 50 to 85 participated. Problems addressed included failure of the glucometer adhesive. Patients sought understanding of graphs, often requiring some assistance with interpretation. Among 70 patients in subsequent testing, 67% were African-American, 59% were women. Nearly one-fourth (23%) indicated that they never check their blood sugars. Previous HG was reported by 67%. In 2 weeks of monitoring, 73% had HG (glucose ≤70 mg/dL), and 42% had serious, clinically significant HG (glucose under 54 mg/dL). Eight patients with HG also had HG by home-based blood glucometry. Nearly a third of daytime prompts were unanswered. In 24% of participants, continuous glucometers became detached. CONCLUSION Continuous glucometry occurred for 2 weeks in an older vulnerable population, but devices posed wearability challenges. Most patients experienced HG, often serious in magnitude. This suggests important opportunities to improve wearability and decrease HG frequency among this population.
Collapse
Affiliation(s)
- Michael Weiner
- Department of Medicine, Indiana University, Indianapolis, Indiana,Center for Health Services Research, Regenstrief Institute, Inc., Indianapolis, Indiana,Center for Health Information and Communication, Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service CIN 13–416, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana,Address for correspondence Michael Weiner, MD, MPH Regenstrief Institute, Inc.1101 West 10th Street, Indianapolis, IN 46202United States
| | - Philip Adeoye
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| | | | - Kunal Bodke
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| | | | - Anietra R. Butler
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| | | | - Lindsay A. Dirlam
- Lifestyle Health and Wellness, Eskenazi Health, Indianapolis, Indiana
| | - Denisha Ferguson
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| | - Amanda L. Keegan
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| | - NiCole R. Keith
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana,Department of Kinesiology, Indiana University, Indianapolis, Indiana
| | - Joy L. Lee
- Department of Medicine, Indiana University, Indianapolis, Indiana,Center for Health Services Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| | - Corrina B. McCorkle
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| | - Daniel G. Pino
- Department of Medicine, Indiana University, Indianapolis, Indiana,Lifestyle Health and Wellness, Eskenazi Health, Indianapolis, Indiana
| | - Mu Shan
- Department of Biostatistics and Health Data Science, Indiana University, Indianapolis, Indiana
| | - Preethi Srinivas
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| | - Qing Tang
- Department of Biostatistics and Health Data Science, Indiana University, Indianapolis, Indiana
| | - Evgenia Teal
- Data Services, Regenstrief Institute, Inc., Indianapolis, Indiana
| | - Wanzhu Tu
- Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana,Department of Biostatistics and Health Data Science, Indiana University, Indianapolis, Indiana
| | - April Savoy
- Center for Health Services Research, Regenstrief Institute, Inc., Indianapolis, Indiana,Center for Health Information and Communication, Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service CIN 13–416, Richard L. Roudebush VA Medical Center, Indianapolis, Indiana,Computer and Information Technology, Purdue School of Engineering and Technology, Indiana University-Purdue University Indianapolis, Indiana
| | - Christopher M. Callahan
- Department of Medicine, Indiana University, Indianapolis, Indiana,Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana,Senior Care, Eskenazi Health, Indianapolis, Indiana
| | - Daniel O. Clark
- Department of Medicine, Indiana University, Indianapolis, Indiana,Center for Aging Research, Regenstrief Institute, Inc., Indianapolis, Indiana
| |
Collapse
|
8
|
Yang W, Zhang X, Chen J, Miao W, Zheng C, Qian X, Geng G. Effects of internet-based interventions on improvement of glycemic control and self-management in older adults with diabetes: Systematic review and meta-analysis. Prim Care Diabetes 2022; 16:475-483. [PMID: 35654678 DOI: 10.1016/j.pcd.2022.05.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Revised: 05/09/2022] [Accepted: 05/22/2022] [Indexed: 11/17/2022]
Abstract
PURPOSE To assess and summarize the effects of internet-based interventions on diabetes control and self-management of older adults with diabetes. METHODS PubMed, Web of Science and three Chinese databases were searched to identified articles published in until December 2021. Clinical trials if they covered the effects of internet-based interventions on diabetes control and self-management of older adults with diabetes were included. All data analysis were performed by Review Manager 5.3. RESULTS Sixteen studies with a total of 5604 participants met the inclusion criteria. Our primary outcomes included HbA1c control and self-management. (1) HbA1c control: results indicated statistically difference and high heterogeneity [Q = 112.9, df = 8, p < 0.001, I2 = 93%], in the subgroup analysis of studies in China, results showed a significant influence of internet-based interventions on HbA1c control [Q = 21.31, df = 5, p = 0.03, I2 = 77%]; (2) self-management: in the subgroup analysis of study duration ≤ 6 months [Q = 84.62, df = 2, p < 0.001, I2 = 98%]. CONCLUSION Internet-based interventions are promising on diabetes control and self-management of older adults with diabetes, but still preliminary due to the heterogeneity of intervention components and the limited number of higher methodological quality trials. AVAILABILITY OF DATA AND MATERIAL Applicable.
Collapse
Affiliation(s)
- Wenwen Yang
- School of Medicine, Nantong University, Nantong, Jiangsu, China
| | - Xiaoyi Zhang
- Affiliated Hospital of Nantong University, Nantong, Jiangsu, China
| | - Jianqun Chen
- Nantong North Nursing Home, Nantong, Jiangsu, China
| | - Weiwei Miao
- School of Medicine, Nantong University, Nantong, Jiangsu, China
| | - Chen Zheng
- School of Medicine, Nantong University, Nantong, Jiangsu, China
| | - Xiangyun Qian
- Nursing Department Nantong Third People's Hospital, Nantong, Jiangsu, China.
| | - Guiling Geng
- School of Medicine, Nantong University, Nantong, Jiangsu, China.
| |
Collapse
|
9
|
Li Z, Jin Y, Lu C, Luo R, Wang J, Liu Y. Effects of patient decision aids in patients with type 2 diabetes mellitus: A systematic review and meta-analysis. Int J Nurs Pract 2021; 27:e12914. [PMID: 33657667 DOI: 10.1111/ijn.12914] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2020] [Revised: 11/13/2020] [Accepted: 12/08/2020] [Indexed: 11/26/2022]
Abstract
AIMS This study aimed to systematically evaluate the effectiveness of patient decision aids on knowledge, decisional conflict and decisional self-efficacy outcomes in patients with diabetes. METHODS A comprehensive database search was performed using the Web of Science, Cochrane Library, PubMed, Embase, PsycINFO (Ovid), CINAHL (EBASCO), CNKI, VIP, Wan Fang Database and the Ottawa Decision Aid Library Inventory (http://decisionaid.ohri.ca/index.html) from inception to 13 October 2019. Two reviewers independently searched databases, screened articles, extracted data and evaluated the risk bias of included studies. Then Rev Man 5.3 software was adopted for statistical analysis. RESULTS Ten articles containing 1,452 people with diabetes were selected. The results of meta-analysis showed that patient decision aids had a positive effect on reducing decisional conflict and improving decisional self-efficacy among patients with type 2 diabetes. Meanwhile, this article also revealed that patient decision aids have beneficial short-term effects on improving knowledge, but there was no significant long-term effect. CONCLUSION Patient decision aids are capable of becoming support tools to improve shared decision making. Further implementation studies are required to transform patient decision aids tools into clinical practice.
Collapse
Affiliation(s)
- Zimeng Li
- School of Nursing, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Yinghui Jin
- Center for Evidence-Based and Translational Medicine, Zhongnan Hospital of Wuhan University, Center for Evidence-Based and Translational Medicine, Wuhan University, Hubei, China
| | - Cui Lu
- Emergency Department, Tianjin TEDA Hospital, Tianjin, China
| | - Ruzhen Luo
- School of Nursing, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Jiayao Wang
- School of Nursing, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| | - Yanhui Liu
- School of Nursing, Tianjin University of Traditional Chinese Medicine, Tianjin, China
| |
Collapse
|
10
|
Weiner M, Cummins J, Raji A, Ofner S, Iglay K, Teal E, Li X, Engel SS, Knapp K, Rajpathak S, Baker J, Chatterjee AK, Radican L. A randomized study on the usefulness of an electronic outpatient hypoglycemia risk calculator for clinicians of patients with diabetes in a safety-net institution. Curr Med Res Opin 2020; 36:583-593. [PMID: 31951747 DOI: 10.1080/03007995.2020.1717451] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
Objective: Hypoglycemia (HG) occurs in up to 60% of patients with diabetes mellitus (DM) each year. We assessed a HG alert tool in an electronic health record system, and determined its effect on clinical practice and outcomes.Methods: The tool applied a statistical model, yielding patient-specific information about HG risk. We randomized outpatient primary-care providers (PCPs) to see or not see the alerts. Patients were assigned to study group according to the first PCP seen during four months. We assessed prescriptions, testing, and HG. Variables were compared by multinomial, logistic, or linear model. ClinicalTrials.gov ID: NCT04177147 (registered on 22 November 2019).Results: Patients (N = 3350) visited 123 intervention PCPs; 3395 patients visited 220 control PCPs. Intervention PCPs were shown 18,645 alerts (mean of 152 per PCP). Patients' mean age was 55 years, with 61% female, 49% black, and 49% Medicaid recipients. Mean baseline A1c and body mass index were similar between groups. During follow-up, the number of A1c and glucose tests, and number of new, refilled, changed, or discontinued insulin prescriptions, were highest for patients with highest risk. Per 100 patients on average, the intervention group had fewer sulfonylurea refills (6 vs. 8; p < .05) and outpatient encounters (470 vs. 502; p < .05), though the change in encounters was not significant. Frequency of HG events was unchanged.Conclusions: Informing PCPs about risk of HG led to fewer sulfonylurea refills and visits. Longer-term studies are needed to assess potential for long-term benefits.
Collapse
Affiliation(s)
- Michael Weiner
- Regenstrief Institute, Inc, Indianapolis, IN, USA
- Indiana University Center for Health Services and Outcomes Research, Indianapolis, IN, USA
- Center for Health Information and Communication, U.S. Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service CIN 13-416, Richard L. Roudebush VA Medical Center, Indianapolis, IN, USA
| | | | | | - Susan Ofner
- Department of Biostatistics, Indiana University, Indianapolis, IN, USA
| | | | - Evgenia Teal
- Regenstrief Institute, Inc, Indianapolis, IN, USA
| | - Xiaochun Li
- Department of Biostatistics, Indiana University, Indianapolis, IN, USA
| | | | | | | | - Jarod Baker
- Regenstrief Institute, Inc, Indianapolis, IN, USA
| | | | | |
Collapse
|
11
|
Abstract
PURPOSE OF REVIEW A patient's prognosis and risk of adverse drug effects are important considerations for individualizing care of older patients with diabetes. This review summarizes the evidence for risk assessment and proposes approaches for clinicians in the context of current clinical guidelines. RECENT FINDINGS Diabetes guidelines vary in their recommendations for how life expectancy should be estimated and used to inform the selection of glycemic targets. Readily available prognostic tools may improve estimation of life expectancy but require validation among patients with diabetes. Treatment decisions based on prognosis are difficult for clinicians to communicate and for patients to understand. Determining hypoglycemia risk involves assessing major risk factors; models to synthesize these factors have been developed. Applying risk assessment to individualize diabetes care is complex and currently relies heavily on clinician judgment. More research is need to validate structured approaches to risk assessment and determine how to incorporate them into patient-centered diabetes care.
Collapse
Affiliation(s)
- Scott J Pilla
- Department of Medicine, Division of General Internal Medicine, The Johns Hopkins University School of Medicine, Baltimore, MD, USA.
- Welch Center for Prevention, Epidemiology & Clinical Research, Baltimore, MD, USA.
| | - Nancy L Schoenborn
- Department of Medicine, Division of Geriatric Medicine and Gerontology, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Nisa M Maruthur
- Department of Medicine, Division of General Internal Medicine, The Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Welch Center for Prevention, Epidemiology & Clinical Research, Baltimore, MD, USA
- Department of Epidemiology, The Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| | - Elbert S Huang
- Division of General Internal Medicine, Department of Medicine, The University of Chicago, Chicago, IL, USA
| |
Collapse
|
12
|
Wieringa TH, Rodriguez-Gutierrez R, Spencer-Bonilla G, de Wit M, Ponce OJ, Sanchez-Herrera MF, Espinoza NR, Zisman-Ilani Y, Kunneman M, Schoonmade LJ, Montori VM, Snoek FJ. Decision aids that facilitate elements of shared decision making in chronic illnesses: a systematic review. Syst Rev 2019; 8:121. [PMID: 31109357 PMCID: PMC6528254 DOI: 10.1186/s13643-019-1034-4] [Citation(s) in RCA: 49] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/22/2018] [Accepted: 04/29/2019] [Indexed: 01/19/2023] Open
Abstract
BACKGROUND Shared decision making (SDM) is a patient-centered approach in which clinicians and patients work together to find and choose the best course of action for each patient's particular situation. Six SDM key elements can be identified: situation diagnosis, choice awareness, option clarification, discussion of harms and benefits, deliberation of patient preferences, and making the decision. The International Patient Decision Aid Standards (IPDAS) require that a decision aid (DA) support these key elements. Yet, the extent to which DAs support these six key SDM elements and how this relates to their impact remain unknown. METHODS We searched bibliographic databases (from inception until November 2017), reference lists of included studies, trial registries, and experts for randomized controlled trials of DAs in patients with cardiovascular, or chronic respiratory conditions or diabetes. Reviewers worked in duplicate and independently selected studies for inclusion, extracted trial, and DA characteristics, and evaluated the quality of each trial. RESULTS DAs most commonly clarified options (20 of 20; 100%) and discussed their harms and benefits (18 of 20; 90%; unclear in two DAs); all six elements were clearly supported in 4 DAs (20%). We found no association between the presence of these elements and SDM outcomes. CONCLUSIONS DAs for selected chronic conditions are mostly designed to transfer information about options and their harms and benefits. The extent to which their support of SDM key elements relates to their impact on SDM outcomes could not be ascertained. SYSTEMATIC REVIEW REGISTRATION PROSPERO registration number: CRD42016050320 .
Collapse
Affiliation(s)
- Thomas H Wieringa
- Department of Medical Psychology, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, the Netherlands.
| | - Rene Rodriguez-Gutierrez
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA.,Division of Endocrinology, Department of Internal Medicine, "Dr. Jose E. González" University Hospital, Autonomous University of Nuevo Leon, Monterrey, Nuevo Leon, Mexico.,Plataforma INVEST Medicina UANL-KER Unit Mayo Clinic, KER Unit México, "Dr. Jose E. González" University Hospital, Autonomous University of Nuevo Leon, Monterrey, Nuevo Leon, Mexico
| | - Gabriela Spencer-Bonilla
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA.,Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA
| | - Maartje de Wit
- Department of Medical Psychology, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, the Netherlands
| | - Oscar J Ponce
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA
| | | | - Nataly R Espinoza
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA
| | | | - Marleen Kunneman
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA.,Medical Decision Making, Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands
| | | | - Victor M Montori
- Knowledge and Evaluation Research Unit, Mayo Clinic, Rochester, MN, USA
| | - Frank J Snoek
- Department of Medical Psychology, Amsterdam Public Health Research Institute, Amsterdam UMC, Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, the Netherlands
| |
Collapse
|
13
|
Karagiannis T, Andreadis P, Manolopoulos A, Malandris K, Avgerinos I, Karagianni A, Tsapas A. Decision aids for people with Type 2 diabetes mellitus: an effectiveness rapid review and meta-analysis. Diabet Med 2019; 36:557-568. [PMID: 30791131 DOI: 10.1111/dme.13939] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 02/19/2019] [Indexed: 12/13/2022]
Abstract
AIMS To perform a rapid review and meta-analysis of randomized controlled trials (RCTs) evaluating patient decision aids (PtDAs) for people with Type 2 diabetes mellitus. METHODS We searched Medline and the Cochrane Library for RCTs assessing PtDAs in people with Type 2 diabetes. PtDAs were defined as tools designed to help people engage in decision-making about healthcare options, such as making treatment choices or setting therapeutic goals. The study selection process was facilitated by an automated screening tool to identify RCTs. We classified outcomes into seven domains and conducted meta-analyses using random effects models. RESULTS We included a total of 15 studies, nine of which were cluster RCTs, that evaluated 10 PtDAs. Thirteen trials compared a PtDA with usual care or usual care plus educational material, whereas two RCTs compared individually tailored vs. non-tailored PtDAs. Meta-analyses showed a favourable effect of PtDAs compared with usual care in reducing decisional conflict [weighted mean difference (WMD) -4.66, 95% confidence interval (CI) -7.93 to -1.39] and in improving knowledge (WMD 20.46, 95% CI 9.13 to 3.77). Use of PtDAs resulted in more active involvement in decision-making during the consultation, although no effect was evident in terms of glycaemic control or self-reported medication adherence. CONCLUSIONS PtDAs for people with Type 2 diabetes can improve the quality of decision-making and increase knowledge transfer. Interpretation of our findings is attenuated due to limitations related to the rapid review approach, including searching only two databases and performing data extraction and risk of bias assessment by a single reviewer.
Collapse
Affiliation(s)
- T Karagiannis
- Clinical Research and Evidence-Based Medicine Unit, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - P Andreadis
- Clinical Research and Evidence-Based Medicine Unit, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - A Manolopoulos
- Clinical Research and Evidence-Based Medicine Unit, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - K Malandris
- Clinical Research and Evidence-Based Medicine Unit, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - I Avgerinos
- Clinical Research and Evidence-Based Medicine Unit, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - A Karagianni
- Clinical Research and Evidence-Based Medicine Unit, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - A Tsapas
- Clinical Research and Evidence-Based Medicine Unit, Aristotle University of Thessaloniki, Thessaloniki, Greece
- Harris Manchester College, University of Oxford, Oxford, UK
| |
Collapse
|
14
|
Gutacker N, Street A. Use of large-scale HRQoL datasets to generate individualised predictions and inform patients about the likely benefit of surgery. Qual Life Res 2017; 26:2497-2505. [PMID: 28567601 PMCID: PMC5548850 DOI: 10.1007/s11136-017-1599-0] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/20/2017] [Indexed: 01/18/2023]
Abstract
Purpose The English NHS has mandated the routine collection of health-related quality of life (HRQoL) data before and after surgery, giving prospective patient information about the likely benefit of surgery. Yet, the information is difficult to access and interpret because it is not presented in a lay-friendly format and does not reflect patients’ individual circumstances. We set out a methodology to generate personalised information to help patients make informed decisions. Methods We used anonymised, pre- and postoperative EuroQol-5D-3L (EQ-5D) data for over 490,000 English NHS patients who underwent primary hip or knee replacement surgery or groin hernia repair between April 2009 and March 2016. We estimated linear regression models to relate changes in EQ-5D utility scores to patients’ own assessment of the success of surgery, and calculated from that minimally important differences for health improvements/deteriorations. Classification tree analysis was used to develop algorithms that sort patients into homogeneous groups that best predict postoperative EQ-5D utility scores. Results Patients were classified into between 55 (hip replacement) to 60 (hernia repair) homogeneous groups. The classifications explained between 14 and 27% of variation in postoperative EQ-5D utility score. Conclusions Patients are heterogeneous in their expected benefit from surgery, and decision aids should reflect this. Large administrative datasets on HRQoL can be used to generate the required individualised predictions to inform patients.
Collapse
Affiliation(s)
- Nils Gutacker
- Centre for Health Economics, University of York, Heslington, YO10 5DD, UK.
| | - Andrew Street
- Centre for Health Economics, University of York, Heslington, YO10 5DD, UK
| |
Collapse
|