1
|
Im J, Kross EK, Engelberg RA, Dotolo DG, Ungar A, Nielsen E, Torrence J, Abedini NC. Applying human-centered design to adapt the Jumpstart Guide for goals-of-care discussions in persons living with dementia. J Am Geriatr Soc 2024; 72:2111-2119. [PMID: 38801253 PMCID: PMC11226381 DOI: 10.1111/jgs.18965] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 04/16/2024] [Accepted: 04/21/2024] [Indexed: 05/29/2024]
Abstract
BACKGROUND Goals-of-care discussions (GOCD) are uncommon in persons living with dementia (PLWD) despite the likelihood of eventual loss of decisional capacity in the population. The Jumpstart Guide, an existing serious illness communication priming tool, can improve GOCD in certain populations, but has not previously been adapted for use among PLWD and their caregivers. METHODS Using human-centered design (HCD), we adapted the Jumpstart Guide for use with PLWD and their caregivers. We conducted qualitative interviews with clinicians and caregivers of PLWD. Six team members conducted qualitative rapid analysis of interviews leading to the development of summary templates and integrative matrices. Four iterations of the Jumpstart Guide led to the final version. RESULTS Thirteen clinicians and 11 caregivers were interviewed. Interviews provided key insights into the unique barriers PLWD and their caregivers face during GOCD, including discomfort with accepting a dementia diagnosis and concern with using "serious illness" to describe dementia, as is commonly done in palliative care. Clinicians described differences in GOCD with PLWD compared to other serious illnesses, and the challenge of getting patients and families to think about future health states. Interviews led to Jumpstart Guide adaptations in the following domains: (1) format and structure, (2) content, and (3) specific language. Suggested changes included prioritizing naming a decision-maker, changing conversation prompts to improve accessibility and understandability, ensuring the Jumpstart Guide could be used with patients as well as their caregivers, and altering language to avoid references to "serious illness" and "abilities." CONCLUSION Using HCD yielded valuable insights from clinicians and caregivers about the unique barriers to conducting GOCD among PLWD and their caregivers. These insights were used to adapt the Jumpstart Guide for use with PLWD and their caregivers, which is currently being tested in a pragmatic randomized controlled trial in outpatient clinics.
Collapse
Affiliation(s)
- Jennifer Im
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle, WA
- Department of Health Systems and Population Health, School of Public Health, University of Washington, Seattle, WA
| | - Erin K. Kross
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle, WA
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle, WA
| | - Ruth A. Engelberg
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle, WA
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle, WA
| | - Danae G. Dotolo
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle, WA
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle, WA
| | - Anna Ungar
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle, WA
| | - Elizabeth Nielsen
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle, WA
| | - Janaki Torrence
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle, WA
| | - Nauzley C. Abedini
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle, WA
- Division of Gerontology and Geriatric Medicine, Department of Medicine, University of Washington, Seattle, WA
| |
Collapse
|
2
|
Ganta T, Kia A, Parchure P, Wang MH, Besculides M, Mazumdar M, Smith CB. Fairness in Predicting Cancer Mortality Across Racial Subgroups. JAMA Netw Open 2024; 7:e2421290. [PMID: 38985468 PMCID: PMC11238025 DOI: 10.1001/jamanetworkopen.2024.21290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/13/2024] [Accepted: 05/10/2024] [Indexed: 07/11/2024] Open
Abstract
Importance Machine learning has potential to transform cancer care by helping clinicians prioritize patients for serious illness conversations. However, models need to be evaluated for unequal performance across racial groups (ie, racial bias) so that existing racial disparities are not exacerbated. Objective To evaluate whether racial bias exists in a predictive machine learning model that identifies 180-day cancer mortality risk among patients with solid malignant tumors. Design, Setting, and Participants In this cohort study, a machine learning model to predict cancer mortality for patients aged 21 years or older diagnosed with cancer between January 2016 and December 2021 was developed with a random forest algorithm using retrospective data from the Mount Sinai Health System cancer registry, Social Security Death Index, and electronic health records up to the date when databases were accessed for cohort extraction (February 2022). Exposure Race category. Main Outcomes and Measures The primary outcomes were model discriminatory performance (area under the receiver operating characteristic curve [AUROC], F1 score) among each race category (Asian, Black, Native American, White, and other or unknown) and fairness metrics (equal opportunity, equalized odds, and disparate impact) among each pairwise comparison of race categories. True-positive rate ratios represented equal opportunity; both true-positive and false-positive rate ratios, equalized odds; and the percentage of predictive positive rate ratios, disparate impact. All metrics were estimated as a proportion or ratio, with variability captured through 95% CIs. The prespecified criterion for the model's clinical use was a threshold of at least 80% for fairness metrics across different racial groups to ensure the model's prediction would not be biased against any specific race. Results The test validation dataset included 43 274 patients with balanced demographics. Mean (SD) age was 64.09 (14.26) years, with 49.6% older than 65 years. A total of 53.3% were female; 9.5%, Asian; 18.9%, Black; 0.1%, Native American; 52.2%, White; and 19.2%, other or unknown race; 0.1% had missing race data. A total of 88.9% of patients were alive, and 11.1% were dead. The AUROCs, F1 scores, and fairness metrics maintained reasonable concordance among the racial subgroups: the AUROCs ranged from 0.75 (95% CI, 0.72-0.78) for Asian patients and 0.75 (95% CI, 0.73-0.77) for Black patients to 0.77 (95% CI, 0.75-0.79) for patients with other or unknown race; F1 scores, from 0.32 (95% CI, 0.32-0.33) for White patients to 0.40 (95% CI, 0.39-0.42) for Black patients; equal opportunity ratios, from 0.96 (95% CI, 0.95-0.98) for Black patients compared with White patients to 1.02 (95% CI, 1.00-1.04) for Black patients compared with patients with other or unknown race; equalized odds ratios, from 0.87 (95% CI, 0.85-0.92) for Black patients compared with White patients to 1.16 (1.10-1.21) for Black patients compared with patients with other or unknown race; and disparate impact ratios, from 0.86 (95% CI, 0.82-0.89) for Black patients compared with White patients to 1.17 (95% CI, 1.12-1.22) for Black patients compared with patients with other or unknown race. Conclusions and Relevance In this cohort study, the lack of significant variation in performance or fairness metrics indicated an absence of racial bias, suggesting that the model fairly identified cancer mortality risk across racial groups. It remains essential to consistently review the model's application in clinical settings to ensure equitable patient care.
Collapse
Affiliation(s)
- Teja Ganta
- Division of Hematology and Medical Oncology, Icahn School of Medicine at Mount Sinai, New York, New York
- Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Arash Kia
- Department of Anesthesiology, Perioperative and Pain Medicine, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Prathamesh Parchure
- Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Min-heng Wang
- Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Melanie Besculides
- Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Madhu Mazumdar
- Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, New York, New York
- Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, New York
| | - Cardinale B. Smith
- Division of Hematology and Medical Oncology, Icahn School of Medicine at Mount Sinai, New York, New York
- Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, New York
| |
Collapse
|
3
|
Takvorian SU, Gabriel P, Wileyto EP, Blumenthal D, Tejada S, Clifton ABW, Asch DA, Buttenheim AM, Rendle KA, Shelton RC, Chaiyachati KH, Fayanju OM, Ware S, Schuchter LM, Kumar P, Salam T, Lieberman A, Ragusano D, Bauer AM, Scott CA, Shulman LN, Schnoll R, Beidas RS, Bekelman JE, Parikh RB. Clinician- and Patient-Directed Communication Strategies for Patients With Cancer at High Mortality Risk: A Cluster Randomized Trial. JAMA Netw Open 2024; 7:e2418639. [PMID: 38949813 PMCID: PMC11217875 DOI: 10.1001/jamanetworkopen.2024.18639] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Accepted: 04/23/2024] [Indexed: 07/02/2024] Open
Abstract
Importance Serious illness conversations (SICs) that elicit patients' values, goals, and care preferences reduce anxiety and depression and improve quality of life, but occur infrequently for patients with cancer. Behavioral economic implementation strategies (nudges) directed at clinicians and/or patients may increase SIC completion. Objective To test the independent and combined effects of clinician and patient nudges on SIC completion. Design, Setting, and Participants A 2 × 2 factorial, cluster randomized trial was conducted from September 7, 2021, to March 11, 2022, at oncology clinics across 4 hospitals and 6 community sites within a large academic health system in Pennsylvania and New Jersey among 163 medical and gynecologic oncology clinicians and 4450 patients with cancer at high risk of mortality (≥10% risk of 180-day mortality). Interventions Clinician clusters and patients were independently randomized to receive usual care vs nudges, resulting in 4 arms: (1) active control, operating for 2 years prior to trial start, consisting of clinician text message reminders to complete SICs for patients at high mortality risk; (2) clinician nudge only, consisting of active control plus weekly peer comparisons of clinician-level SIC completion rates; (3) patient nudge only, consisting of active control plus a preclinic electronic communication designed to prime patients for SICs; and (4) combined clinician and patient nudges. Main Outcomes and Measures The primary outcome was a documented SIC in the electronic health record within 6 months of a participant's first clinic visit after randomization. Analysis was performed on an intent-to-treat basis at the patient level. Results The study accrued 4450 patients (median age, 67 years [IQR, 59-75 years]; 2352 women [52.9%]) seen by 163 clinicians, randomized to active control (n = 1004), clinician nudge (n = 1179), patient nudge (n = 997), or combined nudges (n = 1270). Overall patient-level rates of 6-month SIC completion were 11.2% for the active control arm (112 of 1004), 11.5% for the clinician nudge arm (136 of 1179), 11.5% for the patient nudge arm (115 of 997), and 14.1% for the combined nudge arm (179 of 1270). Compared with active control, the combined nudges were associated with an increase in SIC rates (ratio of hazard ratios [rHR], 1.55 [95% CI, 1.00-2.40]; P = .049), whereas the clinician nudge (HR, 0.95 [95% CI, 0.64-1.41; P = .79) and patient nudge (HR, 0.99 [95% CI, 0.73-1.33]; P = .93) were not. Conclusions and Relevance In this cluster randomized trial, nudges combining clinician peer comparisons with patient priming questionnaires were associated with a marginal increase in documented SICs compared with an active control. Combining clinician- and patient-directed nudges may help to promote SICs in routine cancer care. Trial Registration ClinicalTrials.gov Identifier: NCT04867850.
Collapse
Affiliation(s)
| | - Peter Gabriel
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - E. Paul Wileyto
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Daniel Blumenthal
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Sharon Tejada
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Alicia B. W. Clifton
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Wicked Saints Studios, Medford, Oregon
| | - David A. Asch
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Alison M. Buttenheim
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- School of Nursing, University of Pennsylvania, Philadelphia
| | | | - Rachel C. Shelton
- Department of Sociomedical Sciences, Mailman School of Public Health, Columbia University, New York, New York
| | - Krisda H. Chaiyachati
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Verily Life Sciences, San Francisco, California
| | | | - Susan Ware
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Lynn M. Schuchter
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Pallavi Kumar
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Tasnim Salam
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- New Jersey Department of Health Communicable Disease Service, Trenton, New Jersey
| | - Adina Lieberman
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Daniel Ragusano
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- School of Medicine, American University of the Caribbean, Cupecoy, Sint Maarten
| | - Anna-Marika Bauer
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Critical Path Institute, Tucson, Arizona
| | - Callie A. Scott
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Cohere Health, Ann Arbor, Michigan
| | | | - Robert Schnoll
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Rinad S. Beidas
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, Illinois
| | | | - Ravi B. Parikh
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| |
Collapse
|
4
|
Libert Y, Langhendries C, Choucroun L, Merckaert I. Interventions aiming to improve advance care planning uptake in oncology: a scoping review of recent randomized controlled trials. Curr Opin Oncol 2024; 36:233-247. [PMID: 38742390 PMCID: PMC11155288 DOI: 10.1097/cco.0000000000001045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/16/2024]
Abstract
PURPOSE OF REVIEW The evidence regarding the effectiveness of interventions aiming to improve ACP uptake appears mixed. The purpose of this scoping review is to summarize findings from recent RCTs assessing the efficacy of interventions designed to increase ACP uptake in cancer care. RECENT FINDINGS The PubMed database was searched for studies published between 2021 and 2023. Ten RCTs were identified that evaluated a wide range of interventions aimed at encouraging or facilitating ACP communication among patients with advanced cancer, caregivers, and HCPs. Three RCTs developed ACP facilitation tools for patients and sometimes caregivers. Six RCTs involved integrating trained ACP facilitators into patient care, and one RCT used nudges to encourage oncologists trained in ACP discussions to initiate discussions with patients at highest risk of short-term mortality. Most of the reviewed interventions were based on previously tested intervention approaches or tools. SUMMARY The key effects identified in this review are the facilitation of ACP discussions among patients, caregivers, and HCPs, and the documentation of patient preferences in the medical record. The direct involvement of facilitators in ACP document drafting and inclusion in patient files, as well as interventions prompting end-of-life discussions, contributed to this effect.
Collapse
Affiliation(s)
- Yves Libert
- Université libre de Bruxelles (ULB), Faculté des Sciences Psychologiques et de l’Éducation
- Université libre de Bruxelles (ULB), Hôpital Universitaire de Bruxelles (H.U.B), Institut Jules Bordet, Service de Psychologie (Secteur Psycho-Oncologie)
| | - Caroline Langhendries
- Université libre de Bruxelles (ULB), Faculté des Sciences Psychologiques et de l’Éducation
- Université libre de Bruxelles (ULB), Hôpital Universitaire de Bruxelles (H.U.B), Institut Jules Bordet, Service de Psychologie (Secteur Psycho-Oncologie)
| | - Lisa Choucroun
- Université libre de Bruxelles (ULB), Faculté des Sciences Psychologiques et de l’Éducation
- Université libre de Bruxelles (ULB), Hôpital Universitaire de Bruxelles (H.U.B), Institut Jules Bordet, Service de Psychologie (Secteur Psycho-Oncologie)
| | - Isabelle Merckaert
- Université libre de Bruxelles (ULB), Faculté des Sciences Psychologiques et de l’Éducation
- Université libre de Bruxelles (ULB), Hôpital Universitaire de Bruxelles (H.U.B), Institut Jules Bordet, Service de Psychologie (Secteur Psycho-Oncologie)
- Centre de Psycho-Oncologie, Brussels, Belgium
| |
Collapse
|
5
|
Rodriguez GM, Parikh DA, Kapphahn K, Gupta DM, Fan AC, Shah S, Srinivas S, Teuteberg W, Seevaratnam B, Asuncion K, Chien J, Moore K, Ruiz SM, Patel MI. Coaches Activating, Reaching, and Engaging Patients to Engage in Advance Care Planning: A Randomized Clinical Trial. JAMA Oncol 2024; 10:949-953. [PMID: 38780960 PMCID: PMC11117149 DOI: 10.1001/jamaoncol.2024.1242] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2023] [Accepted: 12/21/2023] [Indexed: 05/25/2024]
Abstract
Importance Advance care planning (ACP) remains low among patients with advanced cancer. Multilevel interventions compared with clinician-level interventions may be more effective in improving ACP. Objective To evaluate whether a multilevel intervention could improve clinician-documented ACP compared with a clinician-level intervention alone. Design, Setting, and Participants This randomized clinical trial, performed from September 12, 2019, through May 12, 2021, included adults with advanced genitourinary cancers at an academic, tertiary hospital. Data analysis was performed by intention to treat from May 1 to August 10, 2023. Intervention Participants were randomized 1:1 to a 6-month patient-level lay health worker structured ACP education along with a clinician-level intervention composed of 3-hour ACP training and integration of a structured electronic health record documentation template (intervention group) or to the clinician-level intervention alone (control group). Main Outcome and Measures The primary outcome was ACP documentation in the electronic health record by the oncology clinician within 12 months after randomization. Secondary, exploratory outcomes included shared decision-making, palliative care use, hospice use, emergency department visits, and hospitalizations within 12 months after randomization. Results Among 402 participants enrolled in the study, median age was 71 years (range, 21-102 years); 361 (89.8%) identified as male. More intervention group participants had oncology clinician-documented ACP than control group participants (82 [37.8%] vs 40 [21.6%]; odds ratio [OR], 2.29; 95% CI, 1.44-3.64). At 12-month follow-up, more intervention than control group participants had palliative care (72 [33.2%] vs 25 [13.5%]; OR, 3.18; 95% CI, 1.91-5.28) and hospice use (49 [22.6%] vs 19 [10.3%]; OR, 2.54; 95% CI, 1.44-4.51). There were no differences in the proportion of participants between groups with an emergency department visit (65 [30.0%] vs 61 [33.0%]; OR, 0.87; 95% CI, 0.57-1.33) or hospitalization (89 [41.0%] vs 85 [46.0%]; OR, 0.82; 95% CI, 0.55-1.22). Intervention group participants had fewer hospitalizations than control group participants (mean [SD] number of hospitalizations per year, 0.87 [1.60] vs 1.04 [1.77]) and a lower risk of hospitalization (incidence rate ratio, 0.80; 95% CI, 0.65-0.98). Conclusions and Relevance In this randomized clinical trial, a multilevel intervention improved oncology clinician-documented ACP compared with a clinician-level intervention alone for patients with genitourinary cancer. The intervention is one approach to effectively increase ACP among patients with cancer. Trial Registration ClinicalTrials.gov Identifier: NCT03856463.
Collapse
Affiliation(s)
- Gladys M. Rodriguez
- Department of Medicine, Feinberg School of Medicine at Northwestern University, Chicago, Illinois
| | - Divya A. Parikh
- Department of Medicine, Stanford University, Stanford, California
- Medical Services, VA Palo Alto Health Care System, Palo Alto, California
| | - Kris Kapphahn
- Quantitative Sciences Unit, Stanford University School of Medicine, Stanford, California
| | - Divya M. Gupta
- Department of Medicine, Feinberg School of Medicine at Northwestern University, Chicago, Illinois
| | - Alice C. Fan
- Department of Medicine, Stanford University, Stanford, California
| | - Sumit Shah
- Department of Medicine, Stanford University, Stanford, California
| | - Sandy Srinivas
- Department of Medicine, Stanford University, Stanford, California
| | | | | | - Khay Asuncion
- Department of Medicine, Stanford University, Stanford, California
| | - Joanne Chien
- Department of Medicine, Stanford University, Stanford, California
| | - Kaidi Moore
- Department of Medicine, Stanford University, Stanford, California
| | - Shann Mika Ruiz
- Department of Medicine, Stanford University, Stanford, California
| | - Manali I. Patel
- Department of Medicine, Stanford University, Stanford, California
- Medical Services, VA Palo Alto Health Care System, Palo Alto, California
| |
Collapse
|
6
|
Kolla L, Parikh RB. Uses and limitations of artificial intelligence for oncology. Cancer 2024; 130:2101-2107. [PMID: 38554271 PMCID: PMC11170282 DOI: 10.1002/cncr.35307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2023] [Revised: 02/19/2024] [Accepted: 03/15/2024] [Indexed: 04/01/2024]
Abstract
Modern artificial intelligence (AI) tools built on high-dimensional patient data are reshaping oncology care, helping to improve goal-concordant care, decrease cancer mortality rates, and increase workflow efficiency and scope of care. However, data-related concerns and human biases that seep into algorithms during development and post-deployment phases affect performance in real-world settings, limiting the utility and safety of AI technology in oncology clinics. To this end, the authors review the current potential and limitations of predictive AI for cancer diagnosis and prognostication as well as of generative AI, specifically modern chatbots, which interfaces with patients and clinicians. They conclude the review with a discussion on ongoing challenges and regulatory opportunities in the field.
Collapse
Affiliation(s)
- Likhitha Kolla
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Ravi B. Parikh
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Corporal Michael J. Crescenz VA Medical Center, Philadelphia, PA, USA
| |
Collapse
|
7
|
Piscitello GM, Rogal S, Schell J, Schenker Y, Arnold RM. Equity in Using Artificial Intelligence Mortality Predictions to Target Goals of Care Documentation. J Gen Intern Med 2024:10.1007/s11606-024-08849-w. [PMID: 38858343 DOI: 10.1007/s11606-024-08849-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Accepted: 05/31/2024] [Indexed: 06/12/2024]
Abstract
BACKGROUND Artificial intelligence (AI) algorithms are increasingly used to target patients with elevated mortality risk scores for goals-of-care (GOC) conversations. OBJECTIVE To evaluate the association between the presence or absence of AI-generated mortality risk scores with GOC documentation. DESIGN Retrospective cross-sectional study at one large academic medical center between July 2021 and December 2022. PARTICIPANTS Hospitalized adult patients with AI-defined Serious Illness Risk Indicator (SIRI) scores indicating > 30% 90-day mortality risk (defined as "elevated" SIRI) or no SIRI scores due to insufficient data. INTERVENTION A targeted intervention to increase GOC documentation for patients with AI-generated scores predicting elevated risk of mortality. MAIN MEASURES Odds ratios comparing GOC documentation for patients with elevated or no SIRI scores with similar severity of illness using propensity score matching and risk-adjusted mixed-effects logistic regression. KEY RESULTS Among 13,710 patients with elevated (n = 3643, 27%) or no (n = 10,067, 73%) SIRI scores, the median age was 64 years (SD 18). Twenty-five percent were non-White, 18% had Medicaid, 43% were admitted to an intensive care unit, and 11% died during admission. Patients lacking SIRI scores were more likely to be younger (median 60 vs. 72 years, p < 0.0001), be non-White (29% vs. 13%, p < 0.0001), and have Medicaid (22% vs. 9%, p < 0.0001). Patients with elevated versus no SIRI scores were more likely to have GOC documentation in the unmatched (aOR 2.5, p < 0.0001) and propensity-matched cohorts (aOR 2.1, p < 0.0001). CONCLUSIONS Using AI predictions of mortality to target GOC documentation may create differences in documentation prevalence between patients with and without AI mortality prediction scores with similar severity of illness. These finding suggest using AI to target GOC documentation may have the unintended consequence of disadvantaging severely ill patients lacking AI-generated scores from receiving targeted GOC documentation, including patients who are more likely to be non-White and have Medicaid insurance.
Collapse
Affiliation(s)
- Gina M Piscitello
- Division of General Internal Medicine, Section of Palliative Care and Medical Ethics, University of Pittsburgh, Pittsburgh, PA, USA.
- Palliative Research Center, University of Pittsburgh, Pittsburgh, PA, USA.
| | - Shari Rogal
- Departments of Medicine and Surgery, University of Pittsburgh, Pittsburgh, PA, USA
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare Center, Pittsburgh, PA, USA
| | - Jane Schell
- Division of General Internal Medicine, Section of Palliative Care and Medical Ethics, University of Pittsburgh, Pittsburgh, PA, USA
- Palliative Research Center, University of Pittsburgh, Pittsburgh, PA, USA
| | - Yael Schenker
- Division of General Internal Medicine, Section of Palliative Care and Medical Ethics, University of Pittsburgh, Pittsburgh, PA, USA
- Palliative Research Center, University of Pittsburgh, Pittsburgh, PA, USA
| | - Robert M Arnold
- Department of Geriatrics and Palliative Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|
8
|
Patel TA, Heintz J, Chen J, LaPergola M, Bilker WB, Patel MS, Arya LA, Patel MI, Bekelman JE, Manz CR, Parikh RB. Spending Analysis of Machine Learning-Based Communication Nudges in Oncology. NEJM AI 2024; 1:10.1056/aioa2300228. [PMID: 39036423 PMCID: PMC11259034 DOI: 10.1056/aioa2300228] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/23/2024]
Abstract
BACKGROUND Serious illness conversations (SICs) in the outpatient setting may improve mood and quality of life among patients with cancer and decrease aggressive end-of-life care. Interventions informed by behavioral economics may increase rates of SICs between oncology clinicians and patients, but the impact of these interventions on end-of-life spending is unknown. METHODS This study is a secondary analysis of a stepped-wedge cluster randomized, controlled trial that involved nine medical oncology practices and their high-risk patients at a large academic institution between June 2019 and April 2020. The study included 1187 patients who were identified by a machine-learning algorithm as high risk of 180-day mortality and who died by December 2020. The patients were randomly assigned to standard of care (controls) or to a behavioral intervention designed to increase clinician-initiated SICs. We abstracted spending - defined as inflation-adjusted costs for acute care (inpatient plus emergency room), office/outpatient care, intravenous systemic therapy, other therapy (e.g., radiation), long-term care, and hospice - from the institution's accounting system, and we captured spending at inpatient, outpatient, and pharmacy settings. To evaluate intervention impacts on spending, we used a two-part model, first using logistic regression to model zero versus nonzero spending and second using generalized linear mixed models with gamma distribution and log-link function to model daily mean spending in the last 180days of life. Models were adjusted for clinic and wedge fixed effects, and they were clustered at the oncologist level. For all patients with at least one SIC within 6 months of death, we also calculated their mean daily spending before and after SIC. RESULTS Median age at death was 68years (interquartile range, 15.5), 317 patients (27%) were Black or of ethnicities other than white, and 448 patients (38%) had an SIC. The intervention was associated with lower unadjusted mean daily spending in the last 6 months of life for the intervention group versus controls ($377.96 vs. $449.92; adjusted mean difference, -$75.33; 95% confidence interval, -$136.42 to -$14.23; P=0.02), translating to $13,747 total adjusted savings per decedent and $13 million in cumulative savings across all decedents in the intervention group. Compared with controls, patients in the intervention group incurred lower mean daily spending for systemic therapy (adjusted difference, -$44.59; P=0.001), office/outpatient care (-$9.62; P=0.001), and other therapy (-$8.65; P=0.04). The intervention was not associated with differences in end-of-life spending for acute care, long-term care, or hospice. Results were consistent for spending in the last 1 and 3 months of life and after adjusting for age, race, and ethnicity. For patients with SICs, mean daily spending decreased by $37.92 following the first SIC ($329.87 vs. $291.95). CONCLUSIONS A machine learning-based, behaviorally informed intervention to prompt SICs led to end-of-life savings among patients with cancer, driven by decreased systemic therapy and outpatient spending. (Funded by the Penn Center for Precision Medicine and the National Institutes of Health; ClinicalTrials.gov number, NCT03984773.).
Collapse
Affiliation(s)
| | - Jonathan Heintz
- Biostatistics Analysis Center, Perelman School of Medicine, University of Pennsylvania Health System, Philadelphia
| | - Jinbo Chen
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | | | - Warren B Bilker
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | | | - Lily A Arya
- University of Pennsylvania, Philadelphia
- University of Pennsylvania Health System, Philadelphia
| | - Manali I Patel
- Stanford University School of Medicine, Stanford, CA
- Veterans Affairs Palo Alto Health Care System, Palo Alto, CA
| | - Justin E Bekelman
- Division of Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia
| | | | - Ravi B Parikh
- Division of Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia
| |
Collapse
|
9
|
Shilling DM, Manz CR, Strand JJ, Patel MI. Let Us Have the Conversation: Serious Illness Communication in Oncology: Definitions, Barriers, and Successful Approaches. Am Soc Clin Oncol Educ Book 2024; 44:e431352. [PMID: 38788187 DOI: 10.1200/edbk_431352] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/26/2024]
Abstract
Serious illness communications are crucial elements of care delivery for patients with cancer. High-quality serious illness communications are composed of open, honest discussions between patients, caregivers, and clinicians regarding patient's communication preferences, expected illness trajectory, prognosis, and risks and benefits of any recommended care. High-quality communication ideally starts at the time of a patients' cancer diagnosis, allows space for and response to patient emotions, elicits patients' values and care preferences, and is iterative and longitudinal. When integrated into cancer care, such communication can result in improved patient experiences with their care, care that matches patients' goals, and reduced care intensity at the end of life. Despite national recommendations for routine integration of these communication into cancer care, a minority of patients with cancer receive such communication. In this chapter, we describe elements of high-quality serious illness communication, patient-, clinician-, institution-, and payer-level barriers, and successful strategies that can routinely integrate such communication into cancer care delivery.
Collapse
Affiliation(s)
- Danielle M Shilling
- Division of Community Internal Medicine, Geriatrics & Palliative Care, Mayo Clinic, Rochester, MN
| | - Christopher R Manz
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA
- Department of Medicine, Harvard Medical School, Boston, MA
| | - Jacob J Strand
- Division of Community Internal Medicine, Geriatrics & Palliative Care, Mayo Clinic, Rochester, MN
| | - Manali I Patel
- Division of Oncology, Stanford University School of Medicine, Stanford, CA
- VA Palo Alto Health Care System, Palo Alto, CA
| |
Collapse
|
10
|
Petrillo LA, Jones KF, El-Jawahri A, Sanders J, Greer JA, Temel JS. Why and How to Integrate Early Palliative Care Into Cutting-Edge Personalized Cancer Care. Am Soc Clin Oncol Educ Book 2024; 44:e100038. [PMID: 38815187 DOI: 10.1200/edbk_100038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/01/2024]
Abstract
Early palliative care, palliative care integrated with oncology care early in the course of illness, has myriad benefits for patients and their caregivers, including improved quality of life, reduced physical and psychological symptom burden, enhanced prognostic awareness, and reduced health care utilization at the end of life. Although ASCO and others recommend early palliative care for all patients with advanced cancer, widespread implementation of early palliative care has not been realized because of barriers such as insufficient reimbursement and a palliative care workforce shortage. Investigators have recently tested several implementation strategies to overcome these barriers, including triggers for palliative care consultations, telehealth delivery, navigator-delivered interventions, and primary palliative care interventions. More research is needed to identify mechanisms to distribute palliative care optimally and equitably. Simultaneously, the transformation of the oncology treatment landscape has led to shifts in the supportive care needs of patients and caregivers, who may experience longer, uncertain trajectories of cancer. Now, palliative care also plays a clear role in the care of patients with hematologic malignancies and may be beneficial for patients undergoing phase I clinical trials and their caregivers. Further research and clinical guidance regarding how to balance the risks and benefits of opioid therapy and safely manage cancer-related pain across this wide range of settings are urgently needed. The strengths of early palliative care in supporting patients' and caregivers' coping and centering decisions on their goals and values remain valuable in the care of patients receiving cutting-edge personalized cancer care.
Collapse
Affiliation(s)
- Laura A Petrillo
- Division of Palliative Care and Geriatrics, Massachusetts General Hospital, Boston, MA
- Harvard Medical School, Boston, MA
| | - Katie Fitzgerald Jones
- Harvard Medical School, Boston, MA
- New England Geriatrics Research, Education, and Clinical Center (GRECC), Jamaica Plain, MA
| | - Areej El-Jawahri
- Harvard Medical School, Boston, MA
- Division of Hematology/Oncology, Massachusetts General Hospital, Boston, MA
| | - Justin Sanders
- Division of Supportive and Palliative Care, McGill University Health Centre, Montreal, CA
- Department of Family Medicine, McGill University, Montreal, CA
| | - Joseph A Greer
- Harvard Medical School, Boston, MA
- Department of Psychiatry, Massachusetts General Hospital, Boston, MA
| | - Jennifer S Temel
- Harvard Medical School, Boston, MA
- Division of Hematology/Oncology, Massachusetts General Hospital, Boston, MA
| |
Collapse
|
11
|
Baird CE, Wulff-Burchfield E, Egan PC, Hugar LA, Vyas A, Trikalinos NA, Liu MA, Bélanger E, Olszewski AJ, Bantis LE, Panagiotou OA. Predictors of high-intensity care at the end of life among older adults with solid tumors: A population-based study. J Geriatr Oncol 2024; 15:101774. [PMID: 38676975 PMCID: PMC11162260 DOI: 10.1016/j.jgo.2024.101774] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2023] [Revised: 03/05/2024] [Accepted: 04/12/2024] [Indexed: 04/29/2024]
Abstract
INTRODUCTION High-intensity end-of-life (EoL) care can be burdensome for patients, caregivers, and health systems and does not confer any meaningful clinical benefit. Yet, there are significant knowledge gaps regarding the predictors of high-intensity EoL care. In this study, we identify risk factors associated with high-intensity EoL care among older adults with the four most common malignancies, including breast, prostate, lung, and colorectal cancer. MATERIALS AND METHODS Using SEER-Medicare data, we conducted a retrospective analysis of Medicare beneficiaries aged 65 and older who died of breast, prostate, lung, or colorectal cancer between 2011 and 2015. We used multivariable logistic regression to identify clinical, demographic, socioeconomic, and geographic predictors of high-intensity EoL care, which we defined as death in an acute care hospital, receipt of any oral or parenteral chemotherapy within 14 days of death, one or more admissions to the intensive care unit within 30 days of death, two or more emergency department visits within 30 days of death, or two or more inpatient admissions within 30 days of death. RESULTS Among 59,355 decedents, factors associated with increased likelihood of receiving high-intensity EoL care were increased comorbidity burden (odds ratio [OR]:1.29; 95% confidence interval [CI]:1.28-1.30), female sex (OR:1.05; 95% CI:1.01-1.09), Black race (OR:1.14; 95% CI:1.07-1.23), Other race/ethnicity (OR:1.20; 95% CI:1.10-1.30), stage III disease (OR:1.11; 95% CI:1.05-1.18), living in a county with >1,000,000 people (OR:1.23; 95% CI:1.16-1.31), living in a census tract with 10%-<20% poverty (OR:1.09; 95% CI:1.03-1.16) or 20%-100% poverty (OR:1.12; 95% CI:1.04-1.19), and having state-subsidized Medicare premiums (OR:1.18; 95% CI:1.12-1.24). The risk of high-intensity EoL care was lower among patients who were older (OR:0.98; 95% CI:0.98-0.99), lived in the Midwest (OR:0.69; 95% CI:0.65-0.75), South (OR:0.70; 95% CI:0.65-0.74), or West (OR:0.81; 95% CI:0.77-0.86), lived in mostly rural areas (OR:0.92; 95% CI:0.86-1.00), and had poor performance status (OR:0.26; 95% CI:0.25-0.28). Results were largely consistent across cancer types. DISCUSSION The risk factors identified in our study can inform the development of new interventions for patients with cancer who are likely to receive high-intensity EoL care. Health systems should consider incorporating these risk factors into decision-support tools to assist clinicians in identifying which patients should be referred to hospice and palliative care.
Collapse
Affiliation(s)
- Courtney E Baird
- Center for Gerontology and Healthcare Research, Department of Health Services, Policy and Practice, Brown University School of Public Health, Providence, RI, USA.
| | - Elizabeth Wulff-Burchfield
- Medical Oncology Division and Palliative Medicine Division, Department of Internal Medicine, University of Kansas School of Medicine, University of Kansas Cancer Center, The University of Kansas Health System, Kansas City, KS, USA
| | - Pamela C Egan
- Department of Medicine, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Lee A Hugar
- Department of Genitourinary Oncology, H. Lee Moffitt Cancer Center, Tampa, FL, USA
| | - Ami Vyas
- University of Rhode Island, College of Pharmacy, Department of Pharmacy Practice, Kingston, RI, USA
| | - Nikolaos A Trikalinos
- Division of Oncology, Department of Medicine, Washington University Medical School Campus, St. Louis, MO, USA; Siteman Cancer Center, St. Louis, MO, USA
| | - Michael A Liu
- Columbia University Medical Center, Herbert Irving Comprehensive Cancer Center, New York, NY, USA
| | - Emmanuelle Bélanger
- Center for Gerontology and Healthcare Research, Department of Health Services, Policy and Practice, Brown University School of Public Health, Providence, RI, USA
| | - Adam J Olszewski
- Department of Medicine, The Warren Alpert Medical School of Brown University, Providence, RI, USA
| | - Leonidas E Bantis
- Department of Biostatistics and Data Science, University of Kansas Medical Center, Kansas City, KS, USA
| | - Orestis A Panagiotou
- Center for Gerontology and Healthcare Research, Department of Health Services, Policy and Practice, Brown University School of Public Health, Providence, RI, USA
| |
Collapse
|
12
|
Zhuang Q, Zhang AY, Cong RSTY, Yang GM, Neo PSH, Tan DS, Chua ML, Tan IB, Wong FY, Eng Hock Ong M, Shao Wei Lam S, Liu N. Towards proactive palliative care in oncology: developing an explainable EHR-based machine learning model for mortality risk prediction. BMC Palliat Care 2024; 23:124. [PMID: 38769564 PMCID: PMC11103848 DOI: 10.1186/s12904-024-01457-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2023] [Accepted: 05/15/2024] [Indexed: 05/22/2024] Open
Abstract
BACKGROUND Ex-ante identification of the last year in life facilitates a proactive palliative approach. Machine learning models trained on electronic health records (EHR) demonstrate promising performance in cancer prognostication. However, gaps in literature include incomplete reporting of model performance, inadequate alignment of model formulation with implementation use-case, and insufficient explainability hindering trust and adoption in clinical settings. Hence, we aim to develop an explainable machine learning EHR-based model that prompts palliative care processes by predicting for 365-day mortality risk among patients with advanced cancer within an outpatient setting. METHODS Our cohort consisted of 5,926 adults diagnosed with Stage 3 or 4 solid organ cancer between July 1, 2017, and June 30, 2020 and receiving ambulatory cancer care within a tertiary center. The classification problem was modelled using Extreme Gradient Boosting (XGBoost) and aligned to our envisioned use-case: "Given a prediction point that corresponds to an outpatient cancer encounter, predict for mortality within 365-days from prediction point, using EHR data up to 365-days prior." The model was trained with 75% of the dataset (n = 39,416 outpatient encounters) and validated on a 25% hold-out dataset (n = 13,122 outpatient encounters). To explain model outputs, we used Shapley Additive Explanations (SHAP) values. Clinical characteristics, laboratory tests and treatment data were used to train the model. Performance was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC), while model calibration was assessed using the Brier score. RESULTS In total, 17,149 of the 52,538 prediction points (32.6%) had a mortality event within the 365-day prediction window. The model demonstrated an AUROC of 0.861 (95% CI 0.856-0.867) and AUPRC of 0.771. The Brier score was 0.147, indicating slight overestimations of mortality risk. Explanatory diagrams utilizing SHAP values allowed visualization of feature impacts on predictions at both the global and individual levels. CONCLUSION Our machine learning model demonstrated good discrimination and precision-recall in predicting 365-day mortality risk among individuals with advanced cancer. It has the potential to provide personalized mortality predictions and facilitate earlier integration of palliative care.
Collapse
Affiliation(s)
- Qingyuan Zhuang
- Division of Supportive and Palliative Care, National Cancer Centre Singapore, 30 Hospital Blvd, Singapore, 168583, Singapore.
- Data Computational Science Core, National Cancer Centre Singapore, Singapore, Singapore.
| | - Alwin Yaoxian Zhang
- Division of Supportive and Palliative Care, National Cancer Centre Singapore, 30 Hospital Blvd, Singapore, 168583, Singapore
| | - Ryan Shea Tan Ying Cong
- Division of Medical Oncology, National Cancer Centre Singapore, Singapore, Singapore
- Data Computational Science Core, National Cancer Centre Singapore, Singapore, Singapore
| | - Grace Meijuan Yang
- Division of Supportive and Palliative Care, National Cancer Centre Singapore, 30 Hospital Blvd, Singapore, 168583, Singapore
- Lien Centre of Palliative Care, Duke-NUS Medical School, Singapore, Singapore
| | - Patricia Soek Hui Neo
- Division of Supportive and Palliative Care, National Cancer Centre Singapore, 30 Hospital Blvd, Singapore, 168583, Singapore
| | - Daniel Sw Tan
- Division of Medical Oncology, National Cancer Centre Singapore, Singapore, Singapore
- Division of Clinical Trials and Epidemiological Sciences, National Cancer Centre Singapore, Singapore, Singapore
| | - Melvin Lk Chua
- Data Computational Science Core, National Cancer Centre Singapore, Singapore, Singapore
- Division of Radiation Oncology, National Cancer Centre Singapore, Singapore, Singapore
| | - Iain Beehuat Tan
- Division of Medical Oncology, National Cancer Centre Singapore, Singapore, Singapore
- Data Computational Science Core, National Cancer Centre Singapore, Singapore, Singapore
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore, Singapore
| | - Fuh Yong Wong
- Division of Radiation Oncology, National Cancer Centre Singapore, Singapore, Singapore
- Department of Cancer Informatics, National Cancer Centre Singapore, Singapore, Singapore
| | - Marcus Eng Hock Ong
- Health Services Research Centre, SingHealth, Singapore
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Sean Shao Wei Lam
- Health Services Research Centre, SingHealth, Singapore
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| |
Collapse
|
13
|
Han R, Acosta JN, Shakeri Z, Ioannidis JPA, Topol EJ, Rajpurkar P. Randomised controlled trials evaluating artificial intelligence in clinical practice: a scoping review. Lancet Digit Health 2024; 6:e367-e373. [PMID: 38670745 PMCID: PMC11068159 DOI: 10.1016/s2589-7500(24)00047-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 03/01/2024] [Accepted: 03/04/2024] [Indexed: 04/28/2024]
Abstract
This scoping review of randomised controlled trials on artificial intelligence (AI) in clinical practice reveals an expanding interest in AI across clinical specialties and locations. The USA and China are leading in the number of trials, with a focus on deep learning systems for medical imaging, particularly in gastroenterology and radiology. A majority of trials (70 [81%] of 86) report positive primary endpoints, primarily related to diagnostic yield or performance; however, the predominance of single-centre trials, little demographic reporting, and varying reports of operational efficiency raise concerns about the generalisability and practicality of these results. Despite the promising outcomes, considering the likelihood of publication bias and the need for more comprehensive research including multicentre trials, diverse outcome measures, and improved reporting standards is crucial. Future AI trials should prioritise patient-relevant outcomes to fully understand AI's true effects and limitations in health care.
Collapse
Affiliation(s)
- Ryan Han
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA; Department of Computer Science, Stanford University, Stanford, CA, USA; University of California Los Angeles-Caltech Medical Scientist Training Program, Los Angeles, CA, USA
| | - Julián N Acosta
- Department of Neurology, Yale School of Medicine, New Haven, CT, USA; Rad AI, San Francisco, CA, USA
| | - Zahra Shakeri
- Institute of Health Policy, Management and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
| | - John P A Ioannidis
- Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, CA, USA; Meta-Research Innovation Center at Stanford, Stanford University, Stanford, CA, USA
| | - Eric J Topol
- Scripps Research Translational Institute, Scripps Research, La Jolla, CA, USA.
| | - Pranav Rajpurkar
- Department of Biomedical Informatics, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
14
|
Herskovits AZ, Newman T, Nicholas K, Colorado-Jimenez CF, Perry CE, Valentino A, Wagner I, Egan B, Gorenshteyn D, Vickers AJ, Pessin MS. Comparing Clinician Estimates versus a Statistical Tool for Predicting Risk of Death within 45 Days of Admission for Cancer Patients. Appl Clin Inform 2024; 15:489-500. [PMID: 38925539 PMCID: PMC11208110 DOI: 10.1055/s-0044-1787185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Accepted: 04/29/2024] [Indexed: 06/28/2024] Open
Abstract
OBJECTIVES While clinical practice guidelines recommend that oncologists discuss goals of care with patients who have advanced cancer, it is estimated that less than 20% of individuals admitted to the hospital with high-risk cancers have end-of-life discussions with their providers. While there has been interest in developing models for mortality prediction to trigger such discussions, few studies have compared how such models compare with clinical judgment to determine a patient's mortality risk. METHODS This study is a prospective analysis of 1,069 solid tumor medical oncology hospital admissions (n = 911 unique patients) from February 7 to June 7, 2022, at Memorial Sloan Kettering Cancer Center. Electronic surveys were sent to hospitalists, advanced practice providers, and medical oncologists the first afternoon following a hospital admission and they were asked to estimate the probability that the patient would die within 45 days. Provider estimates of mortality were compared with those from a predictive model developed using a supervised machine learning methodology, and incorporated routine laboratory, demographic, biometric, and admission data. Area under the receiver operating characteristic curve (AUC), calibration and decision curves were compared between clinician estimates and the model predictions. RESULTS Within 45 days following hospital admission, 229 (25%) of 911 patients died. The model performed better than the clinician estimates (AUC 0.834 vs. 0.753, p < 0.0001). Integrating clinician predictions with the model's estimates further increased the AUC to 0.853 (p < 0.0001). Clinicians overestimated risk whereas the model was extremely well-calibrated. The model demonstrated net benefit over a wide range of threshold probabilities. CONCLUSION The inpatient prognosis at admission model is a robust tool to assist clinical providers in evaluating mortality risk, and it has recently been implemented in the electronic medical record at our institution to improve end-of-life care planning for hospitalized cancer patients.
Collapse
Affiliation(s)
- Adrianna Z. Herskovits
- Department of Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | - Tiffanny Newman
- Department of Strategy and Innovation, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | - Kevin Nicholas
- Department of Strategy and Innovation, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | - Cesar F. Colorado-Jimenez
- Department of Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | - Claire E. Perry
- Department of Strategy and Innovation, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | - Alisa Valentino
- Department of Pathology and Laboratory Medicine, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | - Isaac Wagner
- Department of Strategy and Innovation, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | - Barbara Egan
- Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | | | - Andrew J. Vickers
- Department of Epidemiology and Biostatistics, Memorial Sloan Kettering Cancer Center, New York, New York, United States
| | - Melissa S. Pessin
- Department of Pathology, University of Chicago, Chicago, Illinois, United States
| |
Collapse
|
15
|
Rotenstein L, Wang L, Zupanc SN, Penumarthy A, Laurentiev J, Lamey J, Farah S, Lipsitz S, Jain N, Bates DW, Zhou L, Lakin JR. Looking Beyond Mortality Prediction: Primary Care Physician Views of Patients' Palliative Care Needs Predicted by a Machine Learning Tool. Appl Clin Inform 2024; 15:460-468. [PMID: 38636542 PMCID: PMC11168809 DOI: 10.1055/a-2309-1599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2023] [Accepted: 04/17/2024] [Indexed: 04/20/2024] Open
Abstract
OBJECTIVES To assess primary care physicians' (PCPs) perception of the need for serious illness conversations (SIC) or other palliative care interventions in patients flagged by a machine learning tool for high 1-year mortality risk. METHODS We surveyed PCPs from four Brigham and Women's Hospital primary care practice sites. Multiple mortality prediction algorithms were ensembled to assess adult patients of these PCPs who were either enrolled in the hospital's integrated care management program or had one of several chronic conditions. The patients were classified as high or low risk of 1-year mortality. A blinded survey had PCPs evaluate these patients for palliative care needs. We measured PCP and machine learning tool agreement regarding patients' need for an SIC/elevated risk of mortality. RESULTS Of 66 PCPs, 20 (30.3%) participated in the survey. Out of 312 patients evaluated, 60.6% were female, with a mean (standard deviation [SD]) age of 69.3 (17.5) years, and a mean (SD) Charlson Comorbidity Index of 2.80 (2.89). The machine learning tool identified 162 (51.9%) patients as high risk. Excluding deceased or unfamiliar patients, PCPs felt that an SIC was appropriate for 179 patients; the machine learning tool flagged 123 of these patients as high risk (68.7% concordance). For 105 patients whom PCPs deemed SIC unnecessary, the tool classified 83 as low risk (79.1% concordance). There was substantial agreement between PCPs and the tool (Gwet's agreement coefficient of 0.640). CONCLUSIONS A machine learning mortality prediction tool offers promise as a clinical decision aid, helping clinicians pinpoint patients needing palliative care interventions.
Collapse
Affiliation(s)
- Lisa Rotenstein
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- School of Medicine, University of California, San Francisco, San Francisco, California, United States
| | - Liqin Wang
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
| | - Sophia N. Zupanc
- School of Medicine, University of California, San Francisco, San Francisco, California, United States
- Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute, Boston, Massachusetts, United States
| | - Akhila Penumarthy
- Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute, Boston, Massachusetts, United States
| | - John Laurentiev
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
| | - Jan Lamey
- Brigham and Women's Physician Organization, Brigham and Women's Hospital, Boston, Massachusetts, United States
| | - Subrina Farah
- Department of Biostatistics and Computational Biology, Dana-Farber Cancer Institute, Boston, Massachusetts, United States
| | - Stuart Lipsitz
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
| | - Nina Jain
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
| | - David W. Bates
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
| | - Li Zhou
- Division of General Internal Medicine and Primary Care, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
- Harvard Medical School, Boston, Massachusetts, United States
| | - Joshua R. Lakin
- Harvard Medical School, Boston, Massachusetts, United States
- Department of Psychosocial Oncology and Palliative Care, Dana-Farber Cancer Institute, Boston, Massachusetts, United States
- Division of Palliative Medicine, Brigham and Women's Hospital, Boston, Massachusetts, United States
| |
Collapse
|
16
|
Haimovich AD, Burke RC, Nathanson LA, Rubins D, Taylor RA, Kross EK, Ouchi K, Shapiro NI, Schonberg MA. Geriatric End-of-Life Screening Tool Prediction of 6-Month Mortality in Older Patients. JAMA Netw Open 2024; 7:e2414213. [PMID: 38819823 PMCID: PMC11143461 DOI: 10.1001/jamanetworkopen.2024.14213] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Accepted: 03/31/2024] [Indexed: 06/01/2024] Open
Abstract
Importance Emergency department (ED) visits by older adults with life-limiting illnesses are a critical opportunity to establish patient care end-of-life preferences, but little is known about the optimal screening criteria for resource-constrained EDs. Objectives To externally validate the Geriatric End-of-Life Screening Tool (GEST) in an independent population and compare it with commonly used serious illness diagnostic criteria. Design, Setting, and Participants This prognostic study assessed a cohort of patients aged 65 years and older who were treated in a tertiary care ED in Boston, Massachusetts, from 2017 to 2021. Patients arriving in cardiac arrest or who died within 1 day of ED arrival were excluded. Data analysis was performed from August 1, 2023, to March 27, 2024. Exposure GEST, a logistic regression algorithm that uses commonly available electronic health record (EHR) datapoints and was developed and validated across 9 EDs, was compared with serious illness diagnoses as documented in the EHR. Serious illnesses included stroke/transient ischemic attack, liver disease, cancer, lung disease, and age greater than 80 years, among others. Main Outcomes and Measures The primary outcome was 6-month mortality following an ED encounter. Statistical analyses included area under the receiver operating characteristic curve, calibration analyses, Kaplan-Meier survival curves, and decision curves. Results This external validation included 82 371 ED encounters by 40 505 unique individuals (mean [SD] age, 76.8 [8.4] years; 54.3% women, 13.8% 6-month mortality rate). GEST had an external validation area under the receiver operating characteristic curve of 0.79 (95% CI, 0.78-0.79) that was stable across years and demographic subgroups. Of included encounters, 53.4% had a serious illness, with a sensitivity of 77.4% (95% CI, 76.6%-78.2%) and specificity of 50.5% (95% CI, 50.1%-50.8%). Varying GEST cutoffs from 5% to 30% increased specificity (5%: 49.1% [95% CI, 48.7%-49.5%]; 30%: 92.2% [95% CI, 92.0%-92.4%]) at the cost of sensitivity (5%: 89.3% [95% CI, 88.8-89.9]; 30%: 36.2% [95% CI, 35.3-37.1]). In a decision curve analysis, GEST outperformed serious illness criteria across all tested thresholds. When comparing patients referred to intervention by GEST with serious illness criteria, GEST reclassified 45.1% of patients with serious illness as having low risk of mortality with an observed mortality rate 8.1% and 2.6% of patients without serious illness as having high mortality risk with an observed mortality rate of 34.3% for a total reclassification rate of 25.3%. Conclusions and Relevance The findings of this study suggest that both serious illness criteria and GEST identified older ED patients at risk for 6-month mortality, but GEST offered more useful screening characteristics. Future trials of serious illness interventions for high mortality risk in older adults may consider transitioning from diagnosis code criteria to GEST, an automatable EHR-based algorithm.
Collapse
Affiliation(s)
- Adrian D. Haimovich
- Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Ryan C. Burke
- Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Larry A. Nathanson
- Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - David Rubins
- Department of Internal Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - R. Andrew Taylor
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut
| | - Erin K. Kross
- Division of Pulmonary, Critical Care, and Sleep Medicine, University of Washington, Seattle
- Cambia Palliative Care Center of Excellence at UW Medicine, Seattle, Washington
| | - Kei Ouchi
- Department of Emergency Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
| | - Nathan I. Shapiro
- Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| | - Mara A. Schonberg
- Department of Internal Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts
| |
Collapse
|
17
|
Huguet N, Chen J, Parikh RB, Marino M, Flocke SA, Likumahuwa-Ackman S, Bekelman J, DeVoe JE. Applying Machine Learning Techniques to Implementation Science. Online J Public Health Inform 2024; 16:e50201. [PMID: 38648094 DOI: 10.2196/50201] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 11/15/2023] [Accepted: 03/14/2024] [Indexed: 04/25/2024] Open
Abstract
Machine learning (ML) approaches could expand the usefulness and application of implementation science methods in clinical medicine and public health settings. The aim of this viewpoint is to introduce a roadmap for applying ML techniques to address implementation science questions, such as predicting what will work best, for whom, under what circumstances, and with what predicted level of support, and what and when adaptation or deimplementation are needed. We describe how ML approaches could be used and discuss challenges that implementation scientists and methodologists will need to consider when using ML throughout the stages of implementation.
Collapse
Affiliation(s)
- Nathalie Huguet
- Department of Family Medicine, Oregon Health & Science University, Portland, OR, United States
- BRIDGE-C2 Implementation Science Center for Cancer Control, Oregon Health & Science University, Portland, OR, United States
| | - Jinying Chen
- Section of Preventive Medicine and Epidemiology, Department of Medicine, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, United States
- Data Science Core, Boston University Chobanian & Avedisian School of Medicine, Boston, MA, United States
- iDAPT Implementation Science Center for Cancer Control, Wake Forest School of Medicine, Winston-Salem, NC, United States
| | - Ravi B Parikh
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
| | - Miguel Marino
- Department of Family Medicine, Oregon Health & Science University, Portland, OR, United States
- BRIDGE-C2 Implementation Science Center for Cancer Control, Oregon Health & Science University, Portland, OR, United States
| | - Susan A Flocke
- Department of Family Medicine, Oregon Health & Science University, Portland, OR, United States
- BRIDGE-C2 Implementation Science Center for Cancer Control, Oregon Health & Science University, Portland, OR, United States
| | - Sonja Likumahuwa-Ackman
- Department of Family Medicine, Oregon Health & Science University, Portland, OR, United States
- BRIDGE-C2 Implementation Science Center for Cancer Control, Oregon Health & Science University, Portland, OR, United States
| | - Justin Bekelman
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, Penn Medicine, Philadelphia, PA, United States
| | - Jennifer E DeVoe
- Department of Family Medicine, Oregon Health & Science University, Portland, OR, United States
- BRIDGE-C2 Implementation Science Center for Cancer Control, Oregon Health & Science University, Portland, OR, United States
| |
Collapse
|
18
|
Osmanodja B, Sassi Z, Eickmann S, Hansen CM, Roller R, Burchardt A, Samhammer D, Dabrock P, Möller S, Budde K, Herrmann A. Investigating the Impact of AI on Shared Decision-Making in Post-Kidney Transplant Care (PRIMA-AI): Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2024; 13:e54857. [PMID: 38557315 PMCID: PMC11019425 DOI: 10.2196/54857] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Revised: 02/03/2024] [Accepted: 02/05/2024] [Indexed: 04/04/2024] Open
Abstract
BACKGROUND Patients after kidney transplantation eventually face the risk of graft loss with the concomitant need for dialysis or retransplantation. Choosing the right kidney replacement therapy after graft loss is an important preference-sensitive decision for kidney transplant recipients. However, the rate of conversations about treatment options after kidney graft loss has been shown to be as low as 13% in previous studies. It is unknown whether the implementation of artificial intelligence (AI)-based risk prediction models can increase the number of conversations about treatment options after graft loss and how this might influence the associated shared decision-making (SDM). OBJECTIVE This study aims to explore the impact of AI-based risk prediction for the risk of graft loss on the frequency of conversations about the treatment options after graft loss, as well as the associated SDM process. METHODS This is a 2-year, prospective, randomized, 2-armed, parallel-group, single-center trial in a German kidney transplant center. All patients will receive the same routine post-kidney transplant care that usually includes follow-up visits every 3 months at the kidney transplant center. For patients in the intervention arm, physicians will be assisted by a validated and previously published AI-based risk prediction system that estimates the risk for graft loss in the next year, starting from 3 months after randomization until 24 months after randomization. The study population will consist of 122 kidney transplant recipients >12 months after transplantation, who are at least 18 years of age, are able to communicate in German, and have an estimated glomerular filtration rate <30 mL/min/1.73 m2. Patients with multi-organ transplantation, or who are not able to communicate in German, as well as underage patients, cannot participate. For the primary end point, the proportion of patients who have had a conversation about their treatment options after graft loss is compared at 12 months after randomization. Additionally, 2 different assessment tools for SDM, the CollaboRATE mean score and the Control Preference Scale, are compared between the 2 groups at 12 months and 24 months after randomization. Furthermore, recordings of patient-physician conversations, as well as semistructured interviews with patients, support persons, and physicians, are performed to support the quantitative results. RESULTS The enrollment for the study is ongoing. The first results are expected to be submitted for publication in 2025. CONCLUSIONS This is the first study to examine the influence of AI-based risk prediction on physician-patient interaction in the context of kidney transplantation. We use a mixed methods approach by combining a randomized design with a simple quantitative end point (frequency of conversations), different quantitative measurements for SDM, and several qualitative research methods (eg, records of physician-patient conversations and semistructured interviews) to examine the implementation of AI-based risk prediction in the clinic. TRIAL REGISTRATION ClinicalTrials.gov NCT06056518; https://clinicaltrials.gov/study/NCT06056518. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) PRR1-10.2196/54857.
Collapse
Affiliation(s)
- Bilgin Osmanodja
- Department of Nephrology and Medical Intensive Care, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Zeineb Sassi
- Department of Epidemiology and Preventive Medicine, Medical Sociology, University Regensburg, Regensburg, Germany
| | - Sascha Eickmann
- Department of Epidemiology and Preventive Medicine, Medical Sociology, University Regensburg, Regensburg, Germany
| | - Carla Maria Hansen
- Department of Nephrology and Medical Intensive Care, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Roland Roller
- German Research Center for Artificial Intelligence, Berlin, Germany
| | | | - David Samhammer
- Institute for Systematic Theology II (Ethics), Friedrich-Alexander University Erlangen Nürnberg, Erlangen, Germany
| | - Peter Dabrock
- Institute for Systematic Theology II (Ethics), Friedrich-Alexander University Erlangen Nürnberg, Erlangen, Germany
| | - Sebastian Möller
- German Research Center for Artificial Intelligence, Berlin, Germany
- Quality and Usability Lab, Technical University of Berlin, Berlin, Germany
| | - Klemens Budde
- Department of Nephrology and Medical Intensive Care, Charité - Universitätsmedizin Berlin, Berlin, Germany
| | - Anne Herrmann
- Department of Epidemiology and Preventive Medicine, Medical Sociology, University Regensburg, Regensburg, Germany
- School of Medicine and Public Health, University of Newcastle, Callaghan, Australia
| |
Collapse
|
19
|
Sloss EA, McPherson JP, Beck AC, Guo JW, Scheese CH, Flake NR, Chalkidis G, Staes CJ. Patient and Caregiver Perceptions of an Interface Design to Communicate Artificial Intelligence-Based Prognosis for Patients With Advanced Solid Tumors. JCO Clin Cancer Inform 2024; 8:e2300187. [PMID: 38657194 PMCID: PMC11161249 DOI: 10.1200/cci.23.00187] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2023] [Revised: 02/22/2024] [Accepted: 03/07/2024] [Indexed: 04/26/2024] Open
Abstract
PURPOSE Use of artificial intelligence (AI) in cancer care is increasing. What remains unclear is how best to design patient-facing systems that communicate AI output. With oncologist input, we designed an interface that presents patient-specific, machine learning-based 6-month survival prognosis information designed to aid oncology providers in preparing for and discussing prognosis with patients with advanced solid tumors and their caregivers. The primary purpose of this study was to assess patient and caregiver perceptions and identify enhancements of the interface for communicating 6-month survival and other prognosis information when making treatment decisions concerning anticancer and supportive therapy. METHODS This qualitative study included interviews and focus groups conducted between November and December 2022. Purposive sampling was used to recruit former patients with cancer and/or former caregivers of patients with cancer who had participated in cancer treatment decisions from Utah or elsewhere in the United States. Categories and themes related to perceptions of the interface were identified. RESULTS We received feedback from 20 participants during eight individual interviews and two focus groups, including four cancer survivors, 13 caregivers, and three representing both. Overall, most participants expressed positive perceptions about the tool and identified its value for supporting decision making, feeling less alone, and supporting communication among oncologists, patients, and their caregivers. Participants identified areas for improvement and implementation considerations, particularly that oncologists should share the tool and guide discussions about prognosis with patients who want to receive the information. CONCLUSION This study revealed important patient and caregiver perceptions of and enhancements for the proposed interface. Originally designed with input from oncology providers, patient and caregiver participants identified additional interface design recommendations and implementation considerations to support communication about prognosis.
Collapse
Affiliation(s)
| | - Jordan P. McPherson
- Huntsman Cancer Institute, University of Utah, Salt Lake City, UT
- College of Pharmacy, University of Utah, Salt Lake City, UT
| | - Anna C. Beck
- Huntsman Cancer Institute, University of Utah, Salt Lake City, UT
| | - Jia-Wen Guo
- College of Nursing, University of Utah, Salt Lake City, UT
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT
| | - Carolyn H. Scheese
- College of Nursing, University of Utah, Salt Lake City, UT
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT
| | - Naomi R. Flake
- Clinical & Translational Science Institute, University of Utah, Salt Lake City, UT
| | | | - Catherine J. Staes
- College of Nursing, University of Utah, Salt Lake City, UT
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT
| |
Collapse
|
20
|
Kim MS, Cha H, You SH, Kim S. Thirty-day mortality after palliative radiotherapy in advanced cancer patients: Optimizing end-of-life care in Asia. J Med Imaging Radiat Oncol 2024; 68:307-315. [PMID: 38450953 DOI: 10.1111/1754-9485.13635] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 02/23/2024] [Indexed: 03/08/2024]
Abstract
INTRODUCTION Evidence-based guidelines recommend hypofractionated palliative radiotherapy (PRT); nonetheless, many patients receive prolonged course of PRT. To identify patients with limited benefits from PRT in end-of-life care, we evaluated the pattern of PRT at an Asian institution and factors associated with 30-day mortality after PRT (30dM). METHODS We retrospectively reviewed 228 patients who died after PRT in Yonsei Wonju Severance Christian hospital between October 2014 and March 2022. The associations between clinical factors and survival were assessed using the Cox proportional hazards method. Survival was analysed using the existing models to evaluate their performance in our cohort. RESULTS The median PRT duration was 13 (IQR, 7-15) days. Only 11.4% of the patients were treated with hypofractionated radiotherapy. One-third of the patients (32.9%) could not complete PRT and 39 (17.1%) died during PRT. The 30dM was 31.6%. The median time from PRT to death was 17 (IQR, 11-23) days for the patients who died within 30 days. The number of involved organs (≤2 vs. >2; P < 0.001), albumin level (<3.3 vs. ≥3.3; P = 0.016), admission during PRT (P < 0.001), admission 3 months before PRT (P = 0.036) and ICU care during PRT (P < 0.001) were prognostic factors. A comparison of survival based on the existing models yielded unsatisfactory results in our cohort. CONCLUSION Almost one-third of the patients received PRT in the last 30 days of life. The use of hypofractionation for PRT was low in this Asian population. Further research is necessary to develop a predictive model of early mortality, allowing tailored end-of-life care for Asian patients.
Collapse
Affiliation(s)
- Mi Sun Kim
- Department of Radiation Oncology, Yonsei University Wonju College of Medicine, Wonju, Republic of Korea
| | - Hyejung Cha
- Department of Radiation Oncology, Yonsei University Wonju College of Medicine, Wonju, Republic of Korea
| | - Sei Hwan You
- Department of Radiation Oncology, Yonsei University Wonju College of Medicine, Wonju, Republic of Korea
| | - Sunghyun Kim
- Department of Radiation Oncology, Yonsei University Wonju College of Medicine, Wonju, Republic of Korea
| |
Collapse
|
21
|
Bange EM, Li Y, Kumar P, Doucette A, Gabriel P, Parikh R, Li EH, Mamtani R, Getz KD. The association between telemedicine, advance care planning, and unplanned hospitalizations among high-risk patients with cancer. Cancer 2024; 130:636-644. [PMID: 37987207 PMCID: PMC10922036 DOI: 10.1002/cncr.35116] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2023] [Revised: 08/31/2023] [Accepted: 10/15/2023] [Indexed: 11/22/2023]
Abstract
BACKGROUND Despite the widespread implementation of telemedicine, there are limited data regarding its impact on key components of care for patients with incurable or high-risk cancer. For these patients, high-quality care requires detailed conversations regarding treatment priorities (advance care planning) and clinical care to minimize unnecessary acute care (unplanned hospitalizations). Whether telemedicine affects these outcomes relative to in-person clinic visits was examined among patients with cancer at high risk for 6-month mortality. METHODS This retrospective cohort study included adult patients with cancer with any tumor type treated at the University of Pennsylvania who were newly identified between April 1 and December 31, 2020, to be at high risk for 6-month mortality via a validated machine learning algorithm. Separate modified Poisson regressions were used to assess the occurrence of advance care planning and unplanned hospitalizations for telemedicine as compared to in-person visits. Additional analyses were done comparing telemedicine type (video or phone) as compared to in-person clinic visits. RESULTS The occurrence of advance care planning was similar between telemedicine and in-person visits (6.8% vs. 6.0%; adjusted risk ratio [aRR], 1.25; 95% CI, 0.92-1.69). In regard to telemedicine subtype, patients exposed to video encounters were modestly more likely to have documented advance care planning in comparison to those seen in person (7.5% vs. 6.0%; aRR, 1.48; 95% CI, 1.03-2.11). The 3-month risk for unplanned hospitalization was comparable for telemedicine compared to in-person clinic encounters (21% vs. 18%; aRR, 1.06; 95% CI, 0.81-1.38). CONCLUSIONS In this study, care delivered by telemedicine, compared to in-person clinic visits, produced comparable rates of advance care planning conversations without increasing hospitalizations, which suggests that vulnerable patients can be managed safely by telemedicine.
Collapse
Affiliation(s)
- Erin M Bange
- Memorial Sloan Kettering Cancer Center, New York, New York, USA
| | - Yimei Li
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Pallavi Kumar
- Palliative and Hospice Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Abramson Cancer Center, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Abigail Doucette
- Abramson Cancer Center, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Peter Gabriel
- Abramson Cancer Center, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Ravi Parikh
- Abramson Cancer Center, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Division of Hematology/Oncology, Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania, USA
| | - Eric H Li
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Ronac Mamtani
- Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| | - Kelly D Getz
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Pediatrics, Children's Hospital of Philadelphia, Philadelphia, Pennsylvania, USA
| |
Collapse
|
22
|
Guo LL, Morse KE, Aftandilian C, Steinberg E, Fries J, Posada J, Fleming SL, Lemmon J, Jessa K, Shah N, Sung L. Characterizing the limitations of using diagnosis codes in the context of machine learning for healthcare. BMC Med Inform Decis Mak 2024; 24:51. [PMID: 38355486 PMCID: PMC10868117 DOI: 10.1186/s12911-024-02449-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2023] [Accepted: 01/30/2024] [Indexed: 02/16/2024] Open
Abstract
BACKGROUND Diagnostic codes are commonly used as inputs for clinical prediction models, to create labels for prediction tasks, and to identify cohorts for multicenter network studies. However, the coverage rates of diagnostic codes and their variability across institutions are underexplored. The primary objective was to describe lab- and diagnosis-based labels for 7 selected outcomes at three institutions. Secondary objectives were to describe agreement, sensitivity, and specificity of diagnosis-based labels against lab-based labels. METHODS This study included three cohorts: SickKids from The Hospital for Sick Children, and StanfordPeds and StanfordAdults from Stanford Medicine. We included seven clinical outcomes with lab-based definitions: acute kidney injury, hyperkalemia, hypoglycemia, hyponatremia, anemia, neutropenia and thrombocytopenia. For each outcome, we created four lab-based labels (abnormal, mild, moderate and severe) based on test result and one diagnosis-based label. Proportion of admissions with a positive label were presented for each outcome stratified by cohort. Using lab-based labels as the gold standard, agreement using Cohen's Kappa, sensitivity and specificity were calculated for each lab-based severity level. RESULTS The number of admissions included were: SickKids (n = 59,298), StanfordPeds (n = 24,639) and StanfordAdults (n = 159,985). The proportion of admissions with a positive diagnosis-based label was significantly higher for StanfordPeds compared to SickKids across all outcomes, with odds ratio (99.9% confidence interval) for abnormal diagnosis-based label ranging from 2.2 (1.7-2.7) for neutropenia to 18.4 (10.1-33.4) for hyperkalemia. Lab-based labels were more similar by institution. When using lab-based labels as the gold standard, Cohen's Kappa and sensitivity were lower at SickKids for all severity levels compared to StanfordPeds. CONCLUSIONS Across multiple outcomes, diagnosis codes were consistently different between the two pediatric institutions. This difference was not explained by differences in test results. These results may have implications for machine learning model development and deployment.
Collapse
Affiliation(s)
- Lin Lawrence Guo
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada
| | - Keith E Morse
- Division of Pediatric Hospital Medicine, Department of Pediatrics, Stanford University, Palo Alto, CA, USA
| | - Catherine Aftandilian
- Division of Hematology/Oncology, Department of Pediatrics, Stanford University, Palo Alto, CA, USA
| | - Ethan Steinberg
- Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA
| | - Jason Fries
- Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA
| | - Jose Posada
- Universidad del Norte, Barranquilla, Colombia
| | - Scott Lanyon Fleming
- Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA
| | - Joshua Lemmon
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada
| | - Karim Jessa
- Information Services, The Hospital for Sick Children, Toronto, ON, Canada
| | - Nigam Shah
- Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA
| | - Lillian Sung
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, Canada.
- Division of Haematology/Oncology, The Hospital for Sick Children, 555 University Avenue, M5G1X8, Toronto, ON, Canada.
| |
Collapse
|
23
|
Baxter R, Pusa S, Andersson S, Fromme EK, Paladino J, Sandgren A. Core elements of serious illness conversations: an integrative systematic review. BMJ Support Palliat Care 2024:spcare-2023-004163. [PMID: 37369576 DOI: 10.1136/spcare-2023-004163] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Accepted: 06/05/2023] [Indexed: 06/29/2023]
Abstract
BACKGROUND Ariadne Labs' Serious Illness Care Program (SICP), inclusive of the Serious Illness Conversation Guide (SICG), has been adapted for use in a variety of settings and among diverse population groups. Explicating the core elements of serious illness conversations could support the inclusion or exclusion of certain components in future iterations of the programme and the guide. AIM This integrative systematic review aimed to identify and describe core elements of serious illness conversations in relation to the SICP and/or SICG. DESIGN Literature published between 1 January 2014 and 20 March 2023 was searched in MEDLINE, PsycINFO, CINAHL and PubMed. All articles were evaluated using the Joanna Briggs Institute Critical Appraisal Guidelines. Data were analysed with thematic synthesis. RESULTS A total of 64 articles met the inclusion criteria. Three themes were revealed: (1) serious illness conversations serve different functions that are reflected in how they are conveyed; (2) serious illness conversations endeavour to discover what matters to patients and (3) serious illness conversations seek to align what patients want in their life and care. CONCLUSIONS Core elements of serious illness conversations included explicating the intention, framing, expectations and directions for the conversation. This encompassed discussing current and possible trajectories with a view towards uncovering matters of importance to the patient as a person. Preferences and priorities could be used to inform future preparation and recommendations. Serious illness conversation elements could be adapted and altered depending on the intended purpose of the conversation.
Collapse
Affiliation(s)
- Rebecca Baxter
- Center for Collaborative Palliative Care, Department of Health and Caring Sciences, Linnaeus University-Vaxjo Campus, Vaxjo, Sweden
| | - Susanna Pusa
- Center for Collaborative Palliative Care, Department of Health and Caring Sciences, Linnaeus University-Vaxjo Campus, Vaxjo, Sweden
| | - Sofia Andersson
- Center for Collaborative Palliative Care, Department of Health and Caring Sciences, Linnaeus University-Vaxjo Campus, Vaxjo, Sweden
| | - Erik K Fromme
- Ariadne Labs, Boston, Massachusetts, USA
- Harvard Medical School, Boston, Massachusetts, USA
| | - Joanna Paladino
- Ariadne Labs, Boston, Massachusetts, USA
- Harvard Medical School, Boston, Massachusetts, USA
- Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Anna Sandgren
- Center for Collaborative Palliative Care, Department of Health and Caring Sciences, Linnaeus University-Vaxjo Campus, Vaxjo, Sweden
| |
Collapse
|
24
|
Schell JO, Schenker Y, Piscitello G, Belin SC, Chiu EJ, Zapf RL, Kip PL, Marroquin OC, Donahoe MP, Holder-Murray J, Arnold RM. Implementing a Serious Illness Risk Prediction Model: Impact on Goals of Care Documentation. J Pain Symptom Manage 2023; 66:603-610.e3. [PMID: 37532159 PMCID: PMC10828667 DOI: 10.1016/j.jpainsymman.2023.07.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/08/2023] [Revised: 07/17/2023] [Accepted: 07/22/2023] [Indexed: 08/04/2023]
Abstract
CONTEXT Goals of care conversations can promote high value care for patients with serious illness, yet documented discussions infrequently occur in hospital settings. OBJECTIVES We sought to develop a quality improvement initiative to improve goals of care documentation for hospitalized patients. METHODS Implementation occurred at an academic medical center in Pittsburgh, Pennsylvania. Intervention included integration of a 90-day mortality prediction model grouping patients into low, intermediate, and high risk; a centralized goals of care note; and automated notifications and targeted palliative consults. We compared documented goals of care discussions by risk score before and after implementation. RESULTS Of the 12,571 patients hospitalized preimplementation and 10,761 postimplementation, 1% were designated high risk and 11% intermediate risk of mortality. Postimplementation, goals of care documentation increased for high (17.6%-70.8%, P< 0.0001) and intermediate risk patients (9.6%-28.0%, P < 0.0001). For intermediate risk patients, the percentage of goals of care documentation performed by palliative medicine specialists increased from pre- to postimplementation (52.3%-71.2%, P = 0.0002). For high-risk patients, the percentage of goals of care documentation completed by the primary service increased from pre-to postimplementation (36.8%-47.1%, P = 0.5898, with documentation performed by palliative medicine specialists slightly decreasing from pre- to postimplementation (63.2%-52.9%, P = 0.5898). CONCLUSIONS Implementation of a goals of care initiative using a mortality prediction model significantly increased goals of care documentation especially among high-risk patients. Further study to assess strategies to increase goals of care documentation for intermediate risk patients is needed especially by nonspecialty palliative care.
Collapse
Affiliation(s)
- Jane O Schell
- Section of Palliative Care and Medical Ethics (J.O.S., Y.S., G.P., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; Renal-Electrolyte Division (J.O.S.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; Palliative Research Center (J.O.S., Y.S., G.P., S.C.B., E.J.C., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania.
| | - Yael Schenker
- Section of Palliative Care and Medical Ethics (J.O.S., Y.S., G.P., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; Palliative Research Center (J.O.S., Y.S., G.P., S.C.B., E.J.C., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
| | - Gina Piscitello
- Section of Palliative Care and Medical Ethics (J.O.S., Y.S., G.P., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; Palliative Research Center (J.O.S., Y.S., G.P., S.C.B., E.J.C., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
| | - Shane C Belin
- Palliative Research Center (J.O.S., Y.S., G.P., S.C.B., E.J.C., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
| | - Eric J Chiu
- Palliative Research Center (J.O.S., Y.S., G.P., S.C.B., E.J.C., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
| | - Rachel L Zapf
- Wolff Center (R.L.Z., P.L.K., R.M.A.), UPMC, Pittsburgh, Pennsylvania
| | - Paula L Kip
- Wolff Center (R.L.Z., P.L.K., R.M.A.), UPMC, Pittsburgh, Pennsylvania
| | | | - Michael P Donahoe
- Division of Pulmonary, Allergy, and Critical Care Medicine (M.P.D.), Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
| | - Jennifer Holder-Murray
- Departments of Surgery and Anesthesiology and Perioperative Medicine (J.H.M.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania
| | - Robert M Arnold
- Section of Palliative Care and Medical Ethics (J.O.S., Y.S., G.P., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; Palliative Research Center (J.O.S., Y.S., G.P., S.C.B., E.J.C., R.M.A.), University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania; Wolff Center (R.L.Z., P.L.K., R.M.A.), UPMC, Pittsburgh, Pennsylvania
| |
Collapse
|
25
|
Parikh RB, Schriver E, Ferrell WJ, Wakim J, Williamson J, Khan N, Kopinsky M, Balachandran M, Gabriel PE, Schuchter LM, Patel MS, Shulman LN, Manz CR. Remote Patient-Reported Outcomes and Activity Monitoring to Improve Patient-Clinician Communication Regarding Symptoms and Functional Status: A Randomized Controlled Trial. JCO Oncol Pract 2023; 19:1143-1151. [PMID: 37816198 PMCID: PMC10732505 DOI: 10.1200/op.23.00048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2023] [Revised: 06/08/2023] [Accepted: 08/29/2023] [Indexed: 10/12/2023] Open
Abstract
PURPOSE Routine collection of patient-generated health data (PGHD) may promote earlier recognition of symptomatic and functional decline. This trial assessed the impact of an intervention integrating remote PGHD collection with patient nudges on symptom and functional status understanding between patients with advanced cancer and their oncology team. METHODS This three-arm randomized controlled trial was conducted from November 19, 2020, to December 17, 2021, at a large tertiary oncology practice. We enrolled patients with stage IV GI and lung cancers undergoing chemotherapy. Over 6 months, patients in two intervention arms received PROStep-weekly text message-based symptom surveys and passive activity monitoring using a wearable accelerometer. PGHD were summarized in dashboards given to patients' oncology team before appointments. One intervention arm received an additional text-based active choice prompt to discuss worsening symptoms or functional status with their clinician. Control patients did not receive PROStep. The coprimary outcomes patient perceptions of oncology team symptom and functional understanding at 6 months were measured on a 1-5 Likert scale (5 = high understanding). RESULTS One hundred eight patients enrolled: 55% male, 81% White, and 77% had GI cancers. Patient-reported clinician understanding did not differ between control and intervention arms for symptoms (4.5 v 4.5; P = .87) or functional status (4.5 v 4.3; P = .31). In the intervention arms, combined patient adherence to weekly symptom reports and daily activity monitoring was 64% and 53%, respectively. Intervention patients in the PROStep versus PROStep + active choice arms reported low burden from wearing the accelerometer (mean burden [standard deviation], 2.7 [1.3] v 2.1 [1.3]; P = .15) and completing surveys (2.1 [1.2] v 1.9 [1.3]; P = .44). CONCLUSION Patients receiving PROStep reported high understanding of symptoms and functional status from their oncology team, although this did not differ from controls.
Collapse
Affiliation(s)
- Ravi B. Parikh
- Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA
- Leonard Davis Institute for Health Economics, University of Pennsylvania, Philadelphia, PA
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Corporal Michael J. Crescenz VA Medical Center, Philadelphia, PA
| | - Emily Schriver
- Institute for Biomedical Informatics, University of Pennsylvania, Philadelphia, PA
- Penn Medicine Predictive Healthcare, University of Pennsylvania Health System, Philadelphia, PA
| | - William J. Ferrell
- Leonard Davis Institute for Health Economics, University of Pennsylvania, Philadelphia, PA
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Jonathan Wakim
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Joelle Williamson
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Neda Khan
- Leonard Davis Institute for Health Economics, University of Pennsylvania, Philadelphia, PA
- Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Center for Health Care Innovation, Penn Medicine, Philadelphia, PA
| | - Michael Kopinsky
- Center for Health Care Innovation, Penn Medicine, Philadelphia, PA
| | | | - Peter E. Gabriel
- Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA
| | - Lynn M. Schuchter
- Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA
| | | | | | - Christopher R. Manz
- Division of Population Sciences, Dana-Farber Cancer Institute, Boston, MA
- Harvard Medical School, Harvard University, Boston, MA
| |
Collapse
|
26
|
Lemmon J, Guo LL, Steinberg E, Morse KE, Fleming SL, Aftandilian C, Pfohl SR, Posada JD, Shah N, Fries J, Sung L. Self-supervised machine learning using adult inpatient data produces effective models for pediatric clinical prediction tasks. J Am Med Inform Assoc 2023; 30:2004-2011. [PMID: 37639620 PMCID: PMC10654865 DOI: 10.1093/jamia/ocad175] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 08/03/2023] [Accepted: 08/15/2023] [Indexed: 08/31/2023] Open
Abstract
OBJECTIVE Development of electronic health records (EHR)-based machine learning models for pediatric inpatients is challenged by limited training data. Self-supervised learning using adult data may be a promising approach to creating robust pediatric prediction models. The primary objective was to determine whether a self-supervised model trained in adult inpatients was noninferior to logistic regression models trained in pediatric inpatients, for pediatric inpatient clinical prediction tasks. MATERIALS AND METHODS This retrospective cohort study used EHR data and included patients with at least one admission to an inpatient unit. One admission per patient was randomly selected. Adult inpatients were 18 years or older while pediatric inpatients were more than 28 days and less than 18 years. Admissions were temporally split into training (January 1, 2008 to December 31, 2019), validation (January 1, 2020 to December 31, 2020), and test (January 1, 2021 to August 1, 2022) sets. Primary comparison was a self-supervised model trained in adult inpatients versus count-based logistic regression models trained in pediatric inpatients. Primary outcome was mean area-under-the-receiver-operating-characteristic-curve (AUROC) for 11 distinct clinical outcomes. Models were evaluated in pediatric inpatients. RESULTS When evaluated in pediatric inpatients, mean AUROC of self-supervised model trained in adult inpatients (0.902) was noninferior to count-based logistic regression models trained in pediatric inpatients (0.868) (mean difference = 0.034, 95% CI=0.014-0.057; P < .001 for noninferiority and P = .006 for superiority). CONCLUSIONS Self-supervised learning in adult inpatients was noninferior to logistic regression models trained in pediatric inpatients. This finding suggests transferability of self-supervised models trained in adult patients to pediatric patients, without requiring costly model retraining.
Collapse
Affiliation(s)
- Joshua Lemmon
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON M5G1X8, Canada
| | - Lin Lawrence Guo
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON M5G1X8, Canada
| | - Ethan Steinberg
- Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA 94305, United States
| | - Keith E Morse
- Division of Pediatric Hospital Medicine, Department of Pediatrics, Stanford University, Palo Alto, CA 94304, United States
| | - Scott Lanyon Fleming
- Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA 94305, United States
| | - Catherine Aftandilian
- Division of Hematology/Oncology, Department of Pediatrics, Stanford University, Palo Alto, CA 94304, United States
| | | | - Jose D Posada
- Universidad del Norte, Barranquilla 081007, Colombia
| | - Nigam Shah
- Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA 94305, United States
| | - Jason Fries
- Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA 94305, United States
| | - Lillian Sung
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON M5G1X8, Canada
- Division of Haematology/Oncology, The Hospital for Sick Children, Toronto, ON M5G1X8, Canada
| |
Collapse
|
27
|
Paladino J, Fromme EK, Kilpatrick L, Dingfield L, Teuteberg W, Bernacki R, Jackson V, Sanders JJ, Jacobsen J, Ritchie C, Mitchell S. Lessons Learned About System-Level Improvement in Serious Illness Communication: A Qualitative Study of Serious Illness Care Program Implementation in Five Health Systems. Jt Comm J Qual Patient Saf 2023; 49:620-633. [PMID: 37537096 DOI: 10.1016/j.jcjq.2023.06.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/09/2023] [Revised: 06/26/2023] [Accepted: 06/26/2023] [Indexed: 08/05/2023]
Abstract
BACKGROUND Serious illness communication is a key element of high-quality care, but it is difficult to implement in practice. The Serious Illness Care Program (SICP) is a multifaceted intervention that contributes to more, earlier, and better serious illness conversations and improved patient outcomes. This qualitative study examined the organizational and implementation factors that influenced improvement in real-world contexts. METHODS The authors performed semistructured interviews of 30 health professionals at five health systems that adopted SICP as quality improvement initiatives to investigate the organizational and implementation factors that appeared to influence improvement. RESULTS After SICP implementation across the organizations studied, approximately 4,661 clinicians have been trained in serious illness communication and 56,712 patients had had an electronic health record (EHR)-documented serious illness conversation. Facilitators included (1) visible support from leaders, who financially invested in an implementation team and champions, expressed the importance of serious illness communication as an institutional priority, and created incentives for training and documenting serious illness conversations; (2) EHR and data infrastructure to foster performance improvement and accountability, including an accessible documentation template, a reporting system, and customized data feedback for clinicians; and (3) communication skills training and sustained support for clinicians to problem-solve communication challenges, reflect on communication experiences, and adapt the intervention. Inhibitors included leadership inaction, competing priorities and incentives, variable clinician acceptance of EHR and data tools, and inadequate support for clinicians after training. CONCLUSION Successful implementation appeared to rely on multilevel organizational strategies to prioritize, reward, and reinforce serious illness communication. The insights derived from this research may function as an organizational road map to guide implementation of SICP or related quality initiatives.
Collapse
|
28
|
Guo LL, Calligan M, Vettese E, Cook S, Gagnidze G, Han O, Inoue J, Lemmon J, Li J, Roshdi M, Sadovy B, Wallace S, Sung L. Development and validation of the SickKids Enterprise-wide Data in Azure Repository (SEDAR). Heliyon 2023; 9:e21586. [PMID: 38027579 PMCID: PMC10661187 DOI: 10.1016/j.heliyon.2023.e21586] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/01/2022] [Revised: 09/15/2023] [Accepted: 10/24/2023] [Indexed: 12/01/2023] Open
Abstract
Objectives To describe the processes developed by The Hospital for Sick Children (SickKids) to enable utilization of electronic health record (EHR) data by creating sequentially transformed schemas for use across multiple user types. Methods We used Microsoft Azure as the cloud service provider and named this effort the SickKids Enterprise-wide Data in Azure Repository (SEDAR). Epic Clarity data from on-premises was copied to a virtual network in Microsoft Azure. Three sequential schemas were developed. The Filtered Schema added a filter to retain only SickKids and valid patients. The Curated Schema created a data structure that was easier to navigate and query. Each table contained a logical unit such as patients, hospital encounters or laboratory tests. Data validation of randomly sampled observations in the Curated Schema was performed. The SK-OMOP Schema was designed to facilitate research and machine learning. Two individuals mapped medical elements to standard Observational Medical Outcomes Partnership (OMOP) concepts. Results A copy of Clarity data was transferred to Microsoft Azure and updated each night using log shipping. The Filtered Schema and Curated Schema were implemented as stored procedures and executed each night with incremental updates or full loads. Data validation required up to 16 iterations for each Curated Schema table. OMOP concept mapping achieved at least 80 % coverage for each SK-OMOP table. Conclusions We described our experience in creating three sequential schemas to address different EHR data access requirements. Future work should consider replicating this approach at other institutions to determine whether approaches are generalizable.
Collapse
Affiliation(s)
- Lin Lawrence Guo
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
| | - Maryann Calligan
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
| | - Emily Vettese
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
| | - Sadie Cook
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
| | - George Gagnidze
- Information Management Technology, The Hospital for Sick Children, Toronto, Canada
| | - Oscar Han
- Information Management Technology, The Hospital for Sick Children, Toronto, Canada
| | - Jiro Inoue
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
| | - Joshua Lemmon
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
| | - Johnson Li
- Information Management Technology, The Hospital for Sick Children, Toronto, Canada
| | - Medhat Roshdi
- Information Management Technology, The Hospital for Sick Children, Toronto, Canada
| | - Bohdan Sadovy
- Information Management Technology, The Hospital for Sick Children, Toronto, Canada
| | - Steven Wallace
- Information Management Technology, The Hospital for Sick Children, Toronto, Canada
| | - Lillian Sung
- Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, Canada
- Division of Haematology/Oncology, The Hospital for Sick Children, Toronto, Canada
| |
Collapse
|
29
|
Kang J, Chowdhry AK, Pugh SL, Park JH. Integrating Artificial Intelligence and Machine Learning Into Cancer Clinical Trials. Semin Radiat Oncol 2023; 33:386-394. [PMID: 37684068 PMCID: PMC10880815 DOI: 10.1016/j.semradonc.2023.06.004] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/10/2023]
Abstract
The practice of oncology requires analyzing and synthesizing abundant data. From the patient's workup to determine eligibility to the therapies received to the post-treatment surveillance, practitioners must constantly juggle, evaluate, and weigh decision-making based on their best understanding of information at hand. These complex, multifactorial decisions have a tremendous opportunity to benefit from data-driven machine learning (ML) methods to drive opportunities in artificial intelligence (AI). Within the past 5 years, we have seen AI move from simply a promising opportunity to being used in prospective trials. Here, we review recent efforts of AI in clinical trials that have moved the needle towards improved prediction of actionable outcomes, such as predicting acute care visits, short term mortality, and pathologic extranodal extension. We then pause and reflect on how these AI models ask a different question than traditional statistics models that readers may be more familiar with; how then should readers conceptualize and interpret AI models that they are not as familiar with. We end with what we believe are promising future opportunities for AI in oncology, with an eye towards allowing the data to inform us through unsupervised learning and generative models, rather than asking AI to perform specific functions.
Collapse
Affiliation(s)
- John Kang
- Department of Radiation Oncology, University of Washington, Seattle, WA..
| | - Amit K Chowdhry
- Department of Radiation Oncology, University of Rochester, Rochester, NY
| | - Stephanie L Pugh
- American College of Radiology, NRG Oncology Statistics and Data Management Center, Philadelphia PA
| | - John H Park
- Department of Radiation Oncology, Kansas City VA Medical Center, Kansas City, MO.; Department of Radiology, University of Missouri Kansas City School of Medicine, Kansas City, MO
| |
Collapse
|
30
|
Perni S, Jimenez R, Jagsi R. Optimizing Informed Consent in Cancer Clinical Trials. Semin Radiat Oncol 2023; 33:349-357. [PMID: 37684064 DOI: 10.1016/j.semradonc.2023.06.001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 09/10/2023]
Abstract
The concept of informed consent has evolved considerably over the course of the 20th century, leading to its establishment as a foundational ethical principle for the conduct of biomedical research in the United States. Even though it is now a highly regulated part of cancer research, the process of obtaining informed consent is often impeded by systemic, clinician, and patient factors that require both small- and large-scale intervention. New challenges and considerations continue to emerge due to innovations in clinical trial design, increases in utilization of genomic sequencing, and advances in genomic editing and artificial intelligence. We present a review of the history, policy, pragmatic challenges, and evolving role of the central ethical tenet of informed consent in clinical trials.
Collapse
Affiliation(s)
- Subha Perni
- Division of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX
| | - Rachel Jimenez
- Department of Radiation Oncology, Massachusetts General Hospital, Boston, MA
| | - Reshma Jagsi
- Department of Radiation Oncology, Winship Cancer Institute of Emory University, Atlanta, GA.
| |
Collapse
|
31
|
Sedhom R, Tomita-Barber J, Manz CR, Parikh RB, Gupta A, Hussaini Q, Dougherty D. Creating a culture for change: Lessons from behavioral economics and complexity science to increase serious illness conversations for patients with cancer. Curr Probl Cancer 2023; 47:101020. [PMID: 37863783 DOI: 10.1016/j.currproblcancer.2023.101020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Revised: 08/31/2023] [Accepted: 09/06/2023] [Indexed: 10/22/2023]
Abstract
Patient-centered cancer care requires communication between patients and clinicians about patients' goals, values, and preferences. Serious illness communication improves patient and caregiver outcomes, the value and quality of cancer care, and the well-being of clinicians. Despite these benefits, there are competing factors including time, capacity, bandwidth, and resistance. Health systems and oncology practices have opportunities to invest in pathways that assist patients and clinicians to engage in serious illness conversations. We discuss how applying insights from behavioral economics and complexity science may help clinicians engage in serious illness conversation and improve patient-centered cancer care.
Collapse
Affiliation(s)
- Ramy Sedhom
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA; Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA; Penn Center for Cancer Care Innovation, Abramson Cancer Center, Penn Medicine, Philadelphia, PA.
| | | | - Christopher R Manz
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, MA; Harvard Medical School, Boston, MA
| | - Ravi B Parikh
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA; Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA; Penn Center for Cancer Care Innovation, Abramson Cancer Center, Penn Medicine, Philadelphia, PA
| | - Arjun Gupta
- Division of Hematology, Oncology, and Transplantation, University of Minnesota, Minneapolis, MN
| | - Qasim Hussaini
- Department of Oncology, Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins University, Baltimore, MD
| | - David Dougherty
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA; Penn Center for Cancer Care Innovation, Abramson Cancer Center, Penn Medicine, Philadelphia, PA
| |
Collapse
|
32
|
Rojas JC, Teran M, Umscheid CA. Clinician Trust in Artificial Intelligence: What is Known and How Trust Can Be Facilitated. Crit Care Clin 2023; 39:769-782. [PMID: 37704339 DOI: 10.1016/j.ccc.2023.02.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/29/2023]
Abstract
Predictive analytics based on artificial intelligence (AI) offer clinicians the opportunity to leverage big data available in electronic health records (EHR) to improve clinical decision-making, and thus patient outcomes. Despite this, many barriers exist to facilitating trust between clinicians and AI-based tools, limiting its current impact. Potential solutions are available at both the local and national level. It will take a broad and diverse coalition of stakeholders, from health-care systems, EHR vendors, and clinical educators to regulators, researchers and the patient community, to help facilitate this trust so that the promise of AI in health care can be realized.
Collapse
Affiliation(s)
- Juan C Rojas
- Department of Internal Medicine, Rush University, 1725 West Harrison Street, Suite 010, Chicago, IL 60612, USA.
| | - Mario Teran
- Agency for Healthcare Research and Quality, 5600 Fishers Lane, Mail Stop 06E53A, Rockville, MD 20857, USA
| | - Craig A Umscheid
- Agency for Healthcare Research and Quality, 5600 Fishers Lane, Mail Stop 06E53A, Rockville, MD 20857, USA
| |
Collapse
|
33
|
Bitterman DS, Gensheimer MF, Jaffray D, Pryma DA, Jiang SB, Morin O, Ginart JB, Upadhaya T, Vallis KA, Buatti JM, Deasy J, Hsiao HT, Chung C, Fuller CD, Greenspan E, Cloyd-Warwick K, Courdy S, Mao A, Barnholtz-Sloan J, Topaloglu U, Hands I, Maurer I, Terry M, Curran WJ, Le QT, Nadaf S, Kibbe W. Cancer Informatics for Cancer Centers: Sharing Ideas on How to Build an Artificial Intelligence-Ready Informatics Ecosystem for Radiation Oncology. JCO Clin Cancer Inform 2023; 7:e2300136. [PMID: 38055914 PMCID: PMC10703125 DOI: 10.1200/cci.23.00136] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2023] [Revised: 08/15/2023] [Accepted: 10/16/2023] [Indexed: 12/08/2023] Open
Abstract
In August 2022, the Cancer Informatics for Cancer Centers brought together cancer informatics leaders for its biannual symposium, Precision Medicine Applications in Radiation Oncology, co-chaired by Quynh-Thu Le, MD (Stanford University), and Walter J. Curran, MD (GenesisCare). Over the course of 3 days, presenters discussed a range of topics relevant to radiation oncology and the cancer informatics community more broadly, including biomarker development, decision support algorithms, novel imaging tools, theranostics, and artificial intelligence (AI) for the radiotherapy workflow. Since the symposium, there has been an impressive shift in the promise and potential for integration of AI in clinical care, accelerated in large part by major advances in generative AI. AI is now poised more than ever to revolutionize cancer care. Radiation oncology is a field that uses and generates a large amount of digital data and is therefore likely to be one of the first fields to be transformed by AI. As experts in the collection, management, and analysis of these data, the informatics community will take a leading role in ensuring that radiation oncology is prepared to take full advantage of these technological advances. In this report, we provide highlights from the symposium, which took place in Santa Barbara, California, from August 29 to 31, 2022. We discuss lessons learned from the symposium for data acquisition, management, representation, and sharing, and put these themes into context to prepare radiation oncology for the successful and safe integration of AI and informatics technologies.
Collapse
Affiliation(s)
- Danielle S. Bitterman
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA
- Department of Radiation Oncology, Brigham and Women's Hospital/Dana-Farber Cancer Institute, Boston, MA
| | - Michael F. Gensheimer
- Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA
| | - David Jaffray
- Department of Radiation Physics, M.D. Anderson Cancer Center, Houston, TX
| | - Daniel A. Pryma
- Abramson Cancer Center, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Steve B. Jiang
- Medical Artificial Intelligence and Automation Laboratory and Department of Radiation Oncology, University of Texas Southwestern Medical Center, Dallas, TX
| | - Olivier Morin
- Department of Radiation Oncology, MEDomics Laboratory, University of California San Francisco, San Francisco, CA
| | - Jorge Barrios Ginart
- Department of Radiation Oncology, MEDomics Laboratory, University of California San Francisco, San Francisco, CA
| | - Taman Upadhaya
- Department of Radiation Oncology, MEDomics Laboratory, University of California San Francisco, San Francisco, CA
| | - Katherine A. Vallis
- Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, MA
| | - John M. Buatti
- Department of Oncology, University of Oxford, Oxford, United Kingdom
| | - Joseph Deasy
- Department of Radiation Oncology, University of Iowa Carver College of Medicine, Iowa City, IA
| | - H. Timothy Hsiao
- Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY
| | - Caroline Chung
- Department of Scientific Affairs, American Society for Radiation Oncology, Arlington, VA
| | - Clifton D. Fuller
- Department of Scientific Affairs, American Society for Radiation Oncology, Arlington, VA
| | - Emily Greenspan
- Department of Radiation Oncology, M.D. Anderson Cancer Center, Houston, TX
| | - Kristy Cloyd-Warwick
- Center for Biomedical Informatics and Information Technology, National Cancer Institute, Rockville, MD
| | | | | | - Jill Barnholtz-Sloan
- Department of Radiation Oncology, M.D. Anderson Cancer Center, Houston, TX
- Center for Informatics, Digital Vertical, City of Hope National Comprehensive Cancer Center, Los Angeles, CA
| | - Umit Topaloglu
- Department of Radiation Oncology, M.D. Anderson Cancer Center, Houston, TX
| | - Isaac Hands
- Division of Cancer Epidemiology and Genetics, National Cancer Institute, Rockville, MD
- Cancer Research Informatics Shared Resource Facility, University of Kentucky Markey Cancer Center, Lexington, NY
| | | | | | | | - Quynh-Thu Le
- Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA
| | - Sorena Nadaf
- Department of Radiation Oncology, Emory University, Atlanta, GA
| | - Warren Kibbe
- Cancer Center Informatics Society, Los Angeles, CA
| |
Collapse
|
34
|
Frosch ZAK, Hasler J, Handorf E, DuBois T, Bleicher RJ, Edelman MJ, Geynisman DM, Hall MJ, Fang CY, Lynch SM. Development of a Multilevel Model to Identify Patients at Risk for Delay in Starting Cancer Treatment. JAMA Netw Open 2023; 6:e2328712. [PMID: 37578796 PMCID: PMC10425824 DOI: 10.1001/jamanetworkopen.2023.28712] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2023] [Accepted: 07/05/2023] [Indexed: 08/15/2023] Open
Abstract
Importance Delays in starting cancer treatment disproportionately affect vulnerable populations and can influence patients' experience and outcomes. Machine learning algorithms incorporating electronic health record (EHR) data and neighborhood-level social determinants of health (SDOH) measures may identify at-risk patients. Objective To develop and validate a machine learning model for estimating the probability of a treatment delay using multilevel data sources. Design, Setting, and Participants This cohort study evaluated 4 different machine learning approaches for estimating the likelihood of a treatment delay greater than 60 days (group least absolute shrinkage and selection operator [LASSO], bayesian additive regression tree, gradient boosting, and random forest). Criteria for selecting between approaches were discrimination, calibration, and interpretability/simplicity. The multilevel data set included clinical, demographic, and neighborhood-level census data derived from the EHR, cancer registry, and American Community Survey. Patients with invasive breast, lung, colorectal, bladder, or kidney cancer diagnosed from 2013 to 2019 and treated at a comprehensive cancer center were included. Data analysis was performed from January 2022 to June 2023. Exposures Variables included demographics, cancer characteristics, comorbidities, laboratory values, imaging orders, and neighborhood variables. Main Outcomes and Measures The outcome estimated by machine learning models was likelihood of a delay greater than 60 days between cancer diagnosis and treatment initiation. The primary metric used to evaluate model performance was area under the receiver operating characteristic curve (AUC-ROC). Results A total of 6409 patients were included (mean [SD] age, 62.8 [12.5] years; 4321 [67.4%] female; 2576 [40.2%] with breast cancer, 1738 [27.1%] with lung cancer, and 1059 [16.5%] with kidney cancer). A total of 1621 (25.3%) experienced a delay greater than 60 days. The selected group LASSO model had an AUC-ROC of 0.713 (95% CI, 0.679-0.745). Lower likelihood of delay was seen with diagnosis at the treating institution; first malignant neoplasm; Asian or Pacific Islander or White race; private insurance; and lacking comorbidities. Greater likelihood of delay was seen at the extremes of neighborhood deprivation. Model performance (AUC-ROC) was lower in Black patients, patients with race and ethnicity other than non-Hispanic White, and those living in the most disadvantaged neighborhoods. Though the model selected neighborhood SDOH variables as contributing variables, performance was similar when fit with and without these variables. Conclusions and Relevance In this cohort study, a machine learning model incorporating EHR and SDOH data was able to estimate the likelihood of delays in starting cancer therapy. Future work should focus on additional ways to incorporate SDOH data to improve model performance, particularly in vulnerable populations.
Collapse
Affiliation(s)
- Zachary A. K. Frosch
- Department of Hematology/Oncology, Fox Chase Cancer Center, Philadelphia, Pennsylvania
- Cancer Prevention and Control Research Program, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Jill Hasler
- Department of Biostatistics, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Elizabeth Handorf
- Cancer Prevention and Control Research Program, Fox Chase Cancer Center, Philadelphia, Pennsylvania
- Department of Biostatistics, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Tesla DuBois
- Cancer Prevention and Control Research Program, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Richard J. Bleicher
- Department of Surgical Oncology, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Martin J. Edelman
- Department of Hematology/Oncology, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Daniel M. Geynisman
- Department of Hematology/Oncology, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Michael J. Hall
- Department of Hematology/Oncology, Fox Chase Cancer Center, Philadelphia, Pennsylvania
- Cancer Prevention and Control Research Program, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Carolyn Y. Fang
- Cancer Prevention and Control Research Program, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| | - Shannon M. Lynch
- Cancer Prevention and Control Research Program, Fox Chase Cancer Center, Philadelphia, Pennsylvania
| |
Collapse
|
35
|
Piscitello GM, Rojas JC, Arnold RM. Equity in Using Artificial Intelligence to Target Serious Illness Conversations for Patients With Life-Limiting Illness. J Pain Symptom Manage 2023; 66:e299-e301. [PMID: 37054955 DOI: 10.1016/j.jpainsymman.2023.04.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Accepted: 04/01/2023] [Indexed: 04/15/2023]
Affiliation(s)
- Gina M Piscitello
- Division of General Internal Medicine (G.M.P., R.M.A.), Section of Palliative Care and Medical Ethics, University of Pittsburgh, Pittsburgh, Pennsylvania; Palliative Research Center (G.M.P. R.M.A.), University of Pittsburgh, Pittsburgh, Pennsylvania; Division of Pulmonary and Critical Care Medicine (J.C.R.), Rush University Medical Center, Chicago, IL, USA.
| | - Juan Carlos Rojas
- Division of General Internal Medicine (G.M.P., R.M.A.), Section of Palliative Care and Medical Ethics, University of Pittsburgh, Pittsburgh, Pennsylvania; Palliative Research Center (G.M.P. R.M.A.), University of Pittsburgh, Pittsburgh, Pennsylvania; Division of Pulmonary and Critical Care Medicine (J.C.R.), Rush University Medical Center, Chicago, IL, USA.
| | - Robert M Arnold
- Division of General Internal Medicine (G.M.P., R.M.A.), Section of Palliative Care and Medical Ethics, University of Pittsburgh, Pittsburgh, Pennsylvania; Palliative Research Center (G.M.P. R.M.A.), University of Pittsburgh, Pittsburgh, Pennsylvania; Division of Pulmonary and Critical Care Medicine (J.C.R.), Rush University Medical Center, Chicago, IL, USA.
| |
Collapse
|
36
|
Chalkidis G, McPherson JP, Beck A, Newman MG, Guo JW, Sloss EA, Staes CJ. External Validation of a Machine Learning Model to Predict 6-Month Mortality for Patients With Advanced Solid Tumors. JAMA Netw Open 2023; 6:e2327193. [PMID: 37535359 PMCID: PMC10401299 DOI: 10.1001/jamanetworkopen.2023.27193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Accepted: 06/23/2023] [Indexed: 08/04/2023] Open
Abstract
This prognostic study performed external validation of a machine learning model to predict 6-month mortality among patients with advanced solid tumors.
Collapse
Affiliation(s)
| | - Jordan P McPherson
- Huntsman Cancer Institute, Salt Lake City, Utah
- College of Pharmacy, University of Utah, Salt Lake City
| | - Anna Beck
- Huntsman Cancer Institute, Salt Lake City, Utah
| | | | - Jia-Wen Guo
- College of Nursing, University of Utah, Salt Lake City
- Department of Biomedical Informatics, University of Utah, Salt Lake City
| | | | - Catherine J Staes
- College of Nursing, University of Utah, Salt Lake City
- Department of Biomedical Informatics, University of Utah, Salt Lake City
| |
Collapse
|
37
|
Curtis JR, Lee RY, Brumback LC, Kross EK, Downey L, Torrence J, LeDuc N, Mallon Andrews K, Im J, Heywood J, Brown CE, Sibley J, Lober WB, Cohen T, Weiner BJ, Khandelwal N, Abedini NC, Engelberg RA. Intervention to Promote Communication About Goals of Care for Hospitalized Patients With Serious Illness: A Randomized Clinical Trial. JAMA 2023; 329:2028-2037. [PMID: 37210665 PMCID: PMC10201405 DOI: 10.1001/jama.2023.8812] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 05/05/2023] [Indexed: 05/22/2023]
Abstract
Importance Discussions about goals of care are important for high-quality palliative care yet are often lacking for hospitalized older patients with serious illness. Objective To evaluate a communication-priming intervention to promote goals-of-care discussions between clinicians and hospitalized older patients with serious illness. Design, Setting, and Participants A pragmatic, randomized clinical trial of a clinician-facing communication-priming intervention vs usual care was conducted at 3 US hospitals within 1 health care system, including a university, county, and community hospital. Eligible hospitalized patients were aged 55 years or older with any of the chronic illnesses used by the Dartmouth Atlas project to study end-of-life care or were aged 80 years or older. Patients with documented goals-of-care discussions or a palliative care consultation between hospital admission and eligibility screening were excluded. Randomization occurred between April 2020 and March 2021 and was stratified by study site and history of dementia. Intervention Physicians and advance practice clinicians who were treating the patients randomized to the intervention received a 1-page, patient-specific intervention (Jumpstart Guide) to prompt and guide goals-of-care discussions. Main Outcomes and Measures The primary outcome was the proportion of patients with electronic health record-documented goals-of-care discussions within 30 days. There was also an evaluation of whether the effect of the intervention varied by age, sex, history of dementia, minoritized race or ethnicity, or study site. Results Of 3918 patients screened, 2512 were enrolled (mean age, 71.7 [SD, 10.8] years and 42% were women) and randomized (1255 to the intervention group and 1257 to the usual care group). The patients were American Indian or Alaska Native (1.8%), Asian (12%), Black (13%), Hispanic (6%), Native Hawaiian or Pacific Islander (0.5%), non-Hispanic (93%), and White (70%). The proportion of patients with electronic health record-documented goals-of-care discussions within 30 days was 34.5% (433 of 1255 patients) in the intervention group vs 30.4% (382 of 1257 patients) in the usual care group (hospital- and dementia-adjusted difference, 4.1% [95% CI, 0.4% to 7.8%]). The analyses of the treatment effect modifiers suggested that the intervention had a larger effect size among patients with minoritized race or ethnicity. Among 803 patients with minoritized race or ethnicity, the hospital- and dementia-adjusted proportion with goals-of-care discussions was 10.2% (95% CI, 4.0% to 16.5%) higher in the intervention group than in the usual care group. Among 1641 non-Hispanic White patients, the adjusted proportion with goals-of-care discussions was 1.6% (95% CI, -3.0% to 6.2%) higher in the intervention group than in the usual care group. There was no evidence of differential treatment effects of the intervention on the primary outcome by age, sex, history of dementia, or study site. Conclusions and Relevance Among hospitalized older adults with serious illness, a pragmatic clinician-facing communication-priming intervention significantly improved documentation of goals-of-care discussions in the electronic health record, with a greater effect size in racially or ethnically minoritized patients. Trial Registration ClinicalTrials.gov Identifier: NCT04281784.
Collapse
Affiliation(s)
- J. Randall Curtis
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Robert Y. Lee
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | | | - Erin K. Kross
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Lois Downey
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Janaki Torrence
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Nicole LeDuc
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Kasey Mallon Andrews
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Jennifer Im
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Department of Health Systems and Population Health, University of Washington, Seattle
| | - Joanna Heywood
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - Crystal E. Brown
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| | - James Sibley
- Department of Biobehavioral Nursing and Health Informatics, University of Washington, Seattle
| | - William B. Lober
- Department of Biobehavioral Nursing and Health Informatics, University of Washington, Seattle
- Department of Biomedical Informatics and Medical Education, University of Washington, Seattle
| | - Trevor Cohen
- Department of Biomedical Informatics and Medical Education, University of Washington, Seattle
| | - Bryan J. Weiner
- Department of Health Systems and Population Health, University of Washington, Seattle
- Department of Global Health, University of Washington, Seattle
| | - Nita Khandelwal
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Department of Anesthesiology and Pain Medicine, University of Washington, Seattle
| | - Nauzley C. Abedini
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Gerontology and Geriatric Medicine, Department of Medicine, University of Washington, Seattle
| | - Ruth A. Engelberg
- Cambia Palliative Care Center of Excellence at UW Medicine, University of Washington, Seattle
- Division of Pulmonary, Critical Care, and Sleep Medicine, Department of Medicine, University of Washington, Seattle
| |
Collapse
|
38
|
Haimovich AD, Xu W, Wei A, Schonberg MA, Hwang U, Taylor RA. Automatable end-of-life screening for older adults in the emergency department using electronic health records. J Am Geriatr Soc 2023; 71:1829-1839. [PMID: 36744550 PMCID: PMC10258151 DOI: 10.1111/jgs.18262] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2022] [Revised: 12/20/2022] [Accepted: 01/08/2023] [Indexed: 02/07/2023]
Abstract
BACKGROUND Emergency department (ED) visits are common at the end-of-life, but the identification of patients with life-limiting illness remains a key challenge in providing timely and resource-sensitive advance care planning (ACP) and palliative care services. To date, there are no validated, automatable instruments for ED end-of-life screening. Here, we developed a novel electronic health record (EHR) prognostic model to screen older ED patients at high risk for 6-month mortality and compare its performance to validated comorbidity indices. METHODS This was a retrospective, observational cohort study of ED visits from adults aged ≥65 years who visited any of 9 EDs across a large regional health system between 2014 and 2019. Multivariable logistic regression that included clinical and demographic variables, vital signs, and laboratory data was used to develop a 6-month mortality predictive model-the Geriatric End-of-life Screening Tool (GEST) using five-fold cross-validation on data from 8 EDs. Performance was compared to the Charlson and Elixhauser comorbidity indices using area under the receiver-operating characteristic curve (AUROC), calibration, and decision curve analyses. Reproducibility was tested against data from the remaining independent ED within the health system. We then used GEST to investigate rates of ACP documentation availability and code status orders in the EHR across risk strata. RESULTS A total of 431,179 encounters by 123,128 adults were included in this study with a 6-month mortality rate of 12.2%. Charlson (AUROC (95% CI): 0.65 (0.64-0.69)) and Elixhauser indices (0.69 (0.68-0.70)) were outperformed by GEST (0.82 (0.82-0.83)). GEST displayed robust performance across demographic subgroups and in our independent validation site. Among patients with a greater than 30% mortality risk using GEST, only 5.0% had ACP documentation; 79.0% had a code status previously ordered, of which 70.7% were full code. In decision curve analysis, GEST provided greater net benefit than the Charlson and Elixhauser scores. CONCLUSIONS Prognostic models using EHR data robustly identify high mortality risk older adults in the ED for whom code status, ACP, or palliative care interventions may be of benefit. Although all tested methods identified patients approaching the end-of-life, GEST was most performant. These tools may enable resource-sensitive end-of-life screening in the ED.
Collapse
Affiliation(s)
- Adrian D Haimovich
- Department of Emergency Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - Wenxin Xu
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - Andrew Wei
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| | - Mara A Schonberg
- Department of Internal Medicine, Beth Israel Deaconess Medical Center, Boston, Massachusetts, USA
| | - Ula Hwang
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
- Geriatric Research, Education and Clinical Center, James J. Peters VAMC, Bronx, New York, USA
| | - R Andrew Taylor
- Department of Emergency Medicine, Yale School of Medicine, New Haven, Connecticut, USA
| |
Collapse
|
39
|
Sedhom R, Shulman LN, Parikh RB. Precision Palliative Care as a Pragmatic Solution for a Care Delivery Problem. J Clin Oncol 2023; 41:2888-2892. [PMID: 37084327 PMCID: PMC10414742 DOI: 10.1200/jco.22.02532] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2022] [Revised: 03/01/2023] [Accepted: 03/23/2023] [Indexed: 04/23/2023] Open
Affiliation(s)
- Ramy Sedhom
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, Penn Medicine, Philadelphia, PA
| | - Lawrence N. Shulman
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, Penn Medicine, Philadelphia, PA
| | - Ravi B. Parikh
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, Penn Medicine, Philadelphia, PA
| |
Collapse
|
40
|
Peters PN, Havrilesky LJ, Davidson BA. Guidelines for goals of care discussions in patients with gynecologic cancer. Gynecol Oncol 2023; 174:247-252. [PMID: 37243995 DOI: 10.1016/j.ygyno.2023.05.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 05/07/2023] [Accepted: 05/15/2023] [Indexed: 05/29/2023]
Abstract
This article represents a distillation of literature to provide guidance for goals of care discussions with patients who have gynecologic malignancies. As clinicians who provide surgical care, chemotherapy, and targeted therapeutics, gynecologic oncology clinicians are uniquely positioned to form longitudinal relationships with patients that can enable patient-centered decision making. In this review, we describe optimal timing, components, and best practices for goals of care discussions in gynecologic oncology.
Collapse
Affiliation(s)
- Pamela N Peters
- Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, Duke University Health System, Durham, NC 27710, United States of America.
| | - Laura J Havrilesky
- Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, Duke University Health System, Durham, NC 27710, United States of America
| | - Brittany A Davidson
- Division of Gynecologic Oncology, Department of Obstetrics & Gynecology, Duke University Health System, Durham, NC 27710, United States of America
| |
Collapse
|
41
|
Ma JE, Lowe J, Berkowitz C, Kim A, Togo I, Musser RC, Fischer J, Shah K, Ibrahim S, Bosworth HB, Totten AM, Dolor R. Provider Interaction With an Electronic Health Record Notification to Identify Eligible Patients for a Cluster Randomized Trial of Advance Care Planning in Primary Care: Secondary Analysis. J Med Internet Res 2023; 25:e41884. [PMID: 37171856 DOI: 10.2196/41884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2022] [Revised: 02/17/2023] [Accepted: 03/21/2023] [Indexed: 05/13/2023] Open
Abstract
BACKGROUND Advance care planning (ACP) improves patient-provider communication and aligns care to patient values, preferences, and goals. Within a multisite Meta-network Learning and Research Center ACP study, one health system deployed an electronic health record (EHR) notification and algorithm to alert providers about patients potentially appropriate for ACP and the clinical study. OBJECTIVE The aim of the study is to describe the implementation and usage of an EHR notification for referring patients to an ACP study, evaluate the association of notifications with study referrals and engagement in ACP, and assess provider interactions with and perspectives on the notifications. METHODS A secondary analysis assessed provider usage and their response to the notification (eg, acknowledge, dismiss, or engage patient in ACP conversation and refer patient to the clinical study). We evaluated all patients identified by the EHR algorithm during the Meta-network Learning and Research Center ACP study. Descriptive statistics compared patients referred to the study to those who were not referred to the study. Health care utilization, hospice referrals, and mortality as well as documentation and billing for ACP and related legal documents are reported. We evaluated associations between notifications with provider actions (ie, referral to study, ACP not documentation, and ACP billing). Provider free-text comments in the notifications were summarized qualitatively. Providers were surveyed on their satisfaction with the notification. RESULTS Among the 2877 patients identified by the EHR algorithm over 20 months, 17,047 unique notifications were presented to 45 providers in 6 clinics, who then referred 290 (10%) patients. Providers had a median of 269 (IQR 65-552) total notifications, and patients had a median of 4 (IQR 2-8). Patients with more (over 5) notifications were less likely to be referred to the study than those with fewer notifications (57/1092, 5.2% vs 233/1785, 13.1%; P<.001). The most common free-text comment on the notification was lack of time. Providers who referred patients to the study were more likely to document ACP and submit ACP billing codes (P<.001). In the survey, 11 providers would recommend the notification (n=7, 64%); however, the notification impacted clinical workflow (n=9, 82%) and was difficult to navigate (n=6, 55%). CONCLUSIONS An EHR notification can be implemented to remind providers to both perform ACP conversations and refer patients to a clinical study. There were diminishing returns after the fifth EHR notification where additional notifications did not lead to more trial referrals, ACP documentation, or ACP billing. Creation and optimization of EHR notifications for study referrals and ACP should consider the provider user, their workflow, and alert fatigue to improve implementation and adoption. TRIAL REGISTRATION ClinicalTrials.gov NCT03577002; https://clinicaltrials.gov/ct2/show/NCT03577002.
Collapse
Affiliation(s)
- Jessica E Ma
- Geriatric Research, Education, and Clinical Center, Durham Veterans Affairs Health Care System, Durham, NC, United States
- Division of General Internal Medicine, Department of Medicine, Duke University School of Medicine, Durham, NC, United States
| | - Jared Lowe
- Division of General Medicine & Clinical Epidemiology, Department of Medicine, University of North Carolina School of Medicine, Chapel Hill, NC, United States
| | - Callie Berkowitz
- Division of Hematology and Oncology, Department of Medicine, University of North Carolina School of Medicine, Chapel Hill, NC, United States
| | - Azalea Kim
- Division of General Internal Medicine, Department of Medicine, Duke University School of Medicine, Durham, NC, United States
| | - Ira Togo
- Duke Office of Clinical Research, Durham, NC, United States
| | - R Clayton Musser
- Division of General Internal Medicine, Department of Medicine, Duke University School of Medicine, Durham, NC, United States
| | - Jonathan Fischer
- Department of Community & Family Medicine, Duke University School of Medicine, Durham, NC, United States
- Duke Population Health Management Office, Durham, NC, United States
| | - Kevin Shah
- Department of Medicine, Duke University School of Medicine, Durham, NC, United States
| | - Salam Ibrahim
- Duke Health Performance Services, Duke University Health System, Durham, NC, United States
| | - Hayden B Bosworth
- Division of General Internal Medicine, Department of Medicine, Duke University School of Medicine, Durham, NC, United States
- Department of Community & Family Medicine, Duke University School of Medicine, Durham, NC, United States
- Department of Medicine, Duke University School of Medicine, Durham, NC, United States
- Department of Psychiatry and Behavioral Services, Duke University School of Medicine, Durham, NC, United States
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC, United States
- Center of Innovation to Accelerate Discovery and Practice Transformation, Durham Veterans Affairs Health Care System, Durham, NC, United States
| | - Annette M Totten
- Oregon Rural Practice Based Research Network, Oregon Health & Science University School of Medicine, Portland, OR, United States
| | - Rowena Dolor
- Division of General Medicine & Clinical Epidemiology, Department of Medicine, University of North Carolina School of Medicine, Chapel Hill, NC, United States
| |
Collapse
|
42
|
Manz CR, Zhang Y, Chen K, Long Q, Small DS, Evans CN, Chivers C, Regli SH, Hanson CW, Bekelman JE, Braun J, Rareshide CAL, O'Connor N, Kumar P, Schuchter LM, Shulman LN, Patel MS, Parikh RB. Long-term Effect of Machine Learning-Triggered Behavioral Nudges on Serious Illness Conversations and End-of-Life Outcomes Among Patients With Cancer: A Randomized Clinical Trial. JAMA Oncol 2023; 9:414-418. [PMID: 36633868 PMCID: PMC9857721 DOI: 10.1001/jamaoncol.2022.6303] [Citation(s) in RCA: 29] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
Importance Serious illness conversations (SICs) between oncology clinicians and patients are associated with improved quality of life and may reduce aggressive end-of-life care. However, most patients with cancer die without a documented SIC. Objective To test the impact of behavioral nudges to clinicians to prompt SICs on the SIC rate and end-of-life outcomes among patients at high risk of death within 180 days (high-risk patients) as identified by a machine learning algorithm. Design, Setting, and Participants This prespecified 40-week analysis of a stepped-wedge randomized clinical trial conducted between June 17, 2019, and April 20, 2020 (including 16 weeks of intervention rollout and 24 weeks of follow-up), included 20 506 patients with cancer representing 41 021 encounters at 9 tertiary or community-based medical oncology clinics in a large academic health system. The current analyses were conducted from June 1, 2021, to May 31, 2022. Intervention High-risk patients were identified using a validated electronic health record machine learning algorithm to predict 6-month mortality. The intervention consisted of (1) weekly emails to clinicians comparing their SIC rates for all patients against peers' rates, (2) weekly lists of high-risk patients, and (3) opt-out text messages to prompt SICs before encounters with high-risk patients. Main Outcomes and Measures The primary outcome was SIC rates for all and high-risk patient encounters; secondary end-of-life outcomes among decedents included inpatient death, hospice enrollment and length of stay, and intensive care unit admission and systemic therapy close to death. Intention-to-treat analyses were adjusted for clinic and wedge fixed effects and clustered at the oncologist level. Results The study included 20 506 patients (mean [SD] age, 60.0 [14.0] years) and 41 021 patient encounters: 22 259 (54%) encounters with female patients, 28 907 (70.5%) with non-Hispanic White patients, and 5520 (13.5%) with high-risk patients; 1417 patients (6.9%) died by the end of follow-up. There were no meaningful differences in demographic characteristics in the control and intervention periods. Among high-risk patient encounters, the unadjusted SIC rates were 3.4% (59 of 1754 encounters) in the control period and 13.5% (510 of 3765 encounters) in the intervention period. In adjusted analyses, the intervention was associated with increased SICs for all patients (adjusted odds ratio, 2.09 [95% CI, 1.53-2.87]; P < .001) and decreased end-of-life systemic therapy (7.5% [72 of 957 patients] vs 10.4% [24 of 231 patients]; adjusted odds ratio, 0.25 [95% CI, 0.11-0.57]; P = .001) relative to controls, but there was no effect on hospice enrollment or length of stay, inpatient death, or end-of-life ICU use. Conclusions and Relevance In this randomized clinical trial, a machine learning-based behavioral intervention and behavioral nudges to clinicans led to an increase in SICs and reduction in end-of-life systemic therapy but no changes in other end-of-life outcomes among outpatients with cancer. These results suggest that machine learning and behavioral nudges can lead to long-lasting improvements in cancer care delivery. Trial Registration ClinicalTrials.gov Identifier: NCT03984773.
Collapse
Affiliation(s)
- Christopher R Manz
- Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
| | - Yichen Zhang
- Division of Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Kan Chen
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Qi Long
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Dylan S Small
- Wharton School of the University of Pennsylvania, Philadelphia
| | - Chalanda N Evans
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | | | | | | | - Justin E Bekelman
- Division of Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia
| | | | - Charles A L Rareshide
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Nina O'Connor
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Pallavi Kumar
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania
| | - Lynn M Schuchter
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia
| | - Lawrence N Shulman
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia
| | | | - Ravi B Parikh
- Division of Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia
- Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania
| |
Collapse
|
43
|
Gensheimer MF, Gupta D, Patel MI, Fardeen T, Hildebrand R, Teuteberg W, Seevaratnam B, Asuncion MK, Alves N, Rogers B, Hansen J, DeNofrio J, Shah NH, Parikh D, Neal J, Fan AC, Moore K, Ruiz S, Li C, Khaki AR, Pagtama J, Chien J, Brown T, Tisch AH, Das M, Srinivas S, Roy M, Wakelee H, Myall NJ, Huang J, Shah S, Lee H, Ramchandran K. Use of Machine Learning and Lay Care Coaches to Increase Advance Care Planning Conversations for Patients With Metastatic Cancer. JCO Oncol Pract 2023; 19:e176-e184. [PMID: 36395436 DOI: 10.1200/op.22.00128] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
PURPOSE Patients with metastatic cancer benefit from advance care planning (ACP) conversations. We aimed to improve ACP using a computer model to select high-risk patients, with shorter predicted survival, for conversations with providers and lay care coaches. Outcomes included ACP documentation frequency and end-of-life quality measures. METHODS In this study of a quality improvement initiative, providers in four medical oncology clinics received Serious Illness Care Program training. Two clinics (thoracic/genitourinary) participated in an intervention, and two (cutaneous/sarcoma) served as controls. ACP conversations were documented in a centralized form in the electronic medical record. In the intervention, providers and care coaches received weekly e-mails highlighting upcoming clinic patients with < 2 year computer-predicted survival and no prior prognosis documentation. Care coaches contacted these patients for an ACP conversation (excluding prognosis). Providers were asked to discuss and document prognosis. RESULTS In the four clinics, 4,968 clinic visits by 1,251 patients met inclusion criteria (metastatic cancer with no prognosis previously documented). In their first visit, 28% of patients were high-risk (< 2 year predicted survival). Preintervention, 3% of both intervention and control clinic patients had ACP documentation during a visit. By intervention end (February 2021), 35% of intervention clinic patients had ACP documentation compared with 3% of control clinic patients. Providers' prognosis documentation rate also increased in intervention clinics after the intervention (2%-27% in intervention clinics, P < .0001; 0%-1% in control clinics). End-of-life care intensity was similar in intervention versus control clinics, but patients with ≥ 1 provider ACP edit met fewer high-intensity care measures (P = .04). CONCLUSION Combining a computer prognosis model with care coaches increased ACP documentation.
Collapse
Affiliation(s)
| | - Divya Gupta
- Stanford University School of Medicine, Stanford CA
| | - Manali I Patel
- Stanford University School of Medicine, Stanford CA.,VA Palo Alto Health Care System, Palo Alto, CA
| | | | | | | | | | | | - Nina Alves
- Stanford University School of Medicine, Stanford CA
| | - Brian Rogers
- Stanford University School of Medicine, Stanford CA
| | | | - Jan DeNofrio
- Stanford University School of Medicine, Stanford CA
| | - Nigam H Shah
- Stanford University School of Medicine, Stanford CA
| | - Divya Parikh
- Stanford University School of Medicine, Stanford CA
| | - Joel Neal
- Stanford University School of Medicine, Stanford CA
| | - Alice C Fan
- Stanford University School of Medicine, Stanford CA
| | - Kaidi Moore
- Stanford University School of Medicine, Stanford CA
| | - Shann Ruiz
- Stanford University School of Medicine, Stanford CA
| | - Connie Li
- Stanford University School of Medicine, Stanford CA
| | | | - Judy Pagtama
- Stanford University School of Medicine, Stanford CA
| | - Joanne Chien
- Stanford University School of Medicine, Stanford CA
| | | | | | - Millie Das
- Stanford University School of Medicine, Stanford CA
| | | | - Mohana Roy
- Stanford University School of Medicine, Stanford CA
| | | | | | - Jane Huang
- Stanford University School of Medicine, Stanford CA
| | - Sumit Shah
- Stanford University School of Medicine, Stanford CA
| | - Howard Lee
- Stanford University School of Medicine, Stanford CA
| | | |
Collapse
|
44
|
Parikh RB, Zhang Y, Kolla L, Chivers C, Courtright KR, Zhu J, Navathe AS, Chen J. Performance drift in a mortality prediction algorithm among patients with cancer during the SARS-CoV-2 pandemic. J Am Med Inform Assoc 2023; 30:348-354. [PMID: 36409991 PMCID: PMC9846686 DOI: 10.1093/jamia/ocac221] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Revised: 10/28/2022] [Accepted: 11/03/2022] [Indexed: 11/22/2022] Open
Abstract
Sudden changes in health care utilization during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic may have impacted the performance of clinical predictive models that were trained prior to the pandemic. In this study, we evaluated the performance over time of a machine learning, electronic health record-based mortality prediction algorithm currently used in clinical practice to identify patients with cancer who may benefit from early advance care planning conversations. We show that during the pandemic period, algorithm identification of high-risk patients had a substantial and sustained decline. Decreases in laboratory utilization during the peak of the pandemic may have contributed to drift. Calibration and overall discrimination did not markedly decline during the pandemic. This argues for careful attention to the performance and retraining of predictive algorithms that use inputs from the pandemic period.
Collapse
Affiliation(s)
- Ravi B Parikh
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Corporal Michael J. Crescenz VA Medical Center, Philadelphia, Pennsylvania, USA
| | - Yichen Zhang
- Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Likhitha Kolla
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Corey Chivers
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Katherine R Courtright
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Jingsan Zhu
- Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Amol S Navathe
- Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Corporal Michael J. Crescenz VA Medical Center, Philadelphia, Pennsylvania, USA
| | - Jinbo Chen
- Department of Biostatistics, Epidemiology and Informatics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
45
|
Kolla L, Chen J, Parikh RB. Time of Clinic Appointment and Serious Illness Communication in Oncology. Cancer Control 2023; 30:10732748231170488. [PMID: 37071969 PMCID: PMC10126780 DOI: 10.1177/10732748231170488] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/20/2023] Open
Abstract
INTRODUCTION Serious illness communication in oncology increases goal concordant care. Factors associated with the frequency of serious illness conversations are not well understood. Given prior evidence of the association between suboptimal decision-making and clinic time, we aimed to investigate the relationship between appointment time and the likelihood of serious illness conversations in oncology. METHODS We conducted a retrospective study of electronic health record data from 55 367 patient encounters between June 2019 to April 2020, using generalized estimating equations to model the likelihood of a serious illness conversation across clinic time. RESULTS Documentation rate decreased from 2.1 to 1.5% in the morning clinic session (8am-12pm) and from 1.2% to .9% in the afternoon clinic session (1pm-4pm). Adjusted odds ratios for Serious illness conversations documentation rates were significantly lower for all hours of each session after the earliest hour (adjusted odds ratios .91 [95% CI, .84-.97], P = .006 for overall linear trend). CONCLUSIONS Serious illness conversations between oncologists and patients decrease considerably through the clinic day, and proactive strategies to avoid missed conversations should be investigated.
Collapse
Affiliation(s)
- Likhitha Kolla
- Perelman School of Medicine, Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA, USA
- Perelman School of Medicine, Department of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Jinbo Chen
- Perelman School of Medicine, Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA, USA
| | - Ravi B Parikh
- Perelman School of Medicine, Department of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Perelman School of Medicine, Department of Medical Ethics and Health Policy, University of Pennsylvania, Philadelphia, PA, USA
- Corporal Michael J. Crescenz VA Medical Center, Philadelphia, PA, USA
| |
Collapse
|
46
|
Adusumalli S, Kanter GP, Small DS, Asch DA, Volpp KG, Park SH, Gitelman Y, Do D, Leri D, Rhodes C, VanZandbergen C, Howell JT, Epps M, Cavella AM, Wenger M, Harrington TO, Clark K, Westover JE, Snider CK, Patel MS. Effect of Nudges to Clinicians, Patients, or Both to Increase Statin Prescribing: A Cluster Randomized Clinical Trial. JAMA Cardiol 2023; 8:23-30. [PMID: 36449275 PMCID: PMC9713674 DOI: 10.1001/jamacardio.2022.4373] [Citation(s) in RCA: 14] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Accepted: 09/29/2022] [Indexed: 12/02/2022]
Abstract
Importance Statins reduce the risk of major adverse cardiovascular events, but less than one-half of individuals in America who meet guideline criteria for a statin are actively prescribed this medication. Objective To evaluate whether nudges to clinicians, patients, or both increase initiation of statin prescribing during primary care visits. Design, Setting, and Participants This cluster randomized clinical trial evaluated statin prescribing of 158 clinicians from 28 primary care practices including 4131 patients. The design included a 12-month preintervention period and a 6-month intervention period between October 19, 2019, and April 18, 2021. Interventions The usual care group received no interventions. The clinician nudge combined an active choice prompt in the electronic health record during the patient visit and monthly feedback on prescribing patterns compared with peers. The patient nudge was an interactive text message delivered 4 days before the visit. The combined nudge included the clinician and patient nudges. Main Outcomes and Measures The primary outcome was initiation of a statin prescription during the visit. Results The sample comprised 4131 patients with a mean (SD) age of 65.5 (10.5) years; 2120 (51.3%) were male; 1210 (29.3%) were Black, 106 (2.6%) were Hispanic, 2732 (66.1%) were White, and 83 (2.0%) were of other race or ethnicity, and 933 (22.6%) had atherosclerotic cardiovascular disease. In unadjusted analyses during the preintervention period, statins were prescribed to 5.6% of patients (105 of 1876) in the usual care group, 4.8% (97 of 2022) in the patient nudge group, 6.0% (104 of 1723) in the clinician nudge group, and 4.7% (82 of 1752) in the combined group. During the intervention, statins were prescribed to 7.3% of patients (75 of 1032) in the usual care group, 8.5% (100 of 1181) in the patient nudge group, 13.0% (128 of 981) in the clinician nudge arm, and 15.5% (145 of 937) in the combined group. In the main adjusted analyses relative to usual care, the clinician nudge significantly increased statin prescribing alone (5.5 percentage points; 95% CI, 3.4 to 7.8 percentage points; P = .01) and when combined with the patient nudge (7.2 percentage points; 95% CI, 5.1 to 9.1 percentage points; P = .001). The patient nudge alone did not change statin prescribing relative to usual care (0.9 percentage points; 95% CI, -0.8 to 2.5 percentage points; P = .32). Conclusions and Relevance Nudges to clinicians with and without a patient nudge significantly increased initiation of a statin prescription during primary care visits. The patient nudge alone was not effective. Trial Registration ClinicalTrials.gov Identifier: NCT04307472.
Collapse
Affiliation(s)
| | | | - Dylan S. Small
- Wharton School, University of Pennsylvania, Philadelphia
| | - David A. Asch
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Wharton School, University of Pennsylvania, Philadelphia
| | - Kevin G. Volpp
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
- Wharton School, University of Pennsylvania, Philadelphia
- Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania
| | - Sae-Hwan Park
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Yevgeniy Gitelman
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - David Do
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Damien Leri
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Corinne Rhodes
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | | | - John T. Howell
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Mika Epps
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Ann M. Cavella
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | - Michael Wenger
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | | | - Kayla Clark
- Perelman School of Medicine, University of Pennsylvania, Philadelphia
| | | | | | | |
Collapse
|
47
|
Parikh RB, Hasler JS, Zhang Y, Liu M, Chivers C, Ferrell W, Gabriel PE, Lerman C, Bekelman JE, Chen J. Development of Machine Learning Algorithms Incorporating Electronic Health Record Data, Patient-Reported Outcomes, or Both to Predict Mortality for Outpatients With Cancer. JCO Clin Cancer Inform 2022; 6:e2200073. [PMID: 36480775 PMCID: PMC10166444 DOI: 10.1200/cci.22.00073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022] Open
Abstract
PURPOSE Machine learning (ML) algorithms that incorporate routinely collected patient-reported outcomes (PROs) alongside electronic health record (EHR) variables may improve prediction of short-term mortality and facilitate earlier supportive and palliative care for patients with cancer. METHODS We trained and validated two-phase ML algorithms that incorporated standard PRO assessments alongside approximately 200 routinely collected EHR variables, among patients with medical oncology encounters at a tertiary academic oncology and a community oncology practice. RESULTS Among 12,350 patients, 5,870 (47.5%) completed PRO assessments. Compared with EHR- and PRO-only algorithms, the EHR + PRO model improved predictive performance in both tertiary oncology (EHR + PRO v EHR v PRO: area under the curve [AUC] 0.86 [0.85-0.87] v 0.82 [0.81-0.83] v 0.74 [0.74-0.74]) and community oncology (area under the curve 0.89 [0.88-0.90] v 0.86 [0.85-0.88] v 0.77 [0.76-0.79]) practices. CONCLUSION Routinely collected PROs contain added prognostic information not captured by an EHR-based ML mortality risk algorithm. Augmenting an EHR-based algorithm with PROs resulted in a more accurate and clinically relevant model, which can facilitate earlier and targeted supportive care for patients with cancer.
Collapse
Affiliation(s)
- Ravi B Parikh
- Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA.,Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA.,Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA.,Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA.,Corporal Michael J. Crescenz VA Medical Center, Philadelphia, PA
| | - Jill S Hasler
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, Philadelphia, PA
| | - Yichen Zhang
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA
| | - Manqing Liu
- Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, MA
| | - Corey Chivers
- Penn Medicine, University of Pennsylvania, Philadelphia, PA
| | - William Ferrell
- Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA
| | - Peter E Gabriel
- Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA
| | - Caryn Lerman
- USC Norris Comprehensive Cancer Center, Los Angeles, CA
| | - Justin E Bekelman
- Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA.,Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA.,Penn Center for Cancer Care Innovation, Abramson Cancer Center, University of Pennsylvania, Philadelphia, PA.,Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA
| | - Jinbo Chen
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, Philadelphia, PA
| |
Collapse
|
48
|
Parikh RB, Manz CR, Nelson MN, Ferrell W, Belardo Z, Temel JS, Patel MS, Shea JA. Oncologist Perceptions of Algorithm-Based Nudges to Prompt Early Serious Illness Communication: A Qualitative Study. J Palliat Med 2022; 25:1702-1707. [PMID: 35984992 PMCID: PMC9836678 DOI: 10.1089/jpm.2022.0095] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/15/2022] [Indexed: 01/22/2023] Open
Abstract
Background: Early serious illness conversations (SICs) about goals of care and prognosis improve mood, quality of life, and end-of-life care quality. Algorithm-based behavioral nudges to oncologists increase the frequency and timeliness of such conversations. However, clinicians' perspectives on such nudges are unknown. Design: Qualitative study consisting of semistructured interviews among medical oncology clinicians who participated in a stepped-wedge cluster randomized trial of Conversation Connect, an algorithm-based intervention consisting of behavioral nudges to promote early SICs in the outpatient oncology setting. Results: Of 79 eligible oncology clinicians, 56 (71%) were approached to participate in interviews and 25 (45%) accepted. Key facilitators to algorithm-based nudges included prompting documentation of conversations, peer comparisons, performance reports, and validating norms around early conversations. Barriers included cancer-specific heterogeneity in algorithm performance and the frequency and tone of text messages. Areas of improvement included utilizing different information channels, identifying patients earlier in the disease trajectory, and incorporating patient-targeted messaging that emphasizes the value of early conversations. Conclusions: Oncology clinicians identified key facilitators and barriers to Conversation Connect. These insights inform future algorithm-based supportive care interventions in oncology. Controlled trial (NCT03984773).
Collapse
Affiliation(s)
- Ravi B. Parikh
- Perelman School of Medicine and University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Abramson Cancer Center, University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Corporal Michael J. Crescenz Veterans Affairs Medical Center, Philadelphia, Pennsylvania, USA
| | - Christopher R. Manz
- Dana-Farber Cancer Institute, Boston, Massachusetts, USA
- Harvard Medical School, Boston, Massachusetts, USA
| | - Maria N. Nelson
- Perelman School of Medicine and University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - William Ferrell
- Perelman School of Medicine and University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Zoe Belardo
- Perelman School of Medicine and University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Jennifer S. Temel
- Harvard Medical School, Boston, Massachusetts, USA
- Division of Hematology and Oncology, Massachusetts General Hospital, Boston, Massachusetts, USA
| | - Mitesh S. Patel
- Perelman School of Medicine and University of Pennsylvania, Philadelphia, Pennsylvania, USA
- Penn Medicine Nudge Unit, Philadelphia, Pennsylvania, USA
- Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania, USA
| | - Judy A. Shea
- Perelman School of Medicine and University of Pennsylvania, Philadelphia, Pennsylvania, USA
| |
Collapse
|
49
|
Wolf A, Sant'Anna A, Vilhelmsson A. Using nudges to promote clinical decision making of healthcare professionals: A scoping review. Prev Med 2022; 164:107320. [PMID: 36283484 DOI: 10.1016/j.ypmed.2022.107320] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/16/2022] [Revised: 08/25/2022] [Accepted: 10/18/2022] [Indexed: 10/31/2022]
Abstract
Nudging has been discussed in the context of policy and public health, but not so much within healthcare. This scoping review aimed to assess the empirical evidence on how nudging techniques can be used to affect the behavior of healthcare professionals (HCPs) in clinical settings. A systematic database search was conducted for the period January 2010-December 2020 using the PRISMA extension for Scoping Review checklist. Two reviewers independently screened each article for inclusion. Included articles were reviewed to extract key information about each intervention, including purpose, target behavior, measured outcomes, key findings, nudging strategies, intervention objectives and their theoretical underpinnings. Two independent dimensions, building on Kahneman's System 1 and System 2, were used to describe nudging strategies according to user action and timing of their implementation. Of the included 51 articles, 40 reported statistically significant results, six were not significant and two reported mixed results. Thirteen different nudging strategies were identified aimed at modifying four types of HPCs' behavior: prescriptions and orders, procedure, hand hygiene, and vaccination. The most common nudging strategy employed were defaults or pre-orders, followed by alerts or reminders, and active choice. Many interventions did not require any deliberate action from users, here termed passive interventions, such as automatically changing prescriptions to their generic equivalent unless indicated by the user. Passive nudges may be successful in changing the target outcome but may go unnoticed by the user. Future work should consider the broader ethical implications of passive nudges.
Collapse
Affiliation(s)
- Axel Wolf
- University of Gothenburg, Centre for Person-Centred Care (GPCC), Sweden; University of Gothenburg, Institute of Health and Care Sciences, Sahlgrenska Academy, Sweden
| | | | - Andreas Vilhelmsson
- Lund University, Department of Laboratory Medicine, Division of Occupational and Environmental Medicine, Sweden.
| |
Collapse
|
50
|
Quinn KL, Abdel‐Qadir H, Barrett K, Bartsch E, Beaman A, Biering‐Sørensen T, Colacci M, Cressman A, Detsky A, Gosset A, Lassen MH, Kandel C, Khaykin Y, Lapointe‐Shaw L, Lovblom E, MacFadden DR, Perkins B, Rothman KJ, Skaarup KG, Stall N, Tang T, Yarnell C, Zipursky J, Warkentin MT, Fralick M. Variation in the risk of death due to COVID-19: An international multicenter cohort study of hospitalized adults. J Hosp Med 2022; 17:793-802. [PMID: 36040111 PMCID: PMC9539016 DOI: 10.1002/jhm.12946] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Revised: 06/28/2022] [Accepted: 07/06/2022] [Indexed: 01/08/2023]
Abstract
BACKGROUND There is wide variation in mortality among patients hospitalized with COVID-19. Whether this is related to patient or hospital factors is unknown. OBJECTIVE To compare the risk of mortality for patients hospitalized with COVID-19 and to determine whether the majority of that variation was explained by differences in patient characteristics across sites. DESIGN, SETTING, AND PARTICIPANTS An international multicenter cohort study of hospitalized adults with laboratory-confirmed COVID-19 enrolled from 10 hospitals in Ontario, Canada and 8 hospitals in Copenhagen, Denmark between January 1, 2020 and November 11, 2020. MAIN OUTCOMES AND MEASURES Inpatient mortality. We used a multivariable multilevel regression model to compare the in-hospital mortality risk across hospitals and quantify the variation attributable to patient-level factors. RESULTS There were 1364 adults hospitalized with COVID-19 in Ontario (n = 1149) and in Denmark (n = 215). In Ontario, the absolute risk of in-hospital mortality ranged from 12.0% to 39.8% across hospitals. Ninety-eight percent of the variation in mortality in Ontario was explained by differences in the characteristics of the patients. In Denmark, the absolute risk of inpatients ranged from 13.8% to 20.6%. One hundred percent of the variation in mortality in Denmark was explained by differences in the characteristics of the inpatients. CONCLUSION There was wide variation in inpatient COVID-19 mortality across hospitals, which was largely explained by patient-level factors, such as age and severity of presenting illness. However, hospital-level factors that could have affected care, including resource availability and capacity, were not taken into account. These findings highlight potential limitations in comparing crude mortality rates across hospitals for the purposes of reporting on the quality of care.
Collapse
Affiliation(s)
- Kieran L. Quinn
- Department of Medicine, Sinai Health SystemUniversity of TorontoTorontoOntarioCanada
- Division of Internal Medicine, Temerty Faculty of MedicineUniversity of TorontoTorontoOntarioCanada
- Interdepartmental Division of Palliative Care, Sinai Health SystemUniversity of TorontoTorontoOntarioCanada
| | - Husam Abdel‐Qadir
- Department of Medicine, Division of CardiologyWomen's College HospitalTorontoOntarioCanada
- Department of MedicineUniversity Health NetworkTorontoOntarioCanada
| | - Kali Barrett
- Department of MedicineUniversity Health NetworkTorontoOntarioCanada
- Department of Medicine, Interdepartmental Division of Critical Care MedicineUniversity of TorontoTorontoOntarioCanada
| | - Emily Bartsch
- Department of Medicine, Sinai Health SystemUniversity of TorontoTorontoOntarioCanada
| | - Andrea Beaman
- Department of PharmacyTrillium Health PartnersMississaugaOntarioCanada
| | | | - Michael Colacci
- Department of MedicineUniversity of TorontoTorontoOntarioCanada
| | - Alex Cressman
- Division of Internal Medicine, Temerty Faculty of MedicineUniversity of TorontoTorontoOntarioCanada
| | - Allan Detsky
- Department of Medicine, Sinai Health SystemUniversity of TorontoTorontoOntarioCanada
| | - Alexi Gosset
- Department of MedicineUniversity of TorontoTorontoOntarioCanada
| | - Mats H. Lassen
- Department of MedicineUniversity of TorontoTorontoOntarioCanada
| | - Chris Kandel
- Department of MedicineMichael Garron HospitalTorontoOntarioCanada
| | - Yaariv Khaykin
- Department of MedicineSouthlake Regional Health CentreNewmarketOntarioCanada
| | | | - Erik Lovblom
- Department of Medicine, Lunenfeld‐Tanenbaum Research InstituteMount Sinai HospitalTorontoOntarioCanada
| | - Derek R. MacFadden
- Department of MedicineThe Ottawa Hospital Research InstituteOttawaOntarioCanada
| | - Bruce Perkins
- Department of MedicineUniversity Health NetworkTorontoOntarioCanada
| | - Kenneth J Rothman
- Department of Epidemiology, School of Public HealthBoston UniversityMassachusettsBostonUSA
| | | | - Nathan Stall
- Department of Medicine, Division of General Internal Medicine and GeriatricsSinai Health and the University Health NetworkTorontoOntarioCanada
| | - Terence Tang
- Department of Medicine, Interdepartmental Division of Critical Care MedicineUniversity of TorontoTorontoOntarioCanada
| | - Chris Yarnell
- Department of Medicine, Interdepartmental Division of Critical Care MedicineUniversity of TorontoTorontoOntarioCanada
| | - Jonathan Zipursky
- Department of MedicineSunnybrook Health Sciences CentreTorontoOntarioCanada
| | - Matthew T. Warkentin
- Department of Medicine, Lunenfeld‐Tanenbaum Research InstituteMount Sinai HospitalTorontoOntarioCanada
| | - Mike Fralick
- Department of Medicine, Sinai Health SystemUniversity of TorontoTorontoOntarioCanada
| | | |
Collapse
|