Long-term Effect of Machine Learning-Triggered Behavioral Nudges on Serious Illness Conversations and End-of-Life Outcomes Among Patients With Cancer: A Randomized Clinical Trial

Clinicians' Use of Choice Framing in ICU Family Meetings

OBJECTIVES

To quantify the frequency and patterns of clinicians' use of choice frames when discussing preference-sensitive care with surrogate decision-makers in the ICU.

DESIGN

Secondary sequential content analysis.

SETTING

One hundred one audio-recorded and transcribed conferences between surrogates and clinicians of incapacitated, critically ill adults from a prospective, multicenter cohort study.

SUBJECTS

Surrogate decision-makers and clinicians.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Four coders identified preference-sensitive decision episodes addressed in the meetings, including topics such as mechanical ventilation, renal replacement, and overall goals of care. Prior critical care literature provided specific topics identified as preference-sensitive specific to the critical care context. Coders then examined each decision episode for the types of choice frames used by clinicians. The choice frames were selected a priori based on decision science literature. In total, there were 202 decision episodes across the 101 transcripts, with 20.3% of the decision episodes discussing mechanical ventilation, 19.3% overall goals of care, 14.4% renal replacement therapy, 14.4% post-discharge care (i.e., discharge location such as a skilled nursing facility), and the remaining 32.1% other topics. Clinicians used default framing, in which an option is presented that will be carried out if another option is not actively chosen, more frequently than any other choice frame (127 or 62.9% of decision episodes). Clinicians presented a polar interrogative, or a "yes or no question" to accept or reject a specific care choice, in 43 (21.3%) decision episodes. Clinicians more frequently presented options emphasizing both potential losses and gains rather than either in isolation.

CONCLUSIONS

Clinicians frequently use default framing and polar questions when discussing preference-sensitive choices with surrogate decision-makers, which are known to be powerful nudges. Future work should focus on designing interventions promoting the informed use of these and the other most common choice frames used by practicing clinicians.

Equity in Using Artificial Intelligence Mortality Predictions to Target Goals of Care Documentation

BACKGROUND

Artificial intelligence (AI) algorithms are increasingly used to target patients with elevated mortality risk scores for goals-of-care (GOC) conversations.

OBJECTIVE

To evaluate the association between the presence or absence of AI-generated mortality risk scores with GOC documentation.

DESIGN

Retrospective cross-sectional study at one large academic medical center between July 2021 and December 2022.

PARTICIPANTS

Hospitalized adult patients with AI-defined Serious Illness Risk Indicator (SIRI) scores indicating > 30% 90-day mortality risk (defined as "elevated" SIRI) or no SIRI scores due to insufficient data.

INTERVENTION

A targeted intervention to increase GOC documentation for patients with AI-generated scores predicting elevated risk of mortality.

MAIN MEASURES

Odds ratios comparing GOC documentation for patients with elevated or no SIRI scores with similar severity of illness using propensity score matching and risk-adjusted mixed-effects logistic regression.

KEY RESULTS

Among 13,710 patients with elevated (n = 3643, 27%) or no (n = 10,067, 73%) SIRI scores, the median age was 64 years (SD 18). Twenty-five percent were non-White, 18% had Medicaid, 43% were admitted to an intensive care unit, and 11% died during admission. Patients lacking SIRI scores were more likely to be younger (median 60 vs. 72 years, p < 0.0001), be non-White (29% vs. 13%, p < 0.0001), and have Medicaid (22% vs. 9%, p < 0.0001). Patients with elevated versus no SIRI scores were more likely to have GOC documentation in the unmatched (aOR 2.5, p < 0.0001) and propensity-matched cohorts (aOR 2.1, p < 0.0001).

CONCLUSIONS

Using AI predictions of mortality to target GOC documentation may create differences in documentation prevalence between patients with and without AI mortality prediction scores with similar severity of illness. These finding suggest using AI to target GOC documentation may have the unintended consequence of disadvantaging severely ill patients lacking AI-generated scores from receiving targeted GOC documentation, including patients who are more likely to be non-White and have Medicaid insurance.

[A look into the neighboring discipline: eHealth in oncology]

Digitalization is dramatically changing the entire healthcare system. Keywords such as artificial intelligence, electronic patient files (ePA), electronic prescriptions (eRp), telemedicine, wearables, augmented reality and digital health applications (DiGA) represent the digital transformation that is already taking place. Digital becomes real! This article outlines the state of research and development, current plans and ongoing uses of digital tools in oncology in the first half of 2024. The possibilities for using artificial intelligence and the use of DiGAs in oncology are presented in more detail in this overview according to their stage of development as they already show a noticeable benefit in oncology.

Let Us Have the Conversation: Serious Illness Communication in Oncology: Definitions, Barriers, and Successful Approaches

Serious illness communications are crucial elements of care delivery for patients with cancer. High-quality serious illness communications are composed of open, honest discussions between patients, caregivers, and clinicians regarding patient's communication preferences, expected illness trajectory, prognosis, and risks and benefits of any recommended care. High-quality communication ideally starts at the time of a patients' cancer diagnosis, allows space for and response to patient emotions, elicits patients' values and care preferences, and is iterative and longitudinal. When integrated into cancer care, such communication can result in improved patient experiences with their care, care that matches patients' goals, and reduced care intensity at the end of life. Despite national recommendations for routine integration of these communication into cancer care, a minority of patients with cancer receive such communication. In this chapter, we describe elements of high-quality serious illness communication, patient-, clinician-, institution-, and payer-level barriers, and successful strategies that can routinely integrate such communication into cancer care delivery.

Artificial intelligence to unlock real-world evidence in clinical oncology: A primer on recent advances

PURPOSE

Real world evidence is crucial to understanding the diffusion of new oncologic therapies, monitoring cancer outcomes, and detecting unexpected toxicities. In practice, real world evidence is challenging to collect rapidly and comprehensively, often requiring expensive and time-consuming manual case-finding and annotation of clinical text. In this Review, we summarise recent developments in the use of artificial intelligence to collect and analyze real world evidence in oncology.

METHODS

We performed a narrative review of the major current trends and recent literature in artificial intelligence applications in oncology.

RESULTS

Artificial intelligence (AI) approaches are increasingly used to efficiently phenotype patients and tumors at large scale. These tools also may provide novel biological insights and improve risk prediction through multimodal integration of radiographic, pathological, and genomic datasets. Custom language processing pipelines and large language models hold great promise for clinical prediction and phenotyping.

CONCLUSIONS

Despite rapid advances, continued progress in computation, generalizability, interpretability, and reliability as well as prospective validation are needed to integrate AI approaches into routine clinical care and real-time monitoring of novel therapies.

Applying human-centered design to adapt the Jumpstart Guide for goals-of-care discussions in persons living with dementia

BACKGROUND

Goals-of-care discussions (GOCD) are uncommon in persons living with dementia (PLWD) despite the likelihood of eventual loss of decisional capacity in the population. The Jumpstart Guide, an existing serious illness communication priming tool, can improve GOCD in certain populations, but has not previously been adapted for use among PLWD and their caregivers.

METHODS

Using human-centered design (HCD), we adapted the Jumpstart Guide for use with PLWD and their caregivers. We conducted qualitative interviews with clinicians and caregivers of PLWD. Six team members conducted qualitative rapid analysis of interviews leading to the development of summary templates and integrative matrices. Four iterations of the Jumpstart Guide led to the final version.

RESULTS

Thirteen clinicians and 11 caregivers were interviewed. Interviews provided key insights into the unique barriers PLWD and their caregivers face during GOCD, including discomfort with accepting a dementia diagnosis and concern with using "serious illness" to describe dementia, as is commonly done in palliative care. Clinicians described differences in GOCD with PLWD compared to other serious illnesses, and the challenge of getting patients and families to think about future health states. Interviews led to Jumpstart Guide adaptations in the following domains: (1) format and structure, (2) content, and (3) specific language. Suggested changes included prioritizing naming a decision-maker, changing conversation prompts to improve accessibility and understandability, ensuring the Jumpstart Guide could be used with patients as well as their caregivers, and altering language to avoid references to "serious illness" and "abilities."

CONCLUSION

Using HCD yielded valuable insights from clinicians and caregivers about the unique barriers to conducting GOCD among PLWD and their caregivers. These insights were used to adapt the Jumpstart Guide for use with PLWD and their caregivers, which is currently being tested in a pragmatic randomized controlled trial in outpatient clinics.

Machine Learning to Allocate Palliative Care Consultations During Cancer Treatment

PURPOSE

For patients with advanced cancer, early consultations with palliative care (PC) specialists reduce costs, improve quality of life, and prolong survival. However, capacity limitations prevent all patients from receiving PC shortly after diagnosis. We evaluated whether a prognostic machine learning system could promote early PC, given existing capacity.

METHODS

Using population-level administrative data in Ontario, Canada, we assembled a cohort of patients with incurable cancer who received palliative-intent systemic therapy between July 1, 2014, and December 30, 2019. We developed a machine learning system that predicted death within 1 year of each treatment using demographics, cancer characteristics, treatments, symptoms, laboratory values, and history of acute care admissions. We trained the system in patients who started treatment before July 1, 2017, and evaluated the potential impact of the system on PC in subsequent patients.

RESULTS

Among 560,210 treatments received by 54,628 patients, death occurred within 1 year of 45.2% of treatments. The machine learning system recommended the same number of PC consultations observed with usual care at the 60.0% 1-year risk of death, with a first-alarm positive predictive value of 69.7% and an outcome-level sensitivity of 74.9%. Compared with usual care, system-guided care could increase early PC by 8.5% overall (95% CI, 7.5 to 9.5; P < .001) and by 15.3% (95% CI, 13.9 to 16.6; P < .001) among patients who live 6 months beyond their first treatment, without requiring more PC consultations in total or substantially increasing PC among patients with a prognosis exceeding 2 years.

CONCLUSION

Prognostic machine learning systems could increase early PC despite existing resource constraints. These results demonstrate an urgent need to deploy and evaluate prognostic systems in real-time clinical practice to increase access to early PC.

Association Between Intensity and Timing of Specialty Palliative Care and Hospice Exposure With Quality of End-of-Life Care

Background: Gaps remain in our understanding of the intensity and timing of specialty palliative care (SPC) exposure on end-of-life (EOL) outcomes. Objective: Examine the association between intensity and timing of SPC and hospice (HO) exposure on EOL care outcomes. Design, Settings, Participants: Data for this cohort study were drawn from 2021 adult decedents from Kaiser Permanente Southern California and Colorado (n = 26,251). Caregivers of a decedent subgroup completed a postdeath care experience survey from July to August 2022 (n = 424). Measurements: SPC intensity (inpatient, outpatient, and home-based) and HO exposure in the five years before death were categorized as: (1) No SPC or HO; (2) SPC-only; (3) HO-only; and (4) SPC-HO. Timing of SPC exposure (<90 or 90+ days) before death was stratified by HO enrollment. Death in the hospital and potentially burdensome treatments in the last 14 days of life were extracted from electronic medical records (EMRs) and claims. EOL care experience was obtained from the caregiver survey. Results: Among the EMR cohort, exposure to SPC and HO were: No SPC or HO (38%), SPC-only (14%; of whom, 55% received inpatient SPC only), HO-only (20%), and SPC-HO (28%). For decedents who did not enroll in HO, exposure to SPC 90+ days versus <90 days before death was associated with lower risk of receiving potentially burdensome treatments (adjusted relative risk, aRR: 0.69 [95% confidence interval, CI: 0.62-0.76], p < 0.001) and 23% lower risk of dying in the hospital (aRR: 0.77 [95% CI: 0.73-0.81], p < 0.001). Caregivers of patients in the HO-only (aRR: 1.27 [95% CI: 0.98-1.63], p = 0.07) and SPC-HO cohorts (aRR: 1.19 [95% CI: 0.93-1.52], p = 0.18) tended to report more positive care experience compared to the no SPC or HO cohort. Conclusion: Earlier exposure to SPC was important in reducing potentially burdensome treatments and death in the hospital for decedents who did not enroll in HO. Increasing availability and access to community-based SPC is needed.

Looking Beyond Mortality Prediction: Primary Care Physician Views of Patients' Palliative Care Needs Predicted by a Machine Learning Tool

OBJECTIVES

 To assess primary care physicians' (PCPs) perception of the need for serious illness conversations (SIC) or other palliative care interventions in patients flagged by a machine learning tool for high 1-year mortality risk.

METHODS

 We surveyed PCPs from four Brigham and Women's Hospital primary care practice sites. Multiple mortality prediction algorithms were ensembled to assess adult patients of these PCPs who were either enrolled in the hospital's integrated care management program or had one of several chronic conditions. The patients were classified as high or low risk of 1-year mortality. A blinded survey had PCPs evaluate these patients for palliative care needs. We measured PCP and machine learning tool agreement regarding patients' need for an SIC/elevated risk of mortality.

RESULTS

 Of 66 PCPs, 20 (30.3%) participated in the survey. Out of 312 patients evaluated, 60.6% were female, with a mean (standard deviation [SD]) age of 69.3 (17.5) years, and a mean (SD) Charlson Comorbidity Index of 2.80 (2.89). The machine learning tool identified 162 (51.9%) patients as high risk. Excluding deceased or unfamiliar patients, PCPs felt that an SIC was appropriate for 179 patients; the machine learning tool flagged 123 of these patients as high risk (68.7% concordance). For 105 patients whom PCPs deemed SIC unnecessary, the tool classified 83 as low risk (79.1% concordance). There was substantial agreement between PCPs and the tool (Gwet's agreement coefficient of 0.640).

CONCLUSIONS

 A machine learning mortality prediction tool offers promise as a clinical decision aid, helping clinicians pinpoint patients needing palliative care interventions.

Investigating the Impact of AI on Shared Decision-Making in Post-Kidney Transplant Care (PRIMA-AI): Protocol for a Randomized Controlled Trial

BACKGROUND

Patients after kidney transplantation eventually face the risk of graft loss with the concomitant need for dialysis or retransplantation. Choosing the right kidney replacement therapy after graft loss is an important preference-sensitive decision for kidney transplant recipients. However, the rate of conversations about treatment options after kidney graft loss has been shown to be as low as 13% in previous studies. It is unknown whether the implementation of artificial intelligence (AI)-based risk prediction models can increase the number of conversations about treatment options after graft loss and how this might influence the associated shared decision-making (SDM).

OBJECTIVE

This study aims to explore the impact of AI-based risk prediction for the risk of graft loss on the frequency of conversations about the treatment options after graft loss, as well as the associated SDM process.

METHODS

This is a 2-year, prospective, randomized, 2-armed, parallel-group, single-center trial in a German kidney transplant center. All patients will receive the same routine post-kidney transplant care that usually includes follow-up visits every 3 months at the kidney transplant center. For patients in the intervention arm, physicians will be assisted by a validated and previously published AI-based risk prediction system that estimates the risk for graft loss in the next year, starting from 3 months after randomization until 24 months after randomization. The study population will consist of 122 kidney transplant recipients >12 months after transplantation, who are at least 18 years of age, are able to communicate in German, and have an estimated glomerular filtration rate <30 mL/min/1.73 m2. Patients with multi-organ transplantation, or who are not able to communicate in German, as well as underage patients, cannot participate. For the primary end point, the proportion of patients who have had a conversation about their treatment options after graft loss is compared at 12 months after randomization. Additionally, 2 different assessment tools for SDM, the CollaboRATE mean score and the Control Preference Scale, are compared between the 2 groups at 12 months and 24 months after randomization. Furthermore, recordings of patient-physician conversations, as well as semistructured interviews with patients, support persons, and physicians, are performed to support the quantitative results.

RESULTS

The enrollment for the study is ongoing. The first results are expected to be submitted for publication in 2025.

CONCLUSIONS

This is the first study to examine the influence of AI-based risk prediction on physician-patient interaction in the context of kidney transplantation. We use a mixed methods approach by combining a randomized design with a simple quantitative end point (frequency of conversations), different quantitative measurements for SDM, and several qualitative research methods (eg, records of physician-patient conversations and semistructured interviews) to examine the implementation of AI-based risk prediction in the clinic.

TRIAL REGISTRATION

ClinicalTrials.gov NCT06056518; https://clinicaltrials.gov/study/NCT06056518.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/54857.

Ethical issues in pain and palliation

PURPOSE OF REVIEW

Increased public awareness of ethical issues in pain and palliative care, along with patient advocacy groups, put pressure on healthcare systems and professionals to address these concerns.Our aim is to review the ethics dilemmas concerning palliative care in ICU, artificial intelligence applications in pain therapy and palliative care, and the opioids epidemics.

RECENT FINDINGS

In this focus review, we highlighted state of the art papers that were published in the last 18 months, on ethical issues in palliative care within the ICU, artificial intelligence trajectories, and how opioids epidemics has impacted pain management practices (see Visual Abstract).

SUMMARY

Palliative care in the ICU should involve a multidisciplinary team, to mitigate patients suffering and futility. Providing spiritual support in the ICU is an important aspect of holistic patient care too.Increasingly sophisticated tools for diagnosing and treating pain, as those involving artificial intelligence, might favour disparities in access, cause informed consent problems, and surely, they need prudence and reproducibility.Pain clinicians worldwide continue to face the ethical dilemma of prescribing opioids for patients with chronic noncancer pain. Balancing the need for effective pain relief with the risk of opioid misuse, addiction, and overdose is a very controversial task.

Patient and Caregiver Perceptions of an Interface Design to Communicate Artificial Intelligence-Based Prognosis for Patients With Advanced Solid Tumors

PURPOSE

Use of artificial intelligence (AI) in cancer care is increasing. What remains unclear is how best to design patient-facing systems that communicate AI output. With oncologist input, we designed an interface that presents patient-specific, machine learning-based 6-month survival prognosis information designed to aid oncology providers in preparing for and discussing prognosis with patients with advanced solid tumors and their caregivers. The primary purpose of this study was to assess patient and caregiver perceptions and identify enhancements of the interface for communicating 6-month survival and other prognosis information when making treatment decisions concerning anticancer and supportive therapy.

METHODS

This qualitative study included interviews and focus groups conducted between November and December 2022. Purposive sampling was used to recruit former patients with cancer and/or former caregivers of patients with cancer who had participated in cancer treatment decisions from Utah or elsewhere in the United States. Categories and themes related to perceptions of the interface were identified.

RESULTS

We received feedback from 20 participants during eight individual interviews and two focus groups, including four cancer survivors, 13 caregivers, and three representing both. Overall, most participants expressed positive perceptions about the tool and identified its value for supporting decision making, feeling less alone, and supporting communication among oncologists, patients, and their caregivers. Participants identified areas for improvement and implementation considerations, particularly that oncologists should share the tool and guide discussions about prognosis with patients who want to receive the information.

CONCLUSION

This study revealed important patient and caregiver perceptions of and enhancements for the proposed interface. Originally designed with input from oncology providers, patient and caregiver participants identified additional interface design recommendations and implementation considerations to support communication about prognosis.

Factors Associated With and Characteristics of Patients Receiving Proton Therapy at the End of Life

Purpose

To identify the characteristics, indications, and toxicities among patients receiving proton beam therapy (PBT) in the final year of life at an academic medical center.

Materials and Methods

A retrospective review of patients who received PBT within the final 12 months of life was performed. Electronic medical records were reviewed for patient and treatment details from 2010 to 2019. Patients were followed from the start of PBT until death or last follow-up. Acute (3 months) toxicities were graded using the Common Terminology Criteria for Adverse Events v5.0. Imaging response was assessed using the Response Evaluation Criteria in Solid Tumors v1.1. The χ2 test was used to evaluate factors associated with palliative treatment. Simple logistic regression was used to evaluate factors associated with toxicity.

Results

Bet299 patients were treated at the end of life (EOL) out of 5802 total patients treated with PBT (5.2%). Median age was 68 years (19-94 years), 58% male. The most common cancer was nonsmall cell lung cancer (27%). Patients were treated for symptom palliation alone (11%), durable control (57%), curative intent (16%), local recurrence (14%), or oligometastatic disease (2%). Forty-five percent received reirradiation. Median treatment time was 32 days (1-189 days). Acute toxicity was noted in 85% of the patients (31% G1, 53% G2, 15% G3). Thirteen patients (4%) experienced chronic toxicity. Breast and hematologic malignancy were associated with palliative intent χ2 (1, N = 14) = 17, P = .013; (χ2 (1, N = 14) = 18, P = .009).

Conclusion

The number of patients treated with PBT at the EOL was low compared to all comers. Many of these patients received treatment with definitive doses and concurrent systemic therapy. Some patients spent a large portion of their remaining days on treatment. A prognostic indicator may better optimize patient selection for PBT at the EOL.

Microbiome preterm birth DREAM challenge: Crowdsourcing machine learning approaches to advance preterm birth research

Every year, 11% of infants are born preterm with significant health consequences, with the vaginal microbiome a risk factor for preterm birth. We crowdsource models to predict (1) preterm birth (PTB; <37 weeks) or (2) early preterm birth (ePTB; <32 weeks) from 9 vaginal microbiome studies representing 3,578 samples from 1,268 pregnant individuals, aggregated from public raw data via phylogenetic harmonization. The predictive models are validated on two independent unpublished datasets representing 331 samples from 148 pregnant individuals. The top-performing models (among 148 and 121 submissions from 318 teams) achieve area under the receiver operator characteristic (AUROC) curve scores of 0.69 and 0.87 predicting PTB and ePTB, respectively. Alpha diversity, VALENCIA community state types, and composition are important features in the top-performing models, most of which are tree-based methods. This work is a model for translation of microbiome data into clinically relevant predictive models and to better understand preterm birth.

A Preliminary Validation of an Optimal Cutpoint in Total Number of Patient-Reported Symptoms in Head and Neck Cancer for Effective Alignment of Clinical Resources with Patients' Symptom Burden

Background

Patients with head and neck cancer (HNC) often experience high symptom burden leading to lower quality of life (QoL).

Objective

This study aims to conceptually model optimal cutpoint by examining where total number of patient-reported symptoms exceeds patients' coping capacity, leading to a decline in QoL in patients with HNC.

Methods

Secondary data analysis of 105 individuals with HNC enrolled in a clinical usefulness study of the NYU Electronic Patient Visit Assessment (ePVA)©, a digital patient-reported symptom measure. Patients completed ePVA and European Organization for Research and Treatment of Cancer (EORTC©) QLQ-C30 v3.0. The total number of patient-reported symptoms was the sum of symptoms as identified by the ePVA questionnaire. Analysis of variance (ANOVA) was used to define optimal cutpoint.

Results

Study participants had a mean age of 61.5, were primarily male (67.6%), and had Stage IV HNC (53.3%). The cutpoint of 10 symptoms was associated with significant decline of QoL (F= 44.8, P<.0001), dividing the population into categories of low symptom burden (< 10 symptoms) and high symptom burden (≥ 10 symptoms). Analyses of EORTC© function subscales supported the validity of 10 symptoms as the optimal cutpoint (Physical: F=28.3, P<.0001; Role: F=21.6, P<.0001; Emotional: F=9.5, P=.003; Social: F=33.1, P<.0001).

Conclusions

In HNC, defining optimal cutpoints in the total number of patient-reported symptoms is feasible.

Implications for Practice

Cutpoints in the total number of patient-reported symptoms may identify patients experiencing a high symptom burden from HNC.

Foundational

Using optimal cutpoints of the total number of patient-reported symptoms may help effectively align clinical resources with patients' symptom burden.

Optimizing artificial intelligence in sepsis management: Opportunities in the present and looking closely to the future

Sepsis remains a major challenge internationally for healthcare systems. Its incidence is rising due to poor public awareness and delays in its recognition and subsequent management. In sepsis, mortality increases with every hour left untreated. Artificial intelligence (AI) is transforming worldwide healthcare delivery at present. This review has outlined how AI can augment strategies to address this global disease burden. AI and machine learning (ML) algorithms can analyze vast quantities of increasingly complex clinical datasets from electronic medical records to assist clinicians in diagnosing and treating sepsis earlier than traditional methods. Our review highlights how these models can predict the risk of sepsis and organ failure even before it occurs. This gives providers additional time to plan and execute treatment plans, thereby avoiding increasing complications associated with delayed diagnosis of sepsis. The potential for cost savings with AI implementation is also discussed, including improving workflow efficiencies, reducing administrative costs, and improving healthcare outcomes. Despite these advantages, clinicians have been slow to adopt AI into clinical practice. Some of the limitations posed by AI solutions include the lack of diverse data sets for model building so that they are widely applicable for routine clinical use. Furthermore, the subsequent algorithms are often based on complex mathematics leading to clinician hesitancy to embrace such technologies. Finally, we highlight the need for robust political and regulatory frameworks in this area to achieve the trust and approval of clinicians and patients to implement this transformational technology.

Design of an interface to communicate artificial intelligence-based prognosis for patients with advanced solid tumors: a user-centered approach

OBJECTIVES

To design an interface to support communication of machine learning (ML)-based prognosis for patients with advanced solid tumors, incorporating oncologists' needs and feedback throughout design.

MATERIALS AND METHODS

Using an interdisciplinary user-centered design approach, we performed 5 rounds of iterative design to refine an interface, involving expert review based on usability heuristics, input from a color-blind adult, and 13 individual semi-structured interviews with oncologists. Individual interviews included patient vignettes and a series of interfaces populated with representative patient data and predicted survival for each treatment decision point when a new line of therapy (LoT) was being considered. Ongoing feedback informed design decisions, and directed qualitative content analysis of interview transcripts was used to evaluate usability and identify enhancement requirements.

RESULTS

Design processes resulted in an interface with 7 sections, each addressing user-focused questions, supporting oncologists to "tell a story" as they discuss prognosis during a clinical encounter. The iteratively enhanced interface both triggered and reflected design decisions relevant when attempting to communicate ML-based prognosis, and exposed misassumptions. Clinicians requested enhancements that emphasized interpretability over explainability. Qualitative findings confirmed that previously identified issues were resolved and clarified necessary enhancements (eg, use months not days) and concerns about usability and trust (eg, address LoT received elsewhere). Appropriate use should be in the context of a conversation with an oncologist.

CONCLUSION

User-centered design, ongoing clinical input, and a visualization to communicate ML-related outcomes are important elements for designing any decision support tool enabled by artificial intelligence, particularly when communicating prognosis risk.

Patient Portals to Elicit Essential Patient-Reported Elements of Communication Supporting Person-Centered Oncologic Care: A Pilot Study of the PERSON Approach

PURPOSE

Patient portal technology offers important new opportunities to support person-centered clinician-patient communication.

METHODS

Questionnaires relating to understanding of illness and treatment intent were sent quarterly via portal to all patients scheduled for follow-up in GI medical oncology clinics. For patients in selected clinics, items eliciting health-related values were added. Patient responses were available to all oncology team members in the electronic health record. Workflow and content of clinician-patient discussions about illness, treatment, and care goals stayed within clinicians' discretion. Feasibility (patient response rate), patient understanding, acceptability (three-item patient questionnaire), and efficacy (quality of clinician communication) were evaluated.

RESULTS

From May 2021 through December 2022, a total of 12,233 questionnaires about illness/treatment understanding were sent to 6,325 patients (one to six per patient), with 97% response, including 9,358 with both open- and closed-ended responses. Fewer than 0.1% of patients indicated distress related to the questionnaire/process. Open-ended responses complemented closed-ended answers by revealing prognostic awareness and illness concerns. Of 48 patients approached to complete the full questionnaire including values items via portal, 15 first received and completed them in clinic (5 on iPad, 10 on paper), while 33 received and 27 (82%) completed the portal questionnaire. Patients found the portal process acceptable, and ratings of clinician communication were higher after clinic visits informed by patients' questionnaire responses (average prescore 6.8 v 5.9 post; P = .03).

CONCLUSION

Almost all patients in this large GI cancer cohort responded via the portal about their understanding of illness and treatment goals. Eliciting their personal values by portal was also feasible, accepted by patients, and improved patient ratings of clinicians' communication. Portals represent a promising tool for scaling assessment of essential patient-reported elements of person-centered communication.

Artificial intelligence in oncology: chances and pitfalls

Artificial intelligence (AI) has been available in rudimentary forms for many decades. Early AI programs were successful in niche areas such as chess or handwriting recognition. However, AI methods had little practical impact on the practice of medicine until recently. Beginning around 2012, AI has emerged as an increasingly important tool in healthcare, and AI-based devices are now approved for clinical use. These devices are capable of processing image data, making diagnoses, and predicting biomarkers for solid tumors, among other applications. Despite this progress, the development of AI in medicine is still in its early stages, and there have been exponential technical advancements since 2022, with some AI programs now demonstrating human-level understanding of image and text data. In the past, technical advances have led to new medical applications with a delay of a few years. Therefore, now we might be at the beginning of a new era in which AI will become even more important in clinical practice. It is essential that this transformation is humane and evidence based, and physicians must take a leading role in ensuring this, particularly in hematology and oncology.

Leveraging Goals of Care Interventions to Deliver Personalized Care Near the End of Life

This Viewpoint discusses barriers to and opportunities for incorporating goal of care communications into end-of-life care.

Microbiome Preterm Birth DREAM Challenge: Crowdsourcing Machine Learning Approaches to Advance Preterm Birth Research

Globally, every year about 11% of infants are born preterm, defined as a birth prior to 37 weeks of gestation, with significant and lingering health consequences. Multiple studies have related the vaginal microbiome to preterm birth. We present a crowdsourcing approach to predict: (a) preterm or (b) early preterm birth from 9 publicly available vaginal microbiome studies representing 3,578 samples from 1,268 pregnant individuals, aggregated from raw sequences via an open-source tool, MaLiAmPi. We validated the crowdsourced models on novel datasets representing 331 samples from 148 pregnant individuals. From 318 DREAM challenge participants we received 148 and 121 submissions for our two separate prediction sub-challenges with top-ranking submissions achieving bootstrapped AUROC scores of 0.69 and 0.87, respectively. Alpha diversity, VALENCIA community state types, and composition (via phylotype relative abundance) were important features in the top performing models, most of which were tree based methods. This work serves as the foundation for subsequent efforts to translate predictive tests into clinical practice, and to better understand and prevent preterm birth.

Advanced Care Planning for Hospitalized Patients Following Clinician Notification of Patient Mortality by a Machine Learning Algorithm

Importance

Goal-concordant care is an ongoing challenge in hospital settings. Identification of high mortality risk within 30 days may call attention to the need to have serious illness conversations, including the documentation of patient goals of care.

Objective

To examine goals of care discussions (GOCDs) in a community hospital setting with patients identified as having a high risk of mortality by a machine learning mortality prediction algorithm.

Design, Setting, and Participants

This cohort study took place at community hospitals within 1 health care system. Participants included adult patients with a high risk of 30-day mortality who were admitted to 1 of 4 hospitals between January 2 and July 15, 2021. Patient encounters of inpatients in the intervention hospital where physicians were notified of the computed high risk mortality score were compared with patient encounters of inpatients in 3 community hospitals without the intervention (ie, matched control).

Intervention

Physicians of patients with a high risk of mortality within 30 days received notification and were encouraged to arrange for GOCDs.

Main Outcomes and Measures

The primary outcome was the percentage change of documented GOCDs prior to discharge. Propensity-score matching was completed on a preintervention and postintervention period using age, sex, race, COVID-19 status, and machine learning-predicted mortality risk scores. A difference-in-difference analysis validated the results.

Results

Overall, 537 patients were included in this study with 201 in the preintervention period (94 in the intervention group; 104 in the control group) and 336 patients in the postintervention period. The intervention and control groups included 168 patients per group and were well-balanced in age (mean [SD], 79.3 [9.60] vs 79.6 [9.21] years; standardized mean difference [SMD], 0.03), sex (female, 85 [51%] vs 85 [51%]; SMD, 0), race (White patients, 145 [86%] vs 144 [86%]; SMD 0.006), and Charlson comorbidities (median [range], 8.00 [2.00-15.0] vs 9.00 [2.00 to 19.0]; SMD, 0.34). Patients in the intervention group from preintervention to postintervention period were associated with being 5 times more likely to have documented GOCDs (OR, 5.11 [95% CI, 1.93 to 13.42]; P = .001) by discharge compared with matched controls, and GOCD occurred significantly earlier in the hospitalization in the intervention patients as compared with matched controls (median, 4 [95% CI, 3 to 6] days vs 16 [95% CI, 15 to not applicable] days; P < .001). Similar findings were observed for Black patient and White patient subgroups.

Conclusions and Relevance

In this cohort study, patients whose physicians had knowledge of high-risk predictions from machine learning mortality algorithms were associated with being 5 times more likely to have documented GOCDs than matched controls. Additional external validation is needed to determine if similar interventions would be helpful at other institutions.

Computation of the distribution of model accuracy statistics in machine learning: Comparison between analytically derived distributions and simulation-based methods

Background and Aims

All fields have seen an increase in machine-learning techniques. To accurately evaluate the efficacy of novel modeling methods, it is necessary to conduct a critical evaluation of the utilized model metrics, such as sensitivity, specificity, and area under the receiver operator characteristic curve (AUROC). For commonly used model metrics, we proposed the use of analytically derived distributions (ADDs) and compared it with simulation-based approaches.

Methods

A retrospective cohort study was conducted using the England National Health Services Heart Disease Prediction Cohort. Four machine learning models (XGBoost, Random Forest, Artificial Neural Network, and Adaptive Boost) were used. The distribution of the model metrics and covariate gain statistics were empirically derived using boot-strap simulation (N = 10,000). The ADDs were created from analytic formulas from the covariates to describe the distribution of the model metrics and compared with those of bootstrap simulation.

Results

XGBoost had the most optimal model having the highest AUROC and the highest aggregate score considering six other model metrics. Based on the Anderson-Darling test, the distribution of the model metrics created from bootstrap did not significantly deviate from a normal distribution. The variance created from the ADD led to smaller SDs than those derived from bootstrap simulation, whereas the rest of the distribution remained not statistically significantly different.

Conclusions

ADD allows for cross study comparison of model metrics, which is usually done with bootstrapping that rely on simulations, which cannot be replicated by the reader.