Adaptation and early implementation of the PREdiction model for gene mutations (PREMM5™) for lynch syndrome risk assessment in a diverse population

Usability, Engagement, and Report Usefulness of Chatbot-Based Family Health History Data Collection: Mixed Methods Analysis

BACKGROUND

Family health history (FHx) is an important predictor of a person's genetic risk but is not collected by many adults in the United States.

OBJECTIVE

This study aims to test and compare the usability, engagement, and report usefulness of 2 web-based methods to collect FHx.

METHODS

This mixed methods study compared FHx data collection using a flow-based chatbot (KIT; the curious interactive test) and a form-based method. KIT's design was optimized to reduce user burden. We recruited and randomized individuals from 2 crowdsourced platforms to 1 of the 2 FHx methods. All participants were asked to complete a questionnaire to assess the method's usability, the usefulness of a report summarizing their experience, user-desired chatbot enhancements, and general user experience. Engagement was studied using log data collected by the methods. We used qualitative findings from analyzing free-text comments to supplement the primary quantitative results.

RESULTS

Participants randomized to KIT reported higher usability than those randomized to the form, with a mean System Usability Scale score of 80.2 versus 61.9 (P<.001), respectively. The engagement analysis reflected design differences in the onboarding process. KIT users spent less time entering FHx information and reported more conditions than form users (mean 5.90 vs 7.97 min; P=.04; and mean 7.8 vs 10.1 conditions; P=.04). Both KIT and form users somewhat agreed that the report was useful (Likert scale ratings of 4.08 and 4.29, respectively). Among desired enhancements, personalization was the highest-rated feature (188/205, 91.7% rated medium- to high-priority). Qualitative analyses revealed positive and negative characteristics of both KIT and the form-based method. Among respondents randomized to KIT, most indicated it was easy to use and navigate and that they could respond to and understand user prompts. Negative comments addressed KIT's personality, conversational pace, and ability to manage errors. For KIT and form respondents, qualitative results revealed common themes, including a desire for more information about conditions and a mutual appreciation for the multiple-choice button response format. Respondents also said they wanted to report health information beyond KIT's prompts (eg, personal health history) and for KIT to provide more personalized responses.

CONCLUSIONS

We showed that KIT provided a usable way to collect FHx. We also identified design considerations to improve chatbot-based FHx data collection: First, the final report summarizing the FHx collection experience should be enhanced to provide more value for patients. Second, the onboarding chatbot prompt may impact data quality and should be carefully considered. Finally, we highlighted several areas that could be improved by moving from a flow-based chatbot to a large language model implementation strategy.

Leveraging Electronic Health Record Data to Understand Gaps Underlying the Underdiagnosis of Lynch Syndrome

Using the electronic health record to address the underdiagnosis of Lynch syndrome.

Risk management actions following genetic testing in the Cancer Health Assessments Reaching Many (CHARM) Study: A prospective cohort study

BACKGROUND

Genetic testing can identify cancer risk early, enabling prevention and early detection. We describe use of risk management interventions following genetic testing in the Cancer Health Assessment Reaching Many (CHARM) study. CHARM assessed risk and provided genetic testing to low income, low literacy, and other underserved populations that historically face barriers to accessing cancer genetic services.

METHODS

CHARM was implemented in Kaiser Permanente Northwest (KPNW) and Denver Health (DH) between 2018 and 2020. We identified post-testing screening (mammography, breast MRI, colonoscopy) and surgical (mastectomy, oophorectomy) procedures using electronic health records. We examined utilization in participants who did and did not receive actionable risk management recommendations from study genetic counselors following national guidelines.

RESULTS

CHARM participants were followed for an average of 15.4 months (range: 0.4-27.8 months) after results disclosure. Less than 2% (11/680) received actionable risk management recommendations (i.e., could be completed in the initial years following testing) based on their test result. Among those who received actionable recommendations, risk management utilization was moderate (54.5%, 6/11 completed any procedure) and varied by procedure (mammogram: 0/3; MRI: 2/4; colonoscopy: 4/5; mastectomy: 1/5; oophorectomy: 0/3). Cancer screening and surgery procedures were rare in participants without actionable recommendations.

CONCLUSION

Though the number of participants who received actionable risk management recommendations was small, our results suggest that implementing CHARM's risk assessment and testing model increased access to evidence-based genetic services and provided opportunities for patients to engage in recommended preventive care, without encouraging risk management overuse.

Evaluating cancer genetic services in a safety net system: overcoming barriers for a lasting impact beyond the CHARM research project

Underserved patients face substantial barriers to receiving cancer genetic services. The Cancer Health Assessments Reaching Many (CHARM) study evaluated ways to increase access to genetic testing for individuals in underserved populations at risk for hereditary cancer syndromes (HCS). Here, we report the successful implementation of CHARM in a low-resource environment and the development of sustainable processes to continue genetic risk assessment in this setting. The research team involved key clinical personnel and patient advisors at Denver Health to provide input on study methods and materials. Through iterative and collaborative stakeholder engagement, the team identified barriers and developed solutions that would both facilitate participation in CHARM and be feasible to implement and sustain long term in clinical care. With a focus on infrastructure building, educational modules were developed to increase awareness among referring providers, and standard methods of identifying and managing HCS patients were implemented in the electronic medical record. Three hundred sixty-four DH patients successfully completed the risk assessment tool within the study, and we observed a sustained increase in referrals to genetics for HCS (from 179 in 2017 to 427 in 2021 post-intervention). Implementation of the CHARM study at a low-resourced safety net health system resulted in sustainable improvements in access to cancer genetic risk assessment and services that continue even after the study ended.Trial registration NCT03426878.

Development and Validation of the PREMMplus Model for Multigene Hereditary Cancer Risk Assessment

PURPOSE

With the availability of multigene panel testing (MGPT) for hereditary cancer risk assessment, clinicians need to assess the likelihood of pathogenic germline variants (PGVs) across numerous genes in parallel. This study's aim was to develop and validate a clinical prediction model (PREMMplus) for MGPT risk assessment.

MATERIALS AND METHODS

PREMMplus was developed in a single-institution cohort of 7,280 individuals who had undergone MGPT. Logistic regression models with Least Absolute Shrinkage and Selection Operator regularization were used to examine candidate predictors (age, sex, ethnicity, and personal/family history of 18 cancers/neoplasms) to estimate one's likelihood of carrying PGVs in 19 genes (broadly categorized by phenotypic overlap and/or relative penetrance: 11 category A [APC, BRCA1/2, CDH1, EPCAM, MLH1, MSH2, MSH6, biallelic MUTYH, PMS2, and TP53] and eight category B genes [ATM, BRIP1, CDKN2A, CHEK2, PALB2, PTEN, RAD51C, and RAD51D]). Model performance was validated in nonoverlapping data sets of 8,691 and 14,849 individuals with prior MGPT ascertained from clinic- and laboratory-based settings, respectively.

RESULTS

PREMMplus (score ≥ 2.5%) had 93.9%, 91.7%, and 89.3% sensitivity and 98.3%, 97.5%, and 97.8% negative-predictive value (NPV) for identifying category A gene PGV carriers in the development and validation cohorts, respectively. PREMMplus assessment (score ≥ 2.5%) had 89.9%, 85.6%, and 84.2% sensitivity and 95.0%, 93.5%, and 93.5% NPV, respectively, for identifying category A/B gene PGV carriers. Decision curve analyses support MGPT for individuals predicted to have ≥ 2.5% probability of a PGV.

CONCLUSION

PREMMplus accurately identifies individuals with PGVs in a diverse spectrum of cancer susceptibility genes with high sensitivity/NPV. Individuals with PREMMplus scores ≥ 2.5% should be considered for MGPT.

ORCA, a values-based decision aid for selecting additional findings from genomic sequencing in adults: Efficacy results from a randomized trial

PURPOSE

Individuals having genomic sequencing can choose to be notified about pathogenic variants in genes unrelated to the testing indication. A decision aid can facilitate weighing one's values before making a choice about these additional results.

METHODS

We conducted a randomized trial (N = 231) comparing informed values-choice congruence among adults at risk for a hereditary cancer syndrome who viewed either the Optional Results Choice Aid (ORCA) or web-based additional findings information alone. ORCA is values-focused with a low-literacy design.

RESULTS

Individuals in both arms had informed values-choice congruence (75% and 73% in the decision aid and web-based groups, respectively; odds ratio [OR] = 1.10, 95% CI = 0.58-2.08). Most participants had adequate knowledge (79% and 76% in the decision aid and web-based groups, respectively; OR = 1.20, 95% CI = 0.61-2.34), with no significant difference between groups. Most had information-seeking values (97% and 98% in the decision aid and web-based groups, respectively; OR = 0.59, 95% CI = 0.10-3.61) and chose to receive additional findings.

CONCLUSION

The ORCA decision aid did not significantly improve informed values-choice congruence over web-based information in this cohort of adults deciding about genomic results. Both web-based approaches may be effective for adults to decide about receiving medically actionable additional results.

Literacy-adapted, electronic family history assessment for genetics referral in primary care: patient user insights from qualitative interviews

BACKGROUND

Risk assessment for hereditary cancer syndromes is recommended in primary care, but family history is rarely collected in enough detail to facilitate risk assessment and referral - a roadblock that disproportionately impacts individuals with healthcare access barriers. We sought to qualitatively assess a literacy-adapted, electronic patient-facing family history tool developed for use in diverse, underserved patient populations recruited in the Cancer Health Assessments Reaching Many (CHARM) Study.

METHODS

Interview participants were recruited from a subpopulation of CHARM participants who experienced barriers to tool use in terms of spending a longer time to complete the tool, having incomplete attempts, and/or providing inaccurate family history in comparison to a genetic counselor-collected standard. We conducted semi-structured interviews with participants about barriers and facilitators to tool use and overall tool acceptability; interviews were recorded and professionally transcribed. Transcripts were coded based on a codebook developed using inductive techniques, and coded excerpts were reviewed to identify overarching themes related to barriers and facilitators to family history self-assessment and acceptability of the study tool.

RESULTS

Interviewees endorsed the tool as easy to navigate and understand. However, they described barriers related to family history information, literacy and language, and certain tool functions. Participants offered concrete, easy-to-implement solutions to each barrier. Despite experience barriers to use of the tool, most participants indicated that electronic family history self-assessment was acceptable or preferable in comparison to clinician-collected family history.

CONCLUSIONS

Even for participants who experienced barriers to tool use, family history self-assessment was considered an acceptable alternative to clinician-collected family history. Barriers experienced could be overcome with minor adaptations to the current family history tool.

TRIAL REGISTRATION

This study is a sub-study of the Cancer Health Assessments Reaching Many (CHARM) trial, ClinicalTrials.gov, NCT03426878. Registered 8 February 2018.

Point/Counterpoint: Is It Time for Universal Germline Genetic Testing for all GI Cancers?

Use of germline genetic testing among patients with cancer is increasing because of (1) the availability of multigene panel tests that include multiple cancer susceptibility genes in a single test, (2) decreased costs of these tests and improvements in insurance coverage, and (3) US Food and Drug Administration-approval of genotype-directed therapies such as poly(ADP-ribose) polymerase inhibitors for individuals with certain cancers and pathogenic germline variants in BRCA1 and BRCA2 (with possible benefits with other genes in the homologous repair deficiency pathway). In addition, National Comprehensive Cancer Network guidelines have already endorsed germline genetic testing for all patients with certain cancer types (epithelial ovarian cancer, exocrine pancreatic cancer, and high-grade/metastatic prostate cancer), regardless of age or personal/family history of cancer. Herein, we debate the pros and cons of offering germline multigene panel testing to all patients diagnosed with any GI cancer. The authors agree that it may just be a matter of time before germline multigene panel testing is offered to all patients with cancer; however, this article will highlight some of the benefits, risks, and limitations of this approach so that research can help fill some of the gaps to ensure that genetic medicine continues to be implemented in ways that improve real-world patient care and outcomes.

Laboratory-related outcomes from integrating an accessible delivery model for hereditary cancer risk assessment and genetic testing in populations with barriers to access

PURPOSE

This study aimed to evaluate the laboratory-related outcomes of participants who were offered genomic testing based on cancer family history risk assessment tools.

METHODS

Patients from clinics that serve populations with access barriers, who are screened at risk for a hereditary cancer syndrome based on adapted family history collection tools (the Breast Cancer Genetics Referral Screening Tool and PREMM₅), were offered exome-based panel testing for cancer risk and medically actionable secondary findings. We used descriptive statistics, electronic health record review, and inferential statistics to explore participant characteristics and results, consultations and actions related to pathogenic/likely pathogenic variants identified, and variables predicting category of findings, respectively.

RESULTS

Of all the participants, 87% successfully returned a saliva kit. Overall, 5% had a pathogenic/likely pathogenic cancer risk variant and 1% had a secondary finding. Almost all (14/15, 93%) participants completed recommended consultations with nongenetics providers after an average of 17 months. The recommended actions (eg, breast magnetic resonance imaging) were completed by 17 of 25 participants. Participant personal history of cancer and PREMM₅ score were each associated with the category of findings (history and colon cancer finding, Fisher's exact P = .02; history and breast cancer finding, Fisher's exact P = .01; PREMM₅^TM score; and colon cancer finding, Fisher's exact P < .001).

CONCLUSION

This accessible model of hereditary cancer risk assessment and genetic testing yielded results that were often acted upon by patients and physicians.