1
|
Soni H, Ivanova J, Wilczewski H, Ong T, Ross JN, Bailey A, Cummins M, Barrera J, Bunnell B, Welch B. User Preferences and Needs for Health Data Collection Using Research Electronic Data Capture: Survey Study. JMIR Med Inform 2024; 12:e49785. [PMID: 38917448 PMCID: PMC11234068 DOI: 10.2196/49785] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Revised: 04/10/2024] [Accepted: 05/04/2024] [Indexed: 06/27/2024] Open
Abstract
BACKGROUND Self-administered web-based questionnaires are widely used to collect health data from patients and clinical research participants. REDCap (Research Electronic Data Capture; Vanderbilt University) is a global, secure web application for building and managing electronic data capture. Unfortunately, stakeholder needs and preferences of electronic data collection via REDCap have rarely been studied. OBJECTIVE This study aims to survey REDCap researchers and administrators to assess their experience with REDCap, especially their perspectives on the advantages, challenges, and suggestions for the enhancement of REDCap as a data collection tool. METHODS We conducted a web-based survey with representatives of REDCap member organizations in the United States. The survey captured information on respondent demographics, quality of patient-reported data collected via REDCap, patient experience of data collection with REDCap, and open-ended questions focusing on the advantages, challenges, and suggestions to enhance REDCap's data collection experience. Descriptive and inferential analysis measures were used to analyze quantitative data. Thematic analysis was used to analyze open-ended responses focusing on the advantages, disadvantages, and enhancements in data collection experience. RESULTS A total of 207 respondents completed the survey. Respondents strongly agreed or agreed that the data collected via REDCap are accurate (188/207, 90.8%), reliable (182/207, 87.9%), and complete (166/207, 80.2%). More than half of respondents strongly agreed or agreed that patients find REDCap easy to use (165/207, 79.7%), could successfully complete tasks without help (151/207, 72.9%), and could do so in a timely manner (163/207, 78.7%). Thematic analysis of open-ended responses yielded 8 major themes: survey development, user experience, survey distribution, survey results, training and support, technology, security, and platform features. The user experience category included more than half of the advantage codes (307/594, 51.7% of codes); meanwhile, respondents reported higher challenges in survey development (169/516, 32.8% of codes), also suggesting the highest enhancement suggestions for the category (162/439, 36.9% of codes). CONCLUSIONS Respondents indicated that REDCap is a valued, low-cost, secure resource for clinical research data collection. REDCap's data collection experience was generally positive among clinical research and care staff members and patients. However, with the advancements in data collection technologies and the availability of modern, intuitive, and mobile-friendly data collection interfaces, there is a critical opportunity to enhance the REDCap experience to meet the needs of researchers and patients.
Collapse
Affiliation(s)
- Hiral Soni
- Doxy.me Research, Doxy.me Inc, Charleston, SC, United States
| | - Julia Ivanova
- Doxy.me Research, Doxy.me Inc, Charleston, SC, United States
| | | | - Triton Ong
- Doxy.me Research, Doxy.me Inc, Charleston, SC, United States
| | - J Nalubega Ross
- Doxy.me Research, Doxy.me Inc, Charleston, SC, United States
| | | | - Mollie Cummins
- Doxy.me Research, Doxy.me Inc, Charleston, SC, United States
- College of Nursing, University of Utah, Salt Lake City, UT, United States
| | - Janelle Barrera
- Doxy.me Research, Doxy.me Inc, Charleston, SC, United States
- Department of Psychiatry and Behavioral Neurosciences, University of South Florida, Tampa, FL, United States
| | - Brian Bunnell
- Doxy.me Research, Doxy.me Inc, Charleston, SC, United States
- Department of Psychiatry and Behavioral Neurosciences, University of South Florida, Tampa, FL, United States
| | - Brandon Welch
- Doxy.me Research, Doxy.me Inc, Charleston, SC, United States
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| |
Collapse
|
2
|
Gordon EJ, Gacki-Smith J, Gooden MJ, Waite P, Yacat R, Abubakari ZR, Duquette D, Agrawal A, Friedewald J, Savage SK, Cooper M, Gilbert A, Muhammad LN, Wicklund C. Development of a culturally targeted chatbot to inform living kidney donor candidates of African ancestry about APOL1 genetic testing: a mixed methods study. J Community Genet 2024; 15:205-216. [PMID: 38349598 PMCID: PMC11031529 DOI: 10.1007/s12687-024-00698-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2023] [Accepted: 01/22/2024] [Indexed: 02/25/2024] Open
Abstract
Clinical chatbots are increasingly used to help integrate genetic testing into clinical contexts, but no chatbot exists for Apolipoprotein L1 (APOL1) genetic testing of living kidney donor (LKD) candidates of African ancestry. Our study aimed to culturally adapt and assess perceptions of the Gia® chatbot to help integrate APOL1 testing into LKD evaluation. Ten focus groups and post-focus group surveys were conducted with 54 LKDs, community members, and kidney transplant recipients of African ancestry. Data were analyzed through thematic analysis and descriptive statistics. Key themes about making Gia culturally targeted included ensuring: (1) transparency by providing Black LKDs' testimonials, explaining patient privacy and confidentiality protections, and explaining how genetic testing can help LKD evaluation; (2) content is informative by educating Black LKDs about APOL1 testing instead of aiming to convince them to undergo testing, presenting statistics, and describing how genetic discrimination is legally prevented; and (3) content avoids stigma about living donation in the Black community. Most agreed Gia was neutral and unbiased (82%), trustworthy (82%), and words, phrases, and expressions were familiar to the intended audience (85%). Our culturally adapted APOL1 Gia chatbot was well regarded. Future research should assess how this chatbot could supplement provider discussion prior to genetic testing to scale APOL1 counseling and testing for LKD candidate clinical evaluation.
Collapse
Affiliation(s)
- Elisa J Gordon
- Department of Surgery, Center for Biomedical Ethics and Society, Vanderbilt University Medical Center, 1161 21St Avenue South, D-4314 Medical Center North Nashville, Nashville, TN, 37232-2730, USA.
| | - Jessica Gacki-Smith
- Center for Health Services and Outcomes Research, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Matthew J Gooden
- Center for Health Services and Outcomes Research, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Preeya Waite
- Center for Health Services and Outcomes Research, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Rochell Yacat
- Medstar Georgetown Transplant Institute, Georgetown University Hospital, Washington, DC, USA
| | - Zenab R Abubakari
- Medstar Georgetown Transplant Institute, Georgetown University Hospital, Washington, DC, USA
| | - Debra Duquette
- Medicine, Cardiology Division, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Akansha Agrawal
- Division of Nephrology and Hypertension, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - John Friedewald
- Division of Nephrology and Hypertension, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | | | - Matthew Cooper
- Froedtert Hospital Center for Advanced Care, Froedtert Memorial Lutheran Hospital Children's Hospital of Wisconsin Medical College of Wisconsin, Milwaukee, WI, USA
- Children's Hospital of Wisconsin, Milwaukee, WI, USA
- Medical College of Wisconsin, Milwaukee, WI, USA
- Froedtert Hospital Center for Advanced Care, Milwaukee, WI, USA
| | - Alexander Gilbert
- Medstar Georgetown Transplant Institute, Georgetown University Hospital, Washington, DC, USA
| | - Lutfiyya N Muhammad
- Department of Preventive Medicine, Division of Biostatistics, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| | - Catherine Wicklund
- Department of Obstetrics and Gynecology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA
| |
Collapse
|
3
|
Webster EM, Perez L, Ahsan MD, Levi S, Chandler I, Thomas C, Babagbemi K, Sharaf RN, Frey MK. Integration and usability of a digital cancer risk stratification tool to optimize identification of patients at risk for hereditary cancers: A pilot study. Gynecol Oncol 2024; 183:1-6. [PMID: 38460222 DOI: 10.1016/j.ygyno.2024.02.028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Revised: 02/19/2024] [Accepted: 02/22/2024] [Indexed: 03/11/2024]
Abstract
BACKGROUND Patients with a personal or family history of cancer may have elevated risk of developing future cancers, which often remains unrecognized due to lapses in screening. This pilot study assessed the usability and clinical outcomes of a cancer risk stratification tool in a gynecologic oncology clinic. METHODS New gynecologic oncology patients were prompted to complete a commercially developed personal and family history-based risk stratification tool to assess eligibility for genetic testing using National Comprehensive Cancer Network criteria and estimated lifetime breast cancer risk using the Tyrer-Cuzick model. After use of the risk stratification tool, usability was assessed via completion rate and the System Usability Scale, and health literacy was assessed using the BRIEF Health Literacy Screening Tool. RESULTS 130 patients were prompted to complete the risk stratification tool; 93 (72%) completed the tool. Race and ethnicity and insurance type were not associated with tool completion. The median System Usability Scale score was 83 out of 100 (interquartile range, 60-95). Health literacy positively correlated with perceived usability. Public insurance and race or ethnicity other than non-Hispanic White was associated with lower perceived usability. Sixty (65%) patients met eligibility criteria for genetic testing, and 21 (38% of 56 eligible patients) were candidates for enhanced breast cancer screening based on an estimated lifetime breast cancer risk of ≥20%. CONCLUSIONS A majority of patients completed the digital cancer risk stratification tool. Older age, lower health literacy, public insurance, and race or ethnicity other than non-Hispanic White were associated with lower perceived tool usability.
Collapse
Affiliation(s)
- Emily M Webster
- Weill Cornell Medicine, New York, NY, United States of America.
| | - Luiza Perez
- Weill Cornell Medicine, New York, NY, United States of America
| | | | - Sarah Levi
- Weill Cornell Medicine, New York, NY, United States of America
| | | | - Charlene Thomas
- Weill Cornell Medicine, New York, NY, United States of America
| | - Kemi Babagbemi
- Weill Cornell Medicine, New York, NY, United States of America
| | - Ravi N Sharaf
- Weill Cornell Medicine, New York, NY, United States of America
| | - Melissa K Frey
- Weill Cornell Medicine, New York, NY, United States of America
| |
Collapse
|
4
|
Soni H, Morrison H, Vasilev D, Ong T, Wilczewski H, Allen C, Hughes-Halbert C, Ritchie JB, Narma A, Schiffman JD, Ivanova J, Bunnell BE, Welch BM. User experience of a family health history chatbot: A quantitative analysis. Health Informatics J 2024; 30:14604582241262251. [PMID: 38865081 DOI: 10.1177/14604582241262251] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/13/2024]
Abstract
OBJECTIVE Family health history (FHx) is an important tool in assessing one's risk towards specific health conditions. However, user experience of FHx collection tools is rarely studied. ItRunsInMyFamily.com (ItRuns) was developed to assess FHx and hereditary cancer risk. This study reports a quantitative user experience analysis of ItRuns. METHODS We conducted a public health campaign in November 2019 to promote FHx collection using ItRuns. We used software telemetry to quantify abandonment and time spent on ItRuns to identify user behaviors and potential areas of improvement. RESULTS Of 11,065 users who started the ItRuns assessment, 4305 (38.91%) reached the final step to receive recommendations about hereditary cancer risk. Highest abandonment rates were during Introduction (32.82%), Invite Friends (29.03%), and Family Cancer History (12.03%) subflows. Median time to complete the assessment was 636 s. Users spent the highest median time on Proband Cancer History (124.00 s) and Family Cancer History (119.00 s) subflows. Search list questions took the longest to complete (median 19.50 s), followed by free text email input (15.00 s). CONCLUSION Knowledge of objective user behaviors at a large scale and factors impacting optimal user experience will help enhance the ItRuns workflow and improve future FHx collection.
Collapse
Affiliation(s)
- Hiral Soni
- Doxy.me Research, Doxy.me Inc., Rochester, NY, USA
| | | | | | - Triton Ong
- Doxy.me Research, Doxy.me Inc., Rochester, NY, USA
| | | | - Caitlin Allen
- Biomedical Informatics Center, Public Health and Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Chanita Hughes-Halbert
- Department of Psychiatry and Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Jordon B Ritchie
- Biomedical Informatics Center, Public Health and Sciences, Medical University of South Carolina, Charleston, SC, USA
| | - Alexa Narma
- Doxy.me Research, Doxy.me Inc., Rochester, NY, USA
| | | | | | - Brian E Bunnell
- Doxy.me Research, Doxy.me Inc., Rochester, NY, USA
- Innovation in Mental Health Lab., Department of Psychiatry and Behavioral Neurosciences, University of South Florida, Tampa, FL, USA
| | - Brandon M Welch
- Doxy.me Research, Doxy.me Inc., Rochester, NY, USA
- Biomedical Informatics Center, Public Health and Sciences, Medical University of South Carolina, Charleston, SC, USA
| |
Collapse
|
5
|
Allen CG, Neil G, Halbert CH, Sterba KR, Nietert PJ, Welch B, Lenert L. Barriers and facilitators to the implementation of family cancer history collection tools in oncology clinical practices. J Am Med Inform Assoc 2024; 31:631-639. [PMID: 38164994 PMCID: PMC10873828 DOI: 10.1093/jamia/ocad243] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 10/30/2023] [Accepted: 12/19/2023] [Indexed: 01/03/2024] Open
Abstract
INTRODUCTION This study aimed to identify barriers and facilitators to the implementation of family cancer history (FCH) collection tools in clinical practices and community settings by assessing clinicians' perceptions of implementing a chatbot interface to collect FCH information and provide personalized results to patients and providers. OBJECTIVES By identifying design and implementation features that facilitate tool adoption and integration into clinical workflows, this study can inform future FCH tool development and adoption in healthcare settings. MATERIALS AND METHODS Quantitative data were collected using survey to evaluate the implementation outcomes of acceptability, adoption, appropriateness, feasibility, and sustainability of the chatbot tool for collecting FCH. Semistructured interviews were conducted to gather qualitative data on respondents' experiences using the tool and recommendations for enhancements. RESULTS We completed data collection with 19 providers (n = 9, 47%), clinical staff (n = 5, 26%), administrators (n = 4, 21%), and other staff (n = 1, 5%) affiliated with the NCI Community Oncology Research Program. FCH was systematically collected using a wide range of tools at sites, with information being inserted into the patient's medical record. Participants found the chatbot tool to be highly acceptable, with the tool aligning with existing workflows, and were open to adopting the tool into their practice. DISCUSSION AND CONCLUSIONS We further the evidence base about the appropriateness of scripted chatbots to support FCH collection. Although the tool had strong support, the varying clinical workflows across clinic sites necessitate that future FCH tool development accommodates customizable implementation strategies. Implementation support is necessary to overcome technical and logistical barriers to enhance the uptake of FCH tools in clinical practices and community settings.
Collapse
Affiliation(s)
- Caitlin G Allen
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| | - Grace Neil
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| | - Chanita Hughes Halbert
- Department of Population and Public Health Sciences, University of Southern California, Los Angeles, CA, United States
| | - Katherine R Sterba
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| | - Paul J Nietert
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| | - Brandon Welch
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| | - Leslie Lenert
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| |
Collapse
|
6
|
Suckiel SA, Kelly NR, Odgis JA, Gallagher KM, Sebastin M, Bonini KE, Marathe PN, Brown K, Di Biase M, Ramos MA, Rodriguez JE, Scarimbolo L, Insel BJ, Ferar KDM, Zinberg RE, Diaz GA, Greally JM, Abul-Husn NS, Bauman LJ, Gelb BD, Horowitz CR, Wasserstein MP, Kenny EE. The NYCKidSeq randomized controlled trial: Impact of GUÍA digitally enhanced genetic results disclosure in diverse families. Am J Hum Genet 2023; 110:2029-2041. [PMID: 38006881 PMCID: PMC10716481 DOI: 10.1016/j.ajhg.2023.10.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/09/2023] [Revised: 10/26/2023] [Accepted: 10/27/2023] [Indexed: 11/27/2023] Open
Abstract
Digital solutions are needed to support rapid increases in the application of genetic/genomic tests (GTs) in diverse clinical settings and patient populations. We developed GUÍA, a bilingual digital application that facilitates disclosure of GT results. The NYCKidSeq randomized controlled trial enrolled diverse children with neurologic, cardiac, and immunologic conditions who underwent GTs. The trial evaluated GUÍA's impact on understanding the GT results by randomizing families to results disclosure genetic counseling with GUÍA (intervention) or standard of care (SOC). Parents/legal guardians (participants) completed surveys at baseline, post-results disclosure, and 6 months later. Survey measures assessed the primary study outcomes of participants' perceived understanding of and confidence in explaining their child's GT results and the secondary outcome of objective understanding. The analysis included 551 diverse participants, 270 in the GUÍA arm and 281 in SOC. Participants in the GUÍA arm had significantly higher perceived understanding post-results (OR = 2.8, CI[1.004, 7.617], p = 0.049) and maintained higher objective understanding over time (OR = 1.1, CI[1.004, 1.127], p = 0.038) compared to SOC. There was no impact on perceived confidence. Hispanic/Latino(a) individuals in the GUÍA arm maintained higher perceived understanding (OR = 3.9, CI[1.603, 9.254], p = 0.003), confidence (OR = 2.7, CI[1.021, 7.277], p = 0.046), and objective understanding (OR = 1.1, CI[1.009, 1.212], p = 0.032) compared to SOC. This trial demonstrates that GUÍA positively impacts understanding of GT results in diverse parents of children with suspected genetic conditions and builds a case for utilizing GUÍA to deliver complex results. Continued development and evaluation of digital applications in diverse populations are critical for equitably scaling GT offerings in specialty clinics.
Collapse
Affiliation(s)
- Sabrina A Suckiel
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA.
| | - Nicole R Kelly
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children's Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Jacqueline A Odgis
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Katie M Gallagher
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children's Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Monisha Sebastin
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children's Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Katherine E Bonini
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Priya N Marathe
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Kaitlyn Brown
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children's Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Miranda Di Biase
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children's Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Michelle A Ramos
- Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Institute for Health Equity Research, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Jessica E Rodriguez
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Laura Scarimbolo
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Beverly J Insel
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Kathleen D M Ferar
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Randi E Zinberg
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Obstetrics, Gynecology and Reproductive Science, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - George A Diaz
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Pediatrics, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - John M Greally
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children's Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Noura S Abul-Husn
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Laurie J Bauman
- Department of Psychiatry and Behavioral Sciences, Albert Einstein College of Medicine, Bronx, NY 10467, USA; Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Bruce D Gelb
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Pediatrics, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Mindich Child Health and Development Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Carol R Horowitz
- Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Institute for Health Equity Research, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA
| | - Melissa P Wasserstein
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children's Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY 10467, USA
| | - Eimear E Kenny
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA.
| |
Collapse
|
7
|
Wutz M, Hermes M, Winter V, Köberlein-Neu J. Factors Influencing the Acceptability, Acceptance, and Adoption of Conversational Agents in Health Care: Integrative Review. J Med Internet Res 2023; 25:e46548. [PMID: 37751279 PMCID: PMC10565637 DOI: 10.2196/46548] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 05/10/2023] [Accepted: 07/10/2023] [Indexed: 09/27/2023] Open
Abstract
BACKGROUND Conversational agents (CAs), also known as chatbots, are digital dialog systems that enable people to have a text-based, speech-based, or nonverbal conversation with a computer or another machine based on natural language via an interface. The use of CAs offers new opportunities and various benefits for health care. However, they are not yet ubiquitous in daily practice. Nevertheless, research regarding the implementation of CAs in health care has grown tremendously in recent years. OBJECTIVE This review aims to present a synthesis of the factors that facilitate or hinder the implementation of CAs from the perspectives of patients and health care professionals. Specifically, it focuses on the early implementation outcomes of acceptability, acceptance, and adoption as cornerstones of later implementation success. METHODS We performed an integrative review. To identify relevant literature, a broad literature search was conducted in June 2021 with no date limits and using all fields in PubMed, Cochrane Library, Web of Science, LIVIVO, and PsycINFO. To keep the review current, another search was conducted in March 2022. To identify as many eligible primary sources as possible, we used a snowballing approach by searching reference lists and conducted a hand search. Factors influencing the acceptability, acceptance, and adoption of CAs in health care were coded through parallel deductive and inductive approaches, which were informed by current technology acceptance and adoption models. Finally, the factors were synthesized in a thematic map. RESULTS Overall, 76 studies were included in this review. We identified influencing factors related to 4 core Unified Theory of Acceptance and Use of Technology (UTAUT) and Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) factors (performance expectancy, effort expectancy, facilitating conditions, and hedonic motivation), with most studies underlining the relevance of performance and effort expectancy. To meet the particularities of the health care context, we redefined the UTAUT2 factors social influence, habit, and price value. We identified 6 other influencing factors: perceived risk, trust, anthropomorphism, health issue, working alliance, and user characteristics. Overall, we identified 10 factors influencing acceptability, acceptance, and adoption among health care professionals (performance expectancy, effort expectancy, facilitating conditions, social influence, price value, perceived risk, trust, anthropomorphism, working alliance, and user characteristics) and 13 factors influencing acceptability, acceptance, and adoption among patients (additionally hedonic motivation, habit, and health issue). CONCLUSIONS This review shows manifold factors influencing the acceptability, acceptance, and adoption of CAs in health care. Knowledge of these factors is fundamental for implementation planning. Therefore, the findings of this review can serve as a basis for future studies to develop appropriate implementation strategies. Furthermore, this review provides an empirical test of current technology acceptance and adoption models and identifies areas where additional research is necessary. TRIAL REGISTRATION PROSPERO CRD42022343690; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=343690.
Collapse
Affiliation(s)
- Maximilian Wutz
- Center for Health Economics and Health Services Research, Schumpeter School of Business and Economics, University of Wuppertal, Wuppertal, Germany
| | - Marius Hermes
- Center for Health Economics and Health Services Research, Schumpeter School of Business and Economics, University of Wuppertal, Wuppertal, Germany
| | - Vera Winter
- Center for Health Economics and Health Services Research, Schumpeter School of Business and Economics, University of Wuppertal, Wuppertal, Germany
| | - Juliane Köberlein-Neu
- Center for Health Economics and Health Services Research, Schumpeter School of Business and Economics, University of Wuppertal, Wuppertal, Germany
| |
Collapse
|
8
|
Siglen E, Vetti HH, Augestad M, Steen VM, Lunde Å, Bjorvatn C. Evaluation of the Rosa Chatbot Providing Genetic Information to Patients at Risk of Hereditary Breast and Ovarian Cancer: Qualitative Interview Study. J Med Internet Res 2023; 25:e46571. [PMID: 37656502 PMCID: PMC10504626 DOI: 10.2196/46571] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/17/2023] [Revised: 06/27/2023] [Accepted: 07/20/2023] [Indexed: 09/02/2023] Open
Abstract
BACKGROUND Genetic testing has become an integrated part of health care for patients with breast or ovarian cancer, and the increasing demand for genetic testing is accompanied by an increasing need for easy access to reliable genetic information for patients. Therefore, we developed a chatbot app (Rosa) that is able to perform humanlike digital conversations about genetic BRCA testing. OBJECTIVE Before implementing this new information service in daily clinical practice, we wanted to explore 2 aspects of chatbot use: the perceived utility and trust in chatbot technology among healthy patients at risk of hereditary cancer and how interaction with a chatbot regarding sensitive information about hereditary cancer influences patients. METHODS Overall, 175 healthy individuals at risk of hereditary breast and ovarian cancer were invited to test the chatbot, Rosa, before and after genetic counseling. To secure a varied sample, participants were recruited from all cancer genetic clinics in Norway, and the selection was based on age, gender, and risk of having a BRCA pathogenic variant. Among the 34.9% (61/175) of participants who consented for individual interview, a selected subgroup (16/61, 26%) shared their experience through in-depth interviews via video. The semistructured interviews covered the following topics: usability, perceived usefulness, trust in the information received via the chatbot, how Rosa influenced the user, and thoughts about future use of digital tools in health care. The transcripts were analyzed using the stepwise-deductive inductive approach. RESULTS The overall finding was that the chatbot was very welcomed by the participants. They appreciated the 24/7 availability wherever they were and the possibility to use it to prepare for genetic counseling and to repeat and ask questions about what had been said afterward. As Rosa was created by health care professionals, they also valued the information they received as being medically correct. Rosa was referred to as being better than Google because it provided specific and reliable answers to their questions. The findings were summed up in 3 concepts: "Anytime, anywhere"; "In addition, not instead"; and "Trustworthy and true." All participants (16/16) denied increased worry after reading about genetic testing and hereditary breast and ovarian cancer in Rosa. CONCLUSIONS Our results indicate that a genetic information chatbot has the potential to contribute to easy access to uniform information for patients at risk of hereditary breast and ovarian cancer, regardless of geographical location. The 24/7 availability of quality-assured information, tailored to the specific situation, had a reassuring effect on our participants. It was consistent across concepts that Rosa was a tool for preparation and repetition; however, none of the participants (0/16) supported that Rosa could replace genetic counseling if hereditary cancer was confirmed. This indicates that a chatbot can be a well-suited digital companion to genetic counseling.
Collapse
Affiliation(s)
- Elen Siglen
- Western Norway Familial Cancer Center, Department of Medical Genetics, Haukeland University Hospital, Bergen, Norway
- Faculty of Health Studies, VID Specialized University, Bergen, Norway
| | - Hildegunn Høberg Vetti
- Western Norway Familial Cancer Center, Department of Medical Genetics, Haukeland University Hospital, Bergen, Norway
- Faculty of Health Studies, VID Specialized University, Bergen, Norway
| | - Mirjam Augestad
- Faculty of Health Studies, VID Specialized University, Bergen, Norway
| | - Vidar M Steen
- Western Norway Familial Cancer Center, Department of Medical Genetics, Haukeland University Hospital, Bergen, Norway
- Department of Clinical Science, University of Bergen, Bergen, Norway
| | - Åshild Lunde
- Department of Global Public Health and Primary Care, University of Bergen, Bergen, Norway
| | - Cathrine Bjorvatn
- Western Norway Familial Cancer Center, Department of Medical Genetics, Haukeland University Hospital, Bergen, Norway
- Faculty of Health Studies, VID Specialized University, Bergen, Norway
| |
Collapse
|
9
|
Webster EM, Ahsan MD, Perez L, Levi SR, Thomas C, Christos P, Hickner A, Hamilton JG, Babagbemi K, Cantillo E, Holcomb K, Chapman-Davis E, Sharaf RN, Frey MK. Chatbot Artificial Intelligence for Genetic Cancer Risk Assessment and Counseling: A Systematic Review and Meta-Analysis. JCO Clin Cancer Inform 2023; 7:e2300123. [PMID: 37934933 PMCID: PMC10730073 DOI: 10.1200/cci.23.00123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2023] [Revised: 08/25/2023] [Accepted: 09/11/2023] [Indexed: 11/09/2023] Open
Abstract
PURPOSE Most individuals with a hereditary cancer syndrome are unaware of their genetic status to underutilization of hereditary cancer risk assessment. Chatbots, or programs that use artificial intelligence to simulate conversation, have emerged as a promising tool in health care and, more recently, as a potential tool for genetic cancer risk assessment and counseling. Here, we evaluated the existing literature on the use of chatbots in genetic cancer risk assessment and counseling. METHODS A systematic review was conducted using key electronic databases to identify studies which use chatbots for genetic cancer risk assessment and counseling. Eligible studies were further subjected to meta-analysis. RESULTS Seven studies met inclusion criteria, evaluating five distinct chatbots. Three studies evaluated a chatbot that could perform genetic cancer risk assessment, one study evaluated a chatbot that offered patient counseling, and three studies included both functions. The pooled estimated completion rate for the genetic cancer risk assessment was 36.7% (95% CI, 14.8 to 65.9). Two studies included comprehensive patient characteristics, and none involved a comparison group. Chatbots varied as to the involvement of a health care provider in the process of risk assessment and counseling. CONCLUSION Chatbots have been used to streamline genetic cancer risk assessment and counseling and hold promise for reducing barriers to genetic services. Data regarding user and nonuser characteristics are lacking, as are data regarding comparative effectiveness to usual care. Future research may consider the impact of chatbots on equitable access to genetic services.
Collapse
|
10
|
Suckiel SA, Kelly NR, Odgis JA, Gallagher KM, Sebastin M, Bonini KE, Marathe PN, Brown K, Di Biase M, Ramos MA, Rodriguez JE, Scarimbolo L, Insel BJ, Ferar KD, Zinberg RE, Diaz GA, Greally JM, Abul-Husn NS, Bauman LJ, Gelb BD, Horowitz CR, Wasserstein MP, Kenny EE. The NYCKidSeq randomized controlled trial: Impact of GUÍA digitally enhanced genetic counseling in racially and ethnically diverse families. MEDRXIV : THE PREPRINT SERVER FOR HEALTH SCIENCES 2023:2023.07.05.23292193. [PMID: 37461450 PMCID: PMC10350148 DOI: 10.1101/2023.07.05.23292193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 07/28/2023]
Abstract
Background Digital solutions are needed to support rapid increases in the application of genetic and genomic tests (GT) in diverse clinical settings and patient populations. We developed GUÍA, a bi-lingual web-based platform that facilitates disclosure of GT results. The NYCKidSeq randomized controlled trial evaluated GUÍA's impact on understanding of GT results. Methods NYCKidSeq enrolled diverse children with neurologic, cardiac, and immunologic conditions who underwent GT. Families were randomized to genetic counseling with GUÍA (intervention) or standard of care (SOC) genetic counseling for results disclosure. Parents/legal guardians (participants) completed surveys at baseline, post-results disclosure, and 6-months later. Survey measures assessed the primary study outcomes of perceived understanding of and confidence in explaining their child's GT results and the secondary outcome of objective understanding. We used regression models to evaluate the association between the intervention and the study outcomes. Results The analysis included 551 participants, 270 in the GUÍA arm and 281 in SOC. Participants' mean age was 41.1 years and 88.6% were mothers. Most participants were Hispanic/Latino(a) (46.3%), White/European American (24.5%), or Black/African American (15.8%). Participants in the GUÍA arm had significantly higher perceived understanding post-results (OR=2.8, CI[1.004,7.617], P=0.049) and maintained higher objective understanding over time (OR=1.1, CI[1.004, 1.127], P=0.038) compared to those in the SOC arm. There was no impact on perceived confidence. Hispanic/Latino(a) individuals in the GUÍA arm maintained higher perceived understanding (OR=3.9, CI[1.6, 9.3], P=0.003), confidence (OR=2.7, CI[1.021, 7.277], P=0.046), and objective understanding (OR=1.1, CI[1.009, 1.212], P=0.032) compared to SOC . Conclusions This trial demonstrates that GUÍA positively impacts understanding of GT results in diverse parents of children with suspected genetic conditions. These findings build a case for utilizing GUÍA to deliver complex and often ambiguous genetic results. Continued development and evaluation of digital applications in diverse populations are critical for equitably scaling GT offerings in specialty clinics. Trial Registration Clinicaltrials.gov identifier NCT03738098.
Collapse
Affiliation(s)
- Sabrina A. Suckiel
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Nicole R. Kelly
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children’s Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY
| | - Jacqueline A. Odgis
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Katie M. Gallagher
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children’s Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY
| | - Monisha Sebastin
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children’s Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY
| | - Katherine E. Bonini
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Priya N. Marathe
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Kaitlyn Brown
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children’s Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY
| | - Miranda Di Biase
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children’s Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY
| | - Michelle A. Ramos
- Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY
- Institute for Health Equity Research, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Jessica E. Rodriguez
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Laura Scarimbolo
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Beverly J. Insel
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Kathleen D.M. Ferar
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Randi E. Zinberg
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Obstetrics, Gynecology and Reproductive Science, Icahn School of Medicine at Mount Sinai, New York, NY
| | - George A. Diaz
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Pediatrics, Icahn School of Medicine at Mount Sinai, New York, New York
| | - John M. Greally
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children’s Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY
| | - Noura S. Abul-Husn
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Laurie J. Bauman
- Department of Psychiatry and Behavioral Sciences, Albert Einstein College of Medicine, Bronx, NY
- Department of Pediatrics, Albert Einstein College of Medicine, Bronx, NY
| | - Bruce D. Gelb
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Pediatrics, Icahn School of Medicine at Mount Sinai, New York, New York
- Mindich Child Health and Development Institute, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Carol R. Horowitz
- Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY
- Institute for Health Equity Research, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Melissa P. Wasserstein
- Department of Pediatrics, Division of Pediatric Genetic Medicine, Children’s Hospital at Montefiore/Montefiore Medical Center/Albert Einstein College of Medicine, Bronx, NY
| | - Eimear E. Kenny
- Institute for Genomic Health, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY
| |
Collapse
|
11
|
Allen C. User experience of a family health history chatbot: A quantitative analysis. RESEARCH SQUARE 2023:rs.3.rs-2886804. [PMID: 37205400 PMCID: PMC10187455 DOI: 10.21203/rs.3.rs-2886804/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/21/2023]
Abstract
Objective Family health history (FHx) is an important tool in assessing one's risk towards specific health conditions. However, user experience of FHx collection tools is rarely studied. ItRunsInMyFamily.com (ItRuns) was developed to assess FHx and hereditary cancer risk. This study reports a quantitative user experience analysis of ItRuns. Methods We conducted a public health campaign in November 2019 to promote FHx collection using ItRuns. We used software telemetry to quantify abandonment and time spent on ItRuns to identify user behaviors and potential areas of improvement. Results Of 11065 users who started the ItRuns assessment, 4305 (38.91%) reached the final step to receive recommendations about hereditary cancer risk. Highest abandonment rates were during Introduction (32.82%), Invite Friends (29.03%), and Family Cancer History (12.03%) subflows. Median time to complete the assessment was 636 seconds. Users spent the highest median time on Proband Cancer History (124.00 seconds) and Family Cancer History (119.00 seconds) subflows. Search list questions took the longest to complete (median 19.50 seconds), followed by free text email input (15.00 seconds). Conclusion Knowledge of objective user behaviors at a large scale and factors impacting optimal user experience will help enhance the ItRuns workflow and improve future FHx collection.
Collapse
|
12
|
Wilczewski H, Soni H, Ivanova J, Ong T, Barrera JF, Bunnell BE, Welch BM. Older adults' experience with virtual conversational agents for health data collection. Front Digit Health 2023; 5:1125926. [PMID: 37006821 PMCID: PMC10050579 DOI: 10.3389/fdgth.2023.1125926] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Accepted: 02/21/2023] [Indexed: 03/17/2023] Open
Abstract
IntroductionVirtual conversational agents (i.e., chatbots) are an intuitive form of data collection. Understanding older adults' experiences with chatbots could help identify their usability needs. This quality improvement study evaluated older adults' experiences with a chatbot for health data collection. A secondary goal was to understand how perceptions differed based on length of chatbot forms.MethodsAfter a demographic survey, participants (≥60 years) completed either a short (21 questions), moderate (30 questions), or long (66 questions) chatbot form. Perceived ease-of-use, usefulness, usability, likelihood to recommend, and cognitive load were measured post-test. Qualitative and quantitative analyses were used.ResultsA total of 260 participants reported on usability and satisfaction metrics including perceived ease-of-use (5.8/7), usefulness (4.7/7), usability (5.4/7), and likelihood to recommend (Net Promoter Score = 0). Cognitive load (12.3/100) was low. There was a statistically significant difference in perceived usefulness between groups, with a significantly higher mean perceived usefulness for Group 1 than Group 3. No other group differences were observed. The chatbot was perceived as quick, easy, and pleasant with concerns about technical issues, privacy, and security. Participants provided suggestions to enhance progress tracking, edit responses, improve readability, and have options to ask questions.DiscussionOlder adults found the chatbot to be easy, useful, and usable. The chatbot required low cognitive load demonstrating it could be an enjoyable health data collection tool for older adults. These results will inform the development of a health data collection chatbot technology.
Collapse
Affiliation(s)
| | - Hiral Soni
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
- Correspondence: Hiral Soni
| | - Julia Ivanova
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
| | - Triton Ong
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
| | - Janelle F. Barrera
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
- Department of Psychiatry and Behavioral Neurosciences, University of South Florida, Tampa, FL, United States
| | - Brian E. Bunnell
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
- Department of Psychiatry and Behavioral Neurosciences, University of South Florida, Tampa, FL, United States
| | - Brandon M. Welch
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| |
Collapse
|
13
|
Soni H, Ivanova J, Wilczewski H, Bailey A, Ong T, Narma A, Bunnell BE, Welch BM. Virtual conversational agents versus online forms: Patient experience and preferences for health data collection. Front Digit Health 2022; 4:954069. [PMID: 36310920 PMCID: PMC9606606 DOI: 10.3389/fdgth.2022.954069] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2022] [Accepted: 09/16/2022] [Indexed: 11/07/2022] Open
Abstract
Objective Virtual conversational agents, or chatbots, have emerged as a novel approach to health data collection. However, research on patient perceptions of chatbots in comparison to traditional online forms is sparse. This study aimed to compare and assess the experience of completing a health assessment using a chatbot vs. an online form. Methods A counterbalanced, within-subject experimental design was used with participants recruited via Amazon Mechanical Turk (mTurk). Participants completed a standardized health assessment using a chatbot (i.e., Dokbot) and an online form (i.e., REDCap), each followed by usability and experience questionnaires. To address poor data quality and preserve integrity of mTurk responses, we employed a thorough data cleaning process informed by previous literature. Quantitative (descriptive and inferential statistics) and qualitative (thematic analysis and complex coding query) approaches were used for analysis. Results A total of 391 participants were recruited, 185 of whom were excluded, resulting in a final sample size of 206 individuals. Most participants (69.9%) preferred the chatbot over the online form. Average Net Promoter Score was higher for the chatbot (NPS = 24) than the online form (NPS = 13) at a statistically significant level. System Usability Scale scores were also higher for the chatbot (i.e. 69.7 vs. 67.7), but this difference was not statistically significant. The chatbot took longer to complete but was perceived as conversational, interactive, and intuitive. The online form received favorable comments for its familiar survey-like interface. Conclusion Our findings demonstrate that a chatbot provided superior engagement, intuitiveness, and interactivity despite increased completion time compared to online forms. Knowledge of patient preferences and barriers will inform future design and development of recommendations and best practice for chatbots for healthcare data collection.
Collapse
Affiliation(s)
- Hiral Soni
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States,Correspondence: Hiral Soni
| | - Julia Ivanova
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
| | | | | | - Triton Ong
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
| | - Alexa Narma
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States
| | - Brian E. Bunnell
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States,Department of Psychiatry and Behavioral Neurosciences, Innovation in Mental Health Lab, University of South Florida, Tampa, FL, United States
| | - Brandon M. Welch
- Doxy.me Research, Doxy.me Inc., Rochester, NY, United States,Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| |
Collapse
|
14
|
Digital Future of Emergency Medical Services: Envisioning and Usability of Electronic Patient Care Report System. ADVANCES IN HUMAN-COMPUTER INTERACTION 2022. [DOI: 10.1155/2022/6012241] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Despite the efforts of emerging technologies in the healthcare system, there is still a slower rate of acceleration in prehospital settings compared with the hospitals in digital transformation adaptation. The acknowledgment that digital transformation is significant to healthcare is reflected in planning for the future of digital healthcare. Thus, this study aimed to measure the usability of the electronic patient care report (ePCR) system among emergency medical services (EMS) staff who work in prehospital settings. A descriptive cross-sectional correlation study was used. Two hundred fifty EMS staff who are working in the prehospital setting at Saudi Red Crescent Authority in the Kingdom of Saudi Arabia were surveyed, and the response rate was 79.2% (198). An adapted tool of the Computer System Usability Questionnaire survey was used to collect data. The data were coded numerically and subjected to descriptive and inferential statistical analysis including Pearson’s correlation coefficient using the statistical software (SPSS 21). The majority of the participants rate their ePCR system as “useable” at a high level with a score of 3.41 (SD = 1.021). The overall mean of the ePCR system’s three subscales: system usefulness, information quality, interface quality, and overall satisfaction were 3.39 (SD = 1.152), 3.30 (SD = 1.052), 3.57 (SD = 1.064), and 3.37 (SD = 1.239), respectively. The least liked aspect of ePCR system software was information quality 81 (40.9%). Furthermore, there was a significant correlation between the age of EMS staff and the usability of the ePCR system (r = −0.150
,
). The results suggest that healthcare institutions’ policy and decision-makers pay close attention to performing standardized training for the staff on their ePCR system before going to the field to increase efficiency and productivity. Furthermore, the users in this study identified other system features that, if included, could have enhanced usability, and improved functions and capabilities of the design to meet the EMS staff’s expectations.
Collapse
|
15
|
Bombard Y, Ginsburg GS, Sturm AC, Zhou AY, Lemke AA. Digital health-enabled genomics: Opportunities and challenges. Am J Hum Genet 2022; 109:1190-1198. [PMID: 35803232 PMCID: PMC9300757 DOI: 10.1016/j.ajhg.2022.05.001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/09/2023] Open
Abstract
Digital health solutions, with apps, virtual care, and electronic medical records, are gaining momentum across all medical disciplines, and their adoption has been accelerated, in part, by the COVID-19 pandemic. Personal wearables, sensors, and mobile technologies are increasingly being used to identify health risks and assist in diagnosis, treatment, and monitoring of health and disease. Genomics is a vanguard of digital healthcare as we witness a convergence of the fields of genomic and digital medicine. Spurred by the acute need to increase health literacy, empower patients' preference-sensitive decisions, or integrate vast amounts of complex genomic data into the clinical workflow, there has been an emergence of digital support tools in genomics-enabled care. We present three use cases that demonstrate the application of these converging technologies: digital genomics decision support tools, conversational chatbots to scale the genetic counseling process, and the digital delivery of comprehensive genetic services. These digital solutions are important to facilitate patient-centered care delivery, improve patient outcomes, and increase healthcare efficiencies in genomic medicine. Yet the development of these innovative digital genomic technologies also reveals strategic challenges that need to be addressed before genomic digital health can be broadly adopted. Alongside key evidentiary gaps in clinical and cost-effectiveness, there is a paucity of clinical guidelines, policy, and regulatory frameworks that incorporate digital health. We propose a research agenda, guided by learning healthcare systems, to realize the vision of digital health-enabled genomics to ensure its sustainable and equitable deployment in clinical care.
Collapse
Affiliation(s)
- Yvonne Bombard
- University of Toronto, Genomics Health Services Research Program, St. Michael’s Hospital, Unity Health Toronto, 30 Bond Street, Toronto, ON, M5B 1W8, Canada,Corresponding author
| | - Geoffrey S. Ginsburg
- All of Us Research Program, National Institutes of Health, Bethesda, MD 20892, USA
| | - Amy C. Sturm
- 23andMe, 223 North Mathilda Avenue, Sunnyvale, CA 94086, USA
| | - Alicia Y. Zhou
- Color Health, Inc, 831 Mitten Road, Burlingame, CA 94010, USA
| | - Amy A. Lemke
- Norton Children’s Research Institute, Affiliated with the University of Louisville School of Medicine, 571 South Floyd Street, Louisville, KY 40202, USA
| |
Collapse
|
16
|
Schmidlen T, Jones CL, Campbell-Salome G, McCormick CZ, Vanenkevort E, Sturm AC. Use of a chatbot to increase uptake of cascade genetic testing. J Genet Couns 2022; 31:1219-1230. [PMID: 35616645 DOI: 10.1002/jgc4.1592] [Citation(s) in RCA: 18] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Revised: 04/26/2022] [Accepted: 05/06/2022] [Indexed: 12/18/2022]
Abstract
Successful proband-mediated family communication and subsequent cascade genetic testing uptake requires interventions that present information clearly, in sufficient detail, and with medical authority. To facilitate family communication for patients receiving clinically actionable results via the MyCode® Community Health Initiative, a Family Sharing Tool (FST) and a cascade chatbot were developed. FST is an electronic mechanism allowing patients to share genetic test results with relatives via chatbot. The cascade chatbot describes the proband's result, associated disease risks, and recommended management and captures whether the user is a blood relative or caregiver, sex, and relationship to the proband. FST and cascade chatbot uptake among MyCode® probands and relatives was tracked from August 2018 through February 2020. Cascade genetic testing uptake was collected from testing laboratories as number of cascades per proband. Fifty-eight percent (316/543) of probands consented to FST; 42% (227/543) declined. Receipt preferences were patient electronic health record (EHR) portal (52%), email (29%), and text (19%). Patient EHR portal users (p < 0.001) and younger patients were more likely to consent (p < 0.001). FST was deployed to 308 probands. Fifty-nine percent (183/308) opened; of those, 56% (102/183) used FST to send a cascade chatbot to relatives. These 102 probands shared a cascade chatbot with 377 relatives. Sixty-two percent (235/377) of relatives opened; of these, 69% (161/235) started, and of these, 57% (92/161) completed the cascade chatbot. Cascade genetic testing uptake was significantly greater among relatives of probands who consented to the FST (M = 2.34 cascades, SD = 2.10) than relatives of probands who declined (M = 1.40 cascades, SD = 0.82, p < 0.001). Proband age was not a significant predictor of cascade genetic testing uptake. Further work is needed to better understand factors impacting proband use of FST and relative use of cascade chatbots.
Collapse
Affiliation(s)
| | - Claire L Jones
- Geisinger, Genomic Medicine Institute, Danville, Pennsylvania, USA
| | | | - Cara Z McCormick
- Geisinger, Genomic Medicine Institute, Danville, Pennsylvania, USA
| | - Erin Vanenkevort
- Geisinger, Genomic Medicine Institute, Danville, Pennsylvania, USA
| | - Amy C Sturm
- Geisinger, Genomic Medicine Institute, Danville, Pennsylvania, USA
| |
Collapse
|
17
|
Mavragani A, Frey LJ, Lamy JB, Bellcross C, Morrison H, Schiffman JD, Welch BM. Automated Clinical Practice Guideline Recommendations for Hereditary Cancer Risk Using Chatbots and Ontologies: System Description. JMIR Cancer 2022; 8:e29289. [PMID: 35099392 PMCID: PMC8845001 DOI: 10.2196/29289] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2021] [Revised: 07/30/2021] [Accepted: 12/18/2021] [Indexed: 01/07/2023] Open
Abstract
BACKGROUND Identifying patients at risk of hereditary cancer based on their family health history is a highly nuanced task. Frequently, patients at risk are not referred for genetic counseling as providers lack the time and training to collect and assess their family health history. Consequently, patients at risk do not receive genetic counseling and testing that they need to determine the preventive steps they should take to mitigate their risk. OBJECTIVE This study aims to automate clinical practice guideline recommendations for hereditary cancer risk based on patient family health history. METHODS We combined chatbots, web application programming interfaces, clinical practice guidelines, and ontologies into a web service-oriented system that can automate family health history collection and assessment. We used Owlready2 and Protégé to develop a lightweight, patient-centric clinical practice guideline domain ontology using hereditary cancer criteria from the American College of Medical Genetics and Genomics and the National Cancer Comprehensive Network. RESULTS The domain ontology has 758 classes, 20 object properties, 23 datatype properties, and 42 individuals and encompasses 44 cancers, 144 genes, and 113 clinical practice guideline criteria. So far, it has been used to assess >5000 family health history cases. We created 192 test cases to ensure concordance with clinical practice guidelines. The average test case completes in 4.5 (SD 1.9) seconds, the longest in 19.6 seconds, and the shortest in 2.9 seconds. CONCLUSIONS Web service-enabled, chatbot-oriented family health history collection and ontology-driven clinical practice guideline criteria risk assessment is a simple and effective method for automating hereditary cancer risk screening.
Collapse
Affiliation(s)
| | - Lewis J Frey
- Health Equity and Rural Outreach Innovation Center, Ralph H. Johnson Veterans Affairs Health Care System, Charleston, SC, United States
| | - Jean-Baptiste Lamy
- Université Sorbonne Paris Nord, LIMICS, Sorbonne Université, INSERM, F-93000, Bobigny, France
| | - Cecelia Bellcross
- Department of Human Genetics, Emory University, Atlanta, GA, United States
| | | | - Joshua D Schiffman
- Division of Pediatric Hematology/Oncology, Department of Pediatrics, University of Utah, Salt Lake City, UT, United States.,Family Cancer Assessment Clinic, Huntsman Cancer Institute, University of Utah, Salt Lake City, United States, UT, United States
| | - Brandon M Welch
- Department of Public Health Sciences, Medical University of South Carolina, Charleston, SC, United States
| |
Collapse
|
18
|
Wang C, Paasche-Orlow MK, Bowen DJ, Cabral H, Winter MR, Norkunas Cunningham T, Trevino-Talbot M, Toledo DM, Cortes DE, Campion M, Bickmore T. Utility of a virtual counselor (VICKY) to collect family health histories among vulnerable patient populations: A randomized controlled trial. PATIENT EDUCATION AND COUNSELING 2021; 104:979-988. [PMID: 33750594 PMCID: PMC8113103 DOI: 10.1016/j.pec.2021.02.034] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2020] [Revised: 01/25/2021] [Accepted: 02/16/2021] [Indexed: 05/29/2023]
Abstract
OBJECTIVES This study is a randomized controlled trial comparing the efficacy of a virtual counselor (VICKY) to the My Family Health Portrait (MFHP) tool for collecting family health history (FHx). METHODS A total of 279 participants were recruited from a large safety-net hospital and block randomized by health literacy to use one of the digital FHx tools, followed by a genetic counselor interview. A final sample of 273 participants were included for analyses of primary study aims pertaining to tool concordance, which assessed agreement between tool and genetic counselor. RESULTS Tool completion differed significantly between tools (VICKY = 97%, MFHP = 51%; p < .0001). Concordance between tool and genetic counselor was significantly greater for participants randomized to VICKY compared to MFHP for ascertaining first- and second-degree relatives (ps<.0001), and most health conditions examined. There was significant interaction by health literacy, with greater differences in concordance observed between tools among those with limited literacy. CONCLUSIONS A virtual counselor overcomes many of the literacy-related barriers to using traditional digital tools and highlights an approach that may be important to consider when collecting health histories from vulnerable populations. PRACTICE IMPLICATIONS The usability of digital health history tools will have important implications for the quality of the data collected and its downstream clinical utility.
Collapse
Affiliation(s)
- Catharine Wang
- Department of Community Health Sciences, Boston University School of Public Health, Boston, MA, USA.
| | - Michael K Paasche-Orlow
- Department of Medicine, Boston University School of Medicine and Boston Medical Center, Boston, MA, USA
| | - Deborah J Bowen
- Department of Bioethics and Humanities, School of Medicine, University of Washington, Seattle, WA, USA
| | - Howard Cabral
- Department of Biostatistics, Boston University School of Public Health, Boston, MA, USA
| | - Michael R Winter
- Biostatistics and Epidemiology Data Analytics Center (BEDAC), Boston University School of Public Health, Boston, MA, USA
| | | | - Michelle Trevino-Talbot
- Department of Community Health Sciences, Boston University School of Public Health, Boston, MA, USA
| | - Diana M Toledo
- Department of Pathology and Laboratory Medicine, Dartmouth-Hitchcock Medical Center, Lebanon, NH, USA
| | - Dharma E Cortes
- Department of Psychiatry, Harvard Medical School, Boston, MA, USA; Health Equity Research Lab, Cambridge Health Alliance, Cambridge, MA, USA
| | - MaryAnn Campion
- Department of Genetics, Stanford University, Stanford, CA, USA
| | - Timothy Bickmore
- Khoury College of Computer Sciences, Northeastern University, Boston, MA, USA
| |
Collapse
|