Knowledge and Perceptions of Hepatitis B and Hepatocellular Carcinoma Screening Guidelines Among Trainees: A Tale of Three Centers

Development of a liver disease-specific large language model chat interface using retrieval-augmented generation

BACKGROUND AND AIMS

Large language models (LLMs) have significant capabilities in clinical information processing tasks. Commercially available LLMs, however, are not optimized for clinical uses and are prone to generating hallucinatory information. Retrieval-augmented generation (RAG) is an enterprise architecture that allows the embedding of customized data into LLMs. This approach "specializes" the LLMs and is thought to reduce hallucinations.

APPROACH AND RESULTS

We developed "LiVersa," a liver disease-specific LLM, by using our institution's protected health information-complaint text embedding and LLM platform, "Versa." We conducted RAG on 30 publicly available American Association for the Study of Liver Diseases guidance documents to be incorporated into LiVersa. We evaluated LiVersa's performance by conducting 2 rounds of testing. First, we compared LiVersa's outputs versus those of trainees from a previously published knowledge assessment. LiVersa answered all 10 questions correctly. Second, we asked 15 hepatologists to evaluate the outputs of 10 hepatology topic questions generated by LiVersa, OpenAI's ChatGPT 4, and Meta's Large Language Model Meta AI 2. LiVersa's outputs were more accurate but were rated less comprehensive and safe compared to those of ChatGPT 4.

RESULTS

We evaluated LiVersa's performance by conducting 2 rounds of testing. First, we compared LiVersa's outputs versus those of trainees from a previously published knowledge assessment. LiVersa answered all 10 questions correctly. Second, we asked 15 hepatologists to evaluate the outputs of 10 hepatology topic questions generated by LiVersa, OpenAI's ChatGPT 4, and Meta's Large Language Model Meta AI 2. LiVersa's outputs were more accurate but were rated less comprehensive and safe compared to those of ChatGPT 4.

CONCLUSIONS

In this demonstration, we built disease-specific and protected health information-compliant LLMs using RAG. While LiVersa demonstrated higher accuracy in answering questions related to hepatology, there were some deficiencies due to limitations set by the number of documents used for RAG. LiVersa will likely require further refinement before potential live deployment. The LiVersa prototype, however, is a proof of concept for utilizing RAG to customize LLMs for clinical use cases.

Interventions to Improve Surveillance for Hepatocellular Carcinoma in High-Risk Patients: A Scoping Review

PURPOSE

Hepatocellular carcinoma (HCC) is most often a sequela of chronic liver disease or chronic hepatitis B infection. Among high-risk patients, surveillance for HCC every 6 months is recommended by international guidelines. However, rates of HCC surveillance are suboptimal (11-64%). Barriers at the patient, provider, and healthcare delivery system levels have been identified.

METHODS

We performed a systemic scoping review to identify and characterize interventions to improve HCC surveillance that has previously been evaluated. Searches using key terms in PubMed and Embase were performed to identify studies examining interventions designed to improve the surveillance rate for HCC in patients with cirrhosis or chronic liver disease that were published in English between January 1990 and September 2021.

RESULTS

Included studies (14) had the following study designs: (1) randomized clinical trials (3, 21.4%), (2) quasi-experimental (2, 14.3%), (3) prospective cohort (6, 42.8%), and (4) retrospective cohort (3, 21.4%). Interventions included mailed outreach invitations, nursing outreach, patient education with or without printed materials, provider education, patient navigation, chronic disease management programs, nursing-led protocols for image ordering, automated reminders to physicians and nurses, web-based clinical management tools, HCC surveillance databases, provider compliance reports, radiology-led surveillance programs, subsidized HCC surveillance, and the use of oral medications. It was found that HCC surveillance rates increased after intervention implementation in all studies.

CONCLUSION

Despite improvements in HCC surveillance rates with intervention, compliance remained suboptimal. Further analysis of which interventions yield the greatest increases in HCC surveillance, design of multi-pronged strategies, and improved implementation are needed.

Development of a Liver Disease-Specific Large Language Model Chat Interface using Retrieval Augmented Generation

Background

Large language models (LLMs) have significant capabilities in clinical information processing tasks. Commercially available LLMs, however, are not optimized for clinical uses and are prone to generating incorrect or hallucinatory information. Retrieval-augmented generation (RAG) is an enterprise architecture that allows embedding of customized data into LLMs. This approach "specializes" the LLMs and is thought to reduce hallucinations.

Methods

We developed "LiVersa," a liver disease-specific LLM, by using our institution's protected health information (PHI)-complaint text embedding and LLM platform, "Versa." We conducted RAG on 30 publicly available American Association for the Study of Liver Diseases (AASLD) guidelines and guidance documents to be incorporated into LiVersa. We evaluated LiVersa's performance by comparing its responses versus those of trainees from a previously published knowledge assessment study regarding hepatitis B (HBV) treatment and hepatocellular carcinoma (HCC) surveillance.

Results

LiVersa answered all 10 questions correctly when forced to provide a "yes" or "no" answer. Full detailed responses with justifications and rationales, however, were not completely correct for three of the questions.

Discussions

In this study, we demonstrated the ability to build disease-specific and PHI-compliant LLMs using RAG. While our LLM, LiVersa, demonstrated more specificity in answering questions related to clinical hepatology - there were some knowledge deficiencies due to limitations set by the number and types of documents used for RAG. The LiVersa prototype, however, is a proof of concept for utilizing RAG to customize LLMs for clinical uses and a potential strategy to realize personalized medicine in the future.

Patient-, Provider-, and System-Level Barriers to Surveillance for Hepatocellular Carcinoma in High-Risk Patients in the USA: a Scoping Review

PURPOSE

Hepatocellular carcinoma has a dismal prognosis, except in patients diagnosed early who are candidates for potentially curative therapies. Most HCC cases develop in patients with chronic liver disease. Therefore, expert society guidelines recommend surveillance every 6 months with ultrasound with or without serum alpha-fetoprotein for high-risk patients. However, fewer than 20% of patients in the USA undergo appropriate surveillance.

METHODS

A systematic scoping review was performed with the objective of identifying barriers to screening among high-risk patients in the USA including mapping key concepts in the relevant literature, identifying the main sources and types of evidence available, and identifying gaps in the literature. A total of 43 studies published from 2007 to 2021 were included. Data were extracted and a conceptual framework was created.

RESULTS

Assessment of quantitative studies revealed poor surveillance rates, often below 50%. Three categories of barriers to surveillance were identified: patient-level, provider-level, and system-level barriers. Prevalent patient-level barriers included financial constraints, lack of awareness of surveillance recommendations, and scheduling difficulties. Common provider-level barriers were lack of provider awareness of guidelines for surveillance, difficulty accessing specialty resources, and time constraints in the clinic. System-level barriers included fewer clinic visits and rural/safety-net settings. Proposed interventions include improved patient/provider education, patient navigators, increased community/academic collaboration, and EMR-based reminders.

CONCLUSION

Based on these findings, there is a crucial need to implement and evaluate proposed interventions to improve HCC surveillance.

Disparities in presentation and management of chronic hepatitis B among Hispanics in a diverse safety net system

BACKGROUND

Chronic hepatitis B (HBV) prevalence is highest in foreign-born Asian and African individuals in the US, though Hispanics make up the largest proportion of the immigrant population. Differences in the diagnosis and management of chronic HBV in Hispanics might exist due to the lower awareness of risk. We aim to examine racial/ethnic disparities in the diagnosis, presentation, and immediate management of chronic HBV in a diverse safety net system enriched for Hispanics.

METHODS

In a large urban safety-net hospital system, we retrospectively identified patients with chronic HBV by serological data and categorized them into mutually exclusive self-identified racial/ethnic groups: Hispanics, Asians, Blacks, and Whites. We then examined differences in screening, disease phenotype and severity, follow-up testing, and referral by race/ethnicity.

RESULTS

Among 1063 patients, 302 (28%) were Hispanics, 569 (54%) Asians, 161 (15%) Blacks, and 31 (3%) Whites. More Hispanics (30%) were screened in the acute setting (defined as inpatient or emergency department encounters) than Asians (13%), Blacks (17%), or Whites (23%) (p<0.01). Hispanics also had lower rates of follow-up testing after HBV diagnosis than Asians including HBeAg status (43% vs. 60%, p<0.01) and HBV DNA levels (42% vs. 58%, p<0.01) and lower rates of linkage to specialty care (32% vs. 55%, p<0.01). Among those with available testing, however, the presence of immune-active chronic HBV was infrequent and similar across racial/ethnic groups. 25% of Hispanics had cirrhosis at initial presentation, proportionally higher than other groups (p<0.01).

CONCLUSION

Our results underscore the importance of raising chronic HBV awareness and increasing both screening and linkage to care among Hispanic immigrants in addition to the existing risk groups, with the goal of mitigating downstream liver-related complications.

Seroprevalence of Hepatitis B Among Healthcare Workers in Asia and Africa and Its Association With Their Knowledge and Awareness: A Systematic Review and Meta-Analysis

Introduction

The hepatitis B virus (HBV) is a blood-borne virus that can be transmitted by percutaneous and mucocutaneous contact with infected bodily fluid. Healthcare workers (HCWs) are more exposed to HBV infection. They must have a thorough understanding of HBV infection since they can contract and spread the virus. In this study, we systematically reviewed all published evidence on the seroprevalence of Hepatitis B virus (HBV) infection among HCWs. and synthesize evidence on the association between knowledge and awareness with HBV infection.

Methods

We searched PubMed, EMBASE, Cochrane Library and Scopus for studies reporting on HBV seroprevalence from January 1997 to September 2021 among healthcare workers. We used random-effects meta-analyses to estimate the pool prevalence of HBV infection.

Results

We identified 25 studies that met our inclusion criteria, with data on 10,043 adults from 11 countries and two regions: Africa and Asia. The overall seroprevalence of HBV was 5.0% (95% confidence interval [CI] 3.6%), with Africa reporting higher estimates (5.0%, 95% CI 3.7%) than Asia population (4.0%, 95% CI 1.9%). The highest pooled prevalence estimate in African countries came from studies published in the Cameroon region (8.0%, 95% CI 5–10%) while the lowest came from Ethiopia (4.0%, 95% CI 2.6%). The overall seroprevalence estimates in the African population were significantly higher than those in the Asian group. Studies in Africa found that the average knowledge and seroprevalence were 1.4% and 11.0%, respectively where, eight studies (53.3%) reported good knowledge and seven studies (46.7%) reported average knowledge. In Asia, two studies (40.0%) reported good knowledge, one study (20.0%) reporting average knowledge, and two studies (40.0%) reporting poor knowledge. African studies demonstrated good knowledge despite the fact that their HBV infection rate was higher than 6.7%.

Conclusion

Africa and Asia have the highest seroprevalence of HBV infection. Improving the comparability of epidemiological and clinical studies constitutes an important step forward. More high-quality data is needed to improve the precision of burden estimates.

Systematic Review Registration

PROSPERO CRD42021279905.

Acceptability and Feasibility of Home-Based Hepatitis B Screening Among Haitian Immigrants

Hepatitis B (HBV) is endemic in Haiti, therefore Haitian immigrants should be screened to identify and link affected individuals to care. Current screening approaches are ineffective. We assessed the acceptability and feasibility of home-based screening among Haitian immigrants using community health workers (CHWs). We recruited participants exiting a pragmatic trial evaluating strategies to improve care delivery (NCT02970136). Participants completed an acceptability questionnaire. Blood drawn by CHWs at participants' homes or community sites was tested for hepatitis B surface antigen, hepatitis B surface antibody and hepatitis B core antibody. Of 60 participants, 59 found screening acceptable; 53 had blood drawn. Of those, 45.3% had HBV previously, 49.1% remained susceptible and 5.7% were vaccinated. Respondents cited various reasons community members might find screening unacceptable. The high prior HBV rate highlights the need for effective outreach programs. Home-based HBV screening was both acceptable and feasible among Haitian immigrants.