Primary Care Physicians' Satisfaction With Interoperable Health Information Technology

Importance

Enabling widespread interoperability-the ability of health information technology systems to exchange information and to use that information without special effort-is a primary focus of public policy on health information technology. More information on clinicians' experience using that technology can serve as one measure of the impact of that policy.

Objective

To assess primary care physician perspectives on the state of interoperability.

Design, Setting, and Participants

A cross-sectional survey of family medicine physicians in the US was conducted from December 12, 2021, to October 12, 2022. A sample of family medicine physicians who completed the Continuous Certification Questionnaire (CCQ), a required part of the American Board of Family Medicine certification process, which has a 100% response rate, were invited to participate.

Main Outcomes and Measures

Eighteen items on the CCQ assessed experience accessing and using various information from outside organizations, including medications, immunizations, and allergies.

Results

A total of 2088 physicians (1053 women [50%]; age reported categorically as either ≥50 years or <50 years) completed the CCQ interoperability questions in 2022. Of these respondents, 35% practiced in hospital or health system-owned practices, while 27% practiced in independently owned practices. Eleven percent were very satisfied with their ability to electronically access all 10 types of information from outside organizations included on the questionnaire, and a mean of 70% were at least somewhat satisfied. A total of 23% of family medicine physicians reported information from outside organizations was very easy to use, and an additional 65% reported that information was somewhat easy to use. Only 8% reported that information from different electronic health record (EHR) developers' products was very easy to use compared with 38% who reported information from the same EHR developer's product was very easy to use.

Conclusions and Relevance

This survey study of family medicine physicians found modest and uneven improvement in physicians' experience with interoperability. These findings suggest that substantial heterogeneity in satisfaction by information type, source of information, EHR, practice type, ownership, and patient population necessitates diverse policy and strategies to improve interoperability.

Influence of Prior SARS-CoV-2 Infection on COVID-19 Severity: Evidence from the National COVID Cohort Collaborative

Background

A large share of SARS-CoV-2 infections now occur among previously infected individuals. In this study, we sought to determine whether prior infection modifies disease severity relative to no prior infection.

Methods

We used data from first and second COVID-19 episodes in the National COVID Cohort Collaborative, a nationwide collection of de-identified electronic health records. We used nested logistic regressions of monthly cohorts weighted on the inverse probability of prior infection to assess risk of hospitalization, death, and increased severity in the first versus second infection cohorts.

Results

We included a total of 2,058,274 individuals in the analysis, 147,592 of whom had two recorded infections. The impact of prior infection differed meaningfully between months. Prior infection was largely protective prior to March 2022, with odds ratios (ORs) as low as 0.66 (95% confidence interval: 0.51 to 0.86) in November 2021 for hospitalization. and as low as 0.23 (0.06 to 0.86) in June 2021 for death. However, prior infection was associated with an increased risk of hospitalization and death, mostly after March 2022 when the ORs were as high as 1.87 (1.26 to 2.80) and 2.99 (1.65 to 5.41) in April 2022, respectively. The overall OR for more severe disease was 1.06 (1.03 to 1.10) among previously infected individuals.

Conclusion

In the pandemic's first two years, previously infected patients generally had less severe disease than people without prior infection. During the Omicron era, however, previously infected patients had the same or worse severity of disease as patients without prior infection.

Variation in Family Physicians' Experiences Across Different Electronic Health Record Platforms: a Descriptive Study

BACKGROUND

Electronic health records (EHRs) have been connected to excessive workload and physician burnout. Little is known about variation in physician experience with different EHRs, however.

OBJECTIVE

To analyze variation in reported usability and satisfaction across EHRs.

DESIGN

Internet-based survey available between December 2021 and October 2022 integrated into American Board of Family Medicine (ABFM) certification process.

PARTICIPANTS

ABFM-certified family physicians who use an EHR with at least 50 total responding physicians.

MEASUREMENTS

Self-reported experience of EHR usability and satisfaction.

KEY RESULTS

We analyzed the responses of 3358 physicians who used one of nine EHRs. Epic, athenahealth, and Practice Fusion were rated significantly higher across six measures of usability. Overall, between 10 and 30% reported being very satisfied with their EHR, and another 32 to 40% report being somewhat satisfied. Physicians who use athenahealth or Epic were most likely to be very satisfied, while physicians using Allscripts, Cerner, or Greenway were the least likely to be very satisfied. EHR-specific factors were the greatest overall influence on variation in satisfaction: they explained 48% of variation in the probability of being very satisfied with Epic, 46% with eClinical Works, 14% with athenahealth, and 49% with Cerner.

CONCLUSIONS

Meaningful differences exist in physician-reported usability and overall satisfaction with EHRs, largely explained by EHR-specific factors. User-centric design and implementation, and robust ongoing evaluation are needed to reduce physician burden and ensure excellent experience with EHRs.

How Do Family Physicians Document Patients' Social Needs in Electronic Health Records?

Social needs are critical determinants of patient health, but their capture in clinical records began recently. A representative survey of family physicians showed that 61% of respondents document social needs using notes, with fewer using diagnosis codes or electronic forms. This preference for unstructured documentation may make it difficult to connect patients across organizations or for policymakers and planners to identify geographic variation in needs.

Equitable Prioritization of Health Interventions by Incorporating Financial Risk Protection Weights Into Economic Evaluations

OBJECTIVES

Financial risk protection (FRP), or the prevention of medical impoverishment, is a major objective of health systems, particularly in low- and middle-income countries where the extent of out-of-pocket (OOP) health expenditures can be substantial. We sought to develop a method that allows decision makers to explicitly integrate FRP outcomes into their priority-setting activities.

METHODS

We used literature review to identify 31 interventions in low- and middle-income countries, each of which provided measures of health outcomes, costs, OOP health expenditures averted, and FRP (proxied by OOP health expenditures averted as a percentage of income), all disaggregated by income quintile. We developed weights drawn from the Z-score of each quintile-intervention pair based on the distribution of FRP of all quintile-intervention pairs. We next ranked the interventions by unweighted and weighted health outcomes for each income quintile. We also evaluated how pro-poor they were by, first, ordering the interventions by cost-effectiveness for each quintile and, next, calculating the proportion of interventions each income quintile would be targeted for a given random budget. A ranking was said to be pro-poor if each quintile received the same or higher proportion of interventions than richer quintiles.

RESULTS

Using FRP weights produced a more pro-poor priority setting than unweighted outcomes. Most of the reordering produced by the inclusion of FRP weights occurred in interventions of moderate cost-effectiveness, suggesting that these weights would be most useful as a way of distinguishing moderately cost-effective interventions with relatively high potential FRP.

CONCLUSIONS

This preliminary method of integrating FRP into priority-setting would likely be most suitable to deciding between health interventions with intermediate cost-effectiveness.

Estimated travel time and staffing constraints to accessing the Ethiopian health care system: A two-step floating catchment area analysis

Background

Despite large investments in the public health care system, disparities in health outcomes persist between lower- and upper-income individuals, as well as rural vs urban dwellers in Ethiopia. Evidence from Ethiopia and other low- and middle-income countries suggests that challenges in health care access may contribute to poverty in these settings.

Methods

We employed a two-step floating catchment area to estimate variations in spatial access to health care and in staffing levels at health care facilities. We estimated the average travel time from the population centers of administrative areas and adjusted them with provider-to-population ratios. To test hypotheses about the role of travel time vs staffing, we applied Spearman's rank tests to these two variables against the access score to assess the significance of observed variations.

Results

Among Ethiopia's 11 first-level administrative units, Addis Ababa, Dire Dawa, and Harari had the best access scores. Regions with the lowest access scores were generally poorer and more rural/pastoral. Approximately 18% of the country did not have access to a public health care facility within a two-hour walk. Our results suggest that spatial access and staffing issues both contribute to access challenges.

Conclusion

Investments both in new health facilities and staffing in existing facilities will be necessary to improve health care access within Ethiopia. Because rural and low-income areas are more likely to have poor access, future strategies for expanding and strengthening the health care system should strongly emphasize equity and the role of improved access in reducing poverty.

Residual Disease Burden After Neoadjuvant Therapy Among US Patients With High-Risk HER2-positive Early-Stage Breast Cancer: A Population Effectiveness Model

BACKGROUND

Approximately half of patients with high-risk HER2-positive early-stage breast cancer (ESBC) do not have pathologic complete response (pCR) after neoadjuvant therapy. The residual burden of disease among this population has not been previously quantified.

MATERIALS AND METHODS

We used decision-modeling techniques to simulate recurrence, progression from locoregional to distant cancer, breast cancer-related mortality, and mortality from other causes over a 10-year period in a hypothetical cohort. We derived progression probabilities primarily from the KATHERINE trial of T-DM1 (ado-trastuzumab emtansine) and mortality outcomes from the published literature. Modeled outcomes included recurrences, breast cancer deaths, deaths from other causes, direct medical costs, and costs due to lost productivity. To estimate the residual disease burden, we compared outcomes from a cohort of patients treated with T-DM1 versus a hypothetical cohort with no disease recurrence.

RESULTS

We estimated that 9,300 people would experience incident high-risk HER2-positive ESBC in the United States in 2021 based on cancer surveillance databases, clinical trial data, and expert opinion. We estimated that, in this group, 2,118 would experience disease recurrence, including 1,576 distant recurrences, and 1,358 would experience breast cancer deaths. This residual disease burden resulted in 6,435 life-years lost versus the recurrence-free cohort, and healthcare-related costs totaling $644 million, primarily associated with treating distant cancers.

CONCLUSION

Patients with HER2-positive ESBC who do not achieve pCR after neoadjuvant therapy are at ongoing risk of recurrence despite the effectiveness of neoadjuvant treatment. There is substantial clinical and economic value in further reducing the residual disease burden in this population.

Radiologist Preferences for Artificial Intelligence-Based Decision Support During Screening Mammography Interpretation

BACKGROUND

Artificial intelligence (AI) may improve cancer detection and risk prediction during mammography screening, but radiologists' preferences regarding its characteristics and implementation are unknown.

PURPOSE

To quantify how different attributes of AI-based cancer detection and risk prediction tools affect radiologists' intentions to use AI during screening mammography interpretation.

MATERIALS AND METHODS

Through qualitative interviews with radiologists, we identified five primary attributes for AI-based breast cancer detection and four for breast cancer risk prediction. We developed a discrete choice experiment based on these attributes and invited 150 US-based radiologists to participate. Each respondent made eight choices for each tool between three alternatives: two hypothetical AI-based tools versus screening without AI. We analyzed samplewide preferences using random parameters logit models and identified subgroups with latent class models.

RESULTS

Respondents (n = 66; 44% response rate) were from six diverse practice settings across eight states. Radiologists were more interested in AI for cancer detection when sensitivity and specificity were balanced (94% sensitivity with <25% of examinations marked) and AI markup appeared at the end of the hanging protocol after radiologists complete their independent review. For AI-based risk prediction, radiologists preferred AI models using both mammography images and clinical data. Overall, 46% to 60% intended to adopt any of the AI tools presented in the study; 26% to 33% approached AI enthusiastically but were deterred if the features did not align with their preferences.

CONCLUSION

Although most radiologists want to use AI-based decision support, short-term uptake may be maximized by implementing tools that meet the preferences of dissuadable users.

Implementation of pharmacogenomic clinical decision support for health systems: a cost-utility analysis

We constructed a cost-effectiveness model to assess the clinical and economic value of a CDS alert program that provides pharmacogenomic (PGx) testing results, compared to no alert program in acute coronary syndrome (ACS) and atrial fibrillation (AF), from a health system perspective. We defaulted that 20% of 500,000 health-system members between the ages of 55 and 65 received PGx testing for CYP2C19 (ACS-clopidogrel) and CYP2C9, CYP4F2 and VKORC1 (AF-warfarin) annually. Clinical events, costs, and quality-adjusted life years (QALYs) were calculated over 20 years with an annual discount rate of 3%. In total, 3169 alerts would be fired. The CDS alert program would help avoid 16 major clinical events and 6 deaths for ACS; and 2 clinical events and 0.9 deaths for AF. The incremental cost-effectiveness ratio was $39,477/QALY. A PGx-CDS alert program was cost-effective, under a willingness-to-pay threshold of $100,000/QALY gained, compared to no alert program.

Economic evaluations of health system strengthening activities in low-income and middle-income country settings: a methodological systematic review

Objective

Health system strengthening (HSS) activities should accompany disease-targeting interventions in low/middle-income countries (LMICs). Economic evaluations provide information on how these types of investment might best be balanced but can be challenging. We conducted a systematic review to evaluate how researchers address these economic evaluation challenges.

Methods

We identified studies about economic evaluation of HSS activities in LMICs using a two-stage approach. First, we conducted a broad search to identify areas where economic evaluations of HSS activities were being conducted. Next, we selected specific interventions for more targeted literature review. We extracted study characteristics using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Finally, we summarised authors’ modelling decisions using a framework that examines how models are developed to emphasise generalisability, precision, or realism.

Findings

Our searches produced 1978 studies, out of which we included 36. Most studies used data from prospective trials and calculated cost-effectiveness directly from these trial inputs, rather than using simulation methods. As a group, these studies primarily emphasised precision and realism over generalisability, meaning that their results were best suited to specific settings.

Conclusions

The number of included studies was small. Our findings suggest that most economic evaluations of HSS do not leverage methods like sensitivity analyses or inputs from literature review that would produce more generalisable (but potentially less precise) results. More research into how decision-makers would use economic evaluations to define the expansion path to strengthening health systems would allow for conceptualising impactful work on the economic value of HSS.

Assessing the Economic Value of Clinical Artificial Intelligence: Challenges and Opportunities

OBJECTIVES

Clinical artificial intelligence (AI) is a novel technology, and few economic evaluations have focused on it to date. Before its wider implementation, it is important to highlight the aspects of AI that challenge traditional health technology assessment methods.

METHODS

We used an existing broad value framework to assess potential ways AI can provide good value for money. We also developed a rubric of how economic evaluations of AI should vary depending on the case of its use.

RESULTS

We found that the measurement of core elements of value-health outcomes and cost-are complicated by AI because its generalizability across different populations is often unclear and because its use may necessitate reconfigured clinical processes. Clinicians' productivity may improve when AI is used. If poorly implemented though, AI may also cause clinicians' workload to increase. Some AI has been found to exacerbate health disparities. Nevertheless, AI may promote equity by expanding access to medical care and, when properly trained, providing unbiased diagnoses and prognoses. The approach to assessment of AI should vary based on its use case: AI that creates new clinical possibilities can improve outcomes, but regulation and evidence collection may be difficult; AI that extends clinical expertise can reduce disparities and lower costs but may result in overuse; and AI that automates clinicians' work can improve productivity but may reduce skills.

CONCLUSIONS

The potential uses of clinical AI create challenges for health technology assessment methods originally developed for pharmaceuticals and medical devices. Health economists should be prepared to examine data collection and methods used to train AI, as these may impact its future value.

Abstract P3-16-01: Population effectiveness model of the consequences of recurrence after trastuzumab emtansine (T-DM1) treatment among U.S. patients with high-risk HER2+ early-stage breast cancer (ESBC)

Background: To estimate the long-term consequences of disease recurrence following treatment with adjuvant T-DM1 among U.S. patients with high-risk HER2+ ESBC who did not achieve pathologic complete response (pCR) after neoadjuvant therapy. Methods: A Markov model was used to simulate local/regional and distant recurrence with 10 years of follow-up. This corresponds to the estimated number of U.S. patients with incident high-risk HER2+ ESBC in 2021 (n = 10,000), which was derived from SEER population-based estimates, the NEOSPHERE trial and expert clinical opinion. The probability of recurrence was based on the T-DM1 arm in the KATHERINE trial and long-term results from the HERA trial. We assumed that 80% of patients with any recurrence experience distant recurrence, while the remainder have local/regional recurrence. SEER data and literature review were used to estimate probabilities of survival, distant recurrence secondary to local/regional recurrence, and direct medical costs. We estimated indirect costs were equal to 15% of direct medical costs. Model outcomes included: recurrences, breast cancer-related deaths, non-breast cancer-related deaths, direct medical costs, and indirect costs (all undiscounted). Results were compared to a scenario in which there was no recurrence to estimate population impact. All outcomes were also projected over 10 annual incident cohorts, each with 10 years of follow-up. Results: We estimated the 2021 U.S. patient cohort would experience 2,279 recurrences, including 1,834 distant, and 1,559 breast cancer-related deaths over 10 years, resulting in 7,744 lost years of life and $632 million in additional spending, including $549 million in direct medical costs. Projection to 10 years of incident cohorts would lead to approximately 23,000 recurrences, 16,000 deaths, 77,000 lost years of life and $6 billion in direct medical costs. Conclusions: Patients with HER2+ ESBC who do not achieve pCR after neoadjuvant therapy are at ongoing risk of recurrence despite the effectiveness of treatment with T-DM1. There is substantial clinical and economic value in further reducing the recurrence risk among this population.

Findings for 2021 Cohort Projected over 10 YearsWith recurrenceNo recurrenceDifferenceLocal/regional recurrences4450445Distant recurrence1,83401,834Breast cancer deaths1,55901,559Non-breast cancer deaths416457-41Direct costs$573M$24M$549MIndirect costs$86M$3.6M$82MLife years90,24997,993-7,744Costs$659M$27M$632M

Citation Format: David L Veenstra, Nathaniel Hendrix, Chantal M Dolan, Kathryn Fisher, Deepa Lalla, Nina Hill, Beverly Moy. Population effectiveness model of the consequences of recurrence after trastuzumab emtansine (T-DM1) treatment among U.S. patients with high-risk HER2+ early-stage breast cancer (ESBC) [abstract]. In: Proceedings of the 2021 San Antonio Breast Cancer Symposium; 2021 Dec 7-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2022;82(4 Suppl):Abstract nr P3-16-01.

Abstract P2-11-19: Estimating the long-term risk of recurrence in patients receiving HER2-targeted agents in HER2+ early-stage breast cancer (ESBC)

Background: The APHINITY, KATHERINE and ExteNET pivotal trials demonstrated the benefits of HER2-targeted agents for disease-free survival in HER2+ ESBC. However, heterogeneity in patient populations and study designs make assessment of treatment benefits challenging. The objective of this study was to project absolute treatment benefits over a standardized 10-year time horizon. Methods: An epidemiologic model was developed to simulate numbers of distant recurrences for 1000 patients with newly diagnosed HER2+ ESBC in the ExteNET, APHINITY and KATHERINE patient populations and select subgroups. ExteNET evaluated neratinib vs. placebo from 2009 to 2012. APHINITY evaluated pertuzumab + trastuzumab vs. placebo + trastuzumab + standard adjuvant chemotherapy in high-risk patients without neoadjuvant therapy treated from 2011 to 2014. KATHERINE evaluated ado-trastuzumab emtansine (T-DM1) vs. trastuzumab in patients without pathologic complete response (pCR) following neoadjuvant therapy treated from 2013 to 2016. We used disease-free survival curves from each trial as model inputs to estimate the risk of distant recurrence and extrapolated these results to a standardized 10-year follow-up period. We assumed there were no further treatment benefits to prevention of recurrence beyond the follow-up period of the trial. Survival curve trends beyond each trial’s observation period were informed by the trastuzumab arm of the long-term HERA trial follow-up data. Model outputs were distant recurrences for the control arms for each study and the number of distant recurrences avoided from treatment. Results: Over a 10-year time horizon, treatment benefits ranged from 11 to 65 distant recurrences avoided. Treatment benefit differed according to baseline risk of the trial population and treatment effect, with higher-risk populations and greater treatment effect (e.g. KATHERINE trial) tending to yield greater absolute treatment benefit, with the exception of the ExteNET ITT population as changes in the treatment effect over time (non-proportional hazards) made modeling long-term results more complex. Absolute treatment benefits were similar for the intent-to-treat (ITT) group in the KATHERINE trial and the no pCR group in the ExteNET trial; and the ITT groups in the APHINITY and ExteNET trials. Refinement of model projections and inclusion of local recurrences in a more complex modeling structure are ongoing.

Conclusions: Results of our simulation suggest that patients with HER2+ ESBC experience a range of reductions in distant recurrence from HER2-targeted agents, depending on baseline risk (pCR and hormone receptor status), and treatment effect. Epidemiologic modeling may be a useful method to estimate long-term outcomes and facilitate evaluation across disparate clinical trials.

Table. Modeled benefits of HER2-targeted agents in a cohort of 1000 patients with HER2+ ESBC over a standardized 10-year timeframeSource data inputsModeled outputs (n=1000)Distant recurrences at 10 yearsClinical study characteristicsHazard ratio for distant recurrence from study1Baseline risk: Events in control armTreatment effect: Distant recurrences avoided from treatmentExteNET study (ITT)0.78 (0.60-1.01) at 5 years14318APHINITY study (ITT)0.76 (0.62-0.95) at 6 years10320ExteNET study (HR+; treated within 12 months of trastuzumab receipt)0.57 (0.39-0.83) at 5 years14727APHINITY study (HR+)0.78 (0.51-1.18) at 6 years26111KATHERINE study (no pCR, ITT)0.60 (0.45-0.79) at 3 years26464KATHERINE study (no pCR, HR+)0.57 (0.42-0.75) at 3 years223941ExteNET study (no pCR, HR+; treated within 12 months of trastuzumab receipt)0.61 (0.32-1.11) at 5 years265651Values in parentheses represent 95% confidence intervals. 2Hazard ratio not reported in clinical study publication. Estimated based on reported IDFS hazard ratio and proportion of distant to all invasive recurrences in ITT population. HR = hormone receptor; ITT = intent-to-treat; pCR = pathologic complete response.

Citation Format: David L Veenstra, Nathaniel Hendrix, Chantal M Dolan, Kathryn A Fisher, Deepa Lalla, Nina Oestreicher, Adam Brufsky. Estimating the long-term risk of recurrence in patients receiving HER2-targeted agents in HER2+ early-stage breast cancer (ESBC) [abstract]. In: Proceedings of the 2021 San Antonio Breast Cancer Symposium; 2021 Dec 7-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2022;82(4 Suppl):Abstract nr P2-11-19.

Clarifying the Trade-Offs of Risk-Stratified Screening for Prostate Cancer: A Cost-Effectiveness Study

Cancer risk prediction is necessary for precision early detection, which matches screening intensity to risk. However, practical steps for translating risk predictions to risk-stratified screening policies are not well established. We used a validated population prostate-cancer model to simulate the outcomes of strategies that increase intensity for men at high risk and reduce intensity for men at low risk. We defined risk by the Prompt Prostate Genetic Score (PGS) (Stratify Genomics, San Diego, California), a germline genetic test. We first recalibrated the model to reflect the disease incidence observed within risk strata using data from a large prevention trial where some participants were tested with Prompt PGS. We then simulated risk-stratified strategies in a population with the same risk distribution as the trial and evaluated the cost-effectiveness of risk-stratified screening versus universal (risk-agnostic) screening. Prompt PGS risk-adapted screening was more cost-effective when universal screening was conservative. Risk-stratified strategies improved outcomes at a cost of less than $100,000 per quality-adjusted life year compared with biennial screening starting at age 55 years, but risk stratification was not cost-effective compared with biennial screening starting at age 45. Heterogeneity of risk and fraction of the population within each stratum were also important determinants of cost-effectiveness.

Economic Evaluation of Urine-Based or Magnetic Resonance Imaging Reflex Tests in Men With Intermediate Prostate-Specific Antigen Levels in the United States

OBJECTIVES

For men with intermediate prostate-specific antigen (PSA) levels (4-10 ng/mL), urine-based biomarkers and multiparametric magnetic resonance imaging (MRI) are increasingly used as reflex tests before prostate biopsy. We assessed the cost effectiveness of these reflex tests in the United States.

METHODS

We used an existing microsimulation model of prostate cancer (PCa) progression and survival to predict lifetime outcomes for a hypothetical cohort of 55-year-old men with intermediate PSA levels. Urine-based biomarkers-PCa antigen (PCA3), TMPRSS2:ERG gene fusion (T2:ERG), and the MyProstateScore (MPS) for any PCa and for high-grade (Gleason score ≥7) PCa (MPShg)-were generated using biomarker data from 1112 men presenting for biopsy at 10 United States institutions. MRI results were based on published sensitivity and specificity for high-grade PCa. Costs and utilities were sourced from literature and Medicare reimbursement schedules. Outcome measures included life years, quality-adjusted life years (QALYs), and lifetime medical costs per patient. Incremental cost-effectiveness ratios were empirically calculated on the basis of simulated life histories under different reflex testing strategies.

RESULTS

Biopsying all men provided the most life years and QALYs, followed by reflex testing using MPShg, MPS, MRI, T2:ERG, PCA3, and biopsying no men (QALY range across strategies 15.98-16.09). Accounting for costs, MRI and MPShg were dominated by other strategies. PCA3, T2:ERG, and MPS were likely to be the most cost-effective strategy at willingness-to-pay thresholds of $100 000/QALY, $125 000/QALY, and $150 000/QALY, respectively.

CONCLUSIONS

Using PCA3, T2:ERG, or MPS as reflex tests has greater economic value than MRI, biopsying all men, or biopsying no men with intermediate PSA levels.

Artificial intelligence in breast cancer screening: primary care provider preferences

BACKGROUND

Artificial intelligence (AI) is increasingly being proposed for use in medicine, including breast cancer screening (BCS). Little is known, however, about referring primary care providers' (PCPs') preferences for this technology.

METHODS

We identified the most important attributes of AI BCS for ordering PCPs using qualitative interviews: sensitivity, specificity, radiologist involvement, understandability of AI decision-making, supporting evidence, and diversity of training data. We invited US-based PCPs to participate in an internet-based experiment designed to force participants to trade off among the attributes of hypothetical AI BCS products. Responses were analyzed with random parameters logit and latent class models to assess how different attributes affect the choice to recommend AI-enhanced screening.

RESULTS

Ninety-one PCPs participated. Sensitivity was most important, and most PCPs viewed radiologist participation in mammography interpretation as important. Other important attributes were specificity, understandability of AI decision-making, and diversity of data. We identified 3 classes of respondents: "Sensitivity First" (41%) found sensitivity to be more than twice as important as other attributes; "Against AI Autonomy" (24%) wanted radiologists to confirm every image; "Uncertain Trade-Offs" (35%) viewed most attributes as having similar importance. A majority (76%) accepted the use of AI in a "triage" role that would allow it to filter out likely negatives without radiologist confirmation.

CONCLUSIONS AND RELEVANCE

Sensitivity was the most important attribute overall, but other key attributes should be addressed to produce clinically acceptable products. We also found that most PCPs accept the use of AI to make determinations about likely negative mammograms without radiologist confirmation.

Differences in the Selection of Health State Utility Values by Sponsorship in Published Cost-Effectiveness Analyses

BACKGROUND

Health state utility values (HSUVs) are among the most influential attributes of cost-effectiveness analyses (CEAs). Our objective was to evaluate whether industry-funded studies select systematically different HSUVs as compared with studies without industry funding.

METHODS

Among 10 diseases with high disease burden in the United States, we further identified 31 progressive health states. We then searched the Tufts Medical Center's CEA Registry to identify studies that included HSUVs and were submitted to the registry between 2002 and 2019. Two reviewers mapped the free-text descriptions of health states onto the 31 predefined health states. We analyzed the effect of industry funding on the point estimates of these HSUVs with a beta regression. We also analyzed the difference between related health states within studies by funding source with a linear regression.

RESULTS

After identifying 26,222 HSUVs from 4198 CEAs, we matched 2573 HSUVs to the 31 predefined health states. We observed large variations within each health state: 12 of 31 health states included a range of HSUVs greater than 0.5. The point estimate model showed 1 statistically significant difference of 31 comparisons between studies with any industry funding and those without. The utility difference model found 3 significant differences out of 39 comparisons between CEAs with any industry funding and those without.

LIMITATIONS

Inclusion of unpublished CEAs may have affected our conclusions about the effect of industry funding on selection of HSUVs. We also relied on free-text descriptions of health states available in the CEA Registry and did not include adjustment for multiple comparisons.

CONCLUSION

Limited evidence exists that industry-funded studies select different HSUVs compared to non-industry-funded studies for the health states we considered.

Inferring antenatal care visit timing in low- and middle-income countries: Methods to inform potential maternal vaccine coverage

BACKGROUND

The timing of antenatal care (ANC) visits directly affect health intervention coverage and impact, especially for those interventions requiring strict gestational age windows for administration, such as maternal respiratory syncytial virus (RSV) vaccine. Existing nationally representative population-based surveys do not record the timing of ANC visits beyond the first, limiting the availability of reliable data around timing of subsequent ANC visits in most low- and middle-income countries (LMICs). Here, we describe a model that estimates the timing of ANC visits by gestational age using publicly available multi-country survey data.

METHODS AND FINDINGS

We used the Demographic and Health Surveys (DHS) data from 69 LMICs. We used several factors to estimate the timing of subsequent ANC visits by gestation age: the timing of the first ANC visit (ANC1) in a given pregnancy, derived from the DHS; the country's reported average ANC coverage at each ANC visit (ANC1 through the fourth ANC visit [ANC4]); and the World Health Organization's guidance on recommended ANC visit. We then used the timing of ANC visit by gestation age to predict the coverage of a potential maternal RSV vaccine administered at 24-36 weeks of gestation. We calculated the maternal immunization coverage by summing the number of eligible women vaccinated at any ANC visit divided by the total number of pregnant women. We find, in general, countries with higher ANC1 coverage were predicted to have higher vaccination coverage. In 82% of countries, the modeled vaccine coverage is less than ANC4 coverage.

CONCLUSIONS

The methods illustrated in this paper have implications on the precision of estimating impact and programmatic feasibility of time-critical interventions, especially for pregnant women. The methods can be easily adapted to vaccine demand forecasts models, vaccine impact assessments, and cost-effectiveness analyses and can be adapted to other maternal interventions that have administration timing restrictions.

Provider preferences for resolving uncertainty and avoiding harms in precision medicine: a discrete choice experiment

Background: Substantial uncertainty exists about how providers assess the value of genomic testing. Materials & methods: We developed and administered a discrete choice experiment to a national sample of providers. We analyzed responses using an error components mixed logit model. Results: We received responses from 356 providers. The attributes important to providers were patient health and function, life expectancy, cost, expert agreement, and biomarker prevalence. Providers significantly valued reducing uncertainty only when it eliminated the possibility of decreased life expectancy. Providers valued improving certainty about life expectancy gains from 12 ± 18 to 12 ± 6 months at US$400 (US$200-600) versus US$200 (-US$60-500) for 4 ± 4 to 4 ± 2 years. Conclusion: Providers value resolving uncertainty most when it eliminates the possibility of substantial harm.

[A woman with papules in a tattoo]

A healthy 21-year-old woman presented with multiple, 1-4 mm, skin-coloured papules following the ink pattern of a feather-shaped tattoo; this was diagnosed as a human-papilloma-virus induced cutaneous verrucae plana (flat wart). Minor trauma can result in auto-inoculation or pseudo-koebnerization; it is, therefore, important to not to tattoo through a wart to avoid spreading the disease.

Forecasting Demand for the Typhoid Conjugate Vaccine in Low- and Middle-income Countries

BACKGROUND

The World Health Organization (WHO) released a position paper in March 2018 calling for integration of a novel typhoid conjugate vaccine (TCV) into routine immunization along with catch-up campaigns for children up to age 15. Gavi, the Vaccine Alliance, has committed funding to help resource-constrained countries introduce this vaccine. In this article, the Typhoid Vaccine Acceleration Consortium forecasts demand if WHO recommendations are followed.

METHODS

We built a model of global TCV introductions between 2020 and 2040 to estimate the demand of the vaccine for 133 countries. We estimated each country's year of introduction by examining its estimated incidence of typhoid fever, its history of introducing new vaccines, and any knowledge we have of its engagement with typhoid prevention, including intention to apply for Gavi funding. Our model predicted use in routine infant vaccination as well as campaigns targeting varying proportions of the unvaccinated population up to 15 years of age.

RESULTS

Between 2020 and 2025, demand will predominantly come from African countries, many receiving Gavi support. After that, Asian countries generate most demand until 2030, when campaigns are estimated to end. Demand will then track the birth cohort of participating countries, suggesting an annual routine demand between 90 and 100 million doses. Peak demand is likely to occur between 2023 and 2026, approaching 300 million annual doses if campaign implementation is high.

CONCLUSIONS

In our analysis, target population for catch-up campaigns is the main driver of uncertainty. At peak demand, there is some risk of exceeding presently estimated peak production capacity. Therefore, it will be important to carefully coordinate introductions, especially when accompanied by campaigns targeting large proportions of the eligible population.

Provider preferences for attributes of artificial intelligence in breast cancer screening: A discrete choice experiment

e14118

Background: One of the emerging medical applications of artificial intelligence (AI) is the interpretation of mammograms for breast cancer screening. It is uncertain what attributes would result in acceptance of AI for breast cancer screening (AI BCS) among ordering clinicians. Methods: We performed qualitative interviews to identify the most important attributes of AI BCS for ordering clinicians. We then invited US-based primary care providers (PCPs) to participate in a discrete choice experiment (DCE). The experiment featured 15 choices between radiologist alone and two AI BCS alternatives where respondents traded better metrics on some attributes for worse metrics on others. Responses were analyzed using a mixed logit model adjusting for preference heterogeneity to determine the probability of recommending AI BCS. Results: In qualitative interviews, the six most important attributes to PCPs were AI sensitivity, specificity, radiologist involvement, understandability of AI decision-making, supporting evidence, and diversity of training data. Forty PCPs completed the DCE. Sensitivity was the most important attribute: a 4 percentage point improvement in sensitivity over the average radiologist increased the probability of recommending AI by 0.41 (95% confidence interval (CI), 0.38-0.42). Specificity was approximately half as important. Respondents were indifferent to whether radiologists confirmed all or only screens likely to be abnormal. However, no radiologist involvement reduced the probability of recommendation by 0.31 (95% CI, 0.29-0.31). An AI developed using data from diverse populations increased the probability of recommendation by 0.38 (95% CI, 0.36-0.39). Lastly, an AI that is transparent in the rationale for its decisions increased the probability of recommendation by 0.41 (95% CI, 0.39-0.41). Conclusions: PCPs prefer AI BCS that improves sensitivity versus specificity, and involves radiologists in the confirmation of abnormal screens. Improving sensitivity alone, however, will likely not be sufficient to support widespread PCP acceptance – algorithms will need to be developed with diverse data and more transparent explanations of their decisions.

Medication persistence of targeted immunomodulators for plaque psoriasis: A retrospective analysis using a U.S. claims database

PURPOSE

Studies of medication persistence in plaque psoriasis have shown inconsistent results, likely due to differing definitions of nonpersistence and of the permissible gap between refills. Also, medication persistence information for two recently approved drugs, apremilast and ixekizumab, is limited.

METHODS

We use the Truven Health MarketScan claims database to assess persistence for six drugs: adalimumab, apremilast, etanercept, ixekizumab, secukinumab, and ustekinumab. We define the permissible gap in three ways: 150 days for ustekinumab and 90 days for all other drugs (150/90 model); 120 days for all drugs (120 model); and twice the days' supply for all drugs (days' supply model). To estimate unadjusted persistence, we use Kaplan-Meier curves, and a proportional hazards model to estimate the adjusted risk of non-persistence.

RESULTS

Ustekinumab is most sensitive to changes in the definition of permissible gap, likely because of its longer maintenance dosing interval. Among targeted drug-experienced patients using ustekinumab, median persistence is 358 days (95% confidence interval: 343-371) in the 150/90 model and 189 days (179-199) in the days' supply model. Among targeted drug-experienced patients, median persistence in the days' supply model is longest for ixekizumab and secukinumab at 252 (217-301) and 222 (210-244) days, respectively. We also find that adjusted risk of nonpersistence increases by approximately 1% per year at treatment start.

CONCLUSION

The definition of permissible gap meaningfully changes both absolute and ordinal estimates of medication persistence. Each definition has unique limitations, which should be considered when interpreting persistence data.

Cost-Effectiveness of Targeted Pharmacotherapy for Moderate to Severe Plaque Psoriasis

BACKGROUND

Newer classes of targeted drugs for moderate to severe plaque psoriasis are more effective and more expensive than older classes, posing a difficult and potentially costly decision about whether to use them as initial targeted treatments.

OBJECTIVE

To estimate the clinical and economic outcomes of initial targeted treatment for the following drugs: adalimumab, etanercept, and infliximab (TNFα inhibitors); apremilast (PDE4 inhibitor); ustekinumab (IL-12/23 inhibitor); and ixekizumab, secukinumab, and brodalumab (IL-17 inhibitors).

METHODS

We developed a Markov model to simulate patient outcomes as measured by quality-adjusted life-years (QALYs) and health care costs over a 10-year period. We assumed that patients who fail initial targeted treatment either proceed to subsequent therapy or discontinue targeted treatment. Effectiveness estimates for initial treatment were defined as improvement in Psoriasis Area and Severity Index (PASI) from baseline and derived from a 2018 network meta-analysis. Wholesale acquisition drug costs were discounted by a class-specific, empirically derived rebate percentage off of 2016 costs. We conducted one-way and probabilistic sensitivity analyses to assess uncertainty in results.

RESULTS

The incremental benefits compared with no targeted treatment were, in descending order: ixekizumab 1.68 QALYs (95% credible range [CR] = 1.11-2.02), brodalumab 1.64 QALYs (95% CR = 1.08-1.98), secukinumab 1.51 QALYs (95% CR = 1.00-1.83), ustekinumab 1.43 QALYs (95% CR=0.94-1.74), infliximab 1.27 QALYs (95% CR = 0.89-1.55), adalimumab 1.15 QALYs (95% CR = 0.76-1.44), etanercept 0.97 QALYs (95% CR = 0.61-1.25), and apremilast 0.87 QALYs (95% CR = 0.52-1.17). Costs of care without targeted treatment totaled $66,451, and costs of targeted treatment ranged from $137,080 (apremilast) to $255,422 (ustekinumab). Probabilistic sensitivity analysis results indicated that infliximab and apremilast are likely to be the most cost-effective initial treatments at willingness-to-pay thresholds around $100,000 per QALY, while IL-17 drugs are more likely to be cost-effective at thresholds approaching $150,000 per QALY. Acquisition cost of the initial targeted drug and utility of clinical response were the most influential parameters.

CONCLUSIONS

Our findings suggest that initial targeted treatment with IL-17 inhibitors is the most effective treatment strategy for plaque psoriasis patients who have failed methotrexate and phototherapy. Apremilast, brodalumab, infliximab, ixekizumab, and secukinumab are cost-effective at different willingness-to-pay thresholds. Additional research is needed on whether the effectiveness of targeted agents changes when used after previously targeted agents.

DISCLOSURES

Funding for this study was contributed by the Institute for Clinical and Economic Review (ICER). Ollendorf, Chapman, Pearson, and Kumar are current employees, and Loos and Liu are former employees, of ICER, an independent organization that evaluates the evidence on the value of health care interventions, which is funded by grants from the Laura and John Arnold Foundation, Blue Shield of California Foundation, and the California HealthCare Foundation. ICER's annual policy summit is supported by dues from Aetna, AHIP, Anthem, Alnylam, AstraZeneca, Blue Shield of California, Cambia Health Solutions and MedSavvy, CVS Caremark, Editas, Express Scripts, Genentech, GlaxoSmithKline, Harvard Pilgrim Health Care, Health Care Service Corporation, OmedaRx, United Healthcare, Johnson & Johnson, Kaiser Permanente, Premera Blue Cross, Merck, National Pharmaceutical Council, Takeda, Pfizer, Novartis, Lilly, Humana, Prime Therapeutics, Sanofi, and Spark Therapeutics. Linder owns stock in Amgen, Biogen, and Eli Lilly; has contingent value rights in Sanofi Genzyme (related to alemtuzumab for multiple sclerosis); has received grant support from Astellas Pharma not related to this study and Clintrex, which was supported by AstraZeneca on an unrelated topic; and has received an honorarium from the Society of Healthcare Epidemiology of America (SHEA) as part of the SHEA Antimicrobial Stewardship Research Workshop Planning Committee, an educational activity supported by Merck. No other authors have potential conflicts of interest.

Integrating value of research into NCI Clinical Trials Cooperative Group research review and prioritization: A pilot study

Background

The Institute of Medicine has called for approaches to help maximize the return on investments (ROI) in cancer clinical trials. Value of Research (VOR) is a health economics technique that estimates ROI and can inform research prioritization. Our objective was to evaluate the impact of using VOR analyses on the clinical trial proposal review process within the SWOG cancer clinical trials consortium.

Methods

We used a previously developed minimal modeling approach to calculate VOR estimates for 9 phase II/III SWOG proposals between February 2015 and December 2016. Estimates were presented to executive committee (EC) members (N = 12) who determine which studies are sent to the National Cancer Institute for funding consideration. EC members scored proposals from 1 (best) to 5 based on scientific merit and potential impact before and after receiving VOR estimates. EC members were surveyed to assess research priorities, proposal evaluation process satisfaction, and the VOR process.

Results

Value of Research estimates ranged from −$2.1B to $16.46B per proposal. Following review of VOR results, the EC changed their score for eight of nine proposals. Proposal rankings were different in pre‐ vs postscores (P value: 0.03). Respondents had mixed views of the ultimate utility of VOR for their decisions with most supporting (42%) or neutral (41%) to the idea of adding VOR to the evaluation process.

Conclusions

The findings from this pilot study indicate use of VOR analyses may be a useful adjunct to inform proposal reviews within NCI Cooperative Clinical Trials groups.

Forecasting demand for maternal influenza immunization in low- and lower-middle-income countries

Immunization of pregnant women against seasonal influenza remains limited in low- and lower-middle-income countries despite being recommended by the World Health Organization (WHO). The WHO/PATH Maternal Influenza Immunization Project was created to identify and address obstacles to delivering influenza vaccines to pregnant women in low resource setting. To gain a better understanding of potential demand from this target group, we developed a model simulating pregnant women populations eligible for vaccination during antenatal care (ANC) services in all low- and lower-middle-income countries. We assessed potential vaccine demand in the context of both seasonal and year-round vaccination strategies and identified the ways that immunization programs may be affected by availability gaps in supply linked to current vaccine production cycles and shelf life duration. Results of our analysis, which includes 54 eligible countries in 2015 for New Vaccine Support from Gavi, the Vaccine Alliance, suggest the demand for influenza vaccines could be 7.7 to 16.0 million doses in 2020, and 27.0 to 61.7 million doses by 2029. If current trends in production capacity and actual production of seasonal influenza vaccines were to continue, global vaccine supply would be sufficient to meet this additional demand—although a majority of countries would face implementation issues linked to timing of supply.

The economic impact of childhood acute gastroenteritis on Malawian families and the healthcare system

OBJECTIVES

This prospective cohort study sought to estimate health system and household costs for episodes of diarrhoeal illness in Malawi.

SETTING

Data were collected in two Malawian settings: a rural health centre in Chilumba and an urban tertiary care hospital in Blantyre.

PARTICIPANTS

Children under 5 years of age presenting with diarrhoeal disease between 1 January 2013 and 21 November 2014 were eligible for inclusion. Illnesses attributed to other underlying causes were excluded, as were illnesses commencing more than 2 weeks prior to presentation. Complete data were collected on 514 cases at both the time of the initial visit to the participating healthcare facility and 6 weeks after discharge.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome measure was the total cost of an episode of illness. Costs to the health system were gathered from chart review (drugs and diagnostics) and actual hospital expenditure (staff and facility costs). Household costs, including lost income, were obtained by interview with the parents/guardians of patients.

RESULTS

Total costs in 2014 US$ for rural inpatient, rural outpatient, urban inpatient and urban outpatient were $65.33, $8.89, $60.23 and $14.51, respectively (excluding lost income). Mean household contributions to these costs were 15.8%, 9.8%, 21.3% and 50.6%.

CONCLUSION

This study found significant financial burden from childhood diarrhoeal disease to the healthcare system and to households. The latter face the risk of consequent impoverishment, as the study demonstrates how the costs of seeking treatment bring the income of the majority of families in all income strata below the national poverty line in the month of illness.

Characterizing Pharmacogenomic-Guided Medication Use With a Clinical Data Repository

The extent to which pharmacogenomic-guided medication use has been adopted in various health systems is unclear. To assess the uptake of pharmacogenomic-guided medication use, we determined its frequency across our health system, which does not have a structured testing program. Using a multisite clinical data repository, we identified adult patients' first prescribed medications between January 2011 and December 2013 and investigated the frequency of germline and somatic pharmacogenomic testing, by the Pharmacogenomics Knowledgebase level of the US Food and Drug Administration label information. There were 268,262 medication orders for drugs with germline pharmacogenomic testing information in their drug labels. Pharmacogenomic testing was detected for 1.5% (129/8,718) of medication orders with recommended or required testing. Of the 3,817 medication orders associated with somatic pharmacogenomic testing information in their drug labels, 20% (372/1,819) of required tests were detected. The low rates of detectable pharmacogenomic testing suggest that structured testing programs are required to achieve the success of precision medicine.

Impact of value of research analyses on SWOG’s clinical trial capsule scoring

e18311

Background: The Institute of Medicine has called for approaches to help maximize the return on research investments in cancer clinical trials. Value of Research (VOR) analysis is a health economics technique that estimates the clinical and economic returns for research investments and can inform research prioritization. Our objective was to evaluate the impact of VOR estimates on SWOG executive committee’s (EC) clinical trial proposal review scores. Methods: We used a previously developed minimal modeling approach to calculate per-patient and population-level (based on US cancer incidence) VOR estimates for 9 phase II and III SWOG research proposals between February 2015 and December 2016. VOR estimates were presented to EC members who determine which studies to submit to the National Cancer Institute (NCI) for funding approval. EC members scored proposals on scientific merit and potential impact before and after receiving VOR estimates. Scores ranged from 1 (best) to 5. We used the Wilcoxon signed rank test to evaluate the change in pre-post scores. Results: Study characteristics, VOR estimates, and pre-post scores are below. EC scores changed after receiving VOR estimates for 8 of 9 proposals. There was no association between VOR estimates and the magnitude of the change in proposal scores (all p > 0.2). Proposals with larger planned annual enrollment had larger changes in scores (0.3 more points [95% CI: 0.2, 0.56] per 100 patient increase; p = 0.04). Conclusions: Presenting VOR estimates influenced EC scores for SWOG trial proposals. VOR may add important data to inform reviews of trial proposals within NCI Cooperative Clinical Trials group settings. [Table: see text]

Synergistic anticholinergic and antiserotonergic effects in humans

Animal research suggests an important interactive role for ascending cholinergic and serotonergic systems in modulation of cerebral function. Employing a randomized, double-blind, crossover design, 11 healthy young adults were tested in each of four conditions: (1) placebo, (2) fenfluramine (a serotonin depleting agent), (3) scopolamine (a muscarinic antagonist), and (4) fenfluramine and scopolamine. P3 latency was slowed by the dual drug treatment to an extent greater than the sum of individual drug effects. EEG mean frequency was decreased by behavioral activation, and this decrease was reversed by the combined drug treatment but not by single drugs. In contrast, verbal memory, EEG alpha power, and P3 amplitude were significantly affected only by scopolamine. No drug effects were found for the N1 and P2 potentials. The results provide the first demonstration of combined anticholinergic and antiserotonergic effects in humans, and offer partial support to the concept of an interactive role of cholinergic and serotonergic systems in cerebral mechanisms.