Performance and usability testing of an automated tool for detection of peripheral artery disease using electronic health records

Peripheral artery disease (PAD) is a common cardiovascular disorder that is frequently underdiagnosed, which can lead to poorer outcomes due to lower rates of medical optimization. We aimed to develop an automated tool to identify undiagnosed PAD and evaluate physician acceptance of a dashboard representation of risk assessment. Data were derived from electronic health records (EHR). We developed and compared traditional risk score models to novel machine learning models. For usability testing, primary and specialty care physicians were recruited and interviewed until thematic saturation. Data from 3168 patients with PAD and 16,863 controls were utilized. Results showed a deep learning model that utilized time engineered features outperformed random forest and traditional logistic regression models (average AUCs 0.96, 0.91 and 0.81, respectively), P < 0.0001. Of interviewed physicians, 75% were receptive to an EHR-based automated PAD model. Feedback emphasized workflow optimization, including integrating risk assessments directly into the EHR, using dashboard designs that minimize clicks, and providing risk assessments for clinically complex patients. In conclusion, we demonstrate that EHR-based machine learning models can accurately detect risk of PAD and that physicians are receptive to automated risk detection for PAD. Future research aims to prospectively validate model performance and impact on patient outcomes.

Methods for improving efficiency in quality measurement: the example of pain screening

OBJECTIVE

Collecting unnecessary data when assessing quality of care wastes valuable resources. We evaluated three approaches for estimating quality-measure adherence and determined minimum visit data required to achieve accurate estimates.

DESIGN

We abstracted medical records for calculating physician-level pain screening rates as: visit-specific, using single-visit data for each patient; visit-level average, using data for all patients and visits; and patient-level average, using data from a subset of patients and visits.

SETTING

VA Greater Los Angeles Health-care System, 2006.

PARTICIPANTS

One hundred and six patients with Stage IV solid tumors.

INTERVENTION

Pain screening at every medical encounter, measured by a 0-10 numeric rating scale and reported to the national Medicare insurance program under a 'pay-for-reporting' program.

MAIN OUTCOME MEASURES

Amount of visit data needed to reach the smallest 95% confidence interval (CI) and stable pain screening estimates.

RESULTS

Pain screening occurred at 22% (23/106; 95% CI: 14-30%) of initial visits and 50% (8/16; 95% CI: 25-75%) of single visits. Across all visits, screening adherence averaged 34% when estimated at the visit-level precision and 30% at the patient level. Maximum patient-level precision was reached at visit 4 (95% CI: ± 8%) and visit level at visit 14 (95% CI: ± 6%). Using patient-level and visit-level approaches, estimates stabilized at visits 8 and 11, respectively, and reached within 1 percentage point of the steady-state value at visits 4 and 9.

CONCLUSION

To address low-pain screening among cancer patients, an oncology pain screening measure may be most efficiently evaluated with data from a sample of patients and visits. This approach may be valid for visit-level quality measures in other settings.

Care of vaginal symptoms among HIV-infected women

OBJECTIVE

Gynecologic disease is common in HIV-infected women. We examine the sociodemographic, clinical, and provider factors associated with the care of women with vaginal symptoms.

METHODS

Women enrolled in the HIV Cost and Services Utilization Study (HCSUS), a nationally representative probability sample of HIV-infected adults, were interviewed between January 1996 and April 1997. Women with vaginal symptoms who sought medical attention were asked, "Did your health care provider examine your vaginal area?" Women were also asked if they received medication for their symptoms.

RESULTS

Among 154 women with vaginal symptoms, 127 sought care for their symptoms. Of those who sought care, 48% saw a gynecologist and 52% sought care from nongynecologists, most often their usual HIV care provider. Women who saw a gynecologist for their symptoms were more likely to have received a pelvic examination (92% versus 76%; p =.06) and vaginal fluid collection (98% versus 88%; p =.06) than those who saw their regular HIV provider. Fifteen percent of women received medication for their symptoms without having a pelvic examination; gynecologists were less likely to prescribe without an examination (8% versus 21%; p =.12).

CONCLUSION

Gynecologists are more likely to provide adequate care of vaginal symptoms among HIV-infected women than nongynecologists who were HIV care providers. This specialty difference is consistent with quality of care studies for other medical conditions, but the potential gynecologic complications of inadequate evaluation and treatment warrants further investigation.

Problems in recruiting community-based physicians for health services research

OBJECTIVE

To qualitatively determine factors that are associated with higher participation rates in community-based health services research requiring significant physician participation burden.

MEASUREMENTS

A review of the literature was undertaken using MEDLINE and the Social Science Research Index to identify health services research studies that recruited large community-based samples of individual physicians and in which the participation burden exceeded that of merely completing a survey. Two reviewers abstracted data on the recruitment methods, and first authors were contacted to supplement published information.

MAIN RESULTS

Sixteen studies were identified with participation rates from 2.5% to 91%. Almost all studies used physician recruiters to personally contact potential participants. Recruiters often knew some of the physicians to be recruited, and personal contact with these "known" physicians resulted in greater participation rates. Incentives were generally absent or modest, and at modest levels, did not appear to affect participation rates. Investigators were almost always affiliated with academic institutions, but were divided as to whether this helped or hindered recruitment. HMO-based and minority physicians were more difficult to recruit. Potential participants most often cited time pressures on staff and themselves as the study burden that caused them to decline.

CONCLUSIONS

Physician personal contact and friendship networks are powerful tools for recruitment. Participation rates might improve by including HMO and minority physicians in the recruitment process. Investigators should transfer as much of the study burden from participating physicians to project staff as possible.

National estimates of HIV-related symptom prevalence from the HIV Cost and Services Utilization Study

OBJECTIVES

The objectives of this study were (1) to estimate the prevalence, bothersomeness, and variation of HIV-related symptoms in a nationally representative sample of HIV-infected adults receiving medical care and (2) to evaluate new aggregate measures of symptom frequency and bothersomeness.

METHODS

Beginning in January 1996, 76% of a multistage national probability sample of 4,042 HIV-infected adults receiving medical care were interviewed. Participants endorsed the presence and degree of bothersomeness of 14 HIV-related symptoms during the preceding 6 months. Sex-standardized symptom number and bothersomeness indices were constructed. After sampling weights were incorporated, symptom distributions were compared according to selected characteristics by analysis of variance and multiple linear regression modeling.

RESULTS

Prevalence of specific symptoms in the reference population was as follows: fever/night sweats, 51.1%; diarrhea, 51%; nausea/anorexia, 49.8%; dysesthesias, 48.9%; severe headache, 39.3%; weight loss, 37.1%; vaginal symptoms, 35.6% of women; sinus symptoms, 34.8%; eye trouble, 32.4%; cough/dyspnea, 30.4%; thrush, 27.3%; rash, 24.3%; oral pain, 24.1%; and Kaposi's sarcoma, 4%. Aggregate measures were reliable (Cronbach's alpha > or =0.75) and demonstrated construct validity when compared with other measures of disease severity. After adjustment for CD4 count, both symptom number and bothersomeness varied significantly (P <0.05) by teaching status of care setting, exposure/risk group, educational achievement, sex, annual income, employment, and insurance category. However, the magnitude of variation was small. Symptoms were greatest in women and injection drug users, as well as in persons with lower educational levels, lower income, and Medicare enrollment or those who were followed up at teaching hospitals.

CONCLUSIONS

The prevalence and bothersomeness of HIV-related symptoms are substantial and vary by setting of care and patient characteristics.

Access of vulnerable groups to antiretroviral therapy among persons in care for HIV disease in the United States. HCSUS Consortium. HIV Cost and Services Utilization Study

OBJECTIVE

To employ the behavioral model of health services use in examining the extent to which predisposing, enabling, and need factors explain the treatment of the HIV-positive population in the United States with highly active antiretroviral therapy (HAART).

DATA SOURCE

A national probability sample of 2,776 adults under treatment for human immunodeficiency virus (HIV) infection.

STUDY DESIGN

The article uses data from the baseline and six-month follow-up surveys. The key independent variables describe vulnerable population groups including women, drug users, ethnic minorities, and the less educated. The dependent variable is whether or not a respondent received HAART by December 1996.

DATA COLLECTION

All interviews were conducted using computer-assisted personal interview instruments designed for this study. Ninety-two percent of the baseline interviews were conducted in person and the remainder over the telephone.

PRINCIPAL FINDINGS

A multistage logit regression shows that the predisposing factors that have previously described vulnerable groups in the general population with limited access to medical care also define HIV-positive groups who are less likely to gain early access to HAART including women, injection drug users, African Americans, and the least educated (odds ratios, controlling for need, ranged from 0.35 to 0.59).

CONCLUSIONS

Those HIV-positive persons with the greatest need (defined by a low CD4 count) are most likely to have early access to HAART, which suggests equitable access. However, some predisposing and enabling variables continue to be important as well, suggesting inequitable access, especially for African Americans and lower-income groups. Policymakers and clinicians need to be sensitized to the continued problems of African Americans and other vulnerable populations in gaining access to such potentially beneficial therapies. Higher income, anonymous test sites, and same-day appointments are important enabling resources.

Quality Enhancement Research Initiative for human immunodeficiency virus/acquired immunodeficiency syndrome: framework and plan. HIV-QUERI Executive Committee

The Veterans Health Administration (VHA) sees approximately equal to 17,000 human immunodeficiency virus (HIV)-infected patients each year, which makes it the largest provider of HIV care in the United States. HIV causes chronic progressive disease that leads to early death. Newer combination antiretro viral treatments are effective but expensive and difficult to use. The HIV Quality Enhancement Research Initiative (HIV-QUERI) uses the QUERI process to identify high-risk and high-volume populations (step 1), which includes those already under VHA care for HIV, those who do not know of their infection, and those at risk for HIV. In identifying best practices (step 2), the HIV-QUERI will benefit greatly from existing guidelines for the care of established HIV infection, but gaps in knowledge regarding adherence to medication regimens and cost-effective screening are large. To identify existing practice patterns (step 3), the HIV-QUERI will develop a clean analytic data set based on Immunology Case Registry files and expand it through a survey of veterans. Interventions to improve care (step 4) will include national, regional, and site-specific feedback on performance relative to quality standards, as well as patient-level and provider-level interventions to improve adherence and support medical decision-making. To document that best practices improve outcomes and quality of life (steps 5 and 6), HIV-QUERI will track indicators on an ongoing basis by use of the Immunology Case Registry database and possible future waves of the survey. In addition, we will require that these issues be addressed in evaluations of HIV-QUERI interventions. In the present article, we present these steps within a framework and plan.

Performance measures for guidelines on preventing opportunistic infections in patients infected with human immunodeficiency virus

This article serves as a complement to the 1999 US Public Health Service/Infectious Diseases Society of America guidelines on the prevention of opportunistic infections in persons infected with HIV, published in this issue of Clinical Infectious Diseases [1]. A number of performance measures to assess compliance with the guidelines and to aid in their implementation are proposed.

The communicable disease impact of eliminating publicly funded prenatal care for undocumented immigrants

OBJECTIVES

In 1996, California proposed regulations to eliminate publicly funded prenatal care for undocumented immigrants. Prenatal treatment of sexually transmitted infections (STIs) can prevent STI-related adverse outcomes of pregnancy (AOP). The study assessed the STI-related health and economic impact of the proposed regulations in Los Angeles County (LAC).

METHODS

We modeled excess STI-related AOPs and associated costs that would occur in LAC as if the regulations were implemented in 1995. Using attributable fractions in the exposed for five STIs and their associated AOPs, we calculated excess STI-related AOPs and their costs that would result from the regulations and the degree to which excess morbidity would offset gross savings.

RESULTS

The model indicates that, depending on regulatory level, 74, 110, or 132 excess AOPs would occur subsequent to the regulations, representing lost prevented morbidity. These excess AOPs would cost $5.1, $7.6, or $9.2 million dollars in direct medical expenses, offsetting anticipated savings by 19.2%, 29.0%, or 34.9%. This analysis does not include other costs of these STIs or costs associated with non-STI-related sequelae of diminished prenatal care, all of which could further offset anticipated savings.

CONCLUSIONS

The proposed regulations would likely lead to increased STI-related morbidity and costs, thereby offsetting anticipated savings. Health departments are in a special position to promptly respond to policy issues affecting vulnerable populations. The development of a practical and rational framework for local-level policy assessment can be important for encouraging good scientific approaches that respond to calls for reductions in basic preventive health services.

Tuberculosis in homeless patients: potential for case finding in public emergency departments

STUDY OBJECTIVES

Previous studies have had difficulty evaluating the optimal clinical site for screening homeless patients for active tuberculosis (TB). We hypothesized that homeless patients with TB would not frequently reside in shelters at the time of their diagnosis and would be more likely than other patients with TB to seek care in public hospitals, thus presenting an opportunity for screening radiography.

METHODS

This registry-based survey included 743 consecutive patients with confirmed active TB in Los Angeles County. No therapeutic intervention was involved.

RESULTS

When compared with patients with TB who were not homeless, homeless patients with TB were more likely to be male (93% versus 63%, P<.001), black (44% versus 15%, P<.001), living in the inner city (55% versus 7%, P<.001), and born in the United States (67% versus 32%, P<.001). They were more infectious than other patients with TB as evidenced by a trend toward more cavitary radiographic lesions (24% versus 16%, P=.11) and significantly more positive sputum smears (56% versus 41%, P=.009). Less than a third lived in congregate facilities such as shelters at the time of their diagnosis. Instead, their disease was diagnosed more often at county hospitals (54% versus 23%, P<.001) than patients with TB who were not homeless.

CONCLUSION

Widespread screening for TB in shelters may miss most homeless patients with TB. Because most county-hospital homeless patients with TB initially present to emergency departments and many do not live in shelters, future cost-effectiveness studies should evaluate chest radiograph screening for all homeless ED patients.

Why do symptomatic patients delay obtaining care for tuberculosis?

The resurgence of tuberculosis (TB) has coincided with deteriorating access to care for high-risk populations. We sought to determine what perceived access barriers delayed symptomatic TB patients from obtaining care. In order to do this, we conducted a survey in Los Angeles County, California, using a consecutive sample of patients with active TB as confirmed by the county TB control authority. The measures used in the study were a self-reported delay in seeking care of more than 60 d from symptom onset, a period sufficient to cause skin-test conversion in exposed contacts, and self-reported access barriers. The county TB registry provided supplementary clinical data. We found that one in five of the 248 symptomatic respondents (response rate: 60%) delayed obtaining care for > 60 d (mean = 74 d, SD = 216 d). During the delay, patients exposed an average of eight contacts. As compared with the rest of the sample, delay was more common in those who were unemployed (25% versus 14%), concerned about cost (27% versus 14%), anticipated prolonged waiting-room time (26% versus 14%), believed they could treat themselves (31% versus 14%), anticipated difficulty in getting an appointment (28% versus 16%), were uncertain about where to get care (33% versus 16%), and feared immigration authorities (47% versus 18%) (p < 0.05). Logistic regression revealed that uncertainty about where to get care, unemployment, and belief in the efficacy of self-treatment independently predicted delay > 60 d. Illness severity as measured by chest radiography, sputum smears, and symptoms had little impact on delay. We conclude that because access variables such as lack of employment and knowledge about where to obtain care were more closely associated with clinically significant delay than was severity of illness, these results raise concerns about the equity of access to care among TB patients. The results suggest that improving the availability of services for high-risk groups may substantially reduce TB patients' delay in obtaining care, and thus may limit the spread of the disease.

Defining and implementing medical necessity in Washington State and Oregon

This paper reports on a qualitative study of how health care providers in the states of Washington and Oregon define and implement medical necessity. Based on a series of semi-structured interviews, we found that few insurers or health care plans in our sample attempted to resolve the ambiguities inherent in defining medical necessity. More importantly, our results suggest that physicians in managed care plans were not using general definitions of medical necessity to make clinical decisions, but instead relied on utilization management techniques to guide the use of medical resources. We conclude that medical necessity as an organizing principle for clinical practice decision making is likely to continue to erode in a managed care environment.

Medical necessity and defined coverage benefits in the Oregon Health Plan

The policy debate in Oregon has primarily focused on the Prioritized List of Services. However, little information is available on how defined coverage benefits and managed care affect the role of medical necessity in determining care for Medicaid patients. This issue is important because medical necessity determinations are currently used by many states to limit extraneous health care costs but require resource-intensive oversight, are open to wide variance, and frequently prompt litigation challenging interpretations of what is necessary and what is not. The qualitative study described here addressed whether medical necessity remains a salient and useful concept in the Oregon Health Plan. Our results indicate that defined coverage benefits, as described by the funded portion of the Prioritized List of Services, supplant medical necessity determinations for coverage, while managed care incentives limit the need for medical necessity determinations at the provider level. Clinical choices are, for the most part, guided by providers' judgment within the financial constraints of capitation and by targeted use management techniques. The combination of capitated care and Oregon's defined coverage benefits package has marginalized the use of medical necessity, albeit with consequences for state oversight of Medicaid services.

Do we understand the effects of 'managed care' in ophthalmology? A review and analysis

Little has been published that directly assesses the effect of structures for providing managed care or the effects of capitated, prepaid financing on the cost and quality of eye care services. Managed care organizations use fewer ophthalmologists and may provide more screening for diabetic retinopathy. Studies of nonophthalmologic care show lower patient satisfaction with care, and mixed effects on cost, quality of care, and access to care, but are difficult to generalize to eye care. We reviewed the published peer-reviewed literature about this topic. Notable gaps exist in the knowledge of critical elements of the influence of managed care on providing eye care and on patient outcomes. Existing measures of quality, cost, satisfaction, and access could easily be adapted for use in evaluating the influence of managed care and guiding health care policy.

Effect of California's proposition 187 on the use of primary care clinics

California's ballot proposition 187, passed by the voters in November 1994, threatened to discontinue undocumented immigrants' eligibility for most health services while mandating that health care professionals report suspected undocumented patients to authorities. Although the proposition has not been put into practice, reports suggest that its passage was associated with a decline in the use of health services by some groups. To assess the effects of the passage of Proposition 187 on the use of primary care services, we surveyed a representative sample of California clinics serving low-income groups (n = 129). Using a mailed questionnaire and phone interviews with clinic directors, we obtained qualitative and quantitative data regarding the effects of the passage of the proposition on clinic use. Among primary care clinics statewide and clinics serving predominantly Latino patients, we detected no significant decline in total monthly visits following the election. Nevertheless, half of clinic directors (51%) thought that the number of clinic visits declined after the passage of Proposition 187, and many directors thought that the deterrent effects of the election persisted for weeks to months after the election. Whereas the number of visits probably declined at some clinics, only a small minority of patients at most primary care clinics could have been deterred from seeking care after the passage of Proposition 187.

Potential impact of restricting STD/HIV care for immigrants in Los Angeles County

Legislative restrictions in immigrants' access to health care and local governmental funding shortfalls in the US and Western Europe have raised fears that public clinic patients might delay care for communicable diseases. To help quantify the potential impact of both policies on public clinics providing sexually transmitted disease (STD) services, we surveyed 234 patients from five LA clinics regarding their alternative sources of health care. Of the 215 providing complete information (response rate = 91%), 52 (24%) reported they had no legal rights to reside in the US. Compared to the legal resident control group, illegal immigrants were more likely to indicate that they had no alternative access to medical care (27% vs 44%; P = 0.03). We conclude that for a substantial proportion of patients, particularly illegal immigrants, the STD clinics are indeed essential. Restricting access to these clinics may have unpredictable public health consequences.

Relationship of isoniazid resistance to human immunodeficiency virus infection in patients with tuberculosis

To investigate the relationship between isoniazid resistance and HIV infection in patients with tuberculosis, we evaluated data in the Los Angeles County tuberculosis registry on 1,506 patients for whom drug susceptibility results were available. Among 235 HIV-infected patients, isoniazid resistance was less common than in 1,271 patients who were HIV-seronegative or who had not been tested for HIV, with an unadjusted odds ratio of 0.3. After adjustment for other factors that affect drug resistance (ethnicity, country of birth, prior diagnosis of tuberculosis, and cavitation), the frequency if isoniazid resistance remained lower than that in patients without HIV infection, with an odds ratio of 0.4 (95% confidence interval, 0.2 to 0.8; p = 0.02). We conclude that in Los Angeles, a setting where there is no ongoing outbreak of drug-resistant tuberculosis, isoniazid-resistant tuberculosis is not more common in HIV-infected patients.

Does fear of immigration authorities deter tuberculosis patients from seeking care?

Physician groups are concerned that legislation requiring physicians to report illegal immigrants to immigration authorities will delay curative care. In particular, patients with tuberculosis may delay seeking care for infectious symptoms and spread the disease. We surveyed 313 consecutive patients with active tuberculosis from 95 different facilities to examine the relationship of immigration-related variables, symptoms, and delay in seeking care. Most patients (71%) sought care for symptoms rather than as a result of the efforts of public health personnel to screen high-risk groups or to trace contacts of infectious persons. At least 20% of respondents lacked legal documents allowing them to reside in the United States. Few (6%) feared that going to a physician might lead to trouble with immigration authorities. Those who did were almost 4 times as likely to delay seeking care for more than 2 months, a period of time likely to result in disease transmission. Patients potentially exposed an average of 10 domestic and workplace contacts during the course of the delay. Any legislation that increases undocumented immigrants' fear that health care professionals will report them to immigration authorities may exacerbate the current tuberculosis epidemic.

Proposal for an administrative curriculum in pediatric emergency medicine fellowship training

Administrative tasks make up a significant component of the practice of pediatric emergency medicine (PEM) physicians. Our survey of 10 academic pediatric emergency departments revealed that PEM physicians who are primarily clinical spent an average of 15% of their time on administrative tasks, and PEM physicians whose positions are administrative as well as clinical spent 30 to 60% of their time on administrative tasks. Of the 101 programs responding to our survey of 220 pediatric residency programs, 80% did not address hospital administrative issues, and many that did address these issues allowed these topics only one hour of presentation time per year. It is clear that there is a discrepancy between the demands placed upon PEM physicians to perform administrative tasks and the sparse or nonexistent opportunities for learning about administrative issues during residency training. It is incumbent upon pediatric emergency fellowship programs to provide an inclusive and well-structured administrative curriculum for their trainees. This article suggests a framework for such a curriculum.

Are insulin and hypertension independently related?

Hyperinsulinemia has been proposed as the common pathogenetic mechanism of obesity, non-insulin-dependent diabetes mellitus, and hypertension. We examined the cross-sectional relationship between fasting and postchallenge insulin levels and hypertensive status in a population-based study of 653 men and 784 women, aged 50 to 93 years, in Rancho Bernardo, California. Hypertensive subjects had slightly but not significantly higher fasting plasma insulin levels than did normotensive subjects, but significantly higher postchallenge insulin levels. After stratification for obesity and glucose intolerance, there was no significant difference between the mean age-adjusted fasting or postchallenge insulin levels of those with and those without hypertension in 12 possible subgroups. Analysis of variance of both fasting and postchallenge insulin levels also failed to reveal a significant relationship between insulin and hypertensive status after adjusting for age, sex, body mass index, and diabetes. The current study does not support the hypothesis that insulin is independently associated with hypertension.

Occupational risk of human parvovirus B19 infection for school and day-care personnel during an outbreak of erythema infectiosum

Human parvovirus B19, the cause of erythema infectiosum, has recently been associated with adverse fetal outcomes. During a large outbreak of erythema infectiosum in Connecticut, a survey was conducted on 571 (90%) of 634 school and day-care personnel to determine the risk of acquiring B19 infection. Serologic evidence of B19 infection was determined by using an enzyme-linked immunosorbent assay. Of the school and day-care personnel, 58% had evidence of previous B19 infection. The minimal rate of B19 infection in susceptible personnel during the outbreak was 19%. The risk was increased for teachers and day-care providers who had contact with younger children and with greater numbers of ill children. These results suggest that B19 infection is an occupational risk for school and day-care personnel.