Model-Based ROC Curve: Examining the Effect of Case Mix and Model Calibration on the ROC Plot

Background

The performance of risk prediction models is often characterized in terms of discrimination and calibration. The receiver-operating characteristic (ROC) curve is widely used for evaluating model discrimination. However, when comparing ROC curves across different samples, the effect of case mix makes the interpretation of discrepancies difficult. Further, compared with model discrimination, evaluating model calibration has not received the same level of attention. Current methods for examining model calibration require specification of smoothing or grouping factors.

Methods

We introduce the “model-based” ROC curve (mROC) to assess model calibration and the effect of case mix during external validation. The mROC curve is the ROC curve that should be observed if the prediction model is calibrated in the external population. We show that calibration-in-the-large and the equivalence of mROC and ROC curves are together sufficient conditions for the model to be calibrated. Based on this, we propose a novel statistical test for calibration that, unlike current methods, does not require any subjective specification of smoothing or grouping factors.

Results

Through a stylized example, we demonstrate how mROC separates the effect of case mix and model miscalibration when externally validating a risk prediction model. We present the results of simulation studies that confirm the properties of the new calibration test. A case study on predicting the risk of acute exacerbations of chronic obstructive pulmonary disease puts the developments in a practical context. R code for the implementation of this method is provided.

Conclusion

mROC can easily be constructed and used to interpret the effect of case mix and calibration on the ROC plot. Given the popularity of ROC curves among applied investigators, this framework can further promote assessment of model calibration.

Highlights

Monitoring of Nitrification in Chloraminated Drinking Water Distribution Systems With Microbiome Bioindicators Using Supervised Machine Learning

Many drinking water utilities in the United States using chloramine as disinfectant treatment in their drinking water distribution systems (DWDS) have experienced nitrification episodes, which detrimentally impact the water quality. Identification of potential predictors of nitrification in DWDS may be used to optimize current nitrification monitoring plans and ultimately helps to safeguard drinking water and public health. In this study, we explored the water microbiome from a chloraminated DWDS simulator operated through successive operational schemes of stable and nitrification events and utilized the 16S rRNA gene dataset to generate high-resolution taxonomic profiles for bioindicator discovery. Analysis of the microbiome revealed both an enrichment and depletion of various bacterial populations associated with nitrification. A supervised machine learning approach (naïve Bayes classifier) trained with bioindicator profiles (membership and structure) were used to classify water samples. Performance of each model was examined using the area under the curve (AUC) from the receiver-operating characteristic (ROC) and precision-recall (PR) curves. The ROC- and PR-AUC gradually increased to 0.778 and 0.775 when genus-level membership (i.e., presence and absence) was used in the model and increased significantly using structure (i.e., distribution) dataset (AUCs = 1.000, p < 0.01). Community structure significantly improved the predictive ability of the model beyond that of membership only regardless of the type of data (sequence- or taxonomy-based model) we used to represent the microbiome. In comparison, an ATP-based model (bulk biomass) generated a lower AUCs of 0.477 and 0.553 (ROC and PR, respectively), which is equivalent to a random classification. A combination of eight bioindicators was able to correctly classify 85% of instances (nitrification or stable events) with an AUC of 0.825 (sensitivity: 0.729, specificity: 0.894) on a full-scale DWDS test set. Abiotic-based model using total Chlorine/NH₂Cl and NH₃ generated AUCs of 0.740 and 0.861 (ROC and PR, respectively), corresponding to a sensitivity of 0.250 and a specificity of 0.957. The AUCs increased to > 0.946 with the addition of NO₂^– concentration, which is indicative of nitrification in the DWDS. This research provides evidence of the feasibility of using bioindicators to predict operational failures in the system (e.g., nitrification).

Association Between the Cerebral Autoregulation Index (Pressure Reactivity), Patient's Clinical Outcome, and Quality of ABP(t) and ICP(t) Signals for CA Monitoring

Background and Objectives: The aim of this study was to explore the association between the cerebral autoregulation (CA) index, the pressure reactivity index (PRx), the patient’s clinical outcome, and the quality of arterial blood pressure (ABP(t)) and intracranial blood pressure (ICP(t)) signals by comparing two filtering methods to derive the PRx. Materials and Methods: Data from 60 traumatic brain injury (TBI) patients were collected. Moving averaging and FIR (Finite Impulse Response) filtering were performed on the ABP(t) and ICP(t) signals, and the PRx was estimated from both filtered datasets. Sensitivity, specificity, and receiver-operating characteristic (ROC) curves with the area under the curves (AUCs) were determined using patient outcomes as a reference. The outcome chosen for comparison among the two filtering methods were mortality and survival. Results: The FIR filtering approach, compared with clinical outcome, had a sensitivity of 70%, a specificity of 81%, and a level of significance p = 0.001 with an area under the curve (AUC) of 0.78. The moving average filtering method compared with the clinical outcome had a sensitivity of 58%, a specificity of 72%, and a level of significance p = 0.054, with an area under the curve (AUC) of 0.66. Conclusions: The FIR (optimal) filtering approach was found to be more sensitive for discriminating between two clinical outcomes, namely intact (survival) and impaired (death) cerebral autoregulation for TBI treatment decision making.

Reliability and accuracy of individual Alberta Stroke Program Early CT Score regions using a medical and a smartphone reading system in a telestroke network

Introduction

The aim of this study was to assess individual regions of the Alberta Stroke Program Early CT Score in noncontrast head computed tomography interpretations using a smartphone in a telestroke network, by comparison to a medical monitor.

Methods

The review board of our institution approved this retrospective study. A factorial design with 188 patients, four radiologists and two reading systems was used. Accuracy and reliability were evaluated.

Results

Very good interobserver agreements were observed on the total Alberta Stroke Program Early CT Score for both the medical and smartphone reading systems, with intraclass correlation coefficients of 0.91 and 0.84 respectively. Interobserver agreements were moderate to very good for the medical reading system (all intraclass correlation coefficients >0.74), whereas they were fair to very good for the smartphone (intraclass correlation coefficients ranged from 0.31–0.81). All intraobserver agreements were good (intraclass correlation coefficient >0.64), except for internal capsule (0.48) and M2 (0.55) regions. The areas under the receiver-operating curve ranged from 0.69–0.89 for the medical system, while for the smartphone ranged from 0.44–0.86. No statistical differences were observed between medical and smartphone reading systems for each region (all p > 0.05).

Discussion

If radiologists are better trained in the evaluation of the lesions in the insula, the internal capsule and the M2 regions, the total and the dichotomised Alberta Stroke Program Early CT Score will be more precise. Hence, ruling out contraindications to thrombolysis administration will be improved, allowing assessment of head computed tomography in a telestroke network using a smartphone to be a common practice.

Generalized ROC methods for immunogenicity data analysis of vaccine phase I studies in a seropositive population

Immunogenicity data from phase 1 vaccine studies can be difficult to interpret, especially in seropositive populations and when multiple assays are used. We developed 3 statistical methods (Youden index [YI] threshold, receiver-operating characteristic relative to baseline [ROC-B], and ROC of postdose levels [ROC-P]) to characterize complex immunogenicity data by assessing the proportion of a study population that achieved values above thresholds. The YI method calculates a single threshold per assay. Both ROC methods construct ROC curves for individual assays and surfaces for assay combinations to assess degree of separation of postdose values from a reference distribution; the ROC-B method uses overall predose values as the reference distribution and the ROC-P method uses pooled postdose values. All methods are applicable to a seropositive population with overlapping distributions of baseline and postdose measurements and can evaluate results of multiple assays jointly. The ROC-P method is also applicable when postdose levels are fully separated from baseline levels, as is common in a seronegative population. These methods were demonstrated using data from a phase 1a study of respiratory syncytial virus vaccines formulated with and without an adjuvant in a seropositive population of adults aged ≥60 years. All 3 methods provided a comprehensive assessment of vaccine immunogenicity effects with results presented in easily interpretable formats. In the example data, the methods demonstrated antigen dose response trend and contribution of adjuvant to response in multiple assays individually and jointly where optimal responses in assay combinations (humoral and cellular) are important.

Effect of Reduced Tube Voltage on Diagnostic Accuracy of CT Colonography

The normal tube voltage in computed tomography colonography (CTC) is 120 kV. Some reports indicate that the use of a low tube voltage (lower than 120 kV) technique plays a significant role in reduction of radiation dose. However, to determine whether a lower tube voltage can reduce radiation dose without compromising diagnostic accuracy, an evaluation of images that are obtained while maintaining the volume CT dose index (CTDI_vol) is required. This study investigated the effect of reduced tube voltage in CTC, without modifying radiation dose (i.e. constant CTDI_vol), on image quality. Evaluation of image quality involved the shape of the noise power spectrum, surface profiling with volume rendering (VR), and receiver operating characteristic (ROC) analysis. The shape of the noise power spectrum obtained with a tube voltage of 80 kV and 100 kV was not similar to the one produced with a tube voltage of 120 kV. Moreover, a higher standard deviation was observed on volume-rendered images that were generated using the reduced tube voltages. In addition, ROC analysis revealed a statistically significant drop in diagnostic accuracy with reduced tube voltage, revealing that the modification of tube voltage affects volume-rendered images. The results of this study suggest that reduction of tube voltage in CTC, so as to reduce radiation dose, affects image quality and diagnostic accuracy.

Diagnostic accuracy of stress perfusion CMR in comparison with quantitative coronary angiography: fully quantitative, semiquantitative, and qualitative assessment

OBJECTIVES

This study's primary objective was to determine the sensitivity, specificity, and accuracy of fully quantitative stress perfusion cardiac magnetic resonance (CMR) versus a reference standard of quantitative coronary angiography. We hypothesized that fully quantitative analysis of stress perfusion CMR would have high diagnostic accuracy for identifying significant coronary artery stenosis and exceed the accuracy of semiquantitative measures of perfusion and qualitative interpretation.

BACKGROUND

Relatively few studies apply fully quantitative CMR perfusion measures to patients with coronary disease and comparisons to semiquantitative and qualitative methods are limited.

METHODS

Dual bolus dipyridamole stress perfusion CMR exams were performed in 67 patients with clinical indications for assessment of myocardial ischemia. Stress perfusion images alone were analyzed with a fully quantitative perfusion (QP) method and 3 semiquantitative methods including contrast enhancement ratio, upslope index, and upslope integral. Comprehensive exams (cine imaging, stress/rest perfusion, late gadolinium enhancement) were analyzed qualitatively with 2 methods including the Duke algorithm and standard clinical interpretation. A 70% or greater stenosis by quantitative coronary angiography was considered abnormal.

RESULTS

The optimum diagnostic threshold for QP determined by receiver-operating characteristic curve occurred when endocardial flow decreased to <50% of mean epicardial flow, which yielded a sensitivity of 87% and specificity of 93%. The area under the curve for QP was 92%, which was superior to semiquantitative methods: contrast enhancement ratio: 78%; upslope index: 82%; and upslope integral: 75% (p = 0.011, p = 0.019, p = 0.004 vs. QP, respectively). Area under the curve for QP was also superior to qualitative methods: Duke algorithm: 70%; and clinical interpretation: 78% (p < 0.001 and p < 0.001 vs. QP, respectively).

CONCLUSIONS

Fully quantitative stress perfusion CMR has high diagnostic accuracy for detecting obstructive coronary artery disease. QP outperforms semiquantitative measures of perfusion and qualitative methods that incorporate a combination of cine, perfusion, and late gadolinium enhancement imaging. These findings suggest a potential clinical role for quantitative stress perfusion CMR.

Chronic obstructive pulmonary disease in patients undergoing transcatheter aortic valve implantation: insights on clinical outcomes, prognostic markers, and functional status changes

OBJECTIVES

This study sought to determine the effects of chronic obstructive pulmonary disease (COPD) on clinical outcomes in patients undergoing transcatheter aortic valve implantation (TAVI) and to determine the factors associated with worse outcomes in COPD patients.

BACKGROUND

No data exist on the factors determining poorer outcomes in COPD patients undergoing TAVI.

METHODS

A total of 319 consecutive patients (29.5% with COPD) who underwent TAVI were studied. Functional status was evaluated by New York Heart Association (NYHA) functional class, Duke Activity Status Index, and the 6-min walk test (6MWT) at baseline and at 6 to 12 months. The TAVI treatment was considered futile if the patient either died or did not improve in NYHA functional class at 6-month follow-up.

RESULTS

Survival rates at 1 year were 70.6% in COPD patients and 84.5% in patients without COPD (p = 0.008). COPD was an independent predictor of cumulative mortality after TAVI (hazard ratio: 1.84; 95% confidence interval: 1.08 to 3.13; p = 0.026). Improvement in functional status was observed after TAVI (p < 0.001 for NYHA functional class, Duke Activity Status Index, and 6MWT), but COPD patients exhibited less (p = 0.036) improvement in NYHA functional class. Among COPD patients, a shorter 6MWT distance predicted cumulative mortality (p = 0.013), whereas poorer baseline spirometry results (FEV1 [forced expiratory volume in the first second of expiration]) determined a higher rate of periprocedural pulmonary complications (p = 0.040). The TAVI treatment was futile in 40 COPD patients (42.5%) and a baseline 6MWT distance <170 m best determined the lack of benefit after TAVI (p = 0.002).

CONCLUSIONS

COPD was associated with a higher rate of mortality at mid-term follow-up. Among COPD patients, a higher degree of airway obstruction and a lower exercise capacity determined a higher risk of pulmonary complications and mortality, respectively. TAVI was futile in more than one-third of the COPD patients, and a shorter distance walked at the 6MWT predicted the lack of benefit after TAVI. These results may help to improve the clinical decision-making process in this challenging group of patients.

Association between pharmacokinetics of adalimumab and mucosal healing in patients with inflammatory bowel diseases

BACKGROUND & AIMS

Little is known about the association between pharmacokinetic features of adalimumab and mucosal healing in patients with inflammatory bowel disease (IBD).

METHODS

We conducted a cross-sectional study of 40 patients with Crohn's disease (CD) or ulcerative colitis (UC) who received adalimumab maintenance therapy and underwent endoscopic evaluation of disease activity and pharmacokinetic analysis (measurements of trough levels and antibodies against adalimumab). Patients in clinical remission were identified based on CD activity index scores less than 150 or Mayo scores less than 3 (for those with UC). Patients with mucosal healing were identified based on Mayo endoscopic scores less than 2 (for UC) or the disappearance of all ulcerations (for CD).

RESULTS

The median trough level of adalimumab was higher in patients in clinical remission (6.02 μg/mL) than in patients with active disease (3.2 μg/mL; P = .012). Trough levels of adalimumab were also higher in patients with mucosal healing (6.5 μg/mL) than in patients without (4.2 μg/mL;  P  <  .005). These results did not vary with type of IBD. On multivariate analysis, trough levels of adalimumab (relative risk, 0.62; 95% confidence interval, 0.40-0.94; P = .026) and duration of adalimumab treatment (relative risk, 0.82; 95% confidence interval, 0.68-0.97; P = .026) were associated independently with healing mucosa. An absence of mucosal healing was associated with trough levels of adalimumab less than 4.9 μg/mL (likelihood ratio, 4.3; sensitivity, 66%; specificity, 85%).

CONCLUSIONS

Trough levels of adalimumab are significantly higher in IBD patients who are in clinical remission and in those with mucosal healing. Detection of antibodies against adalimumab predicts a lack of mucosal healing.

Response to peginterferon alfa-2a (40KD) in HBeAg-negative CHB: on-treatment kinetics of HBsAg serum levels vary by HBV genotype

BACKGROUND & AIMS

We investigated whether HBV genotype influences on-treatment HBsAg kinetics and/or the end-of-treatment HBsAg levels associated with long-term virological response in HBeAg-negative chronic hepatitis B patients treated with peginterferon alfa-2a±lamivudine in the Phase III trial.

METHODS

All patients (n=230) who participated in long-term follow-up were included according to the availability of HBsAg level measurements. Long-term virological response was defined as HBV DNA ≤ 10,000cp/ml (1786IU/ml) at 5 years post-treatment. Genotype-specific end-of-treatment HBsAg levels associated with long-term virological response (identified by ROC analysis) were assessed in 199 patients with HBsAg measurements available at baseline and end-of-treatment. HBsAg kinetics according to genotype and long-term virological response were investigated in the 117 patients with additional samples available at weeks 12, 24, and 72.

RESULTS

Baseline HBsAg levels were significantly higher for A than B, C, and D genotypes (p<0.05). On-treatment HBsAg kinetics varied according to HBV genotype. The difference between responders and non-responders was greatest for genotype A from weeks 12-24; for genotypes B and D from baseline to week 12; there was no significant difference over any timeframe for genotype C. High positive predictive values for long-term virological response could be obtained by applying end-of-treatment genotype-specific cut-offs: 75%, 47%, 71%, and 75% for genotypes A (<400IU/ml), B (<50IU/ml), C (<75IU/ml), and D (<1000IU/ml), respectively.

CONCLUSIONS

On-treatment HBsAg kinetics vary between HBV genotypes. Genotype-specific monitoring timeframes and end-of-treatment thresholds could ameliorate response-guided treatment of HBeAg-negative chronic hepatitis B.

Myocardial CT perfusion imaging in a large animal model: comparison of dynamic versus single-phase acquisitions

OBJECTIVES

This study sought to compare dynamic versus single-phase high-pitch computed tomography (CT) acquisitions for the assessment of myocardial perfusion in a porcine model with adjustable degrees of coronary stenosis.

BACKGROUND

The incremental value of the 2 different approaches to CT-based myocardial perfusion imaging remains unclear.

METHODS

Country pigs received stent implantation in the left anterior descending coronary artery, in which an adjustable narrowing (50% and 75% stenoses) was created using a balloon catheter. All animals underwent CT-based rest and adenosine-stress myocardial perfusion imaging using dynamic and single-phase high-pitch acquisitions at both degrees of stenosis. Fluorescent microspheres served as a reference standard for myocardial blood flow. Segmental CT-based myocardial blood flow (MBFCT) was derived from dynamic acquisitions. Segmental single-phase enhancement (SPE) was recorded from high-pitch, single-phase examinations. MBFCT and SPE were compared between post-stenotic and reference segments, and receiver-operating characteristic curve analysis was performed.

RESULTS

Among 6 animals (28 ± 2 kg), there were significant differences of MBFCT and SPE between post-stenotic and reference segments for all acquisitions at 75% stenosis. By contrast, although for 50% stenosis at rest, MBFCT was lower in post-stenotic than in reference segments (0.65 ± 0.10 ml/g/min vs. 0.75 ± 0.16 ml/g/min, p < 0.05), there was no difference for SPE (128 ± 27 Hounsfield units vs. 137 ± 35 Hounsfield units, p = 0.17), which also did not significantly change under adenosine stress. In receiver-operating characteristic curve analyses, segmental MBFCT showed significantly better performance for ischemia prediction at 75% stenosis and stress (area under the curve: 0.99 vs. 0.89, p < 0.05) as well as for 50% stenosis, regardless of adenosine administration (area under the curve: 0.74 vs. 0.57 and 0.88 vs. 0.61, respectively, both p < 0.05).

CONCLUSIONS

At higher degrees of coronary stenosis, both MBFCT and SPE permit an accurate prediction of segmental myocardial hypoperfusion. However, accuracy of MBFCT is higher than that of SPE at 50% stenosis and can be increased by adenosine stress at both degrees of stenosis.

Consensus and update on the definition of on-treatment platelet reactivity to adenosine diphosphate associated with ischemia and bleeding

Dual antiplatelet therapy with aspirin and a P2Y12 receptor blocker is a key strategy to reduce platelet reactivity and to prevent thrombotic events in patients treated with percutaneous coronary intervention. In an earlier consensus document, we proposed cutoff values for high on-treatment platelet reactivity to adenosine diphosphate (ADP) associated with post-percutaneous coronary intervention ischemic events for various platelet function tests (PFTs). Updated American and European practice guidelines have issued a Class IIb recommendation for PFT to facilitate the choice of P2Y12 receptor inhibitor in selected high-risk patients treated with percutaneous coronary intervention, although routine testing is not recommended (Class III). Accumulated data from large studies underscore the importance of high on-treatment platelet reactivity to ADP as a prognostic risk factor. Recent prospective randomized trials of PFT did not demonstrate clinical benefit, thus questioning whether treatment modification based on the results of current PFT platforms can actually influence outcomes. However, there are major limitations associated with these randomized trials. In addition, recent data suggest that low on-treatment platelet reactivity to ADP is associated with a higher risk of bleeding. Therefore, a therapeutic window concept has been proposed for P2Y12 inhibitor therapy. In this updated consensus document, we review the available evidence addressing the relation of platelet reactivity to thrombotic and bleeding events. In addition, we propose cutoff values for high and low on-treatment platelet reactivity to ADP that might be used in future investigations of personalized antiplatelet therapy.

The complex nature of discordant severe calcified aortic valve disease grading: new insights from combined Doppler echocardiographic and computed tomographic study

OBJECTIVES

With concomitant Doppler echocardiography and multidetector computed tomography (MDCT) measuring aortic valve calcification (AVC) load, this study aimed at defining: 1) independent physiologic/structural determinants of aortic valve area (AVA)/mean gradient (MG) relationship; 2) AVC thresholds best associated with severe aortic stenosis (AS); and 3) whether, in AS with discordant MG, severe calcified aortic valve disease is generally detected.

BACKGROUND

Aortic stenosis with discordant markers of severity, AVA in severe range but low MG, is a conundrum, unresolved by outcome studies.

METHODS

Patients (n = 646) with normal left ventricular ejection fraction AS underwent Doppler echocardiography and AVC measurement by MDCT. On the basis of AVA-indexed-to-body surface area (AVAi) and MG, patients were categorized as concordant severity grading (CG) with moderate AS (AVAi >0.6 cm²/m², MG <40 mm Hg), severe AS (AVAi ≤0.6 cm²/m², MG ≥ 40 mm Hg), discordant-severity-grading (DG) with low-MG (AVAi ≤0.6 cm(2)/m(2), MG <40 mm Hg), or high-MG (AVAi >0.6 cm(2)/m(2), MG ≥40 mm Hg).

RESULTS

The MG (discordant in 29%) was strongly determined by AVA and flow but also independently and strongly influenced by AVC-load (p < 0.0001) and systemic arterial compliance (p < 0.0001). The AVC-load (median [interquartile range]) was similar within patients with DG (low-MG: 1,619 [965 to 2,528] arbitrary units [AU]; high-MG: 1,736 [1,209 to 2,894] AU; p = 0.49), higher than CG-moderate-AS (861 [427 to 1,519] AU; p < 0.0001) but lower than CG-severe-AS (2,931 [1,924 to 4,292] AU; p < 0.0001). The AVC-load thresholds separating severe/moderate AS were defined in CG-AS with normal flow (stroke-volume-index >35 ml/m(2)). The AVC-load, absolute or indexed, identified severe AS accurately (area under the curve ≥0.89, sensitivity ≥86%, specificity ≥79%) in men and women. Upon application of these criteria to DG-low MG, at least one-half of the patients were identified as severe calcified aortic valve disease, irrespective of flow.

CONCLUSIONS

Among patients with AS, MG is often discordant from AVA and is determined by multiple factors, valvular (AVC) and non-valvular (arterial compliance) independently of flow. The AVC-load by MDCT, strongly associated with AS severity, allows diagnosis of severe calcified aortic valve disease. At least one-half of the patients with discordant low gradient present with heavy AVC-load reflective of severe calcified aortic valve disease, emphasizing the clinical yield of AVC quantification by MDCT to diagnose and manage these complex patients.

T(1) mapping for the diagnosis of acute myocarditis using CMR: comparison to T2-weighted and late gadolinium enhanced imaging

OBJECTIVES

This study sought to test the diagnostic performance of native T1 mapping in acute myocarditis compared with cardiac magnetic resonance (CMR) techniques such as dark-blood T2-weighted (T2W)-CMR, bright-blood T2W-CMR, and late gadolinium enhancement (LGE) imaging.

BACKGROUND

The diagnosis of acute myocarditis on CMR often requires multiple techniques, including T2W, early gadolinium enhancement, and LGE imaging. Novel techniques such as T1 mapping and bright-blood T2W-CMR are also sensitive to changes in free water content. We hypothesized that these techniques can serve as new and potentially superior diagnostic criteria for myocarditis.

METHODS

We investigated 50 patients with suspected acute myocarditis (age 42 ± 16 years; 22% women) and 45 controls (age 42 ± 14 years; 22% women). CMR at 1.5-T (median 3 days from presentation) included: 1) dark-blood T2W-CMR (short-tau inversion recovery); 2) bright-blood T2W-CMR (acquisition for cardiac unified T2 edema); 3) native T1 mapping (shortened modified look-locker inversion recovery); and 4) LGE. Image analysis included: 1) global T2 signal intensity ratio of myocardium compared with skeletal muscle; 2) myocardial T1 relaxation times; and 3) areas of LGE.

RESULTS

Compared with controls, patients had significantly higher global T2 signal intensity ratios by dark-blood T2W-CMR (1.73 ± 0.27 vs. 1.56 ± 0.15, p < 0.01), bright-blood T2W-CMR (2.02 ± 0.33 vs. 1.84 ± 0.17, p < 0.01), and mean myocardial T1 (1,010 ± 65 ms vs. 941 ± 18 ms, p < 0.01). Receiver-operating characteristic analysis showed clear differences in diagnostic performance. The areas under the curve for each method were: T1 mapping (0.95), LGE (0.96), dark-blood T2 (0.78), and bright-blood T2 (0.76). A T1 cutoff of 990 ms had a sensitivity, specificity, and diagnostic accuracy of 90%, 91%, and 91%, respectively.

CONCLUSIONS

Native T1 mapping as a novel criterion for the detection of acute myocarditis showed excellent and superior diagnostic performance compared with T2W-CMR. It also has a higher sensitivity compared with T2W and LGE techniques, which may be especially useful in detecting subtle focal disease and when gadolinium contrast imaging is not feasible.

Noninvasive assessment of insulin resistance in the liver using the fasting (13)C-glucose breath test

Evaluating hepatic insulin resistance (IR) is the key to making a sensitive an accurate diagnosis of glucose intolerance. However, there is currently no suitable method to perform this procedure. This study was conducted to investigate whether the fasting (13)C-glucose breath test (FGBT) is useful as a convenient and highly sensitive clinical test for evaluating hepatic IR. Healthy nonobese subjects and a disease group consisting of patients with mild glucose intolerance were administered 100 mg (13)C-glucose after an overnight fast. A series of breath samples was collected until 360 minutes after ingestion, and the (13)CO2-to-(12)CO2 ratio was measured using an infrared spectrometer and was plotted as a kinetic curve of (13)C excretion. The area under the curve until 360 minutes (AUC360) of the (13)C excretion kinetic curve of the FGBT reflects the efficiency of energy production in the liver. First, we assessed the correlations between the AUC360 (or the (13)C excretion rate at 120 minutes) and the HOMA-IR and HbA1c levels as standard measurements of IR and diabetes mellitus (DM). There were relatively strong correlation coefficients (r = -0.49 to -0.81, r(2) = 0.24-0.66, P < 0.01; n = 35 males, n = 33 females). Second, we compared the AUC360 of healthy subjects and that of the patients with mild glucose intolerance. The AUC360 of the healthy subjects was consistently higher than that of the patients with mild glucose intolerance. The presence of IR or DM in males and females was diagnosed using cutoff values. The FGBT is a novel glucose metabolism test that can be used conveniently and safely to evaluate the balance of glucose metabolism in the liver. This test has excellent sensitivity for diagnosing alterations in hepatic glucose metabolism, particularly hepatic IR.

Computed tomography angiography and myocardial computed tomography perfusion in patients with coronary stents: prospective intraindividual comparison with conventional coronary angiography

OBJECTIVES

This study sought to determine whether adding myocardial computed tomography perfusion (CTP) to computed tomography angiography (CTA) improves diagnostic performance for coronary stents.

BACKGROUND

CTA of coronary stents has been limited by nondiagnostic studies caused by metallic stent material and coronary motion.

METHODS

CTA and CTP were performed in 91 consecutive patients with stents before quantitative coronary angiography, the reference standard for obstructive stenosis (≥50%). If a coronary stent or vessel was nondiagnostic on CTA, adenosine stress CTP in the corresponding myocardial territory was read for combined CTA/CTP.

RESULTS

Patients had an average of 2.5 ± 1.8 coronary stents (1 to 10), with a diameter of 3.0 ± 0.5 mm. Significantly more patients were nondiagnostic for stent assessment by CTA (22%; mainly due to metal artifacts [75%] or motion [25%]) versus CTP (1%; p < 0.001; severe angina precluded CTP in 1 case). The per-patient diagnostic accuracy of CTA/CTP for stents (87%, 95% confidence interval [CI]: 78% to 93%) was significantly higher than that of CTA alone (71%, 95% CI: 61% to 80%; p < 0.001), mainly because nondiagnostic examinations were significantly reduced (p < 0.001). In the analysis of any coronary artery disease, diagnostic accuracy and nondiagnostic rate were also significantly improved by the addition of CTP (p < 0.001). CTA/CTP (7.9 ± 2.8 mSv) had a significantly lower effective radiation dose than angiography (9.5 ± 5.1 mSv; p = 0.005). The area under the receiver-operating characteristic curve for CTA/CTP (0.82, 95% CI: 0.69 to 0.95) was superior to that for CTA (0.69, 95% CI: 0.57 to 0.82; p < 0.001) in identifying patients requiring stent revascularization.

CONCLUSIONS

Combined coronary CTA and myocardial CTP improves diagnosis of CAD and in-stent restenosis in patients with stents compared with CTA alone. (Coronary Artery Stent Evaluation With 320-Slice Computed Tomography-The CArS 320 Study [CARS-320]; NCT00967876).

Pulmonary edema predictive scoring index (PEPSI), a new index to predict risk of reperfusion pulmonary edema and improvement of hemodynamics in percutaneous transluminal pulmonary angioplasty

OBJECTIVES

This study sought to identify useful predictors for hemodynamic improvement and risk of reperfusion pulmonary edema (RPE), a major complication of this procedure.

BACKGROUND

Percutaneous transluminal pulmonary angioplasty (PTPA) has been reported to be effective for the treatment of chronic thromboembolic pulmonary hypertension (CTEPH). PTPA has not been widespread because RPE has not been well predicted.

METHODS

We included 140 consecutive procedures in 54 patients with CTEPH. The flow appearance of the target vessels was graded into 4 groups (Pulmonary Flow Grade), and we proposed PEPSI (Pulmonary Edema Predictive Scoring Index) = (sum total change of Pulmonary Flow Grade scores) × (baseline pulmonary vascular resistance). Correlations between occurrence of RPE and 11 variables, including hemodynamic parameters, number of target vessels, and PEPSI, were analyzed.

RESULTS

Hemodynamic parameters significantly improved after median observation period of 6.4 months, and the sum total changes in Pulmonary Flow Grade scores were significantly correlated with the improvement in hemodynamics. Multivariate analysis revealed that PEPSI was the strongest factor correlated with the occurrence of RPE (p < 0.0001). Receiver-operating characteristic curve analysis demonstrated PEPSI to be a useful marker of the risk of RPE (cutoff value 35.4, negative predictive value 92.3%).

CONCLUSIONS

Pulmonary Flow Grade score is useful in determining therapeutic efficacy, and PEPSI is highly supportive to reduce the risk of RPE after PTPA. Using these 2 indexes, PTPA could become a safe and common therapeutic strategy for CTEPH.

Hybrid in silico models for drug-induced liver injury using chemical descriptors and in vitro cell-imaging information

Drug-induced liver injury (DILI) is a major adverse drug reaction that accounts for one-third of post-marketing drug withdrawals. Several classifiers for human hepatotoxicity using chemical descriptors with limited prediction accuracies have been published. In this study, we developed predictive in silico models based on a set of 156 DILI positive and 136 DILI negative compounds for DILI prediction. First, models based on a chemical descriptor (CDK, Dragon and MOE) and in vitro cell-imaging endpoints [human hepatocyte imaging assay technology (HIAT) descriptors] were built using random forest (RF) and five-fold cross-validation procedure. Then three hybrid models were built using HIAT and a single type of chemical descriptors. Generally, the models based only on chemical descriptors were poor, with a correct classification rate (CCR) around 0.60 when the default threshold value (i.e. threshold = 0.50) was used. The hybrid models afforded a CCR of 0.73 with a specificity of 0.74 and a better true positive rate (sensitivity of 0.71), which is crucial in drug toxicity screening for the purpose of patient safety. The benefit of hybrid models was even more drastic when stricter classification thresholds were employed (e.g. CCR would be 0.83 when double thresholds (non-toxic < 0.40 and toxic > 0.60) were used for the hybrid model). We have developed rigorously validated hybrid models which can be used in virtual screening of lead compounds with potential hepatotoxicity. Our study also showed a chemical structure and in vitro biological data can be complementary in enhancing the prediction accuracy of human hepatotoxicity and can afford rational mechanistic interpretation.

The Role of Noncriterial Recollection in Estimating Recollection and Familiarity

Noncriterial recollection (ncR) is recollection of details that are irrelevant to task demands. It has been shown to elevate familiarity estimates and to be functionally equivalent to familiarity in the process dissociation procedure (Yonelinas & Jacoby, 1996). However, Toth and Parks (2006) found no ncR in older adults, and hypothesized that this absence was related to older adults' criterial recollection deficit. To test this hypothesis, as well as whether ncR is functionally equivalent to familiarity and increases the subjective experience of familiarity, remember-know and confidence-rating methods were used to estimate recollection and familiarity with young adults, young adults in a divided-attention condition (Experiment 1), and older adults. Supporting Toth and Parks' hypothesis, ncR was found in all groups, but was consistently larger for groups with higher criterial recollection. Response distributions and receiver-operating characteristics revealed further similarities to criterial recollection and suggested that neither the experience nor usefulness of familiarity was enhanced by ncR. Overall, the results suggest that ncR does not differ fundamentally from criterial recollection.