The use of semi-quantitative ultrasound elastosonography in combination with conventional ultrasonography and contrast-enhanced ultrasonography in the assessment of malignancy risk of thyroid nodules with indeterminate cytology

Background

The pre-surgical selection of thyroid nodules with indeterminate cytology (Thy 3 according to British Thyroid Association) after fine-needle aspiration biopsy (FNAB) is currently required in order to reduce unnecessary total thyroidectomy. The objective of our study was to use a surgical series of Thy 3 nodules to evaluate the predictive role of ultrasound elastosonography (USE) and contrast-enhanced ultrasonography (CEUS) in pre-surgical diagnoses of malignancy.

Subjects and methods

We enrolled 63 patients with Thy 3 nodules in which cytological–histological correlation was available. The ELX 2/1 strain index was obtained by means of semi-quantitative USE, which was performed before surgery in addition to conventional ultrasonography (US) and contrast-enhanced US (CEUS) on the Thy 3 nodules. The ELX 2/1 strain index, a five-item US score and both peak (P) index and time to peak (TTP) index from CEUS were correlated with the histological results. After surgical diagnosis, the data were analysed by using a receiver-operating characteristic (ROC) curve.

Results

Histology was benign in 50 and malignant in 13 Thy 3 nodules. No difference in maximal diameter was noted between benign (22.8 ± 1.6 mm) and malignant (18.9 ± 2.9 mm) nodules. Significant correlations were found between histology and cumulative US findings (p=0.005), ELX 2/1 index (p=0.002), P index (p=0.01) and TTP index (p=0.02). On analysing data from US, USE and CEUS, significant ROC areas under the curve were observed (p<0.0001). A cut-off value was set for US (>2), ELX 2/1 (>0.95), P index (<0.99) and TTP index (>0.98) scores. The diagnostic power of the cumulative pre-surgical analysis of Thy 3 nodules with US, USE and CEUS, considering the experimental cut-off points obtained from the ROC curves was: sensitivity 64%, specificity 92%, PPV 75% and accuracy 84%.

Conclusion

The ELX 2/1 index in conjunction with the US score can be useful in orienting surgical strategies in Thy 3 nodules. The information added by CEUS is less sensitive than that provided by US and USE. The use of a cut-off based on histology can reduce thyroidectomy. Observation should be the first choice when not all instrumental results are suspect.

Jackknife variance of the partial area under the empirical receiver operating characteristic curve

Receiver operating characteristic analysis provides an important methodology for assessing traditional (e.g., imaging technologies and clinical practices) and new (e.g., genomic studies, biomarker development) diagnostic problems. The area under the clinically/practically relevant part of the receiver operating characteristic curve (partial area or partial area under the receiver operating characteristic curve) is an important performance index summarizing diagnostic accuracy at multiple operating points (decision thresholds) that are relevant to actual clinical practice. A robust estimate of the partial area under the receiver operating characteristic curve is provided by the area under the corresponding part of the empirical receiver operating characteristic curve. We derive a closed-form expression for the jackknife variance of the partial area under the empirical receiver operating characteristic curve. Using the derived analytical expression, we investigate the differences between the jackknife variance and a conventional variance estimator. The relative properties in finite samples are demonstrated in a simulation study. The developed formula enables an easy way to estimate the variance of the empirical partial area under the receiver operating characteristic curve, thereby substantially reducing the computation burden, and provides important insight into the structure of the variability. We demonstrate that when compared with the conventional approach, the jackknife variance has substantially smaller bias, and leads to a more appropriate type I error rate of the Wald-type test. The use of the jackknife variance is illustrated in the analysis of a data set from a diagnostic imaging study.

Noninvasive cross-sectional imaging of proximal caries using swept-source optical coherence tomography (SS-OCT) in vivo

The aim of this study was to determine the diagnostic accuracy of swept-source optical coherent tomography (SS-OCT) in detecting and estimating the depth of proximal caries in posterior teeth in vivo. SS-OCT images and bitewing radiographs were obtained from 86 proximal surfaces of 53 patients. Six examiners scored the locations according to a caries lesion depth scale (0-4) using SS-OCT and the radiographs. The results were compared with clinical observations obtained after the treatment. SS-OCT could detect the presence of proximal caries in tomograms that were synthesized based on the backscatter signal obtained from the proximal carious lesion through occlusal enamel. SS-OCT showed significantly higher sensitivity and larger area under the receiver operating characteristic curve than radiographs for the detection of cavitated enamel lesions and dentin caries (Student's t -test, p < 0.05). SS-OCT appears to be a more reliable and accurate method than bitewing radiographs for the detection and estimation of the depth of proximal lesions in the clinical environment.

Validation of a composite of suicide items from the Mood and Feelings Questionnaire (MFQ) in offspring of recurrently depressed parents

The Mood and Feelings Questionnaire (MFQ) is widely used for the assessment of depression in adolescents. The main aim of this study was to examine the concurrent and predictive validity of a composite of four MFQ items related to suicidal ideation using an interview measure of suicidal ideation. A prospective 3-wave high-risk study of offspring of parents with recurrent depression was used including 294 families where children were initially aged 9-17 years. Measures included four parent and child rated MFQ items assessing suicide-related ideation (referred to here as the "MFQ-SI") and a clinically-defined interview measure of suicidal ideation. A parent-child combined MFQ-SI subscale performed well as a screening tool against the interview measure of suicidal ideation (baseline area under the curve (AUC) (95% CI):0.92 (0.85-1.00)). Longitudinally, this measure showed reasonable predictive validity against future suicidal ideation (AUC (95% CI):0.73 (0.58-0.88)). Lastly, there was evidence that a child-rated MFQ-SI scale performed better than a parent-rated one in detecting concurrent suicidal ideation. Longitudinally, both parent and child scales showed reasonable predictive validity against future suicidal ideation. In summary, a brief screen using four MFQ items related to suicidal ideation performs well in identifying concurrent and future suicidal ideation in high-risk adolescents.

Influence analysis in quantitative trait loci detection

This paper presents systematic methods for the detection of influential individuals that affect the log odds (LOD) score curve. We derive general formulas of influence functions for profile likelihoods and introduce them into two standard quantitative trait locus detection methods—the interval mapping method and single marker analysis. Besides influence analysis on specific LOD scores, we also develop influence analysis methods on the shape of the LOD score curves. A simulation-based method is proposed to assess the significance of the influence of the individuals. These methods are shown useful in the influence analysis of a real dataset of an experimental population from an F₂ mouse cross. By receiver operating characteristic analysis, we confirm that the proposed methods show better performance than existing diagnostics.

Evaluating probabilistic forecasts with Bayesian signal detection models

We propose the use of signal detection theory (SDT) to evaluate the performance of both probabilistic forecasting systems and individual forecasters. The main advantage of SDT is that it provides a principled way to distinguish the response from system diagnosticity, which is defined as the ability to distinguish events that occur from those that do not. There are two challenges in applying SDT to probabilistic forecasts. First, the SDT model must handle judged probabilities rather than the conventional binary decisions. Second, the model must be able to operate in the presence of sparse data generated within the context of human forecasting systems. Our approach is to specify a model of how individual forecasts are generated from underlying representations and use Bayesian inference to estimate the underlying latent parameters. Given our estimate of the underlying representations, features of the classic SDT model, such as the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC), follow immediately. We show how our approach allows ROC curves and AUCs to be applied to individuals within a group of forecasters, estimated as a function of time, and extended to measure differences in forecastability across different domains. Among the advantages of this method is that it depends only on the ordinal properties of the probabilistic forecasts. We conclude with a brief discussion of how this approach might facilitate decision making.

Variable selection and prediction using a nested, matched case-control study: Application to hospital acquired pneumonia in stroke patients

Matched case-control designs are commonly used in epidemiologic studies for increased efficiency. These designs have recently been introduced to the setting of modern imaging and genomic studies, which are characterized by high-dimensional covariates. However, appropriate statistical analyses that adjust for the matching have not been widely adopted. A matched case-control study of 430 acute ischemic stroke patients was conducted at Massachusetts General Hospital (MGH) in order to identify specific brain regions of acute infarction that are associated with hospital acquired pneumonia (HAP) in these patients. There are 138 brain regions in which infarction was measured, which introduce nearly 10,000 two-way interactions, and challenge the statistical analysis. We investigate penalized conditional and unconditional logistic regression approaches to this variable selection problem that properly differentiate between selection of main effects and of interactions, and that acknowledge the matching. This neuroimaging study was nested within a larger prospective study of HAP in 1915 stroke patients at MGH, which recorded clinical variables, but did not include neuroimaging. We demonstrate how the larger study, in conjunction with the nested, matched study, affords us the capability to derive a score for prediction of HAP in future stroke patients based on imaging and clinical features. We evaluate the proposed methods in simulation studies and we apply them to the MGH HAP study.

Sample size estimation in diagnostic test studies of biomedical informatics

OBJECTIVES

This review provided a conceptual framework of sample size calculations in the studies of diagnostic test accuracy in various conditions and test outcomes.

METHODS

The formulae of sample size calculations for estimation of adequate sensitivity/specificity, likelihood ratio and AUC as an overall index of accuracy and also for testing in single modality and comparing two diagnostic tasks have been presented for desired confidence interval.

RESULTS

The required sample sizes were calculated and tabulated with different levels of accuracies and marginal errors with 95% confidence level for estimating and for various effect sizes with 80% power for purpose of testing as well. The results show how sample size is varied with accuracy index and effect size of interest.

CONCLUSION

This would help the clinicians when designing diagnostic test studies that an adequate sample size is chosen based on statistical principles in order to guarantee the reliability of study.

Diagnostic utility of CPSS vs. CAPS-CA for assessing posttraumatic stress symptoms in children and adolescents

OBJECTIVES

This study compared the diagnostic utility of the symptom part of the child PTSD symptom scale (CPSS) screening instrument with the clinician-administered PTSD scale for children and adolescents (CAPS-CA).

METHODS

The study included a clinical sample of traumatized children and adolescents (mean age 15.1, range 10-18) living in Norway, who were assessed for posttraumatic stress symptoms using the CPSS and the CAPS-CA. Diagnostic utility was investigated using receiver operating characteristic analyses.

RESULTS

The results showed that CPSS reached medium effect sizes (AUC from .63 to .76). The sensitivity was good (.80), but the specificity was relatively low (.56). Kappa between CPSS and CAPS-CA was low (κ=.27).

CONCLUSIONS

Findings suggests that CPSS is a good tool for screening purposes, but not as a diagnostic instrument in an early phase of assessment. Implications and limitations of the findings are discussed.

Frailty Models for Familial Risk with Application to Breast Cancer

In evaluating familial risk for disease we have two main statistical tasks: assessing the probability of carrying an inherited genetic mutation conferring higher risk; and predicting the absolute risk of developing diseases over time, for those individuals whose mutation status is known. Despite substantial progress, much remains unknown about the role of genetic and environmental risk factors, about the sources of variation in risk among families that carry high-risk mutations, and about the sources of familial aggregation beyond major Mendelian effects. These sources of heterogeneity contribute substantial variation in risk across families. In this paper we present simple and efficient methods for accounting for this variation in familial risk assessment. Our methods are based on frailty models. We implemented them in the context of generalizing Mendelian models of cancer risk, and compared our approaches to others that do not consider heterogeneity across families. Our extensive simulation study demonstrates that when predicting the risk of developing a disease over time conditional on carrier status, accounting for heterogeneity results in a substantial improvement in the area under the curve of the receiver operating characteristic. On the other hand, the improvement for carriership probability estimation is more limited. We illustrate the utility of the proposed approach through the analysis of BRCA1 and BRCA2 mutation carriers in the Washington Ashkenazi Kin-Cohort Study of Breast Cancer.

Joint confidence region estimation for area under ROC curve and Youden index

In the field of diagnostic studies, the area under the ROC curve (AUC) serves as an overall measure of a biomarker/diagnostic test's accuracy. Youden index, defined as the overall correct classification rate minus one at the optimal cut-off point, is another popular index. For continuous biomarkers of binary disease status, although researchers mainly evaluate the diagnostic accuracy using AUC, for the purpose of making diagnosis, Youden index provides an important and direct measure of the diagnostic accuracy at the optimal threshold and hence should be taken into consideration in addition to AUC. Furthermore, AUC and Youden index are generally correlated. In this paper, we initiate the idea of evaluating diagnostic accuracy based on AUC and Youden index simultaneously. As the first step toward this direction, this paper only focuses on the confidence region estimation of AUC and Youden index for a single marker. We present both parametric and non-parametric approaches for estimating joint confidence region of AUC and Youden index. We carry out extensive simulation study to evaluate the performance of the proposed methods. In the end, we apply the proposed methods to a real data set.

ActiGraph GT3X+ cut-points for identifying sedentary behaviour in older adults in free-living environments

OBJECTIVES

To determine the ActiGraph GT3X+ cut-points with the highest accuracy for estimating time spent in sedentary behaviour in older adults in free-living environments. ActivPAL(3)™ was used as the reference standard.

DESIGN

Cross-sectional study.

METHODS

37 participants (13 males and 24 females, 73.5 ± 7.3 years old) wore an ActiGraph GT3X+ and an ActivPAL(3)™ for 7 consecutive days. For ActivPAL(3)™, variables were created based on posture. For ActiGraph GT3X+, sedentary behaviour was defined as (1) vector magnitude and (2) vertical axis counts for 1-s, 15-s and 1-min epochs, with cut-points for 1-s epochs of <1 to <10 counts, for 15-s epochs of <1 to <100 counts and for 1-min epochs of <1 to <400 counts. For each of the ActiGraph GT3X+ cut-points, area under the receiver operating characteristic curve (area under the curve), sensitivity, specificity, and percentage correctly classified were calculated. Bias and 95% limits of agreement were calculated using the Bland-Altman method.

RESULTS

The highest areas under the curve were obtained for the vector magnitude cut-points: <1 count/s, <70 counts/15-s, and <200 counts/min; and for the vertical axis cut-points: <1 count/s, <10 counts/15-s and <25 counts/min. Mean biases ranged from -4.29 to 124.28 min/day. The 95% limits of agreement for these cut-points were ± 2 h suggesting great inter-individual variation.

CONCLUSIONS

The results suggest that cut-points are dependent on unit of analyses (i.e. epoch length and axes); cut-points for a given epoch length and axis cannot simply be extrapolated to other epoch lengths. Limitations regarding inter-individual variability and misclassification of standing activity as sitting/lying must be considered.

Time to positivity as prognostic tool in patients with Pseudomonas aeruginosa bloodstream infection

OBJECTIVES

The time to positivity (TTP), measured as the time span between the start of incubation and the alert signal from the blood culture device, has been described as useful tool of prognosis in patients suffering from blood stream infection with Staphylococcus aureus, Escherichia coli and Klebsiella pneumonia. The present study investigates the relationship between TTP and in-hospital mortality in patients with monomicrobial Pseudomonas aeruginosa blood stream infection (PA-BSI).

METHODS

From 2006 until 2012 a retrospective cohort study was undertaken in 3 hospitals in the region surrounding Tübingen, Germany. Seventy-four patients with monomicrobial PA-BSI were studied. TTP and clinical parameters were determined and analyzed by receiver operating characteristic (ROC) analysis and Cox regression.

RESULTS

The in-hospital mortality of our clinical cohort was 33.78%. In multivariate Cox regression, a TTP ≤ 18 h proved to be independently associated with mortality (HR 3.83, P = 0.012) along with SAPS II score (HR 1.04, P = 0.006), cardiac disease (HR 0.33, P = 0.008) and appropriate definitive antimicrobial treatment (HR 0.21, P = 0.013).

CONCLUSIONS

TTP is an easy-to-measure laboratory tool for prognosis in patients with monomicrobial PA-BSI, providing useful information in addition to clinical parameters.

Frequency discrimination and stimulus deviance in the inferior colliculus and cochlear nucleus

Auditory neurons that exhibit stimulus-specific adaptation (SSA) decrease their response to common tones while retaining responsiveness to rare ones. We recorded single-unit responses from the inferior colliculus (IC) where SSA is known to occur and we explored for the first time SSA in the cochlear nucleus (CN) of rats. We assessed an important functional outcome of SSA, the extent to which frequency discriminability depends on sensory context. For this purpose, pure tones were presented in an oddball sequence as standard (high probability of occurrence) or deviant (low probability of occurrence) stimuli. To study frequency discriminability under different probability contexts, we varied the probability of occurrence and the frequency separation between tones. The neuronal sensitivity was estimated in terms of spike-count probability using signal detection theory. We reproduced the finding that many neurons in the IC exhibited SSA, but we did not observe significant SSA in our CN sample. We concluded that strong SSA is not a ubiquitous phenomenon in the CN. As predicted, frequency discriminability was enhanced in IC when stimuli were presented in an oddball context, and this enhancement was correlated with the degree of SSA shown by the neurons. In contrast, frequency discrimination by CN neurons was independent of stimulus context. Our results demonstrated that SSA is not widespread along the entire auditory pathway, and suggest that SSA increases frequency discriminability of single neurons beyond that expected from their tuning curves.

Variation in Variables that Predict Progression from MCI to AD Dementia over Duration of Follow-up

The purpose of this paper is to investigate the relative utility of using neuroimaging, genetic, cerebrospinal fluid (CSF), and cognitive measures to predict progression from mild cognitive impairment (MCI) to Alzheimer's disease (AD) dementia over a follow-up period. The studied subjects were 139 persons with MCI enrolled in the Alzheimer's Disease Neuroimaging Initiative. Predictors of progression to AD included brain volume, ventricular volume, hippocampal volume, APOE ε4 two alleles, Aβ₄₂, p-tau₁₈₁, p-tau₁₈₁/Aβ₄₂, memory, language, and executive function. We employ a combination of Cox regression analyses and time-dependent receiver operating characteristic (ROC) methods to assess the prognostic utility and performance stability of candidate biomarkers. In a demographic-adjusted multivariable Cox model, seven measures- brain volume, hippocampal volume, ventricular volume, APOE ε4 two alleles, Aβ₄₂, Memory composite, Executive function composite - predicted progression to AD. Time-dependent ROC revealed that this multivariable model had an area under the curve of 0.832, 0.788, 0.794, and 0.757 at 12, 18, 24, and 36 months respectively. Supplemental Cox models with time of origin set differentially at 12, 18, 24 and 36 months showed that six measures were significant predictors at 12 months whereas only memory and executive function predicted progression to AD at 18 and 24 months. The authors concluded that baseline volumetric MRI and cognitive measures selectively predict progression from MCI to AD, with cognitive measures remaining predictive even late in the follow-up period. These findings may inform case selection for AD clinical trials.

Impact of expectation-maximization reconstruction iterations on the diagnosis of temporal lobe epilepsy with PET

There is a well known tradeoff between image noise and image sharpness that is dependent on the number of iterations performed in ordered subset expectation maximization (OSEM) reconstruction of PET data. We aim to evaluate the impact of this tradeoff on the sensitivity and specificity of (18)F-FDG PET for the diagnosis of temporal lobe epilepsy. A retrospective blinded reader study was performed on two OSEM reconstructions, using either 2 or 5 iterations, of 32 (18)F-FDG PET studies acquired at our institution for the diagnosis of temporal lobe epilepsy. The sensitivity and specificity of each reconstruction for identifying patients who were ultimately determined to be surgical candidates was assessed using an ROC analysis. The sensitivity of each reconstruction for identifying patients who showed clinical improvement following surgery was also assessed. Our results showed no significant difference between the two reconstructions studied for either the sensitivity and specificity of (18)F-FDG PET for predicting surgical candidacy, or its sensitivity for predicting positive surgical outcomes. This implies that the number of iterations performed during OSEM reconstruction will have little impact on a reader based interpretation of (18)F-FDG PET scans acquired for the diagnosis of temporal lobe epilepsy, and can be determined by physician and institutional preference.

Top-down spatial categorization signal from prefrontal to posterior parietal cortex in the primate

In the present study we characterized the strength and time course of category-selective responses in prefrontal cortex and area 7a of the posterior parietal cortex during a match-to-sample spatial categorization task. A monkey was trained to categorize whether the height of a horizontal sample bar, presented in rectangular frame at one of three vertical locations, was "high" or "low," depending on whether its position was above or below the frame's midline. After the display of this sample bar, and after a delay, choice bars were sequentially flashed in two locations: at the top and at the bottom of the frame ("choice" epoch). If the monkey timed its response to the display of the choice bar that matched the sample bar, he was rewarded. We found that cells in prefrontal cortex discriminated category early after the initial sample bar was shown, and continued to differentiate "up" from "down" trials throughout the delay and choice periods. In contrast, parietal cells did not differentiate category until the choice period. Therefore, our results support the notion of a top-down categorical signal that originates in prefrontal cortex and that is only represented in parietal cortex when it is necessary to express the categorical decision through a movement.

Developing a Tool to Assess Competency to Consent to Psychiatric Hospitalization (KATOC): Reliability and Validity

OBJECTIVE

The justification of informed consent requires that a patient be provided with the information necessary for deciding treatment and able to use such information based on reasonable thinking. The clinical decision to consider anyone who has mental disorder as incompetence without objective assessment does not only encroach human rights of the persons with mental illness, but seriously prevent them from being recovered. Hence the objective assessment of competency is needed in mental health. Our study aimed to develop the Korean Tool of Competency to Consent to Psychiatric Hospitalization and to analyze the reliability and validity of this tool.

METHODS

Totally 98 patients with mental illness who were hospitalized in mental hospital, participated in this study. For the subjects a questionnaire composing of 22 questions of understanding, appreciation, reasoning and expression of a choice was used. To investigate validity of this tool, MMSE-K, insight test, estimated IQ, BPRS were conducted. Its reliability and usefulness were examined with Cronbach's alpha, ICC and ROC analysis respectively and criterion related validation performed.

RESULTS

As results, this tool shows that agreement between raters is relatively high and the confirmatory factor analysis for constructive validation shows that the tool is valid. Also, for criterion related validation, estimated IQ, insight and MMSE are significantly correlated to understanding, appreciation and reasoning. However competence to express a choice did not show any significant correlation with criterion variables, nor showed BPRS any significant correlation with sub-competences.

CONCLUSION

Our study developed the Korean Tool of Competency to Consent to Psychiatric Admission Treatment in the Mentally Ill, verified the reliability and validity of the tool and analyzed the optimum cutoff to distinguish between competence and incompetence in sub-competences. Korean Assessment Tool of Competency to Consent to Psychiatric Hospitalization (KATOC), analyzed the reliability and validity of this tool and presented the cutoff points by subarea. As a result, the reliability and validity of satisfactory levels were verified, the ROC analysis was implemented based on the clinical assessment and the cutoff points were found in understanding, appreciation, expression of a choice and reasoning. Such findings showed that the tool developed by researchers could be very favorably used in Korea.

Measures, Uncertainties, and Significance Test in Operational ROC Analysis

In receiver operating characteristic (ROC) analysis, the sampling variability can result in uncertainties of performance measures. Thus, while evaluating and comparing the performances of algorithms, the measurement uncertainties must be taken into account. The key issue is how to calculate the uncertainties of performance measures in ROC analysis. Our ultimate goal is to perform the significance test in evaluation and comparison using the standard errors computed. From the operational perspective, based on fingerprint-image matching algorithms on large datasets, the measures and their uncertainties are investigated in the three scenarios: 1) the true accept rate (TAR) of genuine scores at a specified false accept rate (FAR) of impostor scores, 2) the TAR and FAR at a given threshold, and 3) the equal error rate. The uncertainties of measures are calculated using the nonparametric two-sample bootstrap based on our extensive studies of bootstrap variability on large datasets. The significance test is carried out to determine whether the difference between the performance of one algorithm and a hypothesized value, or the difference between the performances of two algorithms where the correlation is taken into account is statistically significant. Examples are provided.

Adaptation of an Apple Sooty Blotch and Flyspeck Warning System for the Upper Midwest United States

A warning system for sooty blotch and flyspeck (SBFS) of apple, developed in the southeastern United States, uses cumulative hours of leaf wetness duration (LWD) to predict the timing of the first appearance of signs. In the Upper Midwest United States, however, this warning system has resulted in sporadic disease control failures. The purpose of the present study was to determine whether the warning system's algorithm could be modified to provide more reliable assessment of SBFS risk. Hourly LWD, rainfall, relative humidity (RH), and temperature data were collected from orchards in Iowa, North Carolina, and Wisconsin in 2005 and 2006. Timing of the first appearance of SBFS signs was determined by weekly scouting. Preliminary analysis using scatterplots and boxplots suggested that cumulative hours of RH ≥ 97% could be a useful predictor of SBFS appearance. Receiver operating characteristic curve analysis was used to compare the predictive performance of cumulative LWD and cumulative hours of RH ≥ 97%. Cumulative hours of RH ≥ 97% was a more conservative and accurate predictor than cumulative LWD for 15 site years in the Upper Midwest, but not for four site years in North Carolina. Performance of the SBFS warning system in the Upper Midwest and climatically similar regions may be improved if cumulative hours of RH ≥ 97% were substituted for cumulative LWD to predict the first appearance of SBFS.