COVID-19 pneumonia chest radiographic severity score: variability assessment among experienced and in-training radiologists and creation of a multireader composite score database for artificial intelligence algorithm development

OBJECTIVE

The purpose was to evaluate reader variability between experienced and in-training radiologists of COVID-19 pneumonia severity on chest radiograph (CXR), and to create a multireader database suitable for AI development.

METHODS

In this study, CXRs from polymerase chain reaction positive COVID-19 patients were reviewed. Six experienced cardiothoracic radiologists and two residents classified each CXR according to severity. One radiologist performed the classification twice to assess intraobserver variability. Severity classification was assessed using a 4-class system: normal (0), mild (1), moderate (2), and severe (3). A median severity score (Rad Med) for each CXR was determined for the six radiologists for development of a multireader database (XCOMS). Kendal Tau correlation and percentage of disagreement were calculated to assess variability.

RESULTS

A total of 397 patients (1208 CXRs) were included (mean age, 60 years SD ± 1), 189 men). Interobserver variability between the radiologists ranges between 0.67 and 0.78. Compared to the Rad Med score, the radiologists show good correlation between 0.79-0.88. Residents show slightly lower interobserver agreement of 0.66 with each other and between 0.69 and 0.71 with experienced radiologists. Intraobserver agreement was high with a correlation coefficient of 0.77. In 220 (18%), 707 (59%), 259 (21%) and 22 (2%) CXRs there was a 0, 1, 2 or 3 class-difference. In 594 (50%) CXRs the median scores of the residents and the radiologists were similar, in 578 (48%) and 36 (3%) CXRs there was a 1 and 2 class-difference.

CONCLUSION

Experienced and in-training radiologists demonstrate good inter- and intraobserver agreement in COVID-19 pneumonia severity classification. A higher percentage of disagreement was observed in moderate cases, which may affect training of AI algorithms.

ADVANCES IN KNOWLEDGE

Most AI algorithms are trained on data labeled by a single expert. This study shows that for COVID-19 X-ray severity classification there is significant variability and disagreement between radiologist and between residents.

CT Morphologic Characteristics and Variant Patterns of Interstitial Pulmonary Fibrosis in Systemic Lupus Erythematosus

Purpose

To assess CT features of pulmonary fibrosis in patients with systemic lupus erythematosus (SLE) and to assess the presence of several distinctive patterns of fibrosis associated with connective tissue disease.

Materials and Methods

A cross-sectional retrospective analysis was performed. An institutional clinical database was queried for the years of 2005-2015 to identify CT examination reports of patients with SLE and fibrotic lung disease, which yielded 50 patients (median age, 49 years; age range, 22-71 years; 46 women). CT examination reports were scored by two subspecialty thoracic radiologists using a standard multilevel semiquantitative system. Readers noted the presence or absence of several recently described CT signs of variant patterns of fibrosis in connective tissue disease (the "anterior upper lobe," "straight-edge," and "exuberant honeycombing" signs), as well as two other morphologic characteristics (an "island-like" appearance of areas of well-defined fibrosis with angular margins surrounded by normal lung and confluent regions of lucent lung destruction).

Results

The most common CT patterns were characterized as either fibrotic nonspecific interstitial pneumonia (38%, 19 of 50) or variant fibrosis (44%, 22 of 50). CT signs of variant fibrosis were identified by both readers in up to 62% of patients, with good κ agreement (0.44-0.64); the island-like sign (62%) and anterior upper lobe sign (52%) were most commonly observed. Pulmonary function test results showed correlations with several imaging findings but did not show correlations with CT signs of variant fibrosis.

Conclusion

When present, pulmonary fibrosis in SLE often has a distinctive appearance and may also manifest as several variant fibrotic patterns.Keywords: CT, Lung© RSNA, 2021See also the commentary by White in this issue.

Reducing STAT Portable Chest Radiograph Turnaround Times: A Pilot Study

BACKGROUND

Our health system orders a high number of STAT priority portable chest radiographs (62%) compared to Routine (35%) and Today (3%). Retrospective chart review of 1000 chest radiographs ordered with the STAT priority revealed that 38% of studies did not indicate clinical urgency. Given the high number or STAT priority portable chest radiographs ordered, prioritizing acquisition and interpretation of true STATs has become challenging for technologists and radiologists, leading to process inefficiencies, long turnaround times (TATs), communication failures, and patient-safety errors.

METHODS

A multidisciplinary team analyzed the current pathway for exam order to finalized report, identified failure modes of imaging order to completion process, and developed guidelines for what constitutes a true STAT examination. A new "urgent" order category meeting the definition of true STAT was designed, tested, and implemented over a 9-month period in participating intensive care units RESULTS: Since study implementation, 108 "urgent" examinations were ordered. Median TAT for a STAT examination from order entry to image acquisition dropped from 70 minutes preimplementation to 16 minutes for "urgent" examinations. Median TAT for exam completion to radiologist image interpretation dropped from 520 minutes preimplementation to 14 minutes for "urgent" examinations. Since implementation, "urgent" examinations were found to be more concordant (70%) with the status of a critically ill patient than STAT examinations (62%).

CONCLUSIONS

The complexity of large multispecialty medical centers and lack of direct interaction of the radiologist with clinicians has led to underappreciation of the needs of ordering providers by radiology, and elucidated system limitations of radiology by ordering providers. By involving a team of frontline clinicians, our team standardized the process of identifying, ordering, procuring, interpreting, and communicating results of true STAT examinations. The process created by our team now serves as a template for implementation in other locations and service lines of our hospital.

A combined pulmonary-radiology workshop for visual evaluation of COPD: study design, chest CT findings and concordance with quantitative evaluation

The purposes of this study were: to describe chest CT findings in normal non-smoking controls and cigarette smokers with and without COPD; to compare the prevalence of CT abnormalities with severity of COPD; and to evaluate concordance between visual and quantitative chest CT (QCT) scoring.

METHODS

Volumetric inspiratory and expiratory CT scans of 294 subjects, including normal non-smokers, smokers without COPD, and smokers with GOLD Stage I-IV COPD, were scored at a multi-reader workshop using a standardized worksheet. There were 58 observers (33 pulmonologists, 25 radiologists); each scan was scored by 9-11 observers. Interobserver agreement was calculated using kappa statistic. Median score of visual observations was compared with QCT measurements.

RESULTS

Interobserver agreement was moderate for the presence or absence of emphysema and for the presence of panlobular emphysema; fair for the presence of centrilobular, paraseptal, and bullous emphysema subtypes and for the presence of bronchial wall thickening; and poor for gas trapping, centrilobular nodularity, mosaic attenuation, and bronchial dilation. Agreement was similar for radiologists and pulmonologists. The prevalence on CT readings of most abnormalities (e.g. emphysema, bronchial wall thickening, mosaic attenuation, expiratory gas trapping) increased significantly with greater COPD severity, while the prevalence of centrilobular nodularity decreased. Concordances between visual scoring and quantitative scoring of emphysema, gas trapping and airway wall thickening were 75%, 87% and 65%, respectively.

CONCLUSIONS

Despite substantial inter-observer variation, visual assessment of chest CT scans in cigarette smokers provides information regarding lung disease severity; visual scoring may be complementary to quantitative evaluation.

Cardiac and pericardial calcifications on chest radiographs

Many types of cardiac and pericardial calcifications identified on chest radiographs can be recognized and distinguished based on characteristic locations and appearances. The purpose of this review is to emphasize the importance of detecting cardiac and pericardial calcifications on chest radiographs, and to illustrate and describe the various types of calcifications that may be encountered and how they may be differentiated from one another. Each type of cardiac and pericardial calcification is discussed, its location and appearance described, and its significance explained. Recognizing and understanding these calcifications is important as they are often encountered in daily practice and play an important role in patient care.

Transcription factor AP2 is required for expression of the rat transforming growth factor-alpha gene

DNase I footprinting of the rat TGF alpha promoter in the presence of crude cell nuclear extract revealed three sites of protein-DNA interaction (Fp-A, Fp-B, Fp-C) in the region from -222 to +73. Mutation of specific sites within the Fp-A and Fp-B regions reduced expression of a TGF alpha promoter-reporter gene (TGF alphaLUC) from 50-90% in transiently transfected CHO cells, indicating the importance of protein/DNA interactions at these sites. Since Fp-A contained a perfect AP2 consensus sequence (5'-GCCNNNGGC-3') as its center, we investigated the possibility that AP2 binding is important for TGF alpha promoter activity. A double-stranded oligonucleotide spanning Fp-A displayed a distinct mobility shift in the presence of nuclear extract that was inhibited by an excess of known functional AP2-binding sequence. Moreover, a similar mobility shift occurred in the presence of purified AP2 protein, and the further addition of AP2 antibody produced a supershifted complex. More refined DNase I footprinting of a smaller, oligonucleotide probe in the presence of purified AP2 protein revealed a protected region that included the putative AP2 binding site. Additionally, co-transfection of an AP2 expression vector increased TGF alphaLUC expression 25-fold in Drosophila Schneider cells. These various findings corroborate a role for AP2 in TGF alpha promoter activity. The Fp-B region contains a T5 motif that has been previously suggested to function as an atypical TATA box. An Fp-B oligonucleotide displayed a specific gel mobility shift in the presence of a TATA binding protein (TBP)-TFIIA complex, and the further addition of TBP antibody produced a supershift. These results confirm that protein binding within Fp-B is functionally important, and they also indicate that the T5 motif functions as a TBP binding site.

Characterization of the mouse transforming growth factor alpha gene: its expression during eyelid development and in waved 1 tissues

The spontaneous mouse waved 1 (wa1) mutation is allelic with the transforming growth factor alpha (TGF-alpha) gene and produces phenotypes similar to those of TGF-alpha knockout mice. Here, we show that TGF-alpha mRNA and protein levels are measurable in wa1 tissues but reduced 5- to 30-fold relative to wild type. Because the wa1-coding sequence is identical to that of the normal mRNA, wa1 is not a null mutation. Nuclear run-on analyses revealed decreased transcription of the TGF-alpha gene in wa1 tissues, but the sequence of a 3.2-kb 5' flanking fragment containing the promoter was unaltered. Moreover, pulsed field gel electrophoresis analysis did not reveal alterations within 750 kb upstream or 350 kb downstream of the gene, and chromosome 6 was karyotypically normal. Hence, we speculate that the wa1 mutation may be subtle and/or reside at a greater distance from the TGF-alpha gene. TGF-alpha deficiency elicits a spectrum of variably penetrant eye anomalies in wa1 and knockout mice that are associated with open eyes at birth. We found that late-gestation wa1 and TGF-alpha-null embryos display a significant delay in eyelid closure, although the eyes of most embryos fuse prior to birth. In situ hybridization localized TGF-alpha expression to the advancing margins of the eyelid epithelium and epidermal growth factor receptor expression throughout the eyelid and corneal epithelia. These results suggest that eye problems observed in TGF-alpha-deficient adult mice arise from premature exposure and trauma to open eyes during or following parturition.

Transcriptional and post-transcriptional induction of the TGFalpha gene in transformed rat liver epithelial cells

Although TGFalpha mRNA and protein are frequently elevated in neoplastic cells, neither the level at which deregulation occurs nor the mechanism(s) responsible have been well characterized. As a first step, we examined the induction of TGFalpha mRNA in two series of clonally-derived rat liver epithelial cell lines that were transformed either by exposure to chemical carcinogen or stable transfection of activated Ha-ras. We found that steady-state levels of TGFalpha mRNA in both series of transformed lines were induced 25- to 50-fold over those in the respective normal parental cells. This induction, which occurred without amplification of the TGF alpha gene, was accompanied by at least a five- to 10-fold increase in transcription along the entire length of the gene with no evidence of a transcriptional attenuation or arrest mechanism in the normal cells. Analysis of the TGFalpha promoter and flanking regions did not support a correlation between the extent of methylation and the level of expression, but did reveal several DNase I hypersensitive sites spanning from -14 to +8 kilobases. Two of these sites were differentially observed in cells displaying high and low TGFalpha gene transcription, while a third site correlated with TPA-induced expression. Finally, measurement of TGFalpha mRNA decay in the presence of Actinomycin D revealed a consistent 1.5- to 3.2-fold increase in the half-life of the TGFalpha transcript in the various transformed cell lines. These results indicate that transformation-mediated induction of TGFalpha gene expression in rat liver epithelial cells occurs through both transcriptional and post-transcriptional mechanisms, but is primarily the result of TGFalpha promoter activation.

Protein interactions at Sp1-like sites in the TGF alpha promoter as visualized by in vivo genomic footprinting

Transcription from the rat TGF alpha promoter initiates at two predominant sites (-188 and -58) in a G+C-rich region that does not contain TATA or CAAT motifs. Previous studies using transfected reporter constructs implicated the transcription factor Sp1 in active expression from the promoter, particularly from the -58 site (Chen et al., 1992; Shin et al., 1992). In the present report we have examined the functionality of two adjacent clusters of Sp1-like recognition sites that are located in the upstream portion of the promoter from -300 to -273. A double-stranded oligonucleotide, which spanned this region and contained the putative Sp1 elements, demonstrated similar gel-mobility shifts in the presence of both crude HeLa cells nuclear extract and pure Sp1 protein. Mutations that simultaneously altered several of the overlapping Sp1 elements significantly reduced the gel-mobility shift activity of this oligonucleotide probe and, when introduced into the promoter templates, inhibited transcription in vitro from the proximal -188 start site. To confirm the binding of protein to these sites in cells, we carried out an in vivo genomic footprinting analysis of this portion of the TGF alpha promoter in normal and transformed rat liver epithelial cell lines that express the endogenous gene at varying levels. This analysis revealed clear evidence of protein/DNA interaction at Sp1-like sites in the -300 and -273 region in cells actively expressing the gene but not in a normal, parental cell line that expressed very low levels of TGF alpha mRNA. Collectively, these results corroborate the functional importance of Sp1 binding elements in the -300 to -273 region, and together with previous findings, indicate that two clusters of Sp1 binding sites respectively determine levels of transcription from the -188 and -58 start sites. Our additional finding that Sp1 mRNA and protein were present at similar levels in normal and transformed cells that expressed the endogenous TGF alpha gene at markedly different levels, suggests that the activity of the TGF alpha promoter could be regulated via the accessibility of Sp1 protein.

Insulin inhibits the estrogen-dependent expression of the chicken very low density apolipoprotein II gene in Leghorn male hepatoma cells

Expression of the very low density apolipoprotein II (apoVLDLII) gene in the chicken is absolutely dependent on estrogen. ApoVLDLII mRNA is expressed in the Leghorn male hepatoma (LMH) cell line in response to estrogen in completely defined medium. Addition of serum to these cultures results in a decrease in apoVLDLII mRNA. Data in this report demonstrate that 1 nM insulin has the same inhibitory effect as 10% serum. Insulin inhibits apoVLDLII mRNA in a dose-dependent manner; 100 fM insulin inhibits the estrogen-dependent response by 76%. After transfection of LMH cells with apoVLDLII sequences from an 8.9-kilobase (kb) genomic clone (pApo107) that contains the entire 2.9-kb coding sequence along with approximately 3 kb each of 5'- and 3'-flanking DNA, the estrogen-dependent expression of apoVLDLII mRNA from both the endogenous gene and transfected DNA is reduced by insulin. Furthermore, insulin reduces by more than 90% the estrogen-dependent expression from a chimeric construct, pApoCAT, which contains apoVLDLII sequences -900/+1455 cloned 5' of the bacterial chloramphenicol acetyltransferase (CAT) reporter gene. To determine the specificity of the response, expression of the pApoCAT construct was tested with insulin-like growth factor-I and insulin. Three hundred picomolar insulin inhibits the estrogen-mediated CAT activity by 50%. Insulin-like growth factor-I at this concentration has no effect or slightly increases the estrogen-dependent expression of pApoCAT, suggesting that the observed inhibitory action is mediated by the insulin receptor. Consequently, the LMH cells provide an excellent model system in which to study the molecular mechanism of insulin and estrogen interaction in the regulation of gene expression.

Estrogen-dependent expression of the chicken very low density apolipoprotein II gene in serum-free cultures of LMH cells

The estrogen-responsive Leghorn strain M chicken hepatoma (LMH) cell line provides a model system for studying the estrogen-dependent, liver-specific expression of avian genes. Serum-free culture conditions have been established that allow expression of apolipoprotein B, very low density apolipoprotein II (apoVLDLII), serum albumin, and transferrin at levels detectable by Northern blot analysis. Regulation of apoVLDLII mRNA by estrogen occurred in an appropriate time- and dose-dependent manner in serum-free cultures of the LMH cells. The expression of apoVLDLII mRNA in serum-free culture was at least 100-fold higher than that expressed in cultures containing 10% serum. The level of estrogen receptors in LMH cells cultured with 10% serum was approximately 2000 receptors per cell, and in serum-free culture approximately 1000 receptors per cell. When these cells were transfected with estrogen receptor DNA and cultured in serum-free medium, apoVLDLII mRNA was decreased relative to that expressed in cells transfected with a control plasmid. These results indicate that when the LMH cells are cultured without serum, estrogen receptors are not the limiting factor for the expression of the apoVLDLII gene.

Functional analysis of regulatory regions upstream and in the first intron of the estrogen-responsive chicken very low density apolipoprotein II gene

Estrogen regulates the expression of the yolk protein genes in the chicken liver during periods of egg laying. While all five of these genes, vitellogenins I, II, and III, very low density apolipoprotein II (apo VLDLII), and apolipoprotein B, respond to estrogen, individual controls are superimposed on their coordinate regulation with respect to the kinetics of induction, magnitude of response, and developmental expression. The estrogen-responsive Leghorn strain M hepatoma (LMH) cell line provides a model system for studying the molecular basis of the similarities and differences in the regulation of these genes. The apoVLDLII gene is regulated by estrogen in LMH cells in an appropriate time- and dose-dependent manner. Regulatory regions of the apoVLDLII gene have been identified by transient transfection studies in LMH cells. All four of the sequences previously shown to bind protein between the TAATA motif at -26 and proximal estrogen response element at -171 are essential in regulation of the apoVLDLII gene. Mutation of any single binding region reduces expression by more than 80%, indicating cooperative interactions of proteins across the entire region. While these sequences will direct assembly of a functional transcription complex, we demonstrate that addition of the first intron of the apoVLDLII gene to the promoter construct results in a 4-fold increase in estrogen-dependent expression following transient transfection into LMH cells. Results of deletion analyses indicate that two distinct regions of the intron contribute to this regulation.