Evaluating postcranial macromorphoscopic traits to estimate population variation among modern South Africans

Population overlap and the variation within and among populations have been globally observed but is often difficult to quantify. To achieve this, numerous different methods need to be explored and validated to assist with the creation of an accurate biological profile. The current lack of databases for postcranial macromorphoscopic traits indicates the need to further investigate if the method can be employed repeatably in a forensic context. The current study aimed to assess the prevalence of eleven postcranial macromorphoscopic traits in a South African sample. A total of 271 postcrania of adult black, coloured, and white South Africans were assessed. The intra- and inter-observer agreement ranged from fair to almost perfect except for the accessory transverse foramen of C1, which had poor agreement between observers. Only seven traits differed significantly between at least two of the groups. Univariate and multivariate random forest models were created to test the positive predictive performance of the traits to classify population affinity. The classification accuracies for the univariate models ranged from 33.3% to 53.0% and ranged from 54.6% to 62.1% for the multivariate models. Based on the variable importance, the traits assessing spinous process bifurcation were the most discriminatory variables. The results indicate that the postcranial MMS approach does not outperform current methods employed to estimate population affinity. Further research needs to be done for the method to have practical applicability for medicolegal casework in South Africa.

Dataset on renal tumor diameter assessment by multiple observers in normal-dose and low-dose CT

Computed tomography-based active surveillance is increasingly used to manage small renal tumors, regardless of patient age. However, there is an unmet need for decreasing radiation exposure while maintaining the necessary accuracy and reproducibility in radiographic measurements, allowing for detecting even minor changes in renal mass size. In this article, we present supplementary data from a multiobserver investigation. We explored the accuracy and reproducibility of low-dose CT (75% dose reduction) compared to normal-dose CT in assessing maximum axial renal tumor diameter. Open-access CT datasets from the 2019 Kidney and Kidney Tumor Segmentation Challenge were used. A web-based platform for assessing observer performance was used by six radiologist observers to obtain and provide data on tumor diameters and accompanying viewing settings, in addition to key images of each measurement and an interactive module for exploring diameter measurements. These data can serve as a baseline and inform future studies investigating and validating lower-dose CT protocols for active surveillance of small renal masses.

Observer agreement in single computerized tomography use for diagnosing paediatric head and neck malignancies at Uganda Cancer Institute

BACKGROUND

In the Ugandan setting, investigation for PHNM with CT uses a protocol with both unenhanced and contrast enhanced procedures hence doubling the ionizing radiation exposure. The purpose of this study was to determine the feasibility of single CT procedures in diagnosing PHNM.

METHODS

This was a cross-sectional study using CT images from patients, aged fifteen years and below, investigated for head and neck malignancies at the Uganda Cancer Institute. Three radiologists, observers A, B and C, with 12, 5 and 2 years of experience, respectively, participated in the study. They independently reported contrast enhanced images (protocol A), unenhanced images (protocol B), then both unenhanced and contrast enhanced images (protocol C) in 2 months intervals. Inter- and intra- observer agreement was determined using Gwen's Agreement coefficient.

RESULTS

Seventy-three CT scans of 36 boys and 37 girls, with a median age of 9 (3-13) years, were used. Intra-and inter-observer agreement on primary tumour location ranged from substantial to almost perfect with the highest intra-observer agreement observed when protocols A and C were compared. Inter-observer agreement for tumour calcifications was substantial for protocol A. Observers A and C demonstrated an almost perfect intra-observer agreement when protocols A and C were compared. There was a substantial inter-observer agreement on diagnosis for all protocols.

CONCLUSIONS

In our setting and examining a limited number of CT images, we demonstrated that contrast-enhanced CT scans provide sufficient information with no evidence of additional value of unenhanced images. Using contrast-enhanced images alone reduced the radiation exposure significantly.

Inter- and intra-observer variability of computed tomography-based parenchymal- and ductal diameters in chronic pancreatitis: a multi-observer international study

PURPOSE

The need for incorporation of quantitative imaging biomarkers of pancreatic parenchymal and ductal structures has been highlighted in recent proposals for new scoring systems in chronic pancreatitis (CP). To quantify inter- and intra-observer variability in CT-based measurements of ductal- and gland diameters in CP patients.

MATERIALS AND METHODS

Prospectively acquired pancreatic CT examinations from 50 CP patients were reviewed by 12 radiologists and four pancreatologists from 10 institutions. Assessment entailed measuring maximum diameter in the axial plane of four structures: (1) pancreatic head (PDhead), (2) pancreatic body (PDbody), (3) main pancreatic duct in the pancreatic head (MPDhead), and (4) body (MPDbody). Agreement was assessed by the 95% limits of agreement with the mean (LOAM), representing how much a single measurement for a specific subject may plausibly deviate from the mean of all measurements on the specific subject. Bland-Altman limits of agreement (LoA) were generated for intra-observer pairs.

RESULTS

The 16 observers completed 6400 caliper placements comprising a first and second measurement session. The widest inter-observer LOAM was seen with PDhead (± 9.1 mm), followed by PDbody (± 5.1 mm), MPDhead (± 3.2 mm), and MPDbody (± 2.6 mm), whereas the mean intra-observer LoA width was ± 7.3, ± 5.1, ± 3.7, and ± 2.4 mm, respectively.

CONCLUSION

Substantial intra- and inter-observer variability was observed in pancreatic two-point measurements. This was especially pronounced for parenchymal and duct diameters of the pancreatic head. These findings challenge the implementation of two-point measurements as the foundation for quantitative imaging scoring systems in CP.

Observer agreement of imaging measurements used for evaluation of dentofacial deformity in juvenile idiopathic arthritis

OBJECTIVES

To examine the precision of imaging measures commonly used to assess mandibular morphology in children and adolescents with juvenile idiopathic arthritis (JIA). Secondly, to compare cone-beam computed tomography (CBCT) and magnetic resonance imaging (MRI) in the measurement of condylar height.

METHODS

Those included were children diagnosed with JIA during 2015-18 who had had an MRI, a CBCT of the temporomandibular joints (TMJs) and a lateral cephalogram (ceph) of the head within one month of each other. Agreement within and between observers and methods was examined using Bland-Altman mean-difference plots and 95% limits of agreement (LOA). A 95% LOA within 15% of the sample mean was considered acceptable. Minimal detectable change (MDC) within and between observers was estimated.

RESULTS

90 patients (33 males) were included, with a mean age of 12.8 years. For MRI, intra- and interobserver 95% LOA were relatively narrow for total mandibular length: 9.6% of the sample mean. For CBCT, condylar height, both intra- and interobserver 95% LOA were wide: 16.0 and 28.4% of the sample mean, respectively. For ceph, both intra- and interobserver 95% LOA were narrow for the SNA-angle and gonion angle: 5.9 and 8% of the sample mean, and 6.2 and 6.8%, respectively.

CONCLUSIONS

We have identified a set of precise measurements for facial morphology assessments in JIA, including one MRI-based (total mandibular length), one CBCT-based (condylar height), and three ceph-based. Condylar height was higher for MRI than for CBCT; however, the measurement was too imprecise for clinical use. MDC was also determined for a series of measurements.

Intra and inter observer agreement in the mobility assessment of the upper thoracic costovertebral joints

BACKGROUND

Determining the mobility of the costovertebral joints might be important in patients with neck-upper thoracic pain. Little research has been performed on observer agreement when assessing the mobility of these joints.

PURPOSE

The purpose of this study was to determine intra- and inter-observer agreements when assessing costovertebral joint mobility of the upper three ribs in those with and without neck pain and to compare the difference between the Kappa and AC statistic.

METHODS

Forty-four participants, with and without current neck/upper-thoracic pain, were assessed by two raters. Raters applied a posterior to anterior pressure to the anatomical neck of the first three ribs bilaterally. Mobility was graded: normal, increased, or decreased. An AC1, for nominal data, AC2 for ordinal data, and the Kappa statistics were used to analyze the results. The AC statistics determines chance agreement different than Kappa.

RESULTS

AC1 showed "moderate to very good" (0.74-1.0) intra-rater reliability, while inter-rater reliability showed "fair to good" agreement (0.51-0.79). Using the AC2 Intra-rater reliability was "very good" and "almost perfect" (AC2: 0.93-1.0), while using the AC2 was "good to very good" and "good to almost perfect" (0.76-0.94). Kappa values for intra-rater reliability ranged from "fair to moderate" (0.38-0.54), while inter-rater reliability ranged from "poor to fair" (-0.10-0.26).

CONCLUSION

Posterior/anterior pressure is a reliable method to assess the mobility of the upper costovertebral joints. Assessing costovertebral mobility is important when establishing a movement diagnosis in patients with neck/upper thoracic pain.

Analysing the quality of Swiss National Forest Inventory measurements of woody species richness

BACKGROUND

Under ongoing climate and land-use change, biodiversity is continuously decreasing and monitoring biodiversity is becoming increasingly important. National Forest Inventory (NFI) programmes provide valuable time-series data on biodiversity and thus contribute to assessments of the state and trends in biodiversity, as well as ecosystem functioning. Data quality in this context is of paramount relevance, particularly for ensuring a meaningful interpretation of changes. The Swiss NFI revisits about 8%-10% of its sample plots regularly in repeat surveys to supervise the quality of fieldwork.

METHODS

We analysed the relevance of observer bias with equivalence tests, examined data quality objectives defined by the Swiss NFI instructors, and calculated the pseudo-turnover (PT) of species composition, that is, the percentage of species not observed by both teams. Three attributes of woody species richness from the latest Swiss NFI cycles (3 and 4) were analysed: occurrence of small tree and shrub species (1) on the sample plot and (2) at the forest edge, and (3) main shrub and trees species in the upper storey.

RESULTS

We found equivalent results between regular and repeat surveys for all attributes. Data quality, however, was significantly below expectations in all cases, that is, as much as 20%-30% below the expected data quality limit of 70%-80% (proportion of observations that should not deviate from a predefined threshold). PT values were about 10%-20%, and the PT of two out of three attributes decreased significantly in NFI4. This type of uncertainty - typically caused by a mixture of overlooking and misidentifying species - should be considered carefully when interpreting change figures on species richness estimates from NFI data.

CONCLUSIONS

Our results provide important information on the data quality achieved in Swiss NFIs in terms of the reproducibility of the collected data. The three applied approaches proved to be effective for evaluating the quality of plot-level species richness and composition data in forest inventories and other biodiversity monitoring programmes. As such, they could also be recommended for assessing the quality of biodiversity indices derived from monitoring data.

Subjective and objective evaluations of horses for fit-to-compete or unfit-to-compete judgement

At Fédération Equestre Internationale (FEI) competitions, horses undergo veterinary inspection for judgement of 'fit-to-compete'. However, FEI Veterinary Delegates (VDs) often differ in opinion. The aim of the present study was to evaluate intra- and inter-observer agreements of fit-to-compete judgement and compare these with objective gait analysis measurements. Twelve horses were evaluated by three experienced VDs and one veterinary specialist and video-recorded for re-evaluation later. Simultaneously, quantitative gait analysis measurements were acquired. Inter-observer agreement during live evaluations was fair (κ = 0.395, 58% agreement). Intra-observer agreement between live observations and videos at one and 12 months was 71% and 73% respectively. Sensitivity and specificity of motion symmetry measured with quantitative gait analysis system were 83.3% and 66.7% respectively, against the consensus of all observers as a reference. These findings might suggest that more VDs should be used to adequately judge fit-to-compete. Quantitative-gait-analysis may be useful to support decision making during fit-to-compete judgement.

Optimal Phase of Dynamic Computed Tomography for Reliable Size Measurement of Metastatic Neuroendocrine Tumors of the Liver: Comparison between Pre- and Post-Contrast Phases

Objective

The reliability of size measurements of liver metastases from neuroendocrine tumors (NETs) on contrast-enhanced computed tomography (CT) phases made by different readers may be hampered due to transient, variable rim enhancement in arterial phase (AP) or portal venous phase (PVP) images. We aimed to assess the reliability of tumor size measurements in pre- and post-contrast scans.

Materials and Methods

The study coordinator selected target lesions according to Response Evaluation Criteria in Solid Tumors 1.1 guidelines in 44 consecutive patients with pathologically confirmed NET liver metastases. Two blinded readers measured the longest diameters of target lesions on pre-contrast, AP, and PVP images twice with a 4-week interval. Inter- and intra-observer agreements were evaluated using Bland-Altman plots and 95% limit of agreement (LOA) calculations.

Results

Of the 79 target lesions (approximate mean size of 3 cm), 45 showed rim enhancement. Inter-observer agreement assessed based on LOA was highest in pre-contrast CT images (−6.1–5.7 mm), followed by PVP (−7.9–7.1 mm) and AP (−8.5–7.4 mm) images. Intra-observer agreement showed the same trend: −2.8–2.9 mm and −2.9–2.9 mm for readers 1 and 2, respectively, on pre-contrast CT, −2.8–2.9 mm and −3.0–3.2 mm, respectively, on PVP, and −3.2–4.2 mm and −3.4–3.2 mm, respectively, on AP images. Mean tumor diameters differed significantly among the phases in the following increasing order: pre-contrast CT, PVP, and AP images.

Conclusion

There was better inter- and intra-observer agreement in size measurements of NET liver metastases on precontrast scans than on AP and PVP scans. Pre-contrast CT may be the optimal for measuring NET liver metastases if its accuracy is proven.

Ultrasound assessment of pubertal breast development in girls: intra- and interobserver agreement

BACKGROUND

Clinical assessment of pubertal breast development using Tanner staging is subjective. This has led to the introduction of ultrasound (US), aiming for a more objective analysis. However, information regarding its reliability is lacking.

OBJECTIVE

To examine intra- and interobserver agreement of breast maturity staging using US and to examine the precision of direct measurements of the gland.

MATERIALS AND METHODS

Fifty-seven healthy girls (mean age: 10.9 years, range: 6.1 to 15.9 years) were examined independently by two observers using US of the left breast to score the glandular maturity stage on a 0-5 scale, and to measure the depth and diameter. One observer repeated the examination after 20 to 35 min to assess intra-observer agreement. Cohen's kappa with linear weights was used to examine intra- and interobserver agreement of the US staging, while the measurement precision was analyzed using Bland-Altman plots and 95% limits of agreement.

RESULTS

The agreement of US staging on a 0-5 scale was very good (kappa 0.84; 95% confidence interval [CI] 0.78-0.91) for intra-observer observation and good (kappa 0.71; 95% CI 0.62-0.80) for interobserver observation. Measurements of glandular depth and diameter were unbiased for a single observer, but the variances were large both within and between observers.

CONCLUSION

US using a scale from 0 to 5 is a reliable method to stage the development of glandular breast tissue during puberty in healthy girls. Measurements of glandular depth and diameter were found to be imprecise.

Observer Agreement for Measurements in Videolaryngostroboscopy

OBJECTIVE

This study evaluated the levels of intraobserver and interobserver agreement for measurements of visuoperceptual variables in videolaryngostroboscopic examinations and compared the observers' behavior during independent versus consensus panel rating.

STUDY DESIGN

This is a retrospective study.

SETTING

This study was conducted in a single-center tertiary care facility.

PARTICIPANTS

Sixty-four patients with dysphonia of heterogeneous etiology were included.

EXPOSURE

All subjects underwent a standardized videolaryngostroboscopic examination.

MAIN OUTCOME AND MEASURES

Two experienced and trained observers scored exactly the same examinations, first independently and then on a consensus panel. Specific visuoperceptual variables and the clinical diagnosis (as recommended by the Committee on Phoniatrics and the Phonosurgery Committee of the European Laryngological Society and advised by the American Speech-Language-Hearing Association) were scored. Descriptive and kappa statistics were used.

RESULTS

In general, intraobserver agreement was better than agreement between observers for measurements of several variables. The intrapanel observer agreement levels were slightly higher than the intraobserver agreement levels on the independent rating task. When rating on the consensus panel, the observers deviated considerably from the scores they had previously given on the independent rating task.

CONCLUSION AND RELEVANCE

Observer agreement in videolaryngostroboscopic assessment has important implications not only for the diagnosis and treatment of dysphonic patients but also for the interpretation of the results of scientific studies using videolaryngostroboscopic outcome parameters. The identification of factors that can influence the levels of observer agreement can provide a better understanding of the rating process and its limitations. The results of this study suggest that future research could achieve better agreement levels by rating the visuoperceptual variables in a panel setting.

The compensatory enlargement of the remaining thyroid lobe following hemithyroidectomy is small and without impact on symptom relief

According to previous studies, hemithyroidectomy results in growth of the remaining thyroid lobe by up to 30% in first 12 months after surgery. However, this estimate is based on imprecise methods, high inter- and intra-observer variability, and lack of blinding of the measurements. Furthermore, it is unknown whether enlargement of the remaining hemi-thyroid interferes with the improvement in symptoms after surgery for goiter. We aimed to assess the impact of postoperative thyroid growth on goiter symptom relief following hemithyroidectomy in patients with benign nodular goiter. Outcomes were measured before and 6 months after hemithyroidectomy in 44 patients. Thyroid volumes were determined by two independent and blinded observers using magnetic resonance imaging (MRI). Inter- and intra-observer variability was visualized by Bland-Altman plots. Goiter symptoms were assessed by the Thyroid-Specific Patient-Reported-Outcome Questionnaire (ThyPRO) on a scale from 0 to 100 points. After hemithyroidectomy, the remaining thyroid lobe was 13.7 ± 6.4 mL, and enlarged by a mean of 1.8 mL over 6 months [95% confidence interval (CI) (1.6; 2.1), p < 0.001], corresponding to an increase of 17% [95% CI (12; 22)]. The Goiter Symptom score improved by 27 points [95% CI (21; 34), p < 0.0001] from median 39 points (range 2-86) at baseline, and was unaffected by the compensatory thyroid growth. Six months after hemithyroidectomy, using blinded MRI evaluations, we demonstrated a small but significant postoperative growth of the remaining hemi-thyroid, which did not significantly affect the considerable improvement in goiter symptoms.

Image assessment of Barrett's esophagus using the simplified narrow band imaging classification

BACKGROUND

A simplified narrow band imaging (NBI) classification has been proposed with the objective of integrating multiple classifications of NBI surface patterns in Barrett's esophagus (BE). Little is known about the impact of the simplified NBI classification on the diagnosis of BE when using high-definition magnification endoscopy with NBI (HM-NBI). This study aimed to evaluate (a) the reproducibility of NBI surface patterns and predicted histology and (b) the diagnostic accuracy of interpreting HM-NBI images by using the simplified NBI classification.

METHODS

Two hundred and forty-eight HM-NBI images from macroscopically normal areas in patients with BE were retrieved from endoscopy databases and randomized for review by four endoscopists (two experts, two non-experts). We evaluated inter- and intra-observer agreement of the interpretation of NBI surface patterns and the predicted histology (dysplasia vs. non-dysplasia), as calculated by using κ statistics, and diagnostic values of the prediction.

RESULTS

The overall inter-observer agreements were substantial for mucosal pattern (κ = 0.73) and vascular pattern (κ = 0.71), and almost perfect for predicting dysplastic histology (κ = 0.80). The overall intra-observer agreements were almost perfect for mucosal (κ = 0.84) and vascular patterns (κ = 0.86), and predicting dysplastic histology (κ = 0.89). The mean accuracy in predicting dysplastic histology for all reviewers was 95 % (experts: 96.8 %, non-experts: 93.1 %).

CONCLUSIONS

The simplified NBI classification has the potential to provide high diagnostic reproducibility and accuracy when using HM-NBI.

Evaluation of AMSTAR to assess the methodological quality of systematic reviews in overviews of reviews of healthcare interventions

Background

Overviews of reviews (overviews) compile information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. It is recommended that authors assess and report the methodological quality of SRs in overviews—for example, using A MeaSurement Tool to Assess systematic Reviews (AMSTAR). Currently, there is variation in whether and how overview authors assess and report SR quality, and limited guidance is available. Our objectives were to: examine methodological considerations involved in using AMSTAR to assess the quality of Cochrane and non-Cochrane SRs in overviews of healthcare interventions; identify challenges (and develop potential decision rules) when using AMSTAR in overviews; and examine the potential impact of considering methodological quality when making inclusion decisions in overviews.

Methods

We selected seven overviews of healthcare interventions and included all SRs meeting each overview’s inclusion criteria. For each SR, two reviewers independently conducted AMSTAR assessments with consensus and discussed challenges encountered. We also examined the correlation between AMSTAR assessments and SR results/conclusions.

Results

Ninety-five SRs were included (30 Cochrane, 65 non-Cochrane). Mean AMSTAR assessments (9.6/11 vs. 5.5/11; p < 0.001) and inter-rater reliability (AC1 statistic: 0.84 vs. 0.69; “almost perfect” vs. “substantial” using the Landis & Koch criteria) were higher for Cochrane compared to non-Cochrane SRs. Four challenges were identified when applying AMSTAR in overviews: the scope of the SRs and overviews often differed; SRs examining similar topics sometimes made different methodological decisions; reporting of non-Cochrane SRs was sometimes poor; and some non-Cochrane SRs included other SRs as well as primary studies. Decision rules were developed to address each challenge. We found no evidence that AMSTAR assessments were correlated with SR results/conclusions.

Conclusions

Results indicate that the AMSTAR tool can be used successfully in overviews that include Cochrane and non-Cochrane SRs, though decision rules may be useful to circumvent common challenges. Findings support existing recommendations that quality assessments of SRs in overviews be conducted independently, in duplicate, with a process for consensus. Results also suggest that using methodological quality to guide inclusion decisions (e.g., to exclude poorly conducted and reported SRs) may not introduce bias into the overview process.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-017-0325-5) contains supplementary material, which is available to authorized users.

Validation of a new method for the endoscopic measurement of post-bariatric gastric outlet using a standard guidewire: an observer agreement study

Background and aims

Between 10 and 20% of all patients undergoing bariatric surgery procedures regain weight secondary to a gastrojejunostomy enlargement. The aim of this study was to validate the interobserver agreement while measuring gastric outlet diameters using a new standard guidewire.

Methods

We selected thirty-five videos of consecutive endoscopic procedures on patients undergoing esophagogastroduodenoscopy after a Roux-en-Y gastric bypass procedure. All videos were evaluated by four raters: two expert endoscopists and two trainees. We excluded videos having a slipped Fobi ring or a strictured gastric outlet. Anastomosis diameter was measured using a novel device with standardized markings on a guidewire (Hydra jagwire, Boston Scientific, Natick. MA) as well as the current gold standard defined as a calibrated endoscopic measuring instrument (Olympus America, Center Valley, PA).

Results

We obtained 272 measurements of the gastric outlet. Overall agreement measured through intra-class correlation coefficients for the gold standard was 0.84 (p < 0.01) and 0.83 (p < 0.01) for the new guidewire. Agreement among experts was 0.699 (p < 0.01), while among trainees it was 0.822 (p < 0.01).

Conclusion

The new guidewire demonstrated a high degree of observer reliability, also presenting similar results between expert endoscopists and trainees.

Inter-observer agreement for clinical examinations of foot lesions of sheep

In sheep, the diagnosis of foot lesions is routinely based on physical examination of the hoof. Correct diagnosis is important for the effective treatment, prevention and control of both infectious and non-infectious causes of lameness. Therefore, the aim of this study was to evaluate the level of inter-observer agreement for clinical examination of ovine foot lesions. Eight observers of varying experience, training and occupation performed foot examinations on a total of 1158 sheep from 38 farms across North England and Wales. On each farm, a group of two to four observers independently examined a sample of 24 to 30 sheep to diagnose the presence or absence of specific foot lesions including white line lesions (WL), contagious ovine digital dermatitis (CODD), footrot (FR), inter-digital dermatitis (ID) and toe granuloma (TG). The inter-observer agreement of foot lesion assessments was examined using Fleiss kappa (κ), and Cohen's κ examined the paired agreement between the test standard observer (TSO) and each observer. Scoring differences with the TSO were examined as the percentage of scoring errors and assessed for evidence of systematic scoring bias. With the exception of WL (maximum error rate 33.3%), few scoring differences with the TSO occurred (maximum error rate 3.3%). This suggests that observers can achieve good levels of reliability when diagnosing most of the commonly observed foot conditions associated with lameness in sheep.

Spine Instability Neoplastic Score: agreement across different medical and surgical specialties

BACKGROUND CONTEXT

Spinal instability is an acknowledged complication of spinal metastases; in spite of recent suggested criteria, it is not clearly defined in the literature.

PURPOSE

This study aimed to assess intra and interobserver agreement when using the Spine Instability Neoplastic Score (SINS) by all physicians involved in its management.

STUDY DESIGN

Independent multicenter reliability study for the recently created SINS, undertaken with a panel of medical oncologists, neurosurgeons, radiologists, orthopedic surgeons, and radiation oncologists, was carried out.

PATIENT SAMPLE

Ninety patients with biopsy-proven spinal metastases and magnetic resonance imaging, reviewed at the multidisciplinary tumor board of our institution, were included.

OUTCOME MEASURES

Intraclass correlation coefficient (ICC) was used for SINS score agreement. Fleiss kappa statistic was used to assess agreement on the location of the most affected vertebral level; agreement on the SINS category ("stable," "potentially stable," or "unstable"); and overall agreement with the classification established by tumor board.

METHODS

Clinical data and imaging were provided to 83 specialists in 44 hospitals across 14 Spanish regions. No assessment criteria were pre-established. Each clinician assessed the SINS score twice, with a minimum 6-week interval. Clinicians were blinded to assessments made by other specialists and to their own previous assessment. Subgroup analyses were performed by clinicians' specialty, experience (≤7, 8-13, ≥14 years), and hospital category (four levels according to size and complexity). This study was supported by Kovacs Foundation.

RESULTS

Intra and interobserver agreement on the location of the most affected levels was "almost perfect" (κ>0.94). Intra-observer agreement on the SINS score was "excellent" (ICC=0.77), whereas interobserver agreement was "moderate" (ICC=0.55). Intra-observer agreement in SINS category was "substantial" (k=0.61), whereas interobserver agreement was "moderate" (k=0.42). Overall agreement with the tumor board classification was "substantial" (κ=0.61). Results were similar across specialties, years of experience, and hospital category.

CONCLUSIONS

Agreement on the assessment of metastatic spine instability is moderate. The SINS can help improve communication among clinicians in oncology care.

Inter- and intraobserver agreement of ADC measurements of lung cancer in free breathing, breath-hold and respiratory triggered diffusion-weighted MRI

OBJECTIVE

To prospectively evaluate the inter- and intraobserver agreement of apparent diffusion coefficient (ADC) measurements in free breathing, breath-hold, and respiratory triggered diffusion-weighted imaging (DWI) of lung cancer.

METHODS

Twenty-two patients with lung cancer (tumor size >2cm) underwent DWIs (3.0T) in three imaging methods. Lesion ADCs were measured twice by both of the two independent observers and compared.

RESULTS

No statistical significance was found among methods, though respiratory-triggered DWI tended to have higher ADCs than breath-hold DWI. Great inter- and intraobserver agreement was shown.

CONCLUSION

ADCs had good inter- and intraobserver agreement in all three DWI methods.

Cesarean section scar measurements in non-pregnant women using three-dimensional ultrasound: a repeatability study

OBJECTIVE

To evaluate intra- and inter-observer agreement in measurements of the cesarean scar niche and the residual myometrial thickness (RMT) using 3-dimensional (3D) transvaginal ultrasonography.

STUDY DESIGN

Fifty-eight uterine 3D volumes from women with deep cesarean scar niches were evaluated. 3D volumes were obtained six to fifteen months after a primary cesarean section. Evaluation of the 3D volume was performed in a standardized multiplanar view. Two observers independently obtained RMT, cesarean scar niche depth (D), length (L), width (W), and myometrium adjacent to the scar (M). Differences within and between observers were expressed in mm and were evaluated according to the Bland-Altman method including the calculation of limits of agreement (LOAs).

RESULTS

The intra-observer LOAs in mm were as follows: RMT: -3.7 to 4.0; D: -2.2 to 2.6; L: -3.6 to 4.2; W: -4.0 to 3.7; and M: -3.4 to 4.5. The inter-observer LOAs in mm were as follows: RMT: -3.2 to 4.1; D: -3.3 to 2.2; L: -3.4 to 4.2; W: -3.2 to 4.1; and M: -4.1 to 3.2.

CONCLUSIONS

In non-pregnant women, we found rather wide limits of agreement measuring the cesarean section scar niche and myometrium using 3D volumes. Whether 3D transvaginal ultrasonography provides clinical advantages compared to 2D TVU needs clarification.

Observers' Agreement on Measurements in Fiberoptic Endoscopic Evaluation of Swallowing

This study analyzed the effect that dysphagia etiology, different observers, and bolus consistency might have on the level of agreement for measurements in FEES images reached by independent versus consensus panel rating. Sixty patients were included and divided into two groups according to dysphagia etiology: neurological or head and neck oncological. All patients underwent standardized FEES examination using thin and thick liquid consistencies. Two observers scored the same exams, first independently and then in a consensus panel. Four ordinal FEES variables were analyzed. Statistical analysis was performed using a linear weighted kappa coefficient and Bayesian multilevel model. Intra- and interobserver agreement on FEES measurements ranged from 0.76 to 0.93 and from 0.61 to 0.88, respectively. Dysphagia etiology did not influence observers’ agreement level. However, bolus consistency resulted in decreased interobserver agreement for all measured FEES variables during thin liquid swallows. When rating on the consensus panel, the observers deviated considerably from the scores they had previously given on the independent rating task. Observer agreement on measurements in FEES exams was influenced by bolus consistency, not by dysphagia etiology. Therefore, observer agreement on FEES measurements should be analyzed by taking bolus consistency into account, as it might affect the interpretation of the outcome. Identifying factors that might influence agreement levels could lead to better understanding of the rating process and assist in developing a more precise measurement scale that would ensure higher levels of observer agreement for measurements in FEES exams.

Diagnosis of vertebral fractures in children: is a simplified algorithm-based qualitative technique reliable?

BACKGROUND

Identification of osteoporotic vertebral fractures allows treatment opportunity reducing future risk. There is no agreed standardised method for diagnosing paediatric vertebral fractures.

OBJECTIVE

To evaluate the precision of a modified adult algorithm-based qualitative (ABQ) technique, applicable to children with primary or secondary osteoporosis.

MATERIALS AND METHODS

Three radiologists independently assessed lateral spine radiographs of 50 children with suspected reduction in bone mineral density using a modified ABQ scoring system and following simplification to include only clinically relevant parameters, a simplified ABQ score. A final consensus of all observers using simplified ABQ was performed as a reference standard for fracture characterisation. Kappa was calculated for interobserver agreement of the components of both scoring systems and intraobserver agreement of simplified ABQ based on a second read of 29 randomly selected images.

RESULTS

Interobserver Kappa for modified ABQ scoring for fracture detection, severity and shape ranged from 0.34 to 0.49 Kappa for abnormal endplate and position assessment was 0.27 to 0.38. Inter- and intraobserver Kappa for simplified ABQ scoring for fracture detection and grade ranged from 0.37 to 0.46 and 0.45 to 0.56, respectively. Inter- and intraobserver Kappa for affected endplate ranged from 0.31 to 0.41 and 0.45 to 0.51, respectively. Subjectively, observers' felt simplified ABQ was easier and less time-consuming.

CONCLUSION

Observer reliability of modified and simplified ABQ was similar, with slight to moderate agreement for fracture detection and grade/severity. Due to subjective preference for simplified ABQ, we suggest its use as a semi-objective measure of diagnosing paediatric vertebral fractures.

The Spine Instability Neoplastic Score: an independent reliability and reproducibility analysis

BACKGROUND

Metastatic vertebral instability has not yet been clearly defined in the literature; there still exists a paucity of reliable criteria to assess the risk of vertebral collapse.

PURPOSE

We performed an independent interobserver and intraobserver agreement evaluation of the Spine Instability Neoplastic Score (SINS) and correlated the score with selected clinical cases and the treatment they received.

STUDY DESIGN

Independent reliability study for the newly created SINS.

PATIENT SAMPLE

Thirty patients who underwent either radiotherapy alone or surgery followed by radiotherapy were randomly selected from the orthopedic surgery and radiotherapy department's databases.

OUTCOME MEASURES

Patients were rated and classified for spinal stability using SINS. Intraclass correlation coefficient (ICC) and Fleiss's kappa measures were occupied for reliability analysis.

METHODS

Patients who underwent either radiotherapy alone or surgery followed by radiotherapy were randomly selected and classified for spinal stability using the SINS by orthopedic surgeons and nonorthopedic oncology specialists. ICC and Fleiss's kappa were calculated for inter- and intraobserver agreement. A comparative analysis of SINS and the actual management was also conducted.

RESULTS

Interobserver ICC reliability for the SINS was 0.79; κ values for location, pain, bone quality, alignment, vertebral body collapse, and posterolateral involvement were 0.81, 0.58, 0.21, 0.45, 0.42, and 0.29 respectively. Intraobserver ICC for the SINS scores was 0.96; ICC values for the same components were 0.98, 0.98, 0.87, 0.88, 0.92, and 0.86, respectively. Potentially unstable lesions (SINS score≥7) were operated on in 62.5%.

CONCLUSIONS

SINS seem to be a reproducible tool that could be used equally by multiple specialists to estimate metastatic vertebral stability; however, prospective clinical validation is still pending.

Interobserver agreement of semi-automated and manual measurements of functional MRI metrics of treatment response in hepatocellular carcinoma

PURPOSE

To assess the interobserver agreement in 50 patients with hepatocellular carcinoma (HCC) before and 1 month after intra-arterial therapy (IAT) using two semi-automated methods and a manual approach for the following functional, volumetric and morphologic parameters: (1) apparent diffusion coefficient (ADC), (2) arterial phase enhancement (AE), (3) portal venous phase enhancement (VE), (4) tumor volume, and assessment according to (5) the Response Evaluation Criteria in Solid Tumors (RECIST), and (6) the European Association for the Study of the Liver (EASL).

MATERIALS AND METHODS

This HIPAA-compliant retrospective study had institutional review board approval. The requirement for patient informed consent was waived. Tumor ADC, AE, VE, volume, RECIST, and EASL in 50 index lesions was measured by three observers. Interobserver reproducibility was evaluated using intraclass correlation coefficients (ICC). P<0.05 was considered to indicate a significant difference.

RESULTS

Semi-automated volumetric measurements of functional parameters (ADC, AE, and VE) before and after IAT as well as change in tumor ADC, AE, or VE had better interobserver agreement (ICC=0.830-0.974) compared with manual ROI-based axial measurements (ICC=0.157-0.799). Semi-automated measurements of tumor volume and size in the axial plane before and after IAT had better interobserver agreement (ICC=0.854-0.996) compared with manual size measurements (ICC=0.543-0.596), and interobserver agreement for change in tumor RECIST size was also higher using semi-automated measurements (ICC=0.655) compared with manual measurements (ICC=0.169). EASL measurements of tumor enhancement in the axial plane before and after IAT ((ICC=0.758-0.809), and changes in EASL after IAT (ICC=0.653) had good interobserver agreement.

CONCLUSION

Semi-automated measurements of functional changes assessed by ADC and VE based on whole-lesion segmentation demonstrated better reproducibility than ROI-based axial measurements, or RECIST or EASL measurements.