Performance of intraclass correlation coefficient (ICC) as a reliability index under various distributions in scale reliability studies

A novel software for method comparison: MCS (method comparison software)-assessing agreement between estimated fetal weights calculated by Hadlock I-V formulas and birth weight

INTRODUCTION

The evaluation of the performance of new methods, expected to provide cheaper and faster results than existing (reference) methods in the health field, is based on comparing the results obtained with this new method to those obtained with the existing method. The primary aim of this study is to examine the correlational and absolute agreement between measurement methods in clinical studies using Bland-Altman analysis and methodological (Ordinary Least Squares, Weighted Ordinary Least Squares, Deming, Weighted Deming, Passing-Bablok, Theil-Sen, and Passing-Bablok for Large Data Sets.) methods, and the secondary aim is to compare the accuracy and precision of Hadlock (I-V) formulas used for fetal weight estimation.

MATERIALS AND METHODS

The study was conducted on singleton pregnancies examined in the Prenatal Diagnosis and Treatment Unit of the Department of Obstetrics and Gynecology at Inonu University Faculty of Medicine and who gave birth in the Obstetrics Unit between 01.01.2020 and 01.09.2023, whose gestational ages were confirmed by first-trimester ultrasonography. Estimated fetal weights were calculated using Hadlock (I-V) formulas, and the agreement of these weights with birth weight was evaluated with Bland-Altman method.

RESULTS

The comparison of estimated fetal weights calculated using Hadlock formulas with birth weight was analyzed using Bland-Altman analysis, ICC, and CCC values along with regression analyses. According to the mean difference values obtained by Bland-Altman analysis, the estimated fetal birth weights obtained by the Hadlock IV formula were most consistent with the actual birth weights.

CONCLUSIONS

The estimated fetal weights obtained using the Hadlock IV formula resulted in the closest measurements to the birth weight. This study showcases the efficacy of a new web-based software, Method Comparison Software (MCS), which can be utilized for evaluating the agreement between different methods in clinical measurements.

Analysis of intrarater and interrater reliability of the interactive line protocol for evaluating neuromuscular function in patients with knee injury and observation of psychological state changes

The isokinetic interactive line protocol (ILP) appears to enhance neuromuscular function and may reduce posttraumatic stress and anxiety. However, the reliability of the protocol has not been investigated, nor has it been analyzed in relation to psychological states. A total of 165 participants took part in this study, with ages ranging from 21 to 24 years. The gender distribution was 68.4% male and 31.6% female. There was no significant difference in injury duration between males and females (z=-0.922, P=0.357). The intrarater reliability of the ILP, as indicated by the intraclass correlation coefficient, was high for both concentric and eccentric contractions of the quadriceps (ranging from 0.891 to 0.980). Likewise, the interrater reliability for these contractions was also high (range, 0.873-0.943). Additionally, during the ILP, there was a significant reduction in posttraumatic stress (t=9.795, P<0.001) and anxiety (t=5.965, P<0.001) related to knee reinjury after 2 weeks. Ultimately, the isokinetic game ILP demonstrates high intra- and interrater reliabilities and can be concluded to effectively reduce posttraumatic stress and anxiety when incorporated into retesting.

Reliabilities of three methods used to evaluate computer-assisted mandibular reconstructions using free fibula flaps

Objective

This study compared the reliabilities of three different methods used to calculate surgical deviations after mandibular reconstructions using free fibular flaps.

Study design

This retrospective study involved 35 patients who underwent computer-assisted mandibular reconstructions using free fibula flaps. The deviations between the virtual surgical plans and the postoperative results were independently analyzed by two researchers using three distinct methods. In Method A, the fibular axis, the center of gravity, and the osteotomy plane served as landmarks when measuring surgical deviations. In Methods B and C, manually designated points were used to measure errors in the fibular length and intersegmental angle. The primary outcome variables were the intraclass correlation coefficients (ICCs) that revealed the inter-rater agreements for all three methods.

Results

The use of Method A was associated with good agreement in terms of the fibular length deviation (ICC = 0.765) and intersegmental angle (ICC = 0.897); both were higher than those afforded by Methods B (ICC = 0.158 and 0.108) and C (ICC = 0.406 and 0.463). The measurements of the fibular transfer osteotomy deviation (ICC = 0.888), linear deviation (0.926), and angular deviation (0.958) were very reliable.

Conclusions

Method A afforded the highest reliability in clinical practice when evaluating surgical deviations after mandibular reconstruction using fibular flaps.

The Retest Reliability of Common Tests to Assess Physical Qualities in Adolescent Rugby League Players

PURPOSE

Rigorous evidence concerning the reliability of physical tests among young athletes is lacking. Therefore, this study aimed to comprehensively determine the retest reliability of the most commonly used tests to assess physical qualities in adolescent rugby league players, and to provide the standard error of measurement and minimal detectable change for each test to support objective athlete monitoring.

METHOD

A repeated-measures design was employed with 50 adolescent, schoolboy rugby league players (16.2 [1.3] y) completing the same testing battery across 2 consecutive weeks. Within each week, stature, body mass, and ∑4 skinfold thickness assessments, as well as countermovement jump, medicine ball throw, and 1-repetition maximum bench press, back squat, and prone row tests were conducted on the first testing day. Forty-eight hours later, a second testing day contained the 10- and 20-m linear sprints, 505-Agility Test, L-run Test, and Multistage Fitness Test.

RESULTS

Nonsignificant (P > .05), trivial differences (dav = 0.00-0.16) were evident between trials in all tests. The coefficient of variation ranged from 0.05% to 4.02% across tests. The intraclass correlation ranged from .704 to .999. All tests presented relatively low standard error of measurement, with accompanying minimal detectable change provided.

CONCLUSION

These data can help inform decision making among end-users when implementing tests in a repeated manner and interpreting testing data.

Test-Retest Reliability of a Physical Activity Behavior, Health and Wellbeing Questionnaire in Adolescents

Background

The aim of this study was to examine the test-retest reliability of the physical activity behavior, health and wellbeing questionnaire, in adolescent populations, administered by teachers in school settings, in the Republic of Ireland.

Methods

A cross-sectional, mixed sample of 55 participants (45.5% males: Age, 13.94 (±.40) years) were included. The participants completed the questionnaire on two occasions (T1 and T2), on the same day and time, one week apart following identical procedures. Variables for testing included physical activity behavior (n=13), health (n=11) and wellbeing (n=2). Test-retest reliability of the questionnaire's covariates, including family affluence and physical impairments were also examined.

Results

Systematic error (Bland-Altman plots) was found to be near to zero for each of the physical activity behavior, health and wellbeing variables. The combined mean coefficient of variation was lower for females (10.19%) in comparison to males (13.01%). The combined mean intraclass correlation coefficients were higher for females (0.901) than males (0.822). Similarly, the combined mean Cronbach alpha coefficient were higher for girls (0.908) than boys (0.821).

Conclusions

This study found the physical activity behavior, health and wellbeing questionnaire to be reliable for use in adolescent populations.

Test-retest reliability of a single isometric mid-thigh pull protocol to assess peak force and strength-endurance

The purpose of this study was to examine the test-retest reliability of strength-endurance protocols using isometric mid-thigh pull (IMTP). Twenty-eight participants (23.2 ± 4.9 years) completed two protocols across four testing sessions. Protocol one consisted of 10 maximal IMTP tests lasting 5 seconds each with 10 seconds rest between. Protocol two consisted of a prolonged 60 second maximal IMTP. Data from protocol 1 was analysed in two ways; (a) use of the highest peak value from the first three IMTP efforts, and the lowest peak value from the final three IMTP efforts, and (b) use of the mean peak force from the first three IMTP efforts and mean peak force from the final three IMTP efforts. Data from protocol two used the highest and lowest peak values in the first- and final-15 seconds. Analyses revealed excellent reliability for peak force across all four testing sessions (ICC = 0.94), as well as good test-retest reliability for strength-endurance for protocol 1 (a; ICC = 0.81, b; ICC = 0.79). Test-retest reliability for protocol 2 was poor (ICC = 0.305). Bland-Altman bias values were smaller for protocol 1(a = -8.8 Nm, b = 21.7 Nm) compared to protocol 2 = (119.3 Nm). Our data suggest that 10 maximal IMTP tests performed as described herein is a reliable method for exercise professionals to assess both peak force and strength-endurance in a single, time-efficient protocol.

Three-dimensional anatomical analysis of the optic nerve canal with virtual model based on CBCT

The optic nerve canal (ONC) is a fine skeletal structure that contains the optic nerve. However, it has not been thoroughly examined. This necessitates establishing a baseline knowledge of the geometrical and volumetric parameters of the ONC. The data of twenty patients who received a cone beam computed tomography examination were prepared using a voxel-based segmentation. The measurement was performed separately by two examiners on virtual skull models in stereolithography format in Geomagic Wrap®. The results showed that the volume of the ONC varied between 92.48 mm3 and 162.7 mm3 (M = 123.46 mm3, SD = 26.61 mm3). Sex-specific statistically significant differences in volume were detected only for the right side. The angle of the ONC to the skull base was independent of the diameter of the canal. Both the intrarater and interrater comparisons of the measurements showed high values of reproducibility of the results. This study showed that a virtual anatomical model provides a feasible and reliable method to investigate the ONC. The examination technique could have a wider range of application in anthropology and application in clinical medicine. Advances in the automation of radiological diagnostics and the digital analysis of X-ray images could help to reduce examination times.

Evaluating the added value of concurrent contrast-enhanced diagnostic CT for PSMA-PET/CT Interpretation

RATIONALE AND OBJECTIVES

To determine whether concurrent contrast-enhanced diagnostic CT (DxCT) confers added diagnostic certainty compared to PSMA-PET/CT alone.

MATERIALS AND METHODS

This retrospective multi-reader study analyzed imaging comprising combined F-18-piflufolastat PSMA-PET/CT with diagnostic chest/abdominopelvic CT from prostate cancer patients within the first 6 months of FDA-approval of the PET agent. Six nuclear radiology readers were randomly presented with PSMA-PET/CT studies with or without DxCT and asked to report their diagnostic certainty for PSMA-avid lesions found on PET. Subsequently, readers re-reviewed the same study after an interlude (with the CT if not previously presented and vice-versa) to determine if DxCT altered their diagnostic assessment. Inter-rater concordance was assessed on a subset of images read by all readers. Diagnostic certainties for PSMA-PET/CT with and without DxCT were compared, and the variables for which DxCT may add value were examined.

RESULTS

Good inter-rater concordance across readers was noted for both PET/CT (Finn's coefficient of reliability for overall scan certainty: 0.85,p < 0.01) and combined DxCT-PET/CT (0.59,p < 0.01). Overall certainty and concordance between PET/CT and combined DxCT-PET/CT datasets were similar (overall scan certainty: 92% ± 16 vs. 92% ± 17,p = 0.43), with no significant advantage for adding DxCT across different anatomic locations or clinical parameters. A slight predilection for combined DxCT-PET/CT was noted when interpreting images acquired for the initial staging of prostate cancer (89% ± 16 vs. 93% ± 17,p = 0.08).

CONCLUSION

Good inter-reader concordance can be achieved across different training levels with PSMA-PET/CT. Furthermore, using DxCT concurrent with PSMA-PET/CT does not significantly improve diagnostic certainty for most indications but may be useful for initial staging.

Meta-analysis and sequential analysis of acupuncture compared to carbamazepine in the treatment of trigeminal neuralgia

BACKGROUND

In this randomized controlled trial (RCT) comparing current acupuncture with carbamazepine for trigeminal neuralgia, meta- and sequential analyses were utilized.

AIM

To guide clinical decision making regarding the treatment of trigeminal neuralgia with carbamazepine.

METHODS

The RCT literature on needle comparison was searched in various Chinese biomedical databases including Chinese Biomedical Literature Database, Wanfang Data, VIP Database, as well as international databases such as Excerpt Medica Database, Cochrane Library, PubMed, and Web of Science, along with related clinical registration platforms such as World Health Organization International Clinical Trial Registry Platform, ChiCTR, and Clinical Trials up to 1 April 2020. Risk of bias was evaluated using the Cochrane Collaborative Risk Bias tool, primary outcome measures (pain reduction) were analyzed using STATA meta-analysis, outcome measures were analyzed using trial sequential analysis 0.9.5.10 Beta sequential analysis, GRADE was used to assess the evidence, and adverse reactions were documented.

RESULTS

This study analyzed 16 RCTs with a total of 1231 participants. The meta-analysis revealed a statistically significant difference in pain reduction between acupuncture and carbamazepine [standardized mean difference (SMD) = 1.47; 95% confidence interval (CI): 0.99-1.95], although the quality of evidence was deemed to be of extremely low quality. Cumulative meta-analysis based on the year of publication indicated that carbamazepine treatment first demonstrated a statistically significant difference in pain reduction in 2014 and remained relatively stable over time [SMD = 1.84; 95%CI: 0.22-3.47]. Additionally, the number of adverse events associated with acupuncture was significantly lower compared to carbamazepine.

CONCLUSION

Acupuncture for trigeminal neuralgia is better than analgesia and safer than carbamazepine; however, firm conclusions still require a high-quality, multicenter, large-sample RCT to confirm these findings.

Establishing the Reliability of The Step Test Evaluation of Performance on Stairs (STEPS) in Multiple Sclerosis

OBJECTIVE

To establish the inter- and intra-rater reliability of The Step Test Evaluation of Performance on Stairs (STEPS) for people with multiple sclerosis (PwMS) and examine its relation to clinical mobility measures, cognition, and activity levels.

DESIGN AND SETTING

STEPS performance was rated by 3 raters at the initial visit. Two raters observed the STEPS performance via videotape at the initial visit and then 1 week later. Participants also completed in lab clinical mobility tests and cognitive assessments at their initial visit. Activity levels were tracked for the subsequent 6 months.

PARTICIPANTS

In total, 23 people with relapsing-remitting MS (N=23).

INTERVENTIONS

Not applicable.

MAIN OUTCOME MEASURE(S)

Intraclass correlation coefficients (ICCs) were used to assess intra-rater and inter-rater reliability, while correlation analyses compared STEPS performance with cognition, clinical mobility assessments, and activity levels. The inter-rater reliability analysis among the 3 raters included scoring from only the initial evaluation. For the intra-rater reliability, 2 raters viewed and rated the videotaped session for each of the participants and then repeated the same process 1 week later.

RESULTS

Total STEPS scores demonstrated excellent agreement by ICC for inter- (ICC=0.97) and intra-rater reliability (ICC>0.95) and significant correlations with established clinical mobility assessments in PwMS. Better performance on STEPS was associated with information processing speed and prospective activity levels in PwMS.

CONCLUSIONS

Stair ambulation is a challenging task, integral for mobility and independence, therefore, having a sensitive and valid reliable assessment of stair performance is critical for PwMS. The STEPS assessment is a quick, easily administered, reliable, and valid tool for assessing stair ambulation in PwMS.

Comparison of case-based and lecture-based learning in dental fluorosis diagnostic ability with visual analog scale assessment

OBJECTIVE

This study aimed to compare the impact of case-based learning (CBL) versus lecture-based learning (LBL) on dental students' clinical decision-making regarding DF severity using Visual Analog Scale (VAS) scoring.

METHODS

Eighty first-year graduate dental students were randomly assigned to either the CBL (n = 38) or LBL (n = 42) groups. Both groups received instruction on DF diagnosis, with CBL involving small group sessions analyzing real cases and LBL involving traditional lectures. Effectiveness was assessed by presenting 32 dental fluorosis cases with Thylstrup-Fejerskov Index (TSIF) scores ranging from 0 to 7 through slide presentations to both groups for VAS assessment. Five evaluators of each group randomly selected were asked to repeat the rating 2 weeks later. Statistical analysis included two-way ANOVA for group and gender differences, intra-class correlation coefficient (ICC) for reliability, and Spearman correlation coefficients for validity.

RESULTS

Variations in VAS scores were observed between CBL and LBL groups, with no significant gender impact. Excellent inter- and intra-evaluator agreement was found for VAS scoring in both groups, indicating its reliability. Validation against established indices (such as DI and TSIF) demonstrated strong correlations, with CBL students exhibiting higher correlations.

CONCLUSIONS

CBL enhances students' clinical decision-making and proficiency in DF diagnosis, as evidenced by more consistent and accurate VAS scoring compared to LBL. These findings highlight the importance of innovative educational strategies in dental curricula, with implications for improving training quality and clinical outcomes.

TRIAL REGISTRATION

The study was registered at the Clinical Research Center, Hospital of Stomatology, Wuhan University (Registration code: HGGC-036).

Smartphone-Facilitated In-Situ Hearing Aid Audiometry for Community-Based Hearing Testing

OBJECTIVES

Hearing loss prevalence is increasing, with an estimated 2.5 billion people affected globally by 2050. Scalable service delivery models using innovative technologies and task-shifting are World Health Organization priorities to improve access to hearing care, particularly in low- and middle-income countries. Smartphone-facilitated audiometry in the community using hearing aids covered by noise-attenuating ear cups ("in-situ") could support more accessible hearing care when provided by less trained individuals such as community health workers (CHWs). This study aimed to determine the validity of this method for potential hearing aid fitting. Study objectives included determining the maximum permissible ambient noise level (MPANL), inter-device reliability, clinical threshold accuracy, reliability, and performance in real-world settings.

DESIGN

Experiment 1: 15 normal-hearing adult participants were evaluated to determine MPANLs for circumaural Peltor 3M earcups covering Lexie Lumen hearing aids with smartphone-facilitated in-situ audiometry. MPANLs were calculated by measuring the difference in attenuation between thresholds obtained with standard headphones and in-situ hearing aids. Experiment 2: Pure-tone frequency and intensity output of 14 same-model Lexie Lumen hearing aids were measured to determine inter-device reliability. Pure-tone stimuli were measured and analyzed to determine sound pressure levels in decibels and pure-tone frequency when connected to a test box 2cc coupler. Experiment 3: 85 adult participants were tested in a sound booth to determine the accuracy of automated in-situ pure-tone audiometry (PTA) compared to clinical PTA (500, 1000, 2000, 3000, 4000, 6000 Hz) facilitated by an audiologist. The first 39 participants were tested twice to determine test-retest reliability. Experiment 4: In a community setting, 144 adult participants were tested with automated in-situ audiometry facilitated by CHWs using a smartphone app. These participants were subsequently tested with automated mobile PTA (500, 1000, 2000, 4000 Hz). An additional 44 participants were tested twice to determine test-retest reliability.

RESULTS

Experiment 1: MPANLs of the Peltor 3M earcup-covered hearing aids were higher than standard headphones across all frequencies, ranging from 24 to 47.3 dB SPL. Experiment 2: Inter-device performance reliability was high, with all inter-device differences across all intensities and frequencies less than 3 dB. Frequency output was consistent and differed less than 0.7% between devices. Experiments 3 and 4: 85.2% and 83.3% of automated in-situ audiometry thresholds were within 10 dB of thresholds obtained in the sound booth and in a community setting, respectively. Acceptable test-retest intraclass correlation coefficient (ICC) was evident across all thresholds obtained in a sound booth (ICC = 0.85 to 0.93) and in a community setting (ICC = 0.83 to 0.97).

CONCLUSIONS

Smartphone-facilitated in-situ audiometry allows for reliable and valid community-based testing. A simple smartphone user interface and automated in-situ audiometry allow CHWs with minimal training to facilitate the testing. With the additional capability to program hearing aids via the smartphone after the initial test, this approach would have the potential to support widespread access to personalized hearing aid fittings facilitated by CHWs in low- and middle-income countries. This approach also supports self-fitting options based on in-situ thresholds, enabling testing and fitting via over the counter hearing aids.

Atypical neural encoding of faces in individuals with autism spectrum disorder

Individuals with autism spectrum disorder (ASD) experience pervasive difficulties in processing social information from faces. However, the behavioral and neural mechanisms underlying social trait judgments of faces in ASD remain largely unclear. Here, we comprehensively addressed this question by employing functional neuroimaging and parametrically generated faces that vary in facial trustworthiness and dominance. Behaviorally, participants with ASD exhibited reduced specificity but increased inter-rater variability in social trait judgments. Neurally, participants with ASD showed hypo-activation across broad face-processing areas. Multivariate analysis based on trial-by-trial face responses could discriminate participant groups in the majority of the face-processing areas. Encoding social traits in ASD engaged vastly different face-processing areas compared to controls, and encoding different social traits engaged different brain areas. Interestingly, the idiosyncratic brain areas encoding social traits in ASD were still flexible and context-dependent, similar to neurotypicals. Additionally, participants with ASD also showed an altered encoding of facial saliency features in the eyes and mouth. Together, our results provide a comprehensive understanding of the neural mechanisms underlying social trait judgments in ASD.

Predicting Ki-67 expression in hepatocellular carcinoma: nomogram based on clinical factors and contrast-enhanced ultrasound radiomics signatures

PURPOSE

To develop a contrast-enhanced ultrasound (CEUS) clinic-radiomics nomogram for individualized assessment of Ki-67 expression in hepatocellular carcinoma (HCC).

METHODS

A retrospective cohort comprising 310 HCC individuals who underwent preoperative CEUS (using SonoVue) at three different centers was partitioned into a training set, a validation set, and an external test set. Radiomics signatures indicating the phenotypes of the Ki-67 were extracted from multiphase CEUS images. The radiomics score (Rad-score) was calculated accordingly after feature selection and the radiomics model was constructed. A clinic-radiomics nomogram was established utilizing multiphase CEUS Rad-score and clinical risk factors. A clinical model only incorporated clinical factors was also developed for comparison. Regarding clinical utility, calibration, and discrimination, the predictive efficiency of the clinic-radiomics nomogram was evaluated.

RESULTS

Seven radiomics signatures from multiphase CEUS images were selected to calculate the Rad-score. The clinic-radiomics nomogram, comprising the Rad-score and clinical risk factors, indicated a good calibration and demonstrated a better discriminatory capacity compared to the clinical model (AUCs: 0.870 vs 0.797, 0.872 vs 0.755, 0.856 vs 0.749 in the training, validation, and external test set, respectively) and the radiomics model (AUCs: 0.870 vs 0.752, 0.872 vs 0.733, 0.856 vs 0.729 in the training, validation, and external test set, respectively). Furthermore, both the clinical impact curve and the decision curve analysis displayed good clinical application of the nomogram.

CONCLUSION

The clinic-radiomics nomogram constructed from multiphase CEUS images and clinical risk parameters can distinguish Ki-67 expression in HCC patients and offer useful insights to guide subsequent personalized treatment.

Measurement Properties of Backward Walking and Its Sensitivity and Feasibility in Predicting Falls in People With Multiple Sclerosis

BACKGROUND

People with multiple sclerosis (MS) experience mobility impairments that elevate fall risk, increasing the need to identify clinical measures that accurately predict falls. Backward walking (BW) better differentiates fallers from nonfallers in MS. However, no studies have reported the measurement properties of the backward walking Timed 25-Foot Walk (B-T25-FW) and BW metrics, like BW velocity. Additionally, it is unknown whether BW can predict future falls in MS or its link to activity levels. This study assessed the reliability and responsiveness of B-T25-FW and BW metrics, including BW velocity. It also examined whether BW could predict falls at 3 and 6 months and its association with activity levels.

METHODS

During 2 separate visits, 23 people with MS completed the forward walking Timed 25-Foot Walk (F-T25-FW) and B-T25-FW, as well as forward walking and BW assessments in which spatiotemporal measures were recorded. Test-retest reliability was determined with intraclass correlation coefficients, and minimum detectable changes were calculated. Correlation analyses explored the relationship between BW velocity, B-T25-FW, prospective falls, and activity levels.

RESULTS

B-T25-FW and BW velocity exhibited excellent test-retest reliability. Large effect sizes to interpret clinically meaningful change in the B-T25-FW and BW velocity were also found. Both metrics demonstrated modest negative correlations with falls at 3 and 6 months and correlated strongly with very active minutes at 3- and 6-months post study.

CONCLUSIONS

The B-T25-FW and BW velocity are effective and reliable in clinical use for evaluating functional mobility in people with MS, are sensitive enough to detect subtle changes, and may be a meaningful marker for tracking disease progression and treatment efficacy.

Interrater reliability of photographic assessment of thyroid eye disease using the VISA classification

PURPOSE

To determine the interrater reliability (IRR) of thyroid eye disease (TED) photographic assessment using the VISA classification. To assess whether a VISA grading atlas improves ophthalmology trainees' performance in photographic assessment of TED.

METHODS

A prospective, partially randomized, international study conducted from September 2021 to May 2022. Online study invitation was emailed to a volunteer sample group of 68 ophthalmology college accredited consultants and trainees, and 6 were excluded from the study. Participants were asked to score 10 patient photographs of TED using only the inflammation and motility restriction components of the VISA classification. IRR was compared between groups of practitioners by their level of experience. A clinical activity grading atlas was randomized to 50% of the ophthalmology trainees.

RESULTS

Overall rater ICC was 0.96 for inflammation and 0.99 for motility restriction. No statistically significant difference in IRR between rater groups was identified. Trainees with a grading atlas had the highest IRR for inflammation (ICC = 0.95). Each subcomponent of the inflammation and motility restriction components of VISA classification had an ICC considered good to excellent. The mean overall rater score was 4.6/9 for inflammation and 3.5/12 for motility restriction. For motility restriction there was a reduced mean score variance among all raters when scoring photographs with more severe motility restriction.

CONCLUSION

IRR using the inflammation and motility restriction components of the VISA classification was excellent. A VISA grading atlas improved trainee performance in grading inflammation.

Internal and external validation of an updated ICD-10-CA to AIS-2005 update 2008 algorithm

BACKGROUND

Administrative data are a powerful tool for population-level trauma research but lack the trauma-specific diagnostic and injury severity codes needed for risk-adjusted comparative analyses. The objective of this study was to validate an algorithm to derive Abbreviated Injury Scale (AIS-2005 update 2008) severity scores from Canadian International Classification of Diseases (ICD-10-CA) diagnostic codes in administrative data.

METHODS

This was a retrospective cohort study using data from the 2009 to 2017 Ontario Trauma Registry for the internal validation of the algorithm. This registry includes all patients treated at a trauma center who sustained a moderate or severe injury or were assessed by a trauma team. It contains both ICD-10-CA codes and injury scores assigned by expert abstractors. We used Cohen's kappa (𝜅) coefficient to compare AIS-2005 Update 2008 scores assigned by expert abstractors to those derived using the algorithm and the intraclass correlation coefficient to compare assigned and derived Injury Severity Scores. Sensitivity and specificity for detection of a severe injury (AIS score, ≥ 3) were then calculated. For the external validation of the algorithm, we used administration data to identify adults who either died in an emergency department or were admitted to hospital in Ontario secondary to a traumatic injury (2009-2017). Logistic regression was used to evaluate the discriminative ability and calibration of the algorithm.

RESULTS

Of 41,869 patients in the Ontario Trauma Registry, 41,793 (99.8%) had at least one diagnosis matched to the algorithm. Evaluation of AIS scores assigned by expert abstractors and those derived using the algorithm demonstrated a high degree of agreement in identification of patients with at least one severe injury (𝜅 = 0.75; 95% confidence interval [CI], 0.74-0.76). Likewise, algorithm-derived scores had a strong ability to rule in or out injury with AIS ≥ 3 (specificity, 78.5%; 95% CI, 77.7-79.4; sensitivity, 95.1; 95% CI, 94.8-95.3). There was strong correlation between expert abstractor-assigned and crosswalk-derived Injury Severity Score (intraclass correlation coefficient, 0.80; 95% CI, 0.80-0.81). Among the 130,542 patients identified using administrative data, the algorithm retained its discriminative properties.

CONCLUSION

Our ICD-10-CA to AIS-2005 update 2008 algorithm produces reliable estimates of injury severity and retains its discriminative properties with administrative data. Our findings suggest that this algorithm can be used for risk adjustment of injury outcomes when using population-based administrative data.

LEVEL OF EVIDENCE

Diagnostic Tests/Criteria; Level II.

Psychometric properties of the Chinese version of the health behavior motivation scale: a translation and validation study

Objective

This study's objectives were to translate the Health Behavior Motivation Scale (HBMS) into Chinese and verify the scale's validity and reliability among Chinese healthy adults.

Method

The HBMS scales were translated into Chinese based on Brislin's principles. The Chinese version of HBMS is created through translation, back translation, and cross-cultural adaptation. This investigation implemented the convenience sampling method to conduct a survey on 781 healthy respondents, utilizing the Chinese version of the HBMS and a general demographic questionnaire. We used AMOS (v28.0) and SPSS (v26.0) for statistical analysis. We employed test-retest reliability, split-half reliability, and internal consistency to assess the reliability of the translation questionnaire. Structure validity and content validity were used to assess validity.

Results

The Chinese version of the Health Behavior Motivation Scale (HBMS) had a Cronbach's alpha coefficient of 0.885, and the range of Cronbach's alpha values for each dimension was 0.820-0.885. The scale's test-retest reliability was 0.824, and its split-half reliability was 0.906. Five public factors with a cumulative variance contribution of 56.527% were retrieved from the exploratory factor analysis. Moreover, the factor loading value for each item exceeded 0.4.In confirmatory factor analysis, the indicators were reported as follows: χ2/df = 1.567, GFI = 0.900, CFI = 0.952, IFI = 0.952, TLI = 0.946, AGFI = 0.881, PGFI = 0.757, PNFI = 0.789, RMSEA = 0.039, and the results of the model fit metrics were within the reference range.

Conclusion

The Chinese version of the HBMS exhibits strong discrimination, validity, and reliability. The tool effectively identifies the motivation of healthy people to engage in healthy behaviors. It can be used by healthcare practitioners to assist in the development of follow-up interventions to reduce the prevalence of chronic disease in older people and the incidence of chronic disease in populations of young and middle-aged people.

Adaptation and validation of the encoding of observations using CONSUL-MCC: A self-determination theory-based tool to observe consultations in maternity care

Objective

During autonomy-supportive consultations, professionals use a need-supportive interaction style to facilitate patients' self-regulated behaviour. To improve maternity care professionals' need-supportive interactions, it is important to provide insights into their interaction style. No tool is currently available for measuring need-supportive interactions in maternity care. Therefore, the aim of this study was to adapt the COUNSEL-CCE to evaluate need-supportive interactions in maternity care and to validate their measurements.

Methods

A five-step adaptation and validation process was performed based on the guideline of Sousa and Rojjanasrirat: 1) adaptation of COUNSEL-CCE by two authors independently; 2) development of a consensus-based tool: CONSUL-MCC; 3) qualitative assessment of CONSUL-MCC; 4) pilot testing of CONSUL-MCC in the target population (N = 10) and 5) psychometric testing in the target population (N = 453).

Results

All indicators of the original tool remained relevant. Four items were rephrased, one indicator was added, and all examples were adapted to maternity care. The results of psychometric testing indicated good construct validity. However, the data characteristics made it impossible to prove the presumed factor structure and perform an accurate intraclass correlation.

Conclusions and innovation

COUNSEL-CCE proved to be a new instrument to gain insights into professionals' interactions and be applied to maternity care.

Simple quantitative planimetric measurement of nigrosome-1 for clinical settings

INTRODUCTION

Loss of MRI hyperintense signal in nigrosome-1 (assessed with susceptibility-weighted imaging) is a biomarker for Parkinson's disease (PD). Current clinical practice involves subjectively rating the appearance of nigrosome-1 which is challenging. The study aimed to test and compare a simple method for quantifying nigrosome-1 with the current subjective rating method.

METHODS

Two experienced neuroradiologists measured area of hyperintense signal in nigrosome-1 (quantitative method) and rated nigrosome-1 appearance (as normal, attenuated, or absent; subjective method) in 42 patients encompassing the full spectrum of nigrosome-1 integrity (21 patients aged 55.5 ± 20.9 years with Essential tremor (ET) and a subset of 21 patients aged 69.6 ± 8.6 years with PD). Neuroradiologists were blinded to each other's measurements, clinical notes, and patient group.

RESULTS

Both methods yielded a significant difference between the groups (PD vs ET; p < 0.001). Pooled (across sides) area of nigrosome-1 hyperintense signal was significantly smaller in the PD group (median = 2.1 mm2, range = 0-15.8 mm2) than ET group (median = 8.3 mm2, range = 0-15.7 mm2; p < 0.001). Inter-rater reliability was high to very high for both methods (subjective: weighted kappa = 0.640, p < 0.001; quantitative: W = 0.733, p = 0.004). Our primary hypothesis that area of nigrosome-1 hyperintense signal exhibits higher inter-rater reliability than subjective rating of nigrosome-1 appearance was not supported.

CONCLUSION

The simple quantitative method, used with subjectively rated nigrosome-1 appearance, may improve confidence in longitudinal clinical reporting, when nigrosome-1 is attenuated. However, further work on the incremental diagnostic value of planimetry and bias, repeatability and reproducibility are needed before it can be recommended in clinical practice.

Validation of the Korean Version of the Assessment of Strategies in Families-Effectiveness Scale

BACKGROUND

Family health significantly affects society and the country. Health problems have been identified as the most important cause of family anxiety and conflict in South Korea. Because valid instruments to measure this concept are limited, using the Assessment of Strategies in Families-Effectiveness scale may provide precious insight into family health and nursing practice.

PURPOSE

This study was designed to translate and validate the Korean version of the Assessment of Strategies in Families-Effectiveness (K-ASF-E) scale. This scale was developed based on the framework of systemic organization for assessment and intervention use in the context of family nursing in South Korea.

METHODS

A methodological approach was used to verify the validity and reliability of the K-ASF-E. To determine validity, we first invited four experts to evaluate content validity. Next, we collected data from 1,028 participants using stratified random sampling that considered the background of each participant in terms of region, age, and gender. On the basis of the collected data, we determined the construct validity of the scale using exploratory factor analysis and confirmatory factor analysis. Finally, we used Cronbach's alpha and intraclass correlation coefficient analyses to assess scale reliability.

RESULTS

The K-ASF-E showed item-level content validity index and scale-level content validity index values of 1.00, as assessed by four experts. Using the principal component analysis method with varimax rotation, the exploratory factor analysis verified the validity of a K-ASF-E scale comprising four dimensions and 20 items, including system maintenance (seven items), system change (four items), coherence (six items), and individuation (three items). For confirmatory factor analysis, the fit indices of the factor structure of K-ASF-E indicate appropriate model fitness. The Cronbach's alpha coefficients for each of the subfactors centered on the target were .59-.86, and the intraclass correlation coefficient (range) was .56-.85.

CONCLUSIONS/IMPLICATIONS FOR PRACTICE

The K-ASF-E scale is the first attempt to develop a reliable and valid measurement tool for family effectiveness in South Korea. Initial testing was validated using a large number of stratified randomized clusters. The K-ASF-E scale accurately measures family effectiveness and may be used for family-related research, intervention, and training programs in community and clinical settings.

Interrater Reliability of the Scale for the Assessment and Rating of Ataxia, Berg Balance Scale, and Functional Independence Measure Motor Domain in Individuals With Hereditary Cerebellar Ataxia

OBJECTIVE

To determine the interrater reliability of the Scale for the Assessment and Rating of Ataxia (SARA), Berg Balance Scale (BBS), and motor domain of the FIM (m-FIM) administered by physiotherapists in individuals with a hereditary cerebellar ataxia (HCA).

DESIGN

Participants were assessed by 1 of 4 physiotherapists. Assessments were video-recorded and the remaining 3 physiotherapists scored the scales for each participant. Raters were blinded to each other's scores.

SETTING

Assessments were administered at 3 clinical locations in separate states in Australia.

PARTICIPANTS

Twenty-one individuals (mean age=47.63 years; SD=18.42; 13 male and 8 female) living in the community with an HCA were recruited (N=21).

MAIN OUTCOME MEASURES

Total and single-item scores of the SARA, BBS, and m-FIM were examined. The m-FIM was conducted by interview.

RESULTS

Intraclass coefficients (2,1) for the total scores of the m-FIM (0.92; 95% confidence interval [CI], 0.85-0.96), SARA (0.92; 95% CI, 0.86-0.96), and BBS (0.99; 95% CI, 0.98-0.99) indicated excellent interrater reliability. However, there was inconsistent agreement with the individual items, with SARA item 5 (right side) and item 7 (both sides) demonstrating poor interrater reliability and items 1 and 2 demonstrating excellent reliability.

CONCLUSIONS

The m-FIM (by interview), SARA, and BBS have excellent interrater reliability for use when assessing individuals with an HCA. Physiotherapists could be considered for administration of the SARA in clinical trials. However, further work is required to improve the agreement of the single-item scores and to examine the other psychometric properties of these scales.

The Effect of Direct and Remote Postexercise Ischemic Conditioning on Muscle Soreness and Strength 24 Hours After Eccentric Drop Jumps

ABSTRACT

Lillquist, T, Mahoney, SJ, Kotarsky, C, McGrath, R, Jarajapu, Y, Scholten, SD, and Hackney, KJ. The effect of direct and remote postexercise ischemic conditioning on muscle soreness and strength 24 hours after eccentric drop jumps. J Strength Cond Res 37(9): 1870-1876, 2023-Strategic limb occlusion applied after exercise may facilitate recovery, not only in directly targeted tissue but also in remote areas of the body. The purpose of this study was to determine if postexercise ischemic conditioning (PEIC) applied directly to one leg facilitated recovery in the targeted leg and the contralateral leg that did not receive direct PEIC. Twenty active men participated in a single-blind, randomized, crossover design. Subjects completed 2 paired testing sessions (PEIC and control-SHAM) that included pre-assessments and 24-hour postassessments. Each paired testing session included an eccentric drop jump task, which has been shown to increase lower-body muscle soreness and decrease strength. After each drop jump task, occlusion cuffs were immediately applied. In the PEIC session, ∼198 mm Hg was applied directly to one leg (PEIC-Direct), whereas the contralateral leg received a nonphysiological stimuli of 20 mm Hg (PEIC-Remote). In the control-SHAM session, both legs directly and remotely received the 20 mm Hg pressure. Unilateral pre-assessments and 24-hour postassessments included muscle soreness using a visual analog scale and strength via peak torque assessment across the force-velocity spectrum (flexion/extension 60/60, 120/120, 180/180, 240/240, 300/300 °·s -1 ), and a maximal eccentric extension (30/30 °·s -1 ). Muscle soreness was significantly increased ( p < 0.05) at 24 hours compared with pretreatment except for PEIC-Direct (1.19 ± 0.78 vs. 2.32 ± 1.48, p = 0.096). Across the force-velocity spectrum, there were no significant differences observed between any associated pretest and posttest ( p > 0.05). PEIC applied directly to target leg after eccentric drop jumps attenuated perceived quadriceps muscle soreness 24 hours post; however, there was no effect on muscle strength.

Reliability of online self-report versions of the Australian Treatment Outcome Profile (ATOP) and Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) among people in residential treatment for substance use problems

BACKGROUND

Online self-report measures are resource-efficient and widely used for monitoring substance use, yet few studies have assessed their reliability. This study assessed the reliability of online self-report versions of the Australian Treatment Outcomes Profile (ATOP) and Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) among people seeking treatment for substance use problems.

PROCEDURES

One-hundred and five clients entering residential treatment for substance use problems (Mage = 33.34, 65% male) were recruited from two facilities in Queensland, Australia. Using a repeated-measures design, we compared online self-report with the original interview versions of the ASSIST, which measured (i) lifetime substance use and (ii) past 3-month substance-use and related harms, and the ATOP, which measured (i) past month frequency of substance use and (ii) the typical quantity used per day. Assessments were administered 1-7 days apart.

FINDINGS

The ATOP demonstrated moderate-excellent inter-rater reliability for the past month use (yes/no) for all substance types, but had poor reliability for alcohol and cannabis. ATOP reliability was high-excellent for the total number of days used in the past month for all substances. The ASSIST demonstrated moderate-excellent inter-rater reliability for substance-use and related harms for all substances except tobacco, however was poor for lifetime use for most substances due to greater reporting in the interview assessment.

CONCLUSIONS

Reliable responding was observed for the frequency of substance use in the past month on the ATOP, and past 3-month substance-use and related harms on the ASSIST. These findings support use of online-self report measures a resource-efficient method to monitor substance use. Underreporting of lifetime use was found in self-report online version of the ASSIST, highlighting the need for improved instruction or interviewer/clinician assistance for lifetime use.

Do classical and computerized cognitive tests have equal intrarater reliability in myotonic dystrophy type 1?

Myotonic dystrophy type 1 (DM1) is a multisystemic inherited neuromuscular disease leading to central nervous system symptoms, including cognitive impairments, among multiple other symptoms. However, information is presently lacking regarding the psychometric properties of neuropsychological tests and promising computerized cognitive tests, such as the Cambridge Neuropsychological Test Automated Battery (CANTAB^Ⓡ). This type of information is critical to improve clinical trial readiness and provide knowledge of DM1 natural history. The aims of the present study were (1) to document the intrarater reliability of classic paper-pencil tests assessing visuospatial working memory, cognitive flexibility, attention, episodic memory and apathy, and (2) to compare these findings with their equivalent computerized automated tests from the CANTAB^Ⓡ. Thirty participants were seen twice at four-week intervals. Results showed that the Stroop Color and Word Test (ICC = 0.741-0.869) and the Ruff 2 & 7 (ICC = 0.703-0.871) appear to be reliable paper-and-pencil tests in the DM1 population. For the CANTAB^Ⓡ, a similar observation was made for the Multitasking test (ICC = 0.588-0.792). Further studies should explore the applicability and concurrent validity of the CANTAB® and classic neuropsychological assessments in additional cohorts of DM1 patients.

Translation, cross-cultural adaptation, and psychometric evaluation of the Persian (Farsi) version of the QoLAF (quality of life in patients with anal fistula) questionnaire

INTRODUCTION

The effective treatment of anal fistulas almost always requires surgical intervention, which could be accompanied by post-operative complications, and affect the quality of life of patients. This study aimed to cross-culturally adapt the Persian version of the Quality of Life in patients with Anal Fistula questionnaire and evaluate its validity and reliability.

MATERIALS AND METHODS

Sixty patients with a mean age of 44 years ranging from 21 to 72 years entered the study. Forty-seven participants were men, and thirteen were women. After performing a scientific translation of the questionnaire based on Beaton's guidelines for cross-cultural adaptation and after extensive reviews by experts and specialists, the final version of the questionnaire was obtained. Then, 60 questionnaires (100%) were filled out by the participants (n = 60) and retrieved during a 7 to 21-day period. Data were collected and analyzed. Finally, according to the obtained data, the validity and reliability of the questionnaire were calculated.

RESULTS

Cross-cultural adaptation of the translated questionnaire was verified by the expert committee. The results showed perfect internal consistency (Cronbach alpha = 0.842), and external consistency (intraclass correlation coefficient = 0.800; P<0.001). Spearman correlation coefficient between test and retest was reported to be 0.980 (P-value <0.01), confirming the temporal stability of the translated questionnaire. The interrater reliability based on Cohen's kappa coefficient also demonstrated a perfect degree of agreement between two peer variables (Kappa = 0.889; P<0.001).

CONCLUSION

The Persian translation of the Quality of Life in patients with the Anal Fistula questionnaire was proven to be valid and reliable for the evaluation of the QoL of patients with anal fistula.

Including visual orienting functions into cerebral visual impairment screening: Reliability, variability, and ecological validity

BACKGROUND

Cerebral visual impairment (CVI) is a heterogeneous brain-based visual processing disorder in which basic visual orienting functions (VOF) and higher-order perception can be impaired.

AIMS

To evaluate (1) the test-retest reliability and variability of an eye tracking-based VOF paradigm, and related clinical characteristics, and (2) the relations between VOF (variability) and daily visual functioning and visuoperceptual dimensions.

METHODS AND PROCEDURES

Thirty-three children with CVI (Males=14; mean age=9 years 10 months) underwent eye tracking thrice, completed a visuoperceptual battery, and parents completed the Flemish CVI questionnaire. VOF reliability and variability of reaction time (RTF), fixation duration and accuracy were assessed with intraclass correlation coefficient (ICC), Bland-Altman plots, and coefficient of variation. Relations were analysed with linear mixed models.

OUTCOMES AND RESULTS

Highly salient visual stimuli had good RTF reliability (ICCs=0.75) and triggered less variable VOF. Intermediate and low salience stimuli had poor-to-moderate reliability and triggered more variable VOF. Younger performance age related to more VOF variability. Greater visual (dis)interest, clutter and distance viewing impairments, and a weaker visuoperceptual profile related to slower RTF.

CONCLUSIONS AND IMPLICATIONS

Highly salient stimuli reveal a child's 'optimal' visual performance, whereas intermediate and low salience stimuli uncover VOF variability, which is a key CVI hallmark to detect.

Assessment of health-related quality of life in Australian patients with idiopathic pulmonary fibrosis: a comparison of the EQ-5D-5L and the AQoL-8D

PURPOSE

Idiopathic pulmonary fibrosis (IPF) is a progressive and debilitating chronic lung disease with a high symptom burden, which has a substantial impact on health-related quality of life (HRQoL). Our study aimed to assess the suitability of the EuroQol five-dimension (EQ-5D-5L) and the Assessment of Quality of Life- eight-dimension (AQoL-8D) questionnaires in measuring HRQoL as health state utility values (HSUVs) in an Australian IPF cohort.

METHODS

Data for estimation of health state utility values (HSUVs) were collected from participants of the Australian IPF Registry (AIPFR) using self-administered surveys which included the EQ-5D-5L and the AQoL-8D. Data on lung function and disease specific HRQoL instruments were collected from the AIPFR. Performance of the two instruments was evaluated based on questionnaire practicality, agreement between the two instruments and test performance (internal and construct validity).

RESULTS

Overall completion rates for the EQ-5D-5L and AQoL-8D were 96% and 85%, respectively. Mean (median) HSUVs were 0.65 (0.70) and 0.69 (0.72) for the EQ-5D-5L and AQoL-8D, respectively. There was reasonable agreement between the two instruments based on the Bland-Altman plot mean difference (-0.04) and intraclass correlation coefficient (0.84), however there were some fundamental differences. A larger range of values was observed with the EQ-5D-5L (-0.57-1.00 vs 0.16-1.00). The EQ-5D-5L had a greater divergent sensitivity and efficacy in relation to assessing HSUVs between clinical groupings. The AQoL-8D ,however, had a higher sensitivity to measure psychosocial aspects of HRQoL in IPF.

CONCLUSION

The EQ-5D-5L demonstrated superior performance when compared to AQoL-8D in persons with IPF. This may be attributable to the high symptom burden which is physically debilitating to which the EQ-5D-5L may be more sensitive.

Test-Retest Reliability and Criterion Validity of the Spanish Version of Two Motor Imagery Questionnaires in People With Parkinson Disease

BACKGROUND AND PURPOSE

The Kinesthetic and Visual Imagery Questionnaire (KVIQ) and the Movement Imagery Questionnaire-Revised Second Version (MIQ-RS) are measurement instruments that assess motor imagery vividness. The aim of this study was to examine the validity and reliability of the Spanish KVIQ and MIQ-RS in people with Parkinson disease (PD).

METHODS

A longitudinal descriptive study was conducted following the COSMIN standards. Thirty-five people with idiopathic PD were evaluated twice (7-15 days apart) with the Spanish KVIQ and MIQ-RS. Structural validity, internal consistency, test-retest reliability (ICC), standard error of measurement (SEM), smallest detectable change (SDC), and criterion validity of the MIQ-RS and KVIQ long (KVIQ-20), short (KVIQ-10), and extended (KVIQ-34) versions and their subscales (if pertinent) were tested.

RESULTS

Factor analysis was satisfactory for the MIQ-RS, KVIQ-20, and KVIQ-10, providing evidence of their 2-dimensional structure. Evidence of the structural validity of the KVIQ-34 was not confirmed and thus was analyzed as an overall score. Revelle's ω > 0.9 showed excellent internal consistency. Test-retest reliability was moderate (ICC = 0.58-0.75) and higher for all visual subscales. SEM and SDC were up to 14.39% and 39.89% of the scores, respectively. Criterion validity between questionnaires and subscales was strong (Spearman's r > 0.7).

DISCUSSION AND CONCLUSIONS

The results provide evidence for the validity and reliability of the Spanish MIQ-RS, KVIQ-20, and KVIQ-10 to assess motor imagery vividness in people with PD, whereas the KVIQ-34 should only be interpreted as an overall score. Psychometric, procedural, and practical features of the questionnaires should be considered when applying into clinical practice.Video Abstract available for more insights from the authors (see the Video, Supplemental Digital Content 1, available at: http://links.lww.com/JNPT/A401).

Intraclass Correlation in Paired Associative Stimulation and Metaplasticity

Paired associative stimulation (PAS) is a widely used noninvasive brain stimulation protocol to assess neural plasticity. Its reproducibility, however, has been rarely tested and with mixed results. With two consecutive studies, we aimed to provide further tests and a more systematic assessment of PAS reproducibility. We measured intraclass correlation coefficients (ICCs)-a widely used tool to assess whether groups of measurements resemble each other-in two PAS studies on healthy volunteers. The first study included five PAS sessions recording 10 MEPS every 10 min for an hour post-PAS. The second study included two PAS sessions recording 50 MEPS at 20 and 50 min post-PAS, based on analyses from the first study. In both studies PAS sessions were spaced one week apart. Within sessions ICC was fair to excellent for both studies, yet between sessions ICC was poor for both studies. We suggest that long term meta-plasticity effects (longer than one week) may interfere with between sessions reproducibility.

Reliability of 3D Stereophotogrammetry for Measuring Postoperative Facial Swelling

This study aimed to determine the reliability of three-dimensional (3D) stereophotogrammetry as a measurement instrument for evaluating soft tissue changes in the head and neck area. Twelve patients received a bilateral sagittal split osteotomy (BSSO). Test and retest 3D photographs were captured within the first three postoperative weeks, and a reference 3D photograph was capture at three months postoperatively. Distance measurements, mean and root mean square of the distance map, and volume differences were obtained. Reliability of these parameters was assessed by intraclass correlation coefficients (ICCs), standard error of measurement (SEM), and smallest detectable change (SDC). All distance measurements had an ICC > 0.91, and the distance map parameters and volume differences showed ICCs > 0.89. The neck region presented the largest SEMs (5.09 mL) and SDC (14.1 mL) for the volume difference. In conclusion, 3D stereophotogrammetry is reliable for distance and volume measurements of soft tissues in patients after a BSSO advancement.

Validation of ratings on the six-item Positive and Negative Syndrome Scale obtained via the Simplified Negative and Positive Symptoms Interview among outpatients with schizophrenia

BACKGROUND

The six-item Positive and Negative Syndrome Scale (PANSS-6) is a measure of the severity of core symptoms of schizophrenia, which can be administered via the brief Simplified Negative and Positive Symptoms Interview (SNAPSI). A recent study has confirmed the validity of PANSS-6 ratings as derived by SNAPSI (PANSS-6_SNAPSI) among inpatients with schizophrenia.

AIMS

We aimed to test the validity of PANSS-6_SNAPSI among outpatients with schizophrenia using PANSS-6 ratings extracted from the 30-item PANSS-30 as derived by the Structured Clinical Interview for the Positive and Negative Syndrome Scale (PANSS-6_SCI-PANSS) as a gold standard reference.

METHODS

PANSS-6_SNAPSI and PANSS-6_SCI-PANSS ratings were obtained at two time points by independent raters with established inter-rater reliability. Agreement between PANSS-6_SNAPSI and PANSS-6_SCI-PANSS ratings was estimated via intra-class coefficients (ICCs) and responsiveness over time was quantified using Spearman's rank correlation coefficients. Post hoc "leave-one-out" analyses were carried out, in which each rater in turn was excluded from the ICC calculations.

RESULTS

Seventy-three outpatients with schizophrenia participated in the study (mean age: 38.3 years; 56% males). The ICC for PANSS-6_SNAPSI versus PANSS-6_SCI-PANSS was 0.67 [95%CI = 0.56-0.76] and the Spearman's rank correlation coefficient for responsiveness was 0.40 (p = 0.004). When data from a specific outlying rater were excluded, the ICC for PANSS-6_SNAPSI versus PANSS-6_SCI-PANSS was 0.75 [95% CI = 0.63-0.83] and the Spearman's rank correlation coefficient for responsiveness was 0.55 (p = 0.018).

CONCLUSIONS

We found PANSS-6_SNAPSI ratings to have acceptable clinical validity, suggesting that PANSS-6_SNAPSI can be used for both inpatients and outpatients with schizophrenia.

Translation and Validation of the Intermittent Catheterization Difficulty Questionnaire (ICDQ) into Greek

Purpose

Clean intermittent self-catheterization (CISC) is a safe and effective alternative to managing incomplete bladder emptying in patients afflicted with neurogenic bladder conditions. The Intermittent Catheterisation Difficulty Questionnaire (ICDQ) is a validated questionnaire concerning the assessment of catheter use and patient difficulties during CISC. The present study aimed to translate and validate the ICDQ into the Greek language. A subsequent outcome was to substantiate the requirement of both urologist consultation with patients undergoing CISC and the detailed evaluation of various therapeutic options with the consultation of other specialist physicians.

Material and Methods

The study was undertaken between March 2022 and July 2022 and involved patients in an outpatient department of a Rehabilitation Centre. Sixty-two neurologic patients suffering from spinal cord injury (SCI), multiple sclerosis (MS), cauda equina (CE), and myelitis represented a convenience sample. To calculate test-retest reliability, patients filled out two consecutive questionnaires; an initial one and a second after one week. The socio and demographic circumstances of all participants were evaluated.

Results

The mean ICDQ total score at the test and retest was 5.96±1.28 (mean total score ± standard deviation) and 5.91±1.29, respectively. Evaluation of the data concerning alterations between men and women did not reveal statistically significant differences. Cronbach’s alpha coefficient was 0.94, which proves the inter-correlation between the different topics. An ICC of 0.97 indicated very high intra-rater reliability.

Conclusion

ICDQ, a valid and reliable self-administered screening tool for CISC difficulties faced by patients using this procedure regularly, was successfully translated and validated into Greek. A more detailed evaluation and understanding of these difficulties would enhance the quality of CISC and allow for more suitable treatment and the selection of catheter types used. These improved treatment strategies are possible as repeated use and constant comparison of ICDQ scores determine treatment impact, facilitating treatment regimen modification, should it be required.

Patients' and family members' experiences of psychiatric inpatient services in China: a comparison based on a dyadic design

PURPOSE

Due to the family-oriented cultural and legal context in China, understanding the difference between patients' and family members' experiences of psychiatric services not only enriches perspectives of service quality assessment, but also promotes service utilization. This study aimed to compare experiences of psychiatric inpatient services between patients and their family members in China.

METHODS

The study included 126 dyads of patients and family members consecutively recruited from the psychiatric inpatient department in a large hospital in China. The responsiveness performance questionnaire was used to measure the experiences of psychiatric inpatient services after patient discharge. After adjusting reporting heterogeneity based on vignettes, dyad difference was examined by intraclass correlation coefficients (ICCs) and paired Wilcoxon signed-rank tests with Bonferroni correction in multigroup testing. Subgroup analyses were conducted within strata of four selected clinical and socio-demographic factors, to test their influence on difference pattern of experiences.

RESULTS

Poor consistency was found for all responsiveness items and the total scores among the 126 dyads and in most subgroup analyses (ICC < 0.6). Paired Wilcoxon signed-rank tests found that patients rated lower than their family members on the item of "asking user's opinions" in 126-dyad comparison (P < 0.05) and 3 subgroups related to severe mental disorders and income inequality after Bonferroni correction.

CONCLUSION

Results reveal inconsistent experiences of psychiatric inpatient services within families in China. Moreover, when making medical decisions, family members' opinions, rather than patients', are more frequently taken into consideration, especially on conditions where imbalanced decision-making power exists between patients and their family members. In the future, user experience improvement should pay equal attention to patients and family members, and the benefits of family involvement and patients' rights of shared decision-making should be carefully balanced.

Development, validation and item reduction of a food literacy questionnaire (IFLQ-19) with Australian adults

Background

Food literacy is theorised to improve diet quality, nutrition behaviours, social connectedness and food security. The definition and conceptualisation by Vidgen & Gallegos, consisting of 11 theoretical components within the four domains of planning and managing, selecting, preparing and eating, is currently the most highly cited framework. However, a valid and reliable questionnaire is needed to comprehensively measure this conceptualisation. Therefore, this study draws on existing item pools to develop a comprehensive food literacy questionnaire using item response theory.

Methods

Five hundred Australian adults were recruited in Study 1 to refine a food literacy item pool using principal component analysis (PCA) and item response theory (IRT) which involved detailed item analysis on targeting, responsiveness, validity and reliability. Another 500 participants were recruited in Study 2 to replicate item analysis on validity and reliability on the refined item pool, and 250 of these participants re-completed the food literacy questionnaire to determine its test–retest reliability.

Results

The PCA saw the 171-item pool reduced to 100-items across 19 statistical components of food literacy. After the thresholds of 26 items were combined, responses to the food literacy questionnaire had ordered thresholds (targeting), acceptable item locations (< -0.01 to + 1.53) and appropriateness of the measurement model (n = 92% expected responses) (responsiveness), met outfit mean-squares MSQ (0.48—1.42) (validity) and had high person, item separation (> 0.99) and test–retest (ICC 2,1 0.55–0.88) scores (reliability).

Conclusions

We developed a 100-item food literacy questionnaire, the IFLQ-19 to comprehensively address the Vidgen & Gallegos theoretical domains and components with good targeting, responsiveness, reliability and validity in a diverse sample of Australian adults.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12966-022-01351-8.

Nomogram Based on CT Radiomics Features Combined With Clinical Factors to Predict Ki-67 Expression in Hepatocellular Carcinoma

Objectives

The study developed and validated a radiomics nomogram based on a combination of computed tomography (CT) radiomics signature and clinical factors and explored the ability of radiomics for individualized prediction of Ki-67 expression in hepatocellular carcinoma (HCC).

Methods

First-order, second-order, and high-order radiomics features were extracted from preoperative enhanced CT images of 172 HCC patients, and the radiomics features with predictive value for high Ki-67 expression were extracted to construct the radiomic signature prediction model. Based on the training group, the radiomics nomogram was constructed based on a combination of radiomic signature and clinical factors that showed an independent association with Ki-67 expression. The area under the receiver operating characteristic curve (AUC), calibration curve, and decision curve analysis (DCA) were used to verify the performance of the nomogram.

Results

Sixteen higher-order radiomic features that were associated with Ki-67 expression were used to construct the radiomics signature (AUC: training group, 0.854; validation group, 0.744). In multivariate logistic regression, alfa-fetoprotein (AFP) and Edmondson grades were identified as independent predictors of Ki-67 expression. Thus, the radiomics signature was combined with AFP and Edmondson grades to construct the radiomics nomogram (AUC: training group, 0.884; validation group, 0.819). The calibration curve and DCA showed good clinical application of the nomogram.

Conclusion

The radiomics nomogram developed in this study based on the high-order features of CT images can accurately predict high Ki-67 expression and provide individualized guidance for the treatment and clinical monitoring of HCC patients.

The COVID-19 experience: A survey of RNs

PURPOSE

To explore the psychological factors that impacted RNs during the COVID-19 pandemic.

METHODS

Survey response data were collected from 151 respondents across 25 states and various healthcare system settings.

RESULTS

RNs in clinical settings had slightly more emotional stress than those in academia. Concerns ranged from personal health and family considerations to large-scale sociopolitical issues.

CONCLUSIONS

RNs in all settings experienced at least some emotional stress during the early stages of the pandemic. However, in a majority of respondents, none of the stressors individually or in combination were so great as to encourage departure from the profession.

Development of a histopathological index for skeletal muscle analysis in Rattus norvegicus (Rodentia: Muridae)

Skeletal muscle histopathological changes induced or caused by pathologies in animal models, can impair functionality, being the main focus of therapeutic studies. This study aimed to propose a histopathological index to assess, in a quantitative manner, skeletal muscle changes induced by experimental protocols for Rodentia's models. For the development, evaluation of fit and parsimony, replicability, and sensitivity index, Wistar rats from experiments with the same experimental design, but with different variation factors, were used to achieve different levels of damage. The anterior tibial muscle of these animals was collected, processed histologically, and stained with hematoxylin and eosin. The adjustment and parsimony of the index were availed through Confirmatory Factor Analysis, reproducibility for evaluation of three people trained through the Intra-Class Correlation, and the discrimination capacity through a one-way ANOVA Test. We pointed out the adjustment for the proposed index while the ICC showed high reproducibility (n = 56; k = 3; ICC = 0.9790) and differences in the extent of damage between groups, following the hierarchical association promoted by experimental model stresses. The results show that the proposed index has a good fit and parsimony (χ² = 426.34; p < 0.0001), in addition to being easily replicable by other researchers who know the morphology of muscle tissue and its morphological changes. It is worth mentioning that the development of tools that facilitate histopathological analysis, and that can quantitatively express the findings, are of great importance for the studies of regenerative science, reinforcing the relevance of this study.

Distal clavicle "A-frame" morphology: a reliable intraoperative guide for arthroscopic distal clavicle excision

BACKGROUND

The purpose of this cadaveric study was to describe the characteristics of the "A-frame" morphology of the distal clavicle via computed tomography (CT) to determine whether it can be used as a reliable intraoperative guide for arthroscopic distal clavicle excision.

METHODS

Twenty-eight fresh-frozen human cadaveric clavicles underwent a 3-dimensional CT scan using 1.0-mm cuts. The distance from the most lateral aspect of the clavicle to the point at which the superior cortex of the clavicle paralleled the inferior cortex was measured. Measurements were performed in a blinded fashion by a single author on 2 separate occasions.

RESULTS

The A-frame was present in all specimens (28 of 28). On the first measurement, the mean distance from the distal clavicle to the point at which the A-frame disappeared was 1.00 cm (range, 0.90-1.08 cm; standard deviation, 0.5 mm). On the second measurement, the mean distance was 1.02 cm (range, 0.90-1.11 cm; standard deviation, 0.6 mm). The intrarater reliability between measurement occasions was 0.65 (95% confidence interval, 0.36-0.82; P < .001).

CONCLUSIONS

This study demonstrated that the cross-sectional A-frame morphology of the distal clavicle was consistently visualized on CT scans. The A-frame disappeared 1.00-1.02 cm medial to the most lateral extent of the clavicle on CT scans. The disappearance of the A-frame morphology of the distal clavicle can serve as a reliable intraoperative guide for arthroscopic distal clavicle excision.

Reliability of the PEDro scale: comparison between trials published in predatory and non-predatory journals

BACKGROUND

Lack of effective peer-review process of predatory journals, resulting in more ambiguity in reporting, language and incomplete descriptions of processes might have an impact on the reliability of PEDro scale. The aim of this investigation was to compare the reliability of the PEDro scale when evaluating the methodological quality of RCTs published in predatory (PJs) and non-predatory (NPJs) journals, to more confidently select interventions appropriate for application to practice.

METHODS

A selected sample of RCTs was independently rated by two raters randomly selected among 11 physical therapists. Reliability of each item of the PEDro scale and the total PEDro score were assessed by Cohen's kappa statistic and percent of agreement and by Intraclass Correlation Coefficients (ICC) and the Standard Error of Measurement (SEM), respectively. The Chi-square test was used to compare the rate of agreement between PJs and NPJs.

RESULTS

A total number of 298 RCTs were assessed (119 published in NPJs). Cronbach's alphas were .704 and .845 for trials published in PJs and NPJs, respectively. Kappa values for individual scale items ranged from .14 to .73 for PJs and from .09 to .70 for NPJs. The ICC was .537 (95% CI .425-.634) and .729 (95% CI .632-.803), and SEM was 1.055 and 0.957 for PJs and NPJs, respectively. Inter-rater reliability in discriminating between studies of moderate to high and low quality was higher for NPJs (k = .57) than for PJs (k = .28).

CONCLUSIONS

Interrater reliability of PEDro score of RCTs published in PJs is lower than that of trials published in NPJs, likely also due to ambiguous language and incomplete reporting. This might make the detection of risk of bias more difficult when selecting interventions appropriate for application to practice or producing secondary literature.

Translation, Cross-Cultural Adaptation, and Measurement Properties of the Brazilian-Portuguese Version of the Fear of Falling Avoidance Behavior Questionnaire in Older Adults

The aim of this study was to translate and culturally adapt the Fear of Falling Avoidance Behavior Questionnaire (FFABQ) into Brazilian-Portuguese (FFABQ-B), and to examine its reliability and validity in Brazilian older adults. The FFABQ-B was translated and tested in 10 Brazilian older adults. We assessed 52 community-dwellers, 68.7 (±6.2) years, using the FFABQ-B, BERG Balance Scale, Activities-specific Balance Confidence Scale, Falls Efficacy Scale, 6-Minute Walk Test, Timed Up and Go test, and activity monitor. Internal consistency, test-retest reliability, construct validity, and floor and ceiling effects were analyzed. The FFABQ-B had adequate internal consistency (Cronbach's α = .90) and test-retest reliability (intraclass correlation coefficient = .81; 95% confidence interval [.68, .90]). The FFABQ-B was associated with 6-Minute Walk Test, Timed Up and Go, BERG Balance Scale, physical activity time (p < .05), Activities-specific Balance Confidence scale, and Falls Efficacy Scale (p < .001). The FFABQ-B is both reliable and valid to assess avoidance behavior in activities and participation due to fear of falling in Brazilian community-dwelling older adults.

Acupuncture Methods for Primary Trigeminal Neuralgia: A Systematic Review and Network Meta-Analysis of Randomized Controlled Trials

Background

Primary trigeminal neuralgia (PTN) is a clinical refractory disorder characterized by excruciating pain that severely impacts the quality of life. Several studies have shown that acupuncture can improve PTN pain. However, the comparative efficacy and safety of acupuncture are unknown. Herein, a systematic review was conducted to compare the efficacy and safety of various acupuncture methods for PTN treatment.

Methods

Relevant randomized controlled trials (RCTs) published up to 1 August 2021 were obtained from PubMed, Embase, Cochrane Central Register of Controlled Trials, Web of Science Core Collection, Chinese National Knowledge Infrastructure, Chinese Biomedical Literature Database, CQVIP Database, Wanfang Database, Allied and Alternative Medicine Database, and related registration platforms. Two authors independently selected the studies and obtained data. Cochrane Handbook was used to assess the methodological quality. We put the pain relief as the primary outcome and the response rate and adverse events as the secondary outcomes. Review Manager v5.3, ADDIS v1.16.8, and STATA v15.0 software were used for data analysis. The intraclass correlation coefficient was used to assess the consistency of the two investigators.

Results

A total of 58 RCTs with 4,126 participants were obtained. The meta-analysis indicated that five acupuncture methods were superior to conventional medicine (carbamazepine) in pain reduction intensity and response rate. Meanwhile, electronic acupuncture plus manual acupuncture was the most effective therapy since it reduced pain intensity in 11 methods and improved the response rate in 10 interventions. Moreover, six interventions had acceptable adverse events, and none of the included studies reported severe adverse events. However, most pieces of evidence were ranked as critically low.

Conclusion

These findings show that acupuncture methods can be effective and safe for PTN. Moreover, electronic acupuncture plus manual acupuncture maybe the best acupuncture treatment for PTN and should be administered to PTN patients. However, additional well-designed and high-quality RCTs should be conducted to verify the above findings in the future. The systematic review is registered with CRD42020221456.

Three-dimensional Multistructural Quantitative Photoacoustic and US Imaging of Human Feet in Vivo

Background Monitoring the microcirculation in human feet is crucial in assessing peripheral vascular diseases, such as diabetic foot. However, conventional imaging modalities are more focused on diagnosis in major arteries, and there are limited methods to provide microvascular information in early stages of the disease. Purpose To investigate a three-dimensional (3D) noncontrast bimodal photoacoustic (PA)/US imaging system that visualizes the human foot morphologically and also reliably quantifies podiatric vascular parameters noninvasively. Materials and Methods A clinically relevant PA/US imaging system was combined with a foot scanner to obtain 3D PA and US images of the human foot in vivo. Healthy participants were recruited from September 2020 to June 2021. The collected 3D PA and US images were postprocessed to present structural information about the foot. The quantitative reliability was evaluated in five repeated scans of 10 healthy feet by calculating the intraclass correlation coefficient and minimal detectable change, and the detectability of microvascular changes was tested by imaging 10 healthy feet intentionally occluded with use of a pressure cuff (160 mm Hg). Statistically significant difference is indicated with P values. Results Ten feet from six healthy male volunteers (mean age ± standard deviation, 27 years ± 3) were included. The foot images clearly visualized the structure of the vasculature, bones, and skin and provided such functional information as the total hemoglobin concentration (HbT), hemoglobin oxygen saturation (SO₂), vessel density, and vessel depth. Functional information from five independent measurements of 10 healthy feet was moderately reliable (intraclass correlation coefficient, 0.51-0.74). Significant improvements in HbT (P = .006) and vessel density (P = .046) as well as the retention of SO₂ were observed, which accurately described the microvascular change due to venous occlusion. Conclusion Three-dimensional photoacoustic and US imaging was able to visualize morphologic and physiologic features of the human foot, including the peripheral microvasculature, in healthy volunteers. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Mezrich in this issue.

The Effect of Paddle Stroke Variables Measured by Trainesense SmartPaddle® on the Velocity of the Kayak

(1) Background: This study aimed to compare key variables of paddle stroke measured by a commercial Trainesense SmartPaddle^® against the strain-gauge shaft and investigate how these variables are associated with the velocity of the boat among national-level canoe polo players. (2) Methods: This study involved 14 Finnish national-level canoe polo players. The measurement protocol consisted of three different paddling velocities, which were performed in indoor swimming pools. The velocity of the boat was calculated based on the performance time measured with the laser photocell gate. Canoe polo equipment was used in the study and a SmartPaddle sensor was attached to the paddle blade. A strain-gauge paddle shaft was used as a reference method to examine the validity of SmartPaddle. (3) Results: The stroke rate, force production time, mean and maximal force measured with the strain-gauge paddle shaft correlated strongly (r = 0.84–0.95, p < 0.01) with SmartPaddle. However, the SmartPaddle overestimated the maximum force compared to the strain-gauge shaft. Stroke rate (r = 0.86, p < 0.01), mean force (r = 0.79, p < 0.01), maximal force (r = 0.78, p < 0.01) and total absolute impulse (r = 0.70, p < 0.01) correlated positively and force production time negatively (r = −0.76, p < 0.01) with the velocity of the boat. (4) Conclusions: We conclude that the SmartPaddle provides promising information on stroke key variables when compared to the strain-gauge paddle shaft. The SmartPaddle is a new and interesting tool for biomechanical research and daily kayaking coaching in real open water conditions. However, more research and algorithm development are needed before the SmartPaddle can be used in everyday coaching sessions in kayaking.

Towards a better understanding of deep convolutional neural network processes for recognizing organic chemicals of environmental concern

Deep convolutional neural network (DCNN) has proved to be a promising tool for identifying organic chemicals of environmental concern. However, the uncertainty associated with DCNN predictions remains to be quantified. The training process contains many random configurations, including dataset segmentation, input sequences, and initial weight, etc. Moreover, the DCNN working mechanism is non-linear and opaque. To increase confidence to use this novel approach, persistent, bioaccumulative, and toxic substances (PBTs) were utilized as representative chemicals of environmental concern to estimate the prediction uncertainty under five distinguished datasets and ten different molecular descriptor (MD) arrangements with 111,852 chemicals and 2424 available MDs. An internal correlation coefficient test indicated that the prediction confidence reached 0.98 when a mean of 50 DCNNs' predictions was used instead of a sing DCNN prediction. A threshold for PBT categorization was determined by considering costs between false-negative and false-positive predictions. As revealed by the guided backpropagation-class activation mapping (GBP-CAM) saliency images, only 12% of all selected MDs were activated by DCNN and influenced decision-making process. However, the activated MDs not only varied among chemical classes but also shifted with different DCNNs. Principal component analysis indicated that 2424 MDs could transform into 370 orthogonal variables. Both results suggest that redundancy exists among selected MDs. Yet, DCNN was found to adapt to redundant data by focusing on the most important information for better prediction performance.

Translation, Cultural Adaptation, Validity, and Internal Consistency of the Greek Version of the Individual Workload Perceptions Scale-Revised

Introduction: The working environment in hospitals has been characterized as very important for the improvement of the provided care and the nurses’ job satisfaction. The aim of the current study was translation and cultural adaptation of the Individual Workload Perceptions Scale-Revised (IWPS-R) as well as the investigation of the validity and internal consistency of the scale.

Methods: This is a cross-sectional, descriptive study involving 365 Greek nurses from two large hospitals in Athens, Greece. Nurses completed the Individual Workload Perceptions Scale-Revised, which is a self-administered questionnaire consisting of 29 items on a five-point Likert-type scale. For the translation, the scale was first translated into the Greek language (forward translation) and then into the English language (backward translation) and culturally adapted. For the study of the construct validity, exploratory and confirmatory factor analyses were performed, while the criterion of the convergent validity was between the five factors of the scale. To study the reliability, the method of test-retest was performed while Cronbach’s alpha coefficient was used to study the internal consistency of the scale. Data analysis was performed via the Statistical Package for the Social Sciences (SPSS), version 21.0 (IBM Corp., Armonk, NY).

Results: According to the exploratory factor analysis, the Greek version of the Individual Workload Perceptions Scale-Revised consists of five factors (Manager Support, Peer Support, Unit Support, Workload, and Intent to Stay) explaining 51.4% of the total variance. From the confirmatory factor analysis, the model was equivalent to the original factorial structure of the IWPS-R. Τhe convergent validity revealed a positive correlation between all the domains of the scale (p < 0.001). The test-retest method showed that there are no significant differences between the first and the second measurements (intraclass correlation coefficient [ICC] = 0.990, p < 0.001). The internal consistency was very good (Cronbach’s alpha coefficient = 0.878).

Conclusions: The IWPS-R is a reliable and valid instrument for Greek nurses to measure the perceptions of the nursing working environment.

Reliability of the Preliminary OMERACT Juvenile Idiopathic Arthritis MRI Score (OMERACT JAMRIS-SIJ)

This study reports the reliability of the juvenile idiopathic arthritis magnetic resonance imaging scoring system (JAMRIS-SIJ). The study comprised of eight raters—two rheumatologists and six radiologists—and 30 coronal T1 and Short-Tau Inversion Recovery (STIR) MRI scans of patients with enthesitis-related juvenile spondylarthritis. The median age of patients was 15 years with a mean disease duration of 5 years and 22 (73.3%) of the sample were boys. The inter-rater agreement of scores for each of the JAMRIS-SIJ items was calculated using a two-way random effect, absolute agreement, and single rater intraclass correlation coefficient (ICC 2.1). The ICC was interpreted together with kurtosis, since the ICC is also affected by the distribution of scores in the sample. The eight-rater, single measure inter-rater ICC (and kurtosis) values for JAMRIS-SIJ inflammation and damage components were the following: bone marrow edema (BME), 0.76 (1.2); joint space inflammation, 0.60 (1.8); capsulitis, 0.58 (9.2); enthesitis, 0.20 (0.1); ankylosis, 0.89 (35); sclerosis, 0.53 (4.6); erosion, 0.50 (6.5); fat lesion, 0.40 (21); backfill, 0.38 (38). The inter-rater reliability for BME and ankylosis scores was good and met the a priori set ICC threshold, whereas for the other items it was variable and below the selected threshold. Future directives should focus on refinement of the scores, definitions, and methods of interpretation prior to validation of the JAMRIS-SIJ through the assessment of its measurement properties.

Development of a valid and reliable pterygium surgery assessment scale for ophthalmology residents

BACKGROUND

Microsurgery training has become an important part of ophthalmology teaching and one of the main topics of examination. Accurate and effective evaluation of microsurgery skills is vital for the training and teaching of residents. In this study, we aimed to establish a pterygium surgery assessment scale for use by ophthalmic residents and evaluate its reliability and validity.

METHODS

Based on a literature search, experienced pterygium surgeons developed the preliminary scale according to the standard surgical procedure. The preliminary scale and a questionnaire were sent to teaching and research experts in the field for feedback. Face and content validity and reliability of the scale were determined by rounds of modifications based on expert feedback. For construct validity, existing assessment scales were obtained and a range of factors were tested.

RESULTS

Nineteen expert surgeons completed the questionnaire and modifications were made until all surgeons agreed on the final scale. Good construct validity was found by evaluation against 257 existing scales. For reliability, 280 evaluation scales were completed. Inter- and intra-rater reliability analysis both found Intraclass Correlation Coefficient (ICC) > 0.8 for all items and total scores.

CONCLUSION

The pterygium surgery assessment scale developed in this study has good reliability and validity, and is an effective measurement tool for the evaluation of ophthalmology residents' pterygium surgical skills.