The κ statistic in rehabilitation research: An examination11No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the author(s) or on any organization with which the author(s) is/are associated

Advancing accuracy in breath testing for lung cancer: strategies for improving diagnostic precision in imbalanced data

BACKGROUND

Breath testing using an electronic nose has been recognized as a promising new technique for the early detection of lung cancer. Imbalanced data are commonly observed in electronic nose studies, but methods to address them are rarely reported.

OBJECTIVE

The objectives of this study were to assess the accuracy of electronic nose screening for lung cancer with imbalanced learning and to select the best mechanical learning algorithm.

METHODS

We conducted a case‒control study that included patients with lung cancer and healthy controls and analyzed metabolites in exhaled breath using a carbon nanotube sensor array. The study used five machine learning algorithms to build predictive models and a synthetic minority oversampling technique to address imbalanced data. The diagnostic accuracy of lung cancer was assessed using pathology reports as the gold standard.

RESULTS

We enrolled 190 subjects between 2020 and 2023. A total of 155 subjects were used in the final analysis, which included 111 lung cancer patients and 44 healthy controls. We randomly divided samples into one training set, one internal validation set, and one external validation set. In the external validation set, the summary sensitivity was 0.88 (95% CI 0.84-0.91), the summary specificity was 1.00 (95% CI 0.85-1.00), the AUC was 0.96 (95% CI 0.94-0.98), the pAUC was 0.92 (95% CI 0.89-0.96), and the DOR was 207.62 (95% CI 24.62-924.64).

CONCLUSION

Electronic nose screening for lung cancer is highly accurate. The support vector machine algorithm is more suitable for analyzing chemical sensor data from electronic noses.

The Swedish version of the pain self-efficacy questionnaire short form, PSEQ-2SV: Cultural adaptation and psychometric evaluation in a population of patients with musculoskeletal disorders

OBJECTIVES

Measuring pain self-efficacy is suggested as relevant in patients with musculoskeletal disorders (MSDs) in a primary care setting. However, there is no pain self-efficacy questionnaire (PSEQ) available in Swedish. The aim of this study was to translate and culturally adapt PSEQ-2 to Swedish and evaluate reliability and validity in a population of patients with MSDs.

METHODS

A translation and cultural adaptation together with psychometric evaluation of reliability and validity were performed according to guidelines with three groups of participants with MSDs. First, a convenient sample of 25 individuals were interviewed in the translation process. Next, 61 participants completed the test-retest survey via social media or QR codes in waiting rooms in rehabilitation clinics. Finally, to evaluate construct validity questionnaires were obtained from 132 participants with MSDs in an ongoing randomized controlled trial.

RESULTS

The Swedish version of PSEQ-2 showed adequate face and content validity. The results of 0.805 on Cohen's weighted kappa indicate that the reliability of PSEQ-2SV in a group of adults with MSDs is on the border between substantial and almost perfect. The point estimate regarding relative rank variance, measuring the individual variation within the group, and relative concentration, the systematic change in how the assessments are concentrated on the scale's categories, shows minor systematic differences and some random differences not neglectable. The construct validity of pre-defined hypotheses was met to some degree.

CONCLUSION

The PSEQ-2SV has been accurately linguistically translated and tested for reliability and validity, in a population of MSDs, and is deemed to be able to be used in the clinic and in research. As there were some concerns regarding measurement error and systematic bias, more research could be of value.

A comparison of agreement between actigraphy and polysomnography for assessing sleep during posttraumatic amnesia

STUDY OBJECTIVES

Sleep disturbance often emerges in the early recovery phase following a moderate to severe traumatic brain injury, known as posttraumatic amnesia. Actigraphy is commonly employed to assess sleep, as it is assumed that patients in posttraumatic amnesia (who display confusion, restlessness, and agitation) would better tolerate this measure over gold-standard polysomnography (PSG). This study evaluated the agreement between PSG and actigraphy for determining (sleep/wake time, sleep efficiency, sleep latency, and awakenings) in patients experiencing posttraumatic amnesia. It also compared the epoch-by-epoch sensitivity, specificity, and accuracy between the Actigraph device's 4 wake threshold settings (low, medium, high, and automatic) to PSG.

METHODS

The sample consisted of 24 inpatients recruited from a traumatic brain injury inpatient rehabilitation unit. Ambulatory PSG was recorded overnight at bedside and a Philips Actiwatch was secured to each patient's wrist for the same period.

RESULTS

There were poor correlations between PSG and actigraphy for all parameters (Lin's concordance correlation coefficient = < 0.80). The low threshold displayed the highest correlation with PSG for wake and sleep time, albeit still low. Actigraphy displayed low specificity (ranging from 17.1% to 36.6%). There appears to be a greater disparity between actigraphy and PSG for patients with increased wake time.

CONCLUSIONS

Actigraphy, while convenient, demonstrated poorer performance in determining sleep-wake parameters in patients with significantly disturbed sleep. Ambulatory PSG can provide a clearer understanding of the extent of sleep disturbances in these patients with reduced mobility during early rehabilitation. Study findings can help design future protocols of sleep assessment during posttraumatic amnesia and optimize treatment.

CITATION

Fedele B, McKenzie D, Williams G, Giles R, Olver J. A comparison of agreement between actigraphy and polysomnography for assessing sleep during posttraumatic amnesia. J Clin Sleep Med. 2022;18(11):2605-2616.

IL-6 and TNF-α responses to acute and regular exercise in adult individuals with multiple sclerosis (MS): a systematic review and meta-analysis

Background

In both the general population and people with multiple sclerosis (PwMS), physical exercise is associated with improved mental well-being. Moreover, there is evidence of the possible protection of physical activity against disease progression in multiple sclerosis (MS). However, the question arises if acute or regular exercise has any impact on the immune system in PwMS. To answer this question, we performed a systematic review and meta-analysis on both plasma and serum cytokine levels (IL-6 and TNF-α) before and after acute and regular exercise among PwMS and compared to healthy controls.

Method

We performed an online search via PubMed, EMBASE, SCOPUS, Web of Science, and Cochrane Library till September 2021 to identify original studies on IL-6 and TNF-α changes after acute and regular exercise in PwMS and controls. Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), 11 original studies were included in the meta-analysis. Sensitivity analyses were used to identify the origins of heterogeneity. R 4.0.4 was used to perform the meta-analysis of IL-6 and TNF-α levels before and after acute and regular exercise in PwMS, compared to controls. This study does not qualify for a clinical trial number.

Results

IL-6 levels did neither increase nor decrease after acute and regular exercise in PwMS, and compared to controls (pre- vs. post-intervention: Standardized Mean Difference (SMD) -0.09, 95% CI [−0.29; 0.11], p-value = 0.37, PwMS vs. Control: SMD −0.08, 95% CI [−0.33; 0.16], p-value = 0.47). In PwMS, TNF-α levels decreased after regular exercise and when TNF-α levels of both acute and regular exercise were pooled (pre- vs. post-intervention: SMD −0.51, 95% CI [-0.91; 0.11], p-value = 0.01, PwMS vs. Control: SMD −0.23, 95% CI [−0.66; 0.18], p-value = 0.26). TNF-α levels did neither increase nor decrease after acute and regular exercise in PwMS, when compared to controls.

Conclusion

This systematic review and meta-analysis show that exercise does not lead to significant changes in peripheral levels of IL-6 in PwMS in contrast to the observed response in healthy subjects and other medical contexts. However, regular exercise had a specific anti-inflammatory effect on blood TNF-α levels in PwMS. It remains to be investigated why PwMS display this different exercise-induced pattern of cytokines.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40001-022-00814-9.

Low interobserver and intraobserver reliability using the Matta radiographic system for intraoperative assessment of reduction following acetabular ORIF

INTRODUCTION

The system described by Matta for rating acetabular fracture quality of reduction following ORIF has been used extensively throughout the literature. However, the reliability of this system remains to be validated. We sought to determine the interobserver and intraobserver reliability of this system when used by fellowship-trained pelvic and acetabular surgeons to evaluate intraoperative fluoroscopy.

METHODS

This is a retrospective evaluation of a prospectively collected acetabular fracture database at an academic level I trauma center. The quality of reduction of all acetabular fractures treated with open reduction internal fixation (ORIF) between May 2013 and December 2015 was assessed using three standard intraoperative fluoroscopic views (anteroposterior and two 45˚ oblique Judets). Displacement of ≤1 mm was considered to be an anatomic reduction, 2-3 mm imperfect, and >3 mm poor according to the system described by Matta. A total of 107 acetabular fractures treated with ORIF with complete intraoperative fluoroscopic images during that time period were available for review. Acetabular fracture reductions were reviewed by the operative surgeon at the time of surgery and subsequently reviewed by two fellowship-trained pelvic and acetabular surgeons. All reduction assessments were performed in a blinded fashion. The primary outcome measure was interobserver reliability for assessing reduction quality. This was evaluated using a weighted kappa (κ_w) statistic between each evaluator and the operative surgeon and a generalized kappa (κ_g) for all 3 surgeons. After a 6-week "washout interval," the surgeons reviewed the images again and intraobserver agreement was calculated using a weighted kappa statistic.

RESULTS

Interobserver reliability based on the initial assessment was low (κ_g = 0.09); however, did slightly improve with the second assessment to fair (κ_g = 0.24). Intraobserver reliability ranged from slight (κ_w = 0.20) to moderate (κ_w = 0.53) among the surgeons.

DISCUSSION

Low interobserver and intraobserver reliability was found when quality of reduction was assessed with intraoperative fluoroscopic images by the operative and two other pelvic and acetabular surgeons using the Matta system. Given the importance of an anatomic reduction on functional and radiographic outcomes, an accurate and reliable system for assessing intraoperative quality of reduction is essential.

Exercise-induced increase in blood-based brain-derived neurotrophic factor (BDNF) in people with multiple sclerosis: A systematic review and meta-analysis of exercise intervention trials

Background

Exercise training may affect the blood levels of brain-derived neurotrophic factor (BDNF), but meta-analyses have not yet been performed comparing pre- and post-intervention BDNF concentrations in patients with multiple sclerosis (PwMS).

Objective

To perform a meta-analysis to study the influence of exercise on BDNF levels and define components that modulate them across clinical trials of exercise training in adults living with multiple sclerosis (MS).

Method

Five databases (PubMed, EMBASE, Cochrane Library, PEDro database, CINAHL) were searched up to June 2021. According to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses, we included 13 articles in the meta-analysis, including 271 subjects. To investigate sources of heterogeneity, subgroup analysis, meta-regression, and sensitivity analysis were conducted. We performed the meta-analysis to compare pre- and post-exercise peripheral levels of BDNF in PwMS.

Results

Post-exercise concentrations of serum BDNF were significantly higher than pre-intervention levels (Standardized Mean Difference (SMD): 0.33, 95% CI: [0.04; 0.61], p-value = 0.02). Meta-regression indicated that the quality of the included studies based on the PEDro assessment tool might be a source of heterogeneity, while no significant effect was found for chronological age and disease severity according to the expanded disability status scale.

Conclusion

This systematic review and meta-analysis shows that physical activity increases peripheral levels of BDNF in PwMS. More research on the effect of different modes of exercise on BDNF levels in PwMS is warranted.

The validity, reliability and feasibility of four instruments for assessing the consciousness of stroke patients in a neurological intensive care unit compared

Background

Early rehabilitation is the foundation for recovery for those admitted to an intensive care unit. Appropriate assessment of consciousness is needed before any rehabilitative intervention begins.

Methods

This prospective study compared the validity, reliability and applicability of the sedation-agitation scale, the Richmond Agitation-Sedation Scale, the motor activity assessment scale and the Glasgow Coma Scale in a working neurological intensive care unit. Eighty-three stroke patients were assessed with the four scales by the same 3 raters acting independently: a senior physician, a senior therapist and a trainee. That generated 996 assessment records for comparison.

Results

Good agreement (r=0.98–0.99) was found among the sedation-agitation scale, the Richmond Agitation-Sedation Scale, the motor activity assessment scale scores, but the Glasgow Coma Scale ratings correlated less well (r=0.72–0.76) with the others. Consistent results were also found among the three raters. After stratification of the ratings by age, gender, level of consciousness and Acute Physiology and Chronic Health Evaluation score, the scales reported significant differences among the levels of consciousness and among those with different Acute Physiology and Chronic Health Evaluation results, but not with different age or gender strata.

Conclusions

The four instruments tested are all reliable enough and feasible for use as a tool for consciousness screening in a neurological intensive care unit.

A method comparison of three immunoassays for detection of neutralizing antibodies against SARS-CoV-2 receptor-binding domain in individuals with adenovirus type-5-vectored COVID-19 vaccination

Objective

Detecting neutralizing antibodies targeting receptor‐binding domain (RBD) is important for the assessment of humoral protection and vaccine efficacy after vaccination. We compared the performance of three surrogate immunoassays for detection of neutralizing antibodies targeting RBD.

Methods

We analyzed 115 serum samples obtained from individuals with Ad5‐vectored COVID‐19 vaccination using two competitive enzyme‐linked immunosorbent assays (Wantai BioPharm and Synthgene Medical Technology) and one competitive chemiluminescence assay (YHLO Biotech). Performance evaluation and methodology comparison were performed according to the Clinical and Laboratory Standards Institute related guidelines.

Results

The precision met the manufacturers’ statements. The linear range of the WANTAI was 0.0625–0.545 U/ml and the YHLO was 0.260–242.4 U/ml. The WANTAI’s limit of blank (LoB) and limit of detection (LoD) were 0.03 and 0.06 U/ml, respectively. The YHLO’s LoB and LoD were 0.048 and 0.211 U/ml, respectively. The correlations of semi‐quantitative results of Synthgene with quantitative results of YHLO (ρ = 0.566) and WANTAI (ρ = 0.512) were medium. For YHLO and WANTAI, there was a good agreement (0.62) and a strong correlation (ρ = 0.931). Passing–Bablok analysis and Bland‐Altman plot showed a positive bias (112.3%) of the YHLO compared to the WANTAI. The exclusion of samples >50 U/ml did not decrease bias.

Conclusion

These findings contribute to a deeper understanding of surrogate viral neutralization assays and provide useful data for future comparison studies.

Can Vertical Laminar Fracture Further Discriminate Fracture Severity Between Thoracolumbar AO Type A3 and A4 Fractures?

OBJECTIVE

To determine whether vertical laminar fracture (VLF) can distinguish between AO type A3 and A4 fractures.

METHODS

In a retrospective review of 111 consecutive acute thoracolumbar burst fractures, 5 reviewers independently analyzed computed tomography scans to classify fractures into A3 or A4 and to identify VLF. The following computed tomography parameters were measured: spinal canal stenosis >50%, anterior vertebral height ratio <50%, load sharing score >6, and local kyphosis >20°. We calculated the diagnostic performance of VLF in detecting A4 fracture. We compared the proportion of fractures with positive bony parameters, neurological deficit, dural tears, and surgical treatment between A3, A4 with VLF, and A4 without VLF.

RESULTS

VLF was present in 62/75 (83%) A4 fractures and 2/36 (5.5%) A3 fractures (P < 0.0001). VLF yielded a high specificity of 94% (95% confidence interval 81%-99%) and moderately high sensitivity of 83% (95% CI 72%-91%) in detecting A4 fractures. A significantly higher proportion of A4 fractures with VLF had neurological deficit (24% vs. 0, P = 0.05), spinal canal stenosis >50% (25% vs. 0, P = 0.04), and anterior vertebral height ratio <50% (24% vs. 0, P = 0.05) than A4 fractures with no VLF. Interrater and intrarater κ values for VLF and AO standard criterion were excellent (>0.85).

CONCLUSIONS

We found VLF to be highly specific, sensitive, and reliable in detecting A4 fractures. A higher proportion of A4 fractures with VLF had radiographic parameters and neurological deficit than A4 fractures with no VLF. VLF could be used as a severity modifier to further discriminate A3 and A4 fractures regarding severity and potentially guide treatment decision making.

Simplified evaluation of CONsciousness disorders (SECONDs) in individuals with severe brain injury: A validation study

BACKGROUND

The Coma Recovery Scale-Revised (CRS-R) is the gold standard to assess severely brain-injured patients with prolonged disorders of consciousness (DoC). However, the amount of time needed to complete this examination may limit its use in clinical settings.

OBJECTIVE

We aimed to validate a new faster tool to assess consciousness in individuals with DoC.

METHODS

This prospective validation study introduces the Simplified Evaluation of CONsciousness Disorders (SECONDs), a tool composed of 8 items: arousal, localization to pain, visual fixation, visual pursuit, oriented behaviors, command-following, and communication (both intentional and functional). A total of 57 individuals with DoC were assessed on 2 consecutive days by 3 blinded examiners: one CRS-R and one SECONDs were performed on 1 day, whereas 2 SECONDs were performed on the other day. A Mann-Whitney U test was used to compare the duration of administration of the SECONDs versus the CRS-R, and weighted Fleiss' kappa coefficients were used to assess inter-/intra-rater reliability as well as concurrent validity.

RESULTS

In the 57 participants, the SECONDs was about 2.5 times faster to administer than the CRS-R. The comparison of the CRS-R versus the SECONDs on the same day or the best of the 3 SECONDs led to "substantial" or "almost perfect" agreement (kappa coefficients ranging from 0.78 to 0.85). Intra-/inter-rater reliability also showed almost perfect agreement (kappa coefficients from 0.85 to 0.91 and 0.82 to 0.85, respectively).

CONCLUSIONS

The SECONDs appears to be a fast, reliable and easy-to-use scale to diagnose DoC and may be a good alternative to other scales in clinical settings where time constraints preclude a more thorough assessment.

Translation and Validation of the TANGO Nocturia Screening Tool into Greek

Purpose

The present study aimed to translate the Targeting the individual's Aetiology of Nocturia to Guide Outcomes Questionnaire (TANGO) into the Greek language to create TANGO-Gr. A relative reliability study in prospective samples of community dwellers and rehabilitation centre residents was undertaken in order to validate the translation. In terms of nocturia severity, these groups were considered to be representative of patients at both ends of the scale, and therefore suitable validation purposes.

Patients and Methods

The prospective descriptive study took place between 07 and 09/2020. Fifty residents in a rehabilitation centre and thirty-seven community dwellers were included in the study. All participants had more than one episode of nocturia per night. They were asked to complete the newly translated TANGO-Gr Questionnaire twice (test-retest procedure). TANGO-Gr includes three possible answers ("yes", "no", and "DK/NA") to facilitate increased validity. The study group evaluated the socio and demographic characteristics, the level of independence and the comprehensive frailty of all participants.

Results

Both groups satisfactorily completed questionnaire responses. The overall Cronbach's alpha coefficient was 0.753 (community-dwelling people: 0.776; rehabilitation centre residents: 0.531). A test-retest statistical analysis to determine reliability found an overall median Kappa of 0.88 (IQR: 0.79-0.94) in community dwellers and 0.91 (IQR: 0.84-1.00) in rehabilitation centre residents, thereby confirming an almost perfect respective agreement.

Conclusion

The validity of the newly translated TANGO-Gr Questionnaire was proven. It should be considered to be a reliable, self-administered screening tool suitable for clinical practice, and therefore invaluable for Greek-speaking people. The study also determined that certain environment-related variables may contribute to nocturia in rehabilitation centre populations, which is a finding which invites further study.

Interrater and Intrarater Reliability of Musculoskeletal Ultrasonographic Findings for the Common Extensor Tendon of the Elbow

Objectives:

This retrospective case study evaluated the interrater and intrarater reliability of seven common extensor tendon pathologic features on musculoskeletal ultrasonography (MSK-US).

Materials and Methods:

A cohort of 50 patients were imaged due to presenting with atraumatic nonradicular lateral elbow pain. Three experienced and two novice readers rated the images on two separate occasions, and AC1 and kappa coefficients were calculated for each feature.

Results:

The interrater reliability was fair with respect to fascial thickening/scarring (AC1 = 0.26), tearing (AC1 = 0.35), tendon thickening (AC1 = 0.38), and intratendinous calcification (AC1 = 0.33); substantial for enthesophytes (AC1 = 0.80); and near complete for hyperemia (AC1 = 0.83) and hypoechogenicity (AC1 = 0.92). Intrarater reliability was moderate for fascial thickening/scarring (κ = 0.48), tearing (κ = 0.41), tendon thickening (0.47), intratendinous calcification (κ = 0.56), and hypoechogenicity (κ = 0.47); substantial for hyperemia (κ = 0.71); and almost perfect for enthesophytes (κ = 0.86).

Conclusion:

MSK-US may be a reliable tool to determine soft tissue changes in common extensor tendon pathology.

Agreement Between Physical Therapists in Diagnosing Benign Paroxysmal Positional Vertigo

BACKGROUND AND PURPOSE

Individuals with benign paroxysmal positional vertigo (BPPV) are frequently referred to physical therapy for management, but little is known on how reliable therapists are at diagnosing BPPV. The purpose of the study was to examine the agreement between physical therapists in identifying nystagmus and diagnosing BPPV.

METHODS

Thirty-eight individuals with complaints of positional vertigo, 19 from each of 2 clinics (clinics 1 and 2) that specialize in vestibular rehabilitation, had eye movements recorded using video goggles during positioning tests including supine-to-sit, supine roll, and Dix-Hallpike tests. Three therapists from each of the clinics independently observed videos, documented nystagmus characteristics of each testing position, and made a diagnosis for each case. Kappa (κ) statistics were calculated between therapists within each clinic for nystagmus identification and diagnosis.

RESULTS

Clinic 1 therapists demonstrated substantial to almost perfect agreement in identifying nystagmus during positional tests (κ = 0.68-1, P < 0.005). Clinic 2 therapists showed moderate to almost perfect agreement for presence of nystagmus (κ = 0.57-1, P < 0.005). Therapists at both sites had almost perfect agreement of diagnosis side, canal, and mechanism (κ = 0.81-1, P < 0.005).

DISCUSSION AND CONCLUSION

Therapists utilized observations from multiple positional tests to determine diagnoses. This was evident by occasional disagreement in nystagmus presence and characteristics, but agreement in diagnosis, including ruling out BPPV. The results may not be generalizable to all physical therapists or therapists' ability to diagnose central and atypical nystagmus presentations. Experienced physical therapists demonstrated strong agreement in diagnosing common forms of BPPV.Video Abstract available for more insight from the authors (see the Video Supplemental Digital Content 1, available at: http://links.lww.com/JNPT/A340).

A comparative performance analysis of the International Classification of Functioning, Disability and Health and the Item-Perspective Classification framework for classifying the content of patient reported outcome measures

BACKGROUND

Standardized coding of the content presented in patient reported outcome measures can be achieved using classification frameworks, and the resulting data can be used for ascertaining content validity or comparative analyses. The International Classification of Functioning (ICF) is a framework with a detailed conceptual structure that has been successfully utilized for such purposes through established coding procedures. The Item Perspective Classification (IPC) framework is a newly developed relational coding system that classifies the respondent perspective and conceptual domains addressed in items. The purpose of this study was to compare and describe the performance of these two frameworks when used alone, and in conjunction, for the generation of data pertaining to the content of patient reported outcome measures.

METHODS

Six health-related quality of life questionnaires with a total of 159 items were classified by two raters using the Item Perspective Classification framework in conjunction with the International Classification of Functioning. Framework performance indicators included: classification capacity (percent of items amenable to successful classification), coding efficiency (number of codes required to classify items), and content overlap detection (percent of items sharing identical classification codes with at least one other item). Inter-rater reliability of item coding was determined using Krippendorff's alpha.

RESULTS

Classification capacity of the IPC framework was 97%, coding efficiency 26, and content overlap detection was 95%; whereas respective values for the ICF were 68%, 114, and 58%. When used in conjunction values were 63%, 129, and 30%. Krippendorff's alpha exceeded 0.97 for all 3 classification indices.

CONCLUSION

Inter-rater agreement on classification data was excellent. The IPC framework provided a unique classification of the respondent's judgment during item response and classified more items using fewer categories, indicated greater content overlap across items and was able to describe the relationship between multiple concepts presented within the context of a single item. The ICF provided a unique classification of item content relating to aspects of disability and generated more detailed and precise descriptions. A combined approach provided a rich description (detailed codes) with each framework providing complementary information. The benefits of this approach in instrument development and content validation require further investigation.

Reliability and validity of the Patient Benefit Assessment Scale for Hospitalised Older Patients (P-BAS HOP)

BACKGROUND

The Patient Benefit Assessment Scale for Hospitalised Older Patients (P-BAS HOP) is a tool which is capable of both identifying the priorities of the individual patient and measuring the outcomes relevant to him/her, resulting in a Patient Benefit Index (PBI) with range 0-3, indicating how much benefit the patient had experienced from the admission. The aim of this study was to evaluate the reliability, validity, responsiveness and interpretability of the P-BAS HOP.

METHODS

A longitudinal study among hospitalised older patients with a baseline interview during hospitalisation and a follow-up by telephone 3 months after discharge. Test-retest reliability of the baseline and follow-up questionnaire were tested. Percentage of agreement, Cohen's kappa with quadratic weighting and maximum attainable kappa were calculated per item. The PBI was calculated for both test and retest of baseline and follow-up and compared with Intraclass Correlation Coefficient (ICC). Construct validity was tested by evaluating pre-defined hypotheses comparing the priority of goals with experienced symptoms or limitations at admission and the achievement of goals with progression or deterioration of other constructs. Responsiveness was evaluated by correlating the PBI with the anchor question 'How much did you benefit from the admission?'. This question was also used to evaluate the interpretability of the PBI with the visual anchor-based minimal important change distribution method.

RESULTS

Reliability was tested with 53 participants at baseline and 72 at follow-up. Mean weighted kappa of the baseline items was 0.38. ICC between PBI of the test and retest was 0.77. Mean weighted kappa of the follow-up items was 0.51. ICC between PBI of the test and retest was 0.62. For the construct validity, tested in 451 participants, all baseline hypotheses were confirmed. From the follow-up hypotheses, tested in 344 participants, five of seven were confirmed. The Spearman's correlation coefficient between the PBI and the anchor question was 0.51. The optimal cut-off point was 0.7 for 'no important benefit' and 1.4 points for 'important benefit' on the PBI.

CONCLUSIONS

Although the concept seems promising, the reliability and validity of the P-BAS HOP appeared to be not yet satisfactory. We therefore recommend adapting the P-BAS HOP.

Breath biopsy of breast cancer using sensor array signals and machine learning analysis

Breast cancer causes metabolic alteration, and volatile metabolites in the breath of patients may be used to diagnose breast cancer. The objective of this study was to develop a new breath test for breast cancer by analyzing volatile metabolites in the exhaled breath. We collected alveolar air from breast cancer patients and non-cancer controls and analyzed the volatile metabolites with an electronic nose composed of 32 carbon nanotubes sensors. We used machine learning techniques to build prediction models for breast cancer and its molecular phenotyping. Between July 2016 and June 2018, we enrolled a total of 899 subjects. Using the random forest model, the prediction accuracy of breast cancer in the test set was 91% (95% CI: 0.85–0.95), sensitivity was 86%, specificity was 97%, positive predictive value was 97%, negative predictive value was 97%, the area under the receiver operating curve was 0.99 (95% CI: 0.99–1.00), and the kappa value was 0.83. The leave-one-out cross-validated discrimination accuracy and reliability of molecular phenotyping of breast cancer were 88.5 ± 12.1% and 0.77 ± 0.23, respectively. Breath tests with electronic noses can be applied intraoperatively to discriminate breast cancer and molecular subtype and support the medical staff to choose the best therapeutic decision.

Agreement and reliability of a symptom modification test cluster for patients with subacromial pain syndrome

OBJECTIVES

To identify if a plausible theoretical construct exists for a test cluster in a group of patients with subacromial pain syndrome presenting with glenohumeral internal rotation deficit (GIRD); determine the intertester agreement and reliability of the proposed test cluster; determine if there are any meaningful relationships between the test cluster and the component tests for the entire sample; and determine if there are any differences in disability on the Dutch version of the shoulder pain and disability index between participants with a positive and negative test cluster.

METHODS

This study is a retrospective secondary analysis of data that were collected to determine the interrater agreement and reliability of clinical tests for assessment of patients with shoulder pain in primary care.

RESULTS

The test cluster total agreement and negative specific agreement was 87.8 and 90.4%, respectively. The prevalence-adjusted bias-adjusted kappa for the test cluster was substantial at 0.76. There were statistically significant meaningful relationships (≥0.50) between GIRD and the test cluster for Tester A (Phi = 0.71, p < .01) and Tester B (Phi = 0.82, p < .01). No differences in disability were identified between those with a positive and negative test cluster.

CONCLUSION

The test cluster described in this study may be a reliable means of identifying a subgroup of patients with subacromial pain syndrome related to GIRD. Future research should look to validate this test cluster prospectively.

Experimental Studies of Inter-Rater Agreement in Traditional Chinese Medicine: A Systematic Review

Objectives: It has been recommended that clinical trials of Traditional Chinese Medicine (TCM) would be more ecologically valid if its characteristic mode of diagnostic reasoning were integrated into their design. In that context, however, it is also widely held that demonstrating a high level of agreement on initial TCM diagnoses is necessary for the replicability that the biomedical paradigm requires for the conclusions from such trials. Our aim was to review, summarize, and critique quantitative experimental studies of inter-rater agreement in TCM, and some of their underlying assumptions. Design: Systematic electronic searches were conducted for articles that reported a quantitative measure of inter-rater agreement across a number of rating choices based on examinations of human subjects in person by TCM practitioners, and published in English language peer-reviewed journals. Publications in languages other than English were not included, nor those appearing in other than peer-reviewed journals. Predefined categories of information were extracted from full texts by two investigators working independently. Each article was scored for methodological quality. Outcome measures: Design features across all studies and levels of inter-rater agreement across studies that reported the same type of outcome statistic were compared. Results: Twenty-one articles met inclusion criteria. Fourteen assessed inter-rater agreement on TCM diagnoses, two on diagnostic signs found upon traditional TCM examination, and five on novel rating schemes derived from TCM theory and practice. Raters were students of TCM colleges or graduates of TCM training programs with 3 or more years experience and licensure. Type of outcome statistic varied. Mean rates of pairwise agreement averaged 57% (median 65, range 19-96) across the 9 studies reporting them. Mean Cohen's kappa averaged 0.34 (median 0.34, range 0.07-0.59) across the seven studies reporting them. Meta-analysis was not possible due to variations in study design and outcome statistics. High risks of bias and confounding, and deficits in statistical reporting were common. Conclusions: With a few exceptions, the levels of agreement were low to moderate. Most studies had significant deficits of both methodology and reporting. Results overall suggest a few design features that might contribute to higher levels of agreement. These should be studied further with better experimental controls and more thorough reporting of outcomes. In addition, methods of complex systems analysis should be explored to more adequately model the relationship between clinical outcomes, and the series of diagnoses and treatments that are the norm in actual TCM practice.

Reproducibility protocol for diagnostic procedures in Manual/Musculoskeletal Medicine

The International Academy of Manual/Musculoskeletal Medicine (IAMMM) has completely revised the protocol for reproducibility studies of diagnostic procedures in Manual/Musculoskeletal Medicine (M/M Medicine). The protocol was and is aimed at the practitioners in the field of M/M Medicine. This IAMMM protocol can be used in a very practical way and makes it feasible to perform reproducibility studies equally well in private practices and clinics for M/M Medicine with two or more physicians and as by educational boards of the societies of M/M Medicine. This IAMMM protocol provides practical solutions for sample size calculations in reproducibility studies using kappa statistics. Step by step, many different statistical aspects of reproducibility studies are explained, resulting in a very structured protocol format of how to perform a reproducibility study using overall agreement and the kappa value as statistical outcome.

Brain activity during dual task gait and balance in aging and age-related neurodegenerative conditions: A systematic review

The aims of this systematic review were to investigate (1) real-time brain activity during DT gait and balance, (2) whether changes in brain activity correlate with changes in behavioral outcomes in older adults and people with age-related neurodegenerative conditions. PubMed, PsycINFO, and Web of Science were searched from 2009 to 2019 using the keywords dual task, brain activity, gait, balance, aging, neurodegeneration, and other related search terms. A total of 15 articles were included in this review. Functional near-infrared spectroscopy and electroencephalogram measures demonstrated that older adults had higher brain activity, particularly in the prefrontal cortex (PFC), compared to young adults during dual task gait and balance. Similar neurophysiological results were observed in people with age-related neurodegenerative conditions. Few studies demonstrated a relationship between increased brain activity and better behavioral outcomes. This systematic review supports the notion that aging and age-related neurodegenerative conditions are associated with neuronal network changes, resulting in increased brain activity specifically in the PFC. Further studies are warranted to assess the relationship between increased PFC activation during dual task gait and balance and behavioral outcomes to better optimize the rehabilitation interventions.

The Correlation Between Tenosynovitis Pattern on Two-Phase Bone Scintigraphy and Clinical Manifestation in Patients with Suspected Rheumatoid Arthritis

Purpose

To investigate the correlation between the tenosynovitis pattern on two-phase bone scintigraphy (2P-BS) and clinical manifestation in patients with suspected rheumatoid arthritis (RA).

Method

2P-BS including technetium-99m-methylene diphosphonate blood pool and bone phase imaging in 402 consecutive patients with clinically suspected RA were retrospectively reviewed. According to 2010 RA Classification Criteria, patients were grouped as RA and non-RA. Visual assessment of all fingers, toes, wrists, and ankles on 2P-BS was performed. Clinical suspected tenosynovitis was evaluated on physical examination. Rheumatoid factor, anti-cyclic citrullinated protein antibody, C-reactive protein, and estimated sedimentation rate were obtained. Radiographic findings were also used to define early and established arthritis.

Results

Tenosynovitis pattern was detected in 12.7% (51/402 patients) on 2P-BS. A total of 94.1% (48/51) were diagnosed as RA vs. 5.9% (3/51) as non-RA. Of the 48 RA patients with positive 2P-BS finding, 85.4% (41/48) had early arthritis and 14.6% (7/48) had established arthritis. On physical examination, tenosynovitis was suspected in 21.9% (88/402). A total of 56.8% (50/88) belonged to the RA group and 43.2% (38/88) to the non-RA group. The tenosynovitis pattern of 2P-BS and physical examination showed statistical difference and moderate agreement. The positive tenosynovitis pattern on 2P-BS represented up to 26.408 of odds ratio which was highest among the RA-associated factors.

Conclusion

Tenosynovitis pattern on 2P-BS was more commonly detected in the RA group and was more frequently associated with early arthritis pattern. Therefore, 2P-BS could give additional information for the detection of subclinical tenosynovitis in early or preclinical RA patients.

Interrater agreement and reliability of clinical tests for assessment of patients with shoulder pain in primary care

Background: There is limited information about the agreement and reliability of clinical shoulder tests. Objectives: To assess the interrater agreement and reliability of clinical shoulder tests in patients with shoulder pain treated in primary care. Methods: Patients with a primary report of shoulder pain underwent a set of 21 clinical shoulder tests twice on the same day, by pairs of independent physical therapists. The outcome parameters were observed and specific interrater agreement for positive and negative scores, and interrater reliability (Cohen's kappa (κ)). Positive and negative interrater agreement values of ≥0.75 were regarded as sufficient for clinical use. For Cohen's κ, the following classification was used: <0.20 poor, 0.21-0.40 fair, 0.41-0.60 moderate, 0.61-0.80 good, 0.81-1.00 very good reliability. Participating clinics were randomized in two groups; with or without a brief practical session on how to conduct the tests. Results: A total of 113 patients were assessed in 12 physical therapy practices by 36 physical therapists. Positive and negative interrater agreement values were both sufficient for 1 test (the Full Can Test), neither sufficient for 5 tests, and only sufficient for either positive or negative agreement for 15 tests. Interrater reliability was fair for 11 tests, moderate for 9 tests, and good for 1 test (the Full Can Test). An additional brief practical session did not result in better agreement or reliability. Conclusion: Clinicians should be aware that interrater agreement and reliability for most shoulder tests is questionable and their value in clinical practice limited.

Reliability of the Spanish version of the wheelchair skills test 4.2 for manual wheelchair users with spinal cord injury

Study design: Cross-sectional.Objectives: The majority of people with a spinal cord injury (SCI) are dependent on wheelchair for their mobility. Approximately, 36% of wheelchair users reported that obstacles such as curbs, uneven terrain, flooring surfaces and thresholds were barriers to mobility. Several studies have shown that assessment and training of wheelchair skills leads to improvements in those skills. The purpose of our study was to translate the Wheelchair Skill Test (4.2) and its report form into Spanish and then determine the inter-rater reliability of the WST 4.2 for manual wheelchairs operated by their users.Setting: Rehabilitation Unit, FLENI Institute, Buenos Aires, Argentina.Methods: The translation was performed by a physical therapist with advanced English language skills and specialized in the treatment of SCI subjects. We administrated and video-recorded the WST 4.2 manual Spanish version in 11 SCI subjects. Two physical therapists received specific training for administering the test and scoring the record. The reliability of the total percentage WST score were statistically quantified by intraclass correlation coefficient (ICC).Results: ICC values for Interrater were 0.998 (p < .0001). 17 out of the 32 skills had a 100 percentage of agreement. Percentage of agreement in the three skills that presented less rating agreement was 73%, 81 % and 82 %, respectively.Conclusion: The results show that the Spanish version of WST 4.2 is a reliable assessment tool to evaluate the skills capacity of spinal cord manual wheelchair users.Implications for rehabilitationWheelchair users require a proficient management of various wheelchair skills to achieve maximum independence in daily life. Determining which wheelchair skills should be addressed during the rehabilitation process is of great importance for their correct training. The WST 4.2 is an appropriate assessment tool to determine the functional capacity of wheelchair users.Making available the WST 4.2 in the Spanish language and demonstrating its reliability in this language allows its use in the Spanish-speaking world. A reliable wheelchair skills test is needed in the Spanish language.

Exercise-induced increase in brain-derived neurotrophic factor in human Parkinson's disease: a systematic review and meta-analysis

Background

Animal models of exercise and Parkinson’s disease (PD) have found that the physiologic use of exercise may interact with the neurodegenerative disease process, likely mediated by brain derived neurotrophic factor (BDNF). No reviews so far have assessed the methodologic quality of available intervention studies or have bundled the effect sizes of individual studies on exercise-induced effects on BDNF blood levels in human PD.

Research design and methods

We searched MEDLINE, EMBASE, Cochrane Library, PsycINFO and PubMed from inception to June 2017.

Results

Data aggregated from two randomized controlled trials and four pre-experimental studies with a total of 100 ambulatory patients with idiopathic PD (Hoehn/Yahr ≤3) found improvements in BDNF blood concentration levels in all 6 studies (two RCTs and 4 pre-experimental studies). Pooled BDNF level change scores from the 2 RCTs resulted in a significant homogeneous summary effect size (Standardized Mean Difference 2.06, 95% CI 1.36 to 2.76), and a significant heterogeneous SES for the motor part of the UPDRS-III examination (MD -5.53, 95% CI -10.42 to -0.64). Clinical improvements were noted in all studies using a variety of outcome measures.

Limitations

The evidence-base consists primarily of small studies with low to moderate methodological quality.

Conclusions

This review provides preliminary evidence for the effectiveness of physical exercise treatments for persons with PD on BDNF blood levels. Further research is needed.

Electronic supplementary material

The online version of this article (10.1186/s40035-018-0112-1) contains supplementary material, which is available to authorized users.

Do Children and Adolescents With Inflammatory Bowel Disease Complete Clinical Disease Indices Similar to Physicians?

OBJECTIVES

The degree to which children and adolescents with inflammatory bowel disease (IBD) complete clinical disease activity indices in accordance with their physician is indefinite. Therefore, we investigated the agreement between patient- and physician-based clinical indices in children and adolescents with a previous diagnosis of IBD.

METHODS

In this cross-sectional study, IBD patients (8-18 years) were included prospectively. Patients completed a patient-based short Pediatric Crohn's Disease Activity Index (shPCDAI) for Crohn disease or the Pediatric Ulcerative Colitis Activity Index (PUCAI) for ulcerative or indeterminate colitis. Physicians completed the original physician-based shPCDAI or PUCAI. Agreement was calculated with linear weighted kappa.

RESULTS

In total, 154 pairs of clinical indices were collected: 89 pairs of shPCDAI's (median age at assessment 15.6 years, 61% men) and 55 pairs of PUCAI's (median age at assessment 14.0 years, 44% men). The shPCDAI disease activity category only fairly agreed between patient- and physician-based indices (kappa: 0.40 [95% confidence interval 0.24-0.55], P < 0.001), with perfect agreement in 58% of pairs. In the majority of disagreement (81%), patients scored in a higher shPCDAI disease activity category. The PUCAI disease activity category substantially agreed between patient- and physician-based indices (kappa: 0.64 [95% confidence interval 0.45-0.83], P < 0.001), with perfect agreement in 78% of pairs. In the majority of disagreement (75%), patients scored in a higher PUCAI disease activity category.

CONCLUSIONS

Patient- and physician-based shPCDAI and PUCAI do not always agree, particularly the shPCDAI, and therefore, should not be interpreted equivalently in management and research on children and adolescents with IBD.

Immunologic detection of Giardia duodenalis in a specific pathogen–free captive olive baboon (Papio cynocephalus anubis) colony

Several commercial Giardia immunoassays were evaluated in baboons for sensitivity and specificity as well as ease of use in a large specific pathogen–free (SPF) colony. An additional objective was to identify the assemblage(s) of Giardia duodenalis present in this baboon colony. A direct immunofluorescent antibody test (IFAT) was used as the reference test. Tests evaluated were a patient-side rapid test for dogs and cats, a human rapid test, and a well-plate ELISA designed for use with humans. Test sensitivities and specificities were compared using the McNemar paired t-test and were further evaluated for agreement using an unweighted Cohen kappa statistic. When compared to the IFAT reference, both human tests were more sensitive than the veterinary test. Based on PCR and sequencing of the G. duodenalis small-subunit ribosomal RNA and glutamate dehydrogenase loci, assemblage AI was present in this baboon colony. We found that 10 of the 110 (9%) baboons in this SPF colony were infected with a zoonotic strain of G. duodenalis.

Validation of a modified early warning score-linked Situation-Background-Assessment-Recommendation communication tool: A mixed methods study

AIMS AND OBJECTIVES

To develop and validate a modified Situation-Background-Assessment-Recommendation communication tool incorporating components of the Cape Town modified early warning score vital signs chart for reporting early signs of clinical deterioration.

BACKGROUND

Reporting early signs of physiological and clinical deterioration could prevent "failure to rescue" or unexpected intensive care admission, cardiac arrest or death. A structured communication tool incorporating physiological and clinical parameters allows nurses to provide pertinent information about a deteriorating patient in a logical order.

DESIGN

Mixed methods instrument development and validation.

METHODS

We used a sequential three-phase method: cognitive interviews, content validation and inter-rater reliability testing to validate a self-designed communication tool. Participants were purposively selected expert nurses and doctors in government sector hospitals in Cape Town.

RESULTS

Cognitive interviews with five experts prompted most changes to the communication tool: 15/42 (35.71%) items were modified. Content validation of a revised tool was high by a predetermined ≥70% of 18 experts: 4/49 (8.2%) items were modified. Inter-rater reliability testing by two nurses indicated substantial to full agreement (Cohen's kappa .61-1) on 37/45 (82%) items. The one item achieving slight agreement (Cohen's kappa .20) indicated a difference in clinical judgement. The high overall percentage agreement (82%) suggests that the modified items are sound. Overall, 45 items remained on the validated tool.

CONCLUSION

The first modified early warning score-linked Situation-Background-Assessment-Recommendation communication tool developed in South Africa was found to be valid and reliable in a local context.

RELEVANCE TO CLINICAL PRACTICE

Nurses in South Africa can use the validated tool to provide doctors with pertinent information about a deteriorating patient in a logical order to prevent a serious adverse event. Our findings provide a reference for other African countries to develop and validate communication tools for reporting early signs of clinical deterioration.

The Gross Motor Function Classification System Family Report Questionnaire: reliability between special-education teachers and caregivers

AIM

The aim of this study was to investigate the reliability of the Thai Gross Motor Function Classification System Family Report Questionnaire (GMFCS-FR) and the possibility of special-education teachers and caregivers in the community using this system in children with cerebral palsy (CP).

METHOD

The reliability was examined by two teachers and two caregivers who classified 21 children with CP aged 2 to 12 years. A GMFCS-FR workshop was organized for raters. The teachers and caregivers classified the mobility of 362 children. The rater reliability was analysed using the weighted kappa coefficient. The possibility of using the GMFCS-FR is reported. The reliability of using the GMFCS-FR in the community was analysed by the intraclass correlation coefficient.

RESULTS

The intrarater reliability ranged from 0.91 to 1.00. The interrater reliability between teachers was 0.85 (95% confidence interval [CI] 0.69-0.97) and between caregivers was 0.84 (95% CI 0.70-0.97). Ninety-seven percent of raters used the Thai GMFCS-FR correctly. The overall intraclass correlation coefficient between raters was 0.90 (95% CI 0.88-0.92).

INTERPRETATION

The Thai GMFCS-FR is a reliable system for classifying the motor function of young children with CP by teachers and caregivers in the community.

Interrater Agreement in the Radiologic Characterization of Ruptured Intracranial Aneurysms Based on Computed Tomography Angiography

OBJECTIVE

To determine interrater agreement in the initial radiologic characterization of ruptured intracranial aneurysms based on computed tomography angiography (CTA) with special emphasis on the rater's level of experience.

METHODS

One junior and one senior rater of 5 high-volume neurovascular tertiary centers evaluated anonymized CTA images of 30 consecutive patients with aneurysmal subarachnoid hemorrhage. Each rater described location, side, size, and morphology in a standardized manner. Interrater variability was analyzed using intraclass correlation and Fleiss' kappa analysis.

RESULTS

There was a high level of agreement for location (κ = 0.76, 95% confidence interval [CI] 0.74-0.79), side (κ = 0.95, CI 0.91-0.99), maximum diameter (intraclass correlation coefficient [ICC] 0.81, CI 0.70-0.90), and dome (ICC 0.78, CI 0.66-0.88) of intracranial aneurysms. In contrast, a lower level of agreement was observed for aneurysms' neck diameter (ICC 0.39, CI 0.28-0.58), the presence of multiple aneurysms (κ = 0.35, CI 0.30-0.40), and aneurysm morphology (blister κ = 0.11, CI -0.05 to 0.07; fusiform κ = 0.54, CI 0.48-0.60; multilobular, κ = 0.39 CI 0.33-0.45). The interrater agreement in the senior rater group was greater than in the junior rater group.

CONCLUSIONS

Interrater agreement confirms the benefit of CTA as initial diagnostic imaging in ruptured intracranial aneurysms but not for aneurysm morphology and presence of multiple aneurysms. A trend towards greater interrater agreement between more experienced raters was noticed.

Reliability of physical examination tests for the diagnosis of knee disorders: Evidence from a systematic review

Clinicians often rely on physical examination tests to guide them in the diagnostic process of knee disorders. However, reliability of these tests is often overlooked and may influence the consistency of results and overall diagnostic validity. Therefore, the objective of this study was to systematically review evidence on the reliability of physical examination tests for the diagnosis of knee disorders. A structured literature search was conducted in databases up to January 2016. Included studies needed to report reliability measures of at least one physical test for any knee disorder. Methodological quality was evaluated using the QAREL checklist. A qualitative synthesis of the evidence was performed. Thirty-three studies were included with a mean QAREL score of 5.5 ± 0.5. Based on low to moderate quality evidence, the Thessaly test for meniscal injuries reached moderate inter-rater reliability (k = 0.54). Based on moderate to excellent quality evidence, the Lachman for anterior cruciate ligament injuries reached moderate to excellent inter-rater reliability (k = 0.42 to 0.81). Based on low to moderate quality evidence, the Tibiofemoral Crepitus, Joint Line and Patellofemoral Pain/Tenderness, Bony Enlargement and Joint Pain on Movement tests for knee osteoarthritis reached fair to excellent inter-rater reliability (k = 0.29 to 0.93). Based on low to moderate quality evidence, the Lateral Glide, Lateral Tilt, Lateral Pull and Quality of Movement tests for patellofemoral pain reached moderate to good inter-rater reliability (k = 0.49 to 0.73). Many physical tests appear to reach good inter-rater reliability, but this is based on low-quality and conflicting evidence. High-quality research is required to evaluate the reliability of knee physical examination tests.

Minimal important improvement thresholds for the six-minute walk test in a knee arthroplasty cohort: triangulation of anchor- and distribution-based methods

Background

The 6-minute walk test (6MWT) is a commonly used metric for measuring change in mobility after knee arthroplasty, however, what is considered an improvement after surgery has not been defined. The determination of important change in an outcome assessment tool is controversial and may require more than one approach. This study, nested within a combined randomised and observational trial, aimed to define a minimal important improvement threshold for the 6MWT in a knee arthroplasty cohort through a triangulation of methods including patient-perceived anchor-based thresholds and distribution-based thresholds.

Methods

Individuals with osteoarthritis performed a 6MWT pre-arthroplasty then at 10 and 26 weeks post-surgery. Each rated their perceived improvement in mobility post-surgery on a 7-point transition scale anchored from “much better” to “much worse”. Based on these responses the cohort was dichotomised into ‘improved’ and ‘not improved’. The thresholds for patient-perceived improvements were then identified using two receiver operating curve methods producing sensitivity and specificity indices. Distribution-based change thresholds were determined using two methods utilising effect size (ES). Agreement between the anchor- and distribution-based methods was assessed using kappa.

Results

One hundred fifty-eight from 166 participants in the randomised cohort and 222 from 243 in the combined randomised and observational cohort were included at 10 and 26 weeks, respectively. The slightly or more patient-perceived improvement threshold at 26 weeks (an absolute improvement of 26 m) was the only one to demonstrate sensitivity and specificity results both better than chance. At 10- and 26-weeks, the ES based on the mean change score divided by the baseline standard deviation (SD), was an absolute change of 24.5 and 37.9 m, respectively. The threshold based on a moderate ES (a 0.5 SD of the baseline score) was a change of 55.0 and 55.4 m at 10- and 26-weeks, respectively. The level of agreement between the 26-week anchor-based and distribution-based minimal absolute changes was very good (k = 0.88 (95 % CI 0.81 0.95)).

Conclusion

A valid threshold of improvement for the 6MWT can only be proposed for changes identified from baseline to 26 weeks post-surgery. The level of agreement between anchor- and distribution-based methods indicates that a true minimal or more threshold of meaningful improvement following surgery is likely within the ranges proposed by the triangulation of all four methods, that is, 26 to 55 m.

Intratester and intertester reliability of the movement system impairment-based classification for patients with knee pain

BACKGROUND

The methods to standardize the test items used for classification of patients with knee pain based on the movement system impairment (MSI) approach have been established. To our knowledge, no study has concentrated on establishing reliability for proposed classification for knee pain problems.

OBJECTIVE

The aim of the study was to assess intra- and intertester reliability of the knee MSI classification in patients with knee pain.

DESIGN

A cross-sectional methodological study.

SETTING

Rasul Akram Hospital.

PARTICIPANTS

Ninety-six subjects with knee pain aged 18-65 years.

METHODS

In order to examine intertester reliability, all three testers assessed the symptoms, signs and the MSI diagnosis of subjects with knee pain simultaneously. In order to assess intratester reliability, the procedure was exactly repeated after a one-week intersession period. Kappa values and percentages of agreement were calculated to analyze the reliability level.

RESULTS

The kappa values for intra- and intertester reliability of the symptom items ranged from 0.83 to 1.00 and 0.00 to 0.83, respectively. For the sign items, the kappa values ranged from 0.18 to 1.00 and 0.00 to 0.82, respectively. Finally, the kappa values of intra- and intertester reliability for patients' classification judgments ranged from 0.66 to 0.71, and 0.48 to 0.58, respectively.

CONCLUSION

The results of the present study indicate that intertester reliability for the symptoms, signs and classification judgments of patients with knee pain based on the MSI approach seemed generally acceptable. However, for intratester reliability, lower levels of the system were observed, probably due to different pain levels or pain behavior between test and retest sessions.

Is Satisfaction with the Acute-Care Experience Higher amongst Consumers Treated in the Private Sector? A Survey of Public and Private Sector Arthroplasty Recipients

Background

Consumer satisfaction with the acute-care experience could reasonably be expected to be higher amongst those treated in the private sector compared to those treated in the public sector given the former relies on high-level satisfaction of its consumers and their subsequent recommendations to thrive. The primary aims of this study were to determine, in a knee or hip arthroplasty cohort, if surgery in the private sector predicts greater overall satisfaction with the acute-care experience and greater likelihood to recommend the same hospital. A secondary aim was to determine whether satisfaction across a range of service domains is also higher in the private sector.

Methods

A telephone survey was conducted 35 days post-surgery. The hospital cohort comprised eight public and seven private high-volume arthroplasty providers. Consumers rated overall satisfaction with care out of 100 and likeliness to recommend their hospital on a 5-point Likert scale. Additional Likert-style questions were asked covering specific service domains. Generalized estimating equation models were used to analyse overall satisfaction (dichotomised as ≥ 90 or < 90) and future recommendations for care (dichotomised as ‘definitely recommend’ or ‘other’), whilst controlling for covariates. The proportions of consumers in each sector reporting the best Likert response for each individual domain were compared using non-parametric tests.

Results

457 survey respondents (n = 210 private) were included. Less patient-reported joint impairment pre-surgery [OR 1.03 (95% CI 1.01–1.05)] and absence of an acute complication (OR 2.13 95% CI 1.41–3.23) significantly predicted higher overall satisfaction. Hip arthroplasty [OR 1.84 (1.1–2.96)] and an absence of an acute complication [OR 2.31 (1.28–4.17] significantly predicted greater likelihood for recommending the hospital. The only care domains where the private out-performed the public sector were hospitality (46.7 vs 35.6%, p <0.01) and frequency of surgeon visitation (76.4 vs 65.8%, p = 0.03).

Conclusions

Arthroplasty consumers treated in the private sector are not more satisfied with their acute-care experience nor are they more likely to recommend their hospital provider. Rather, avoidance of complications in either sector appears to result in improved satisfaction as well as a greater likelihood that patients would recommend their hospital provider.

The Community Integration Questionnaire - Revised: Australian normative data and measurement of electronic social networking

BACKGROUND/AIM

Consideration of the relationship between meaningful participation, health and wellbeing underpins occupational therapy intervention, and drives measurement of community integration following acquired brain injury (ABI). However, utility of community integration measures has been limited to date by lack of normative data against which to compare outcomes, and none examine the growing use of electronic social networking (ESN) for social participation. This research had four aims: (i) develop and pilot items assessing ESN to add to the Community Integration Questionnaire, producing the Community Integration Questionnaire-Revised (CIQ-R); (ii) examine factor structure of the CIQ-R; (iii) collect Australian CIQ-R normative data; and (iv) assess test-retest reliability of the revised measure.

METHODS

SETTING

Australia.

PARTICIPANTS

A convenience sample of adults without ABI (N = 124) was used to develop and pilot ESN items. A representative general population sample of adults without ABI aged 18-64 years (N = 1973) was recruited to gather normative CIQ-R data.

DESIGN

Cross-sectional survey.

MAIN MEASURES

Demographic items and the CIQ-R.

RESULTS

The CIQ-R demonstrated acceptable psychometric properties, with minor modification to the original scoring based on the factor analyses provided. Large representative general population CIQ-R normative data have been established, detailing contribution of a range of independent demographic variables to community integration.

CONCLUSION

The addition of electronic social networking items to the CIQ-R offers a contemporary method of assessing community integration following ABI. Normative CIQ-R data enhance the understanding of community integration in the general population, allowing occupational therapists and other clinicians to make more meaningful comparisons between groups.

Risk factors for foot ulceration in adults with end-stage renal disease on dialysis: study protocol for a prospective observational cohort study

BACKGROUND

Adults with end-stage renal disease treated with dialysis experience a high burden of foot ulceration and lower extremity amputation. However, the risk factors for foot ulceration in the dialysis population are incompletely understood due to the lack of high-quality prospective evidence. This article outlines the design of a prospective observational cohort study, which aims to investigate the risk factors for foot ulceration in adults on dialysis.

METHODS/DESIGN

This study will recruit 430 participants with end-stage renal disease on dialysis from satellite and home-therapy dialysis units across multiple health organisations in Melbourne, Victoria, Australia. Data collection at baseline will include a participant interview, medical record review, completion of a health-status questionnaire and a non-invasive foot assessment. Twenty participants will also be recruited to a reliability study to evaluate the reproducibility of testing procedures. Primary outcome data includes: new foot ulcer(s). Secondary outcome data includes: number of new foot ulcers, time to onset of new foot ulcer(s), new lower extremity amputation(s), episodes of infection of the foot or lower extremity, episodes of osteomyelitis, foot-related hospitalisations, revascularisation procedure(s) of the lower extremity, new podiatry interventions, kidney transplantation, and mortality. Participants will be assessed at baseline, and at 12 months they will be evaluated for the primary and secondary outcomes. Multivariate Cox proportional hazards models will be used to assess predictors of new foot ulceration and time to event secondary outcomes. Logistic regression will be used for binary outcomes including prevalence of foot ulcerations.

DISCUSSION

This is the first multi-centre prospective observational cohort study to investigate risk factors for foot ulceration in adults with end-stage renal disease on dialysis. This study will improve on prior studies by using prospective methods, multi-centre recruitment, statistical methods to control for confounding variables, and a pre-specified sample size estimation. The findings can inform the design of future trials evaluating the effectiveness of clinical interventions, which may lead to improved patient outcomes in the dialysis setting.

The caries assessment spectrum and treatment (CAST) instrument: its reproducibility in clinical studies

A new caries assessment instrument, the Caries Assessment Spectrum and Treatment (CAST), was developed. It covers carious lesion progression from no lesion, sealants and restorations to lesions in enamel and dentine, advanced stages in pulpal and tooth-surrounding tissues, and tooth loss owing to dental caries, in nine codes. The objective of this study was to determine the reproducibility of the CAST instrument in primary and permanent dentitions, using three age groups. Two epidemiological surveys were conducted in Brazil, covering three age groups: 2-6-year-old and 6-9-year-old children and 19-30-year-old adults. Four trained and calibrated examiners performed the examinations. Reproducibility was calculated for intra- and inter-examiner at surface and tooth levels and expressed as unweighted kappa-coefficient value (κ) and percentage of agreement (Po) for CAST codes (0-7) and for the categories healthy (0-2) versus diseased (3-7), and non-cavitated (0-3) versus cavitated (4-7) teeth. Using CAST codes (0-7) for the 2-6-year-old age group in primary dentitions, inter-examiner consistency was κ = 0.74 and Po was 98.3%. In the 6-9-year-old age group in primary dentitions, inter-examiner consistency ranged from κ = 0.68 to κ = 0.86 and Po was ≥ 93.7%. In the 19-30-year-old age group inter-examiner consistency was κ = 0.87 and Po was 94.1%. The reproducibility of the CAST instrument for use in the primary dentition of 2-6-year olds and of 6-9-year olds was 'substantial' to 'almost perfect'. The reproducibility for its use in the permanent dentition of 19-30-year olds was 'almost perfect'. The CAST instrument can reliably be applied in epidemiological studies covering these ages.

The intra- and inter-rater reliability of component analysis of rise from supine in the children with typical development and developmental delay

The purpose of this study was to determine the intra- and inter-rater reliability of classifying the movement patterns of rising from supine to stand in the children with typical development (TD) and mild to moderate developmental delay (DD). Sixty-eight children with TD and 20 children with DD aged 2 through 6 years were videotaped during rising. Two trained pediatric physical therapists independently viewed each videotape and classified the movement patterns of upper extremities (UE), trunk/axial (AX) and lower extremities (LE) regions using descriptive categories developed by previous researchers. Kappa statistic and average percentage of agreement were calculated to determine reliability. The average agreement rate of intra-rater ranged from 90% to 97% in TD group for three regions, and 79% to 89% in DD group; the agreement rate between raters ranged from 82% to 95% in TD group for three regions, and 71% to 87% in DD group. Using kappa statistic guidelines, high intra- and inter-rater reliabilities (k>0.81) were found in TD group, except inter-rater reliability for LE. Substantial intra- and inter-reliabilities (0.61<k<0.81) were found in DD group. As comparing the reliabilities of three regions, the lowest intra- and inter-rater reliabilities were found at the LE region in both groups. As taking age into consideration and dividing each group into subgroups by 2-year interval, in DD group, the intra-rater reliabilities were higher in younger participants (aged 2-4 years), except for the LE region, but the results of inter-rater reliabilities were variable. For the children with TD, the levels of inter-rater reliability increased for the UE region and decreased for LE region in older age, but the results for these two regions were reverse for the children with DD. The findings of this study suggested that the complexities and difficulties affecting reliabilities in classifying the movement patterns of rising were related to developmental capability, age and body region. In order to improve intra- and inter-rater reliabilities, extra training is needed in examining the children with DD, particularly for the UE and LE regions.

General motor function assessment scale--reliability of a Norwegian version

PURPOSE

The General Motor Function assessment scale (GMF) measures activity-related dependence, pain and insecurity among older people in frail health. The aim of the present study was to translate the GMF into a Norwegian version (N-GMF) and establish its reliability and clinical feasibility.

METHODS

The procedure used in translating the GMF was a forward and backward process, testing a convenience sample of 30 frail elderly people with it. The intra-rater reliability tests were performed by three physiotherapists, and the inter-reliability test was done by the same three plus nine independent colleagues. The statistical analyses were performed with a pairwise analysis for intra- and inter-rater reliability, using Cronbach's α, Percentage Agreement (PA), Svensson's rank transformable method and Cohen's κ.

RESULTS

The Cronbach's α coefficients for the different subscales of N-GMF were 0.68 for Dependency, 0.73 for Pain and 0.75 for Insecurity. Intra-rater reliability: The variation in the PA for the total score was 40-70% in Dependence, 30-40% in Pain and 30-60% in Insecurity. The Relative Rank Variant (RV) indicated a modest individual bias and an augmented rank-order agreement coefficient ra of 0.96, 0.96 and 0.99, respectively. The variation in the κ statistics was 0.27-0.62 for Dependence, 0.17-0.35 for Pain and 0.13-0.47 for Insecurity. Inter-rater reliability: The PA between different testers in Dependence, Pain and Insecurity was 74%, 89% and 74%, respectively. The augmented rank-order agreement coefficients were: for Dependence r(a) = 0.97; for Pain, r(a) = 0.99; and for Insecurity, r(a) = 0.99.

CONCLUSION

The N-GMF is a fairly reliable instrument for use with frail elderly people, with intra-rater and inter-rater reliability moderate in Dependence and slight to fair in Pain and Insecurity. The clinical usefulness was stressed in regard to its main focus, the frail elderly, and for communication within a multidisciplinary team. Implications for Rehabilitation The Norwegian-General Motor Function Assessment Scale (N-GMF) is a reliable instrument. The N-GMF is an instrument for screening and assessment of activity-related dependence, pain and insecurity in frail older people. The N-GMF may be used as a tool of communication in a multidisciplinary team.

Pitfalls in the use of kappa when interpreting agreement between multiple raters in reliability studies

OBJECTIVE

To compare different reliability coefficients (exact agreement, and variations of the kappa (generalised, Cohen's and Prevalence Adjusted and Biased Adjusted (PABAK))) for four physiotherapists conducting visual assessments of scapulae.

DESIGN

Inter-therapist reliability study.

SETTING

Research laboratory.

PARTICIPANTS

30 individuals with no history of neck or shoulder pain were recruited with no obvious significant postural abnormalities.

MAIN OUTCOME MEASURES

Ratings of scapular posture were recorded in multiple biomechanical planes under four test conditions (at rest, and while under three isometric conditions) by four physiotherapists.

RESULTS

The magnitude of discrepancy between the two therapist pairs was 0.04 to 0.76 for Cohen's kappa, and 0.00 to 0.86 for PABAK. In comparison, the generalised kappa provided a score between the two paired kappa coefficients. The difference between mean generalised kappa coefficients and mean Cohen's kappa (0.02) and between mean generalised kappa and PABAK (0.02) were negligible, but the magnitude of difference between the generalised kappa and paired kappa within each plane and condition was substantial; 0.02 to 0.57 for Cohen's kappa and 0.02 to 0.63 for PABAK, respectively.

CONCLUSIONS

Calculating coefficients for therapist pairs alone may result in inconsistent findings. In contrast, the generalised kappa provided a coefficient close to the mean of the paired kappa coefficients. These findings support an assertion that generalised kappa may lead to a better representation of reliability between three or more raters and that reliability studies only calculating agreement between two raters should be interpreted with caution. However, generalised kappa may mask more extreme cases of agreement (or disagreement) that paired comparisons may reveal.

The performance of standardized patients in portraying clinical scenarios in speech-language therapy

BACKGROUND

Standardized patients (SPs) are frequently included in the clinical preparation of students in the health sciences. An acknowledged benefit of using SPs is the opportunity to provide a standardized method by which students can demonstrate and develop their competency. Relatively little is known, however, about the capacity of SPs to offer an accurate and standardized performance across a speech-language therapy student cohort.

AIMS

To investigate the accuracy, reproducibility (consistent performance of each SP across student interviews) and replicability (consistent performance of a number of SPs across each scenario) of SPs portraying three scenarios, each as a parent of a child presenting with a speech disorder.

METHODS & PROCEDURES

Forty-four speech-language therapy students interviewed four SPs to gain a case history. All interviews were videotaped. The accuracy of SP portrayal of key features of each scenario was scored by an expert rater and two other raters. Data were analysed to determine levels of accuracy, reproducibility and replicability, and inter-rater reliability was evaluated.

OUTCOMES & RESULTS

SPs were found to have moderate to high levels of accuracy across the three scenarios. There were no significant differences in the performances of each individual SP across interviews or between all SPs on each scenario, indicating that reproducibility and replicability were achieved. Overall inter-rater reliability between raters across all scenarios was greater than 80%.

CONCLUSIONS & IMPLICATIONS

The results would seem to indicate that SPs can present in a standardized manner within a speech-language therapy context, confirming the value of their inclusion in clinical education programmes. Suggestions for improving the training of SPs in order to maintain accuracy are highlighted.

Satisfactory reliability among nursing students using the instrument PVC ASSESS to evaluate management of peripheral venous catheters

PURPOSE

Nursing students should be given opportunities to participate in clinical audits during their education. However, audit tools are seldom tested for reliability among nursing students. The aim of this study was to present reliability among nursing students using the instrument PVC assess to assess management of peripheral venous catheters (PVCs) and PVC-related signs of thrombophlebitis.

METHODS

PVC assess was used to assess 67 inserted PVCs in 60 patients at ten wards at a university hospital. One group of nursing students (n=4) assessed PVCs at the bedside (inter-rater reliability) and photographs of these PVCs were taken. Another group of students (n=3) assessed the PVCs in the photographs after 4 weeks (test-retest reliability). To determine reliability, proportion of agreement [P(A)] and Cohen's kappa coefficient (κ) were calculated.

RESULTS

For bedside assessment of PVCs, P(A) ranged from good to excellent (0.80-1.0) in 55% of the 26 PVC assess items that were tested. P(A) was poor (<0.70) for two items: "adherence of inner dressing to the skin" and "PVC location." In 81% of the items, κ was between moderate and almost perfect: moderate (n=5), substantial (n=3), almost perfect (n=5). For edema at insertion site and two items on PVC dressing, κ was fair (0.21-0.40). Regarding test-retest reliability, P(A) varied between good and excellent (0.81-1) in 85%-95% of the items, and the κ ranged between moderate and almost perfect (0.41-1) in 90%-95%.

CONCLUSIONS

PVC assess demonstrated satisfactory reliability among nursing students. However, students need training in how to use the instrument before assessing PVCs.

Appropriateness and reliability testing of the modified Richmond Agitation-Sedation Scale in Spanish patients with advanced cancer

CONTEXT

A tool to quantify agitation severity and sedation level in patients with advanced cancer is needed.

OBJECTIVES

To test the appropriateness and reliability of the Richmond Agitation-Sedation Scale (RASS) in Spanish patients with advanced cancer.

METHODS

The original RASS was translated into Spanish according to the standard guidelines. Face validity was assessed by members of the palliative care team, and interrater reliability was assessed, using a weighted kappa, from observations of patients admitted to the palliative care unit. The association between scores of the RASS, Ramsay Sedation Scale, and Glasgow Coma Scale was evaluated using Spearman's ρ.

RESULTS

Three hundred twenty-two observations were performed in 156 patients: 116 observations were performed for delirious patients, 76 observations for sedated patients, and 130 observations for patients admitted for other symptom control. The weighted kappa values were practically equal to or greater than 0.90 between nurses and nurses and physicians. The agreement level between observers for each RASS score was roughly 90%. The correlation between the RASS and the Ramsay and Glasgow Scale values was analyzed for 196 observations recorded in 80 patients. The sedation scale of the RASS had a strong correlation with both the Ramsay (Spearman's ρ, -0.89; P < 0.001) and the Glasgow Coma Scales (Spearman's ρ, 0.85; P < 0.001).

CONCLUSION

These data support the use of the RASS in Spanish patients with advanced cancer.