From qualitative data to a measurement instrument: A clarification and elaboration of choices made in the development of the Patient Scale of the Patient and Observer Scar Assessment Scale (POSAS) 3.0

PURPOSE

To clarify and elaborate on the choices that were made in the development of the Patient Scale of the Patient and Observer Scar Assessment Scale 3.0 (POSAS 3.0), based upon the rich information obtained from patients during focus groups and pilot tests.

METHODS

The discussions described in this paper are a reflection of the focus group study and pilot tests that were conducted in order to develop the Patient Scale of the POSAS3.0. The focus groups took place in the Netherlands and Australia and included 45 participants. Pilot tests were performed with 15 participants in Australia, the Netherlands, and the United Kingdom.

RESULTS

We discussed the selection, wording and merging of 17 included items. Additionally, the reason for exclusion of 23 characteristics are given.

CONCLUSION

Based upon the unique and rich material of patient input obtained, two versions of the Patient Scale of the POSAS3.0 were developed: the Generic version, and the Linear scar version. The discussions and decisions taken during the development are informative for a good understanding of the POSAS 3.0 and are indispensable as a background for future translations and cross-cultural adaptations.

Development of a simple standardized scoring system for assessing large vessel vasculitis by 18F-FDG PET-CT and differentiation from atherosclerosis

PURPOSE

This study is to develop a structured approach to distinguishing large-artery vasculitis from atherosclerosis using 18-fluorodeoxyglucose positron emission tomography combined with low-dose computed tomography (FDG PET/CT).

METHODS

FDG PET/CT images of 60 patients were evaluated, 30 having biopsy-proven giant cell arteritis (GCA; the most common form of large-artery vasculitis), and 30 with severe atherosclerosis. Images were evaluated by 12 nuclear medicine physicians using 5 criteria: FDG uptake pattern (intensity, distribution, circularity), the degree of calcification, and co-localization of calcifications with FDG-uptake. Criteria that passed agreement, and reliability tests were subsequently analysed for accuracy using receiver operator curve (ROC) analyses. Criteria that showed discriminative ability were then combined in a multi-component scoring system. Both initial and final 'gestalt' conclusion were also reported by observers before and after detailed examination of the images.

RESULTS

Agreement and reliability analyses disqualified 3 of the 5 criteria, leaving only FDG uptake intensity compared to liver uptake and arterial wall calcification for potential use in a scoring system. ROC analysis showed an area under the curve (AUC) of 0.90 (95%CI 0.87-0.92) for FDG uptake intensity. Degree of calcification showed poor discriminative ability on its own (AUC of 0.62; 95%CI 0.58-0.66). When combining presence of calcification with FDG uptake intensity into a 6-tiered scoring system, the AUC remained similar at 0.91 (95%CI 0.88-0.93). After exclusion of cases with arterial prostheses, the AUC increased to 0.93 (95%CI 0.91-0.95). The accuracy of the 'gestalt' conclusion was initially 89% (95%CI 86-91%) and increased to 93% (95%CI 91-95%) after detailed image examination.

CONCLUSION

Standardised assessment of arterial wall FDG uptake intensity, preferably combined with assessment of arterial calcifications into a scoring method, enables accurate, but not perfect, distinction between large artery vasculitis and atherosclerosis.

Baseline PET radiomics outperforms the IPI risk score for prediction of outcome in diffuse large B-cell lymphoma

The objective of this study is to externally validate the clinical positron emission tomography (PET) model developed in the HOVON-84 trial and to compare the model performance of our clinical PET model using the international prognostic index (IPI). In total, 1195 patients with diffuse large B-cell lymphoma (DLBCL) were included in the study. Data of 887 patients from 6 studies were used as external validation data sets. The primary outcomes were 2-year progression-free survival (PFS) and 2-year time to progression (TTP). The metabolic tumor volume (MTV), maximum distance between the largest lesion and another lesion (Dmaxbulk), and peak standardized uptake value (SUVpeak) were extracted. The predictive values of the IPI and clinical PET model (MTV, Dmaxbulk, SUVpeak, performance status, and age) were tested. Model performance was assessed using the area under the curve (AUC), and diagnostic performance, using the positive predictive value (PPV). The IPI yielded an AUC of 0.62. The clinical PET model yielded a significantly higher AUC of 0.71 (P < .001). Patients with high-risk IPI had a 2-year PFS of 61.4% vs 51.9% for those with high-risk clinical PET, with an increase in PPV from 35.5% to 49.1%, respectively. A total of 66.4% of patients with high-risk IPI were free from progression or relapse vs 55.5% of patients with high-risk clinical PET scores, with an increased PPV from 33.7% to 44.6%, respectively. The clinical PET model remained predictive of outcome in 6 independent first-line DLBCL studies, and had higher model performance than the currently used IPI in all studies.

Development of the Patient Scale of the Patient and Observer Scar Assessment Scale (POSAS) 3.0: a qualitative study

PURPOSE

The Patient and Observer Scar Assessment Scale (POSAS) is widely used for measurements of scar quality. This encompasses visual, tactile and sensory characteristics of the scar. The Patient Scale of previous POSAS versions was lacking input from patients. Therefore, the aim of this study was to develop the POSAS3.0, Patient Scale with involvement of adults patients with all scar types, complying with the highest clinimetric standards.

METHODS

From February 2018 to April 2019, a series of six focus group interviews were performed in the Netherlands and Australia to identify scar quality characteristics that adults with scars consider to be important. All focus groups were transcribed, anonymized and analysed using a thematic analysis. Relevant characteristics were formulated into items, resulting in a Dutch and English version of the Patient Scale. These drafts were pilot tested in Australia, the Netherlands and the United Kingdom, and refined accordingly.

RESULTS

A total of 21 relevant scar quality characteristics were identified during the focus groups. Two distinct versions of the POSAS3.0, Patient Scale were developed. The Generic version contains 16 items and can be used for all scar types, except linear scars. The Linear Scar version of the Patient Scale contains the same 16 items, with an extra item referring to the widening of scar margins. All included items are rated on a verbal rating scale with five response options.

CONCLUSION

Two versions of the POSAS3.0 Patient Scale were developed. Further field tests are being performed to establish the measurement properties and scoring algorithm of the scales.

A clinimetric assessment of the validity and reliability of 3D technology for scar surface area measurement

INTRODUCTION

The quality of scars has become an important outcome of burn care. Objective scar assessment through scar surface area measurement enables quantification of scar formation and evaluation of treatment efficacy. 3D technology has proven valid and reliable but often remains cumbersome, expensive, and time-consuming. 3D technology with depth sensors on mobile devices has become available and might surpass these limitations. This study provides a clinimetric assessment of the validity and reliability of a 3D system with a depth sensor for scar surface area measurement.

METHODS

A technology involving a depth sensor mounted on a mobile device was used. Images and analyses were made with a custom-made software application. A standardized one-keyframe image capturing procedure was followed. To assess validity, stickers with predefined dimensions (8.01 cm² - 77.70 cm²) were imaged in a single observer setting on various body parts of healthy volunteers. To assess reliability, hypertrophic scars, keloids, and normotrophic scars were imaged and rated by two observers independently. Data are expressed as mean (+/-SD), Coefficient of Variation (CV), Intraclass Correlation Coefficients (ICC), and Limits of Agreements (LoA).

RESULTS

Eighty stickers placed on 20 healthy volunteers showed validity with CV between 0.62%- 1.67% for observer A and 0.75%- 1.19% for observer B. For the reliability study, 69 scars on 36 patients were included. Mean scar surface area ranged from 0.83 cm² to 155.59 cm². Mean scar surface area measurement was 13.83 cm² (SD 23.06) for observer A and 13.59 cm² (SD 23.31) for observer B. Adjusted interobserver CV for trained observers is estimated as 5.59%, with corresponding LoA = 0 ± 0.15 x mean surface area. Interobserver ICCs were 0.99-1.00.

CONCLUSION

This 3D technology with a depth sensor for measuring scar surface area provides valid and reliable data and thereby surpasses expensive and time-consuming 3D cameras.

Aberrant patterns of PET response during treatment for DLBCL patients with MYC gene rearrangements

Purpose

MYC gene rearrangements in diffuse large B-cell lymphoma (DLBCL) patients are associated with poor prognosis. Our aim was to compare patterns of 2[¹⁸F]fluoro-2-deoxy-D-glucose positron emission tomography computed tomography (PET/CT) response in MYC + and MYC- DLBCL patients.

Methods

Interim PET/CT (I-PET) and end of treatment PET/CT (EoT-PET) scans of 81 MYC + and 129 MYC- DLBCL patients from 2 HOVON trials were reviewed using the Deauville 5-point scale (DS). DS1-3 was regarded as negative and DS4-5 as positive. Standardized uptake values (SUV) and metabolic tumor volume (MTV) were quantified at baseline, I-PET, and EoT-PET. Negative (NPV) and positive predictive values (PPV) were calculated using 2-year overall survival.

Results

MYC + DLBCL patients had significantly more positive EoT-PET scans than MYC- patients (32.5 vs 15.7%, p = 0.004). I-PET positivity rates were comparable (28.8 vs 23.8%). In MYC + patients 23.2% of the I-PET negative patients converted to positive at EoT-PET, vs only 2% for the MYC- patients (p = 0.002). Nine (34.6%) MYC + DLBCL showed initially uninvolved localizations at EoT-PET, compared to one (5.3%) MYC- patient. A total of 80.8% of EoT-PET positive MYC + patients showed both increased lesional SUV and MTV compared to I-PET. In MYC- patients, 31.6% showed increased SUV and 42.1% showed increased MTV. NPV of I-PET and EoT-PET was high for both MYC subgroups (81.8–94.1%). PPV was highest at EoT-PET for MYC + patients (61.5%).

Conclusion

MYC + DLBCL patients demonstrate aberrant PET response patterns compared to MYC- patients with more frequent progression during treatment after I-PET negative assessment and new lesions at sites that were not initially involved.

Trial registration number and date of registration

HOVON-84: EudraCT: 2006–005,174-42, retrospectively registered 01–08-2008. HOVON-130: EudraCT: 2014–002,654-39, registered 26–01-2015

Supplementary Information

The online version contains supplementary material available at 10.1007/s00259-021-05498-7.

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study

BACKGROUND

Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values.

METHODS

We conducted a 3-round Delphi study involving 52 panelists.

RESULTS

Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements (n = 5), standards on preferred statistical methods for reliability (n = 3) and measurement error (n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee.

CONCLUSION

We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.

The Submental Nasal Appearance Scale for the Assessment of Repaired Unilateral Complete Cleft Lip: A Validation Study

Objective:

To reassess reliability and validity of the Submental Nasal Appearance Scale (SNAS) compared to the preliminary pilot study, for assessment of patient photographs with repaired unilateral cleft lip and palate (UCLP). When utilizing the SNAS, 3 nasal features (1. nasal outline; 2. alar base position; 3. nostril axis) must be graded according to symmetry between the cleft and noncleft side using a 5-point scale with reference photographs for each feature. The mean score calculated from the graded features reflects the overall degree of nasal symmetry, which is considered an important goal when repairing UCLP.

Design:

Fifty patient photographs were selected and cropped, displaying the submental view. Six raters assessed these photographs using the SNAS and a separate 5-point scale to assess the overall submental appearance. Interrater reliability was determined for both methods and correlation was calculated between these as an indication of construct validity.

Setting:

Amsterdam UMC, location VUmc, Amsterdam, The Netherlands.

Patients:

Six- to 9-year-old patients with repaired UCLP.

Results:

Interrater reliability of 0.73 and 0.48 was found for the SNAS and overall appearance assessment, respectively, while in the pilot study values of 0.79 and 0.69 were found. Correlation of 0.59 and 0.74 was found in the current and pilot study, respectively, between the SNAS and overall appearance assessment.

Conclusions:

The SNAS is a reliable tool to assess nasal symmetry from the submental perspective. Reliability of the SNAS is higher compared to grading overall appearance, but validity of the SNAS was less well supported.

Moral competence, moral teamwork and moral action - the European Moral Case Deliberation Outcomes (Euro-MCD) Instrument 2.0 and its revision process

Background

Clinical Ethics Support (CES) services are offered to support healthcare professionals in dealing with ethically difficult situations. Evaluation of CES is important to understand if it is indeed a supportive service in order to inform and improve future implementation of CES. Yet, methods to measure outcomes of CES are scarce. In 2014, the European Moral Case Deliberation Outcomes Instrument (Euro-MCD) was developed to measure outcomes of Moral Case Deliberation (MCD). To further validate the instrument, we tested it in field studies and revised it. This paper presents the Euro-MCD 2.0 and describes the revision process.

Methods

The revision process comprised an iterative dialogue among the authors as Euro-MCD-project team, including empirical findings from six Euro-MCD field-studies and input from European experts in CES and theory. Empirical findings contained perceptions and experiences of MCD outcomes among healthcare professionals who participated in MCDs in various settings in Norway, Sweden and the Netherlands. Theoretical viewpoints on CES, literature on goals of CES and MCD and ethics theory guided the interpretation of the empirical findings and final selection of MCD outcomes.

Results

The Euro-MCD 2.0 Instrument includes three domains: Moral Competence, Moral Teamwork and Moral Action. Moral Competence consists of items about moral sensitivity, analytical skills and virtuous attitude. Moral Teamwork includes open dialogue and supportive relationships and Moral Action refers to moral decision-making and responsible care. During the revision process, we made decisions about adding and reformulating items as well as decreasing the number from 26 to 15 items. We also altered the sentence structure of items to assess the current status of outcomes (e.g. ‘now’) instead of an assumed improvement over time (e.g. ‘better’) and we omitted the question about perceived importance.

Conclusions

The Euro-MCD 2.0 is shorter, less complex and more strongly substantiated by an integration of empirical findings, theoretical reflections and dialogues with participants and experts. Use of the Euro-MCD 2.0 will facilitate evaluation of MCD and can thereby monitor and foster implementation and quality of MCD. The Euro-MCD 2.0 will strengthen future research on evaluation of outcomes of MCD.

Can the Combined Use of Two Screening Instruments Improve the Predictive Power of Dependency in (Instrumental) Activities of Daily Living, Mortality and Hospitalization in Old Age?

BACKGROUND

Due to differences in the definition of frailty, many different screening instruments have been developed. However, the predictive validity of these instruments among community-dwelling older people remains uncertain.

OBJECTIVE

To investigate whether combined (i.e. sequential or parallel) use of available frailty instruments improves the predictive power of dependency in (instrumental) activities of daily living ((I)ADL), mortality and hospitalization.

DESIGN, SETTING AND PARTICIPANTS

A prospective cohort study with two-year follow-up was conducted among pre-frail and frail community-dwelling older people in the Netherlands.

MEASUREMENTS

Four combinations of two highly specific frailty instruments (Frailty Phenotype, Frailty Index) and two highly sensitive instruments (Tilburg Frailty Indicator, Groningen Frailty Indicator) were investigated. We calculated sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for all single instruments as well as for the four combinations, sequential and parallel.

RESULTS

2,420 individuals participated (mean age 76.3 ± 6.6 years, 60.5% female) in our study. Sequential use increased the levels of specificity, as expected, whereas the PPV hardly increased. Parallel use increased the levels of sensitivity, although the NPV hardly increased.

CONCLUSIONS

Applying two frailty instruments sequential or parallel might not be a solution for achieving better predictions of frailty in community-dwelling older people. Our results show that the combination of different screening instruments does not improve predictive validity. However, as this is one of the first studies to investigate the combined use of screening instruments, we recommend further exploration of other combinations of instruments among other study populations.

Validation of two PROMIS item banks for measuring social participation in the Dutch general population

BACKGROUND

The Patient-Reported Outcomes Measurement Information System (PROMIS) item banks 'Ability to Participate in Social Roles and Activities' (35 items) and 'Satisfaction with Social Roles and Activities' (44 items) were developed to measure (satisfaction with) participation more efficiently and precisely than current instruments, by using Computerized Adaptive Testing (CAT). We validated these item banks in a Dutch general population.

METHODS

Participants in an internet panel completed both item banks. Unidimensionality, local dependence, monotonicity, Graded Response Model item fit, Differential Item Functioning (DIF) for age, gender, education, region, ethnicity, and language (Dutch compared to US Social Supplement), and reliability were assessed.

RESULTS

A representative Dutch sample of 1002 people participated. We found for the Ability to Participate and Satisfaction with Participation item banks, respectively, sufficient unidimensionality (CFI: 0.971, 0.960; TLI: 0.970, 0.958; RMSEA: 0.108, 0.108), no local dependence, sufficient monotonicity (H: 0.75, 0.73), good item fit (2 out of 35 items, 1 out of 44 items with S-X2p-value < 0.001). No DIF was found. We found a reliability of at least 0.90 with simulated CATs in 86% and 94% of the participants with on average 4.7 (range 2-12) and 4.3 (range 3-12) items, respectively.

DISCUSSION

The PROMIS participation item banks showed sufficient psychometric properties in a general Dutch population and can be used as CAT. PROMIS CATs allow reliable and valid measurement of participation in an efficient and user-friendly way with limited administration time.

Reliability of the Animated Activity Questionnaire for assessing activity limitations of patients with hip and knee osteoarthritis

OBJECTIVE

The aim of the study was to determine the internal consistency, test-retest reliability and measurement error of the Animated Activity Questionnaire (AAQ) for assessing activity limitations in hip and knee osteoarthritis (HKOA) patients.

METHODS

A total of 1,177 patients, from six countries (the Netherlands, UK, France, Denmark, Italy and Spain), completed the AAQ, a questionnaire consisting of videos displaying 17 activities with 3-5 levels of performance, from which patients choose the video that best matches their own performance. Unidimensionality was assessed by means of confirmatory factor analysis (CFA), using the following fit indices: the Tucker-Lewis index (TFI) >0.95, comparative fit index (CFI) >0.95, and root mean square error of approximation (RMSEA) <0.06. Cronbach's alpha was computed. In 238 patients who completed the AAQ twice, the intra-class correlation coefficient (ICC) was calculated for test-retest reliability. The standard error of measurement (SEM) and the smallest detectable change (SDC) were calculated as parameters of measurement error.

RESULTS

The fit indices for unidimensionality were CFI 0.957, TLI 0.950, and RMSEA 0.144. Cronbach's alpha was 0.95. ICC for test-retest reliability was 0.93 (95% confidence interval 0.91 to 0.95), ranging from 0.85 to 0.98 across countries. SEM and SDC were 4.9 and 13.6, respectively, on a scale from 0 to 100, and ranging from 2.7 to 6.7, and from 7.5 to 18.4, respectively, across countries. The AAQ appeared to measure slightly more precisely in patients with knee problems and patients without prosthesis.

CONCLUSION

The AAQ seemed to be unidimensional, and showed good internal consistency and test-retest reliability. The SDC indicated that changes in scores of at least 14% indicate real improvement in activity limitations.

Uvulopalatopharyngoplasty with or without tonsillectomy in the treatment of adult obstructive sleep apnea - A systematic review

BACKGROUND

The most commonly performed surgical procedure for obstructive sleep apnea (OSA) is uvulopalatopharyngoplasty with or without tonsillectomy (UPPP ± TE). However, there is currently no review solely focusing on clinically relevant effects of standard UPPP technique with or without tonsillectomy as a monotherapy in patients with OSA.

METHODS

A systematic review and meta-analysis were performed to assess the effects of isolated UPPP ± TE in patients with OSA. Studies of any design referring to adult patients with obstructive sleep apnea diagnosed via polysomnography or comparable objective measures were considered, in which isolated "standard" UPPP ± TE was performed.

RESULTS

Forty-eight studies were included for the qualitative analysis. All but one study demonstrated a reduction in the frequency of respiratory events and success/response rates ranged from 35 to 95.2%. In the six studies that reported pre- and postoperative mean scores of the Epworth Sleepiness Scale (ESS), a reduction in sleepiness scores was demonstrated. Data addressing the effect of UPPP ± TE in comparison to no treatment or control were available from two randomized controlled trials (RCT). When pooling the data, UPPP ± TE was significantly more effective in reducing the apnea-hypopnea index (AHI) and Epworth Sleepiness Scale (ESS) (large effect): an AHI mean difference (MD) of -18.59 (95% CI -34.14, -3.04) and an ESS MD of -5.37 (95% CI -7,03, -3.72). Data addressing effect of UPPP ± TE in comparison to baseline was available from three RCT. When pooling the data, the AHI was reduced from a mean 35.4 to 17.9 (49.5% reduction); a MD of -20.41, 95% CI -32.78, -8.04 (-1.80, -1.15) (large effect). Various additional beneficial effects of UPPP ± TE were demonstrated including improvement in sexual function, ventricular function, sleep stages, serum lipid, depressive disorder and driving performance.

CONCLUSION

UPPP ± TE reduces respiratory events and daytime sleepiness in adult patients with OSA and UPPP ± TE is superior to non-treated controls in this regard. Further research is needed to establish the long-term benefit, the impact on cardiovascular morbidity and the role of UPPP ± TE in the variety of available treatment options for OSA.

COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures

PURPOSE

The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties.

METHODS

For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis.

RESULTS

Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system.

CONCLUSIONS

The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.

COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study

BACKGROUND

Content validity is the most important measurement property of a patient-reported outcome measure (PROM) and the most challenging to assess. Our aims were to: (1) develop standards for evaluating the quality of PROM development; (2) update the original COSMIN standards for assessing the quality of content validity studies of PROMs; (3) develop criteria for what constitutes good content validity of PROMs, and (4) develop a rating system for summarizing the evidence on a PROM's content validity and grading the quality of the evidence in systematic reviews of PROMs.

METHODS

An online 4-round Delphi study was performed among 159 experts from 21 countries. Panelists rated the degree to which they (dis)agreed to proposed standards, criteria, and rating issues on 5-point rating scales ('strongly disagree' to 'strongly agree'), and provided arguments for their ratings.

RESULTS

Discussion focused on sample size requirements, recording and field notes, transcribing cognitive interviews, and data coding. After four rounds, the required 67% consensus was reached on all standards, criteria, and rating issues. After pilot-testing, the steering committee made some final changes. Ten criteria for good content validity were defined regarding item relevance, appropriateness of response options and recall period, comprehensiveness, and comprehensibility of the PROM.

DISCUSSION

The consensus-based COSMIN methodology for content validity is more detailed, standardized, and transparent than earlier published guidelines, including the previous COSMIN standards. This methodology can contribute to the selection and use of high-quality PROMs in research and clinical practice.

COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study

BACKGROUND

Content validity is the most important measurement property of a patient-reported outcome measure (PROM) and the most challenging to assess. Our aims were to: (1) develop standards for evaluating the quality of PROM development; (2) update the original COSMIN standards for assessing the quality of content validity studies of PROMs; (3) develop criteria for what constitutes good content validity of PROMs, and (4) develop a rating system for summarizing the evidence on a PROM's content validity and grading the quality of the evidence in systematic reviews of PROMs.

METHODS

An online 4-round Delphi study was performed among 159 experts from 21 countries. Panelists rated the degree to which they (dis)agreed to proposed standards, criteria, and rating issues on 5-point rating scales ('strongly disagree' to 'strongly agree'), and provided arguments for their ratings.

RESULTS

Discussion focused on sample size requirements, recording and field notes, transcribing cognitive interviews, and data coding. After four rounds, the required 67% consensus was reached on all standards, criteria, and rating issues. After pilot-testing, the steering committee made some final changes. Ten criteria for good content validity were defined regarding item relevance, appropriateness of response options and recall period, comprehensiveness, and comprehensibility of the PROM.

DISCUSSION

The consensus-based COSMIN methodology for content validity is more detailed, standardized, and transparent than earlier published guidelines, including the previous COSMIN standards. This methodology can contribute to the selection and use of high-quality PROMs in research and clinical practice.

The Cleft Aesthetic Rating Scale for 18-Year-Old Unilateral Cleft Lip and Palate Patients: A Tool for Nasolabial Aesthetics Assessment

OBJECTIVE

To develop a reliable and easy-to-use method to assess the nasolabial appearance of 18-year-old patients with unilateral cleft lip and palate (CLP).

DESIGN

Retrospective analysis of nasolabial aesthetics using a 5-point ordinal scale and newly developed photographic reference scale: the Cleft Aesthetic Rating Scale (CARS). Three cleft surgeons and 20 medical students scored the nasolabial appearance on standardized frontal photographs.

SETTING

VU University Medical Center, Amsterdam.

PATIENTS

Inclusion criteria: 18-year-old patients, unilateral cleft lip and palate, available photograph of the frontal view.

EXCLUSION CRITERIA

history of facial trauma, congenital syndromes affecting facial appearance. Eighty photographs were available for scoring.

MAIN OUTCOME MEASURES

The interobserver and intraobserver reliability of the CARS for 18-year-old patients when used by cleft surgeons and medical students.

RESULTS

The interobserver reliability for the nose and lip together was 0.64 for the cleft surgeons and 0.61 for the medical students. There was an intraobserver reliability of 0.75 and 0.78 from the surgeons and students, respectively, on the nose and lip together. No significant difference was found between the cleft surgeons and medical students in the way they scored the nose ( P = 0.22) and lip ( P = 0.72).

CONCLUSIONS

The Cleft Aesthetic Rating Scale for 18-year-old patients has a substantial overall estimated reliability when the average score is taken from three or more cleft surgeons or medical students assessing the nasolabial aesthetics of CLP patients.

COSMIN guideline for systematic reviews of patient-reported outcome measures

PURPOSE

Systematic reviews of patient-reported outcome measures (PROMs) differ from reviews of interventions and diagnostic test accuracy studies and are complex. In fact, conducting a review of one or more PROMs comprises of multiple reviews (i.e., one review for each measurement property of each PROM). In the absence of guidance specifically designed for reviews on measurement properties, our aim was to develop a guideline for conducting systematic reviews of PROMs.

METHODS

Based on literature reviews and expert opinions, and in concordance with existing guidelines, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) steering committee developed a guideline for systematic reviews of PROMs.

RESULTS

A consecutive ten-step procedure for conducting a systematic review of PROMs is proposed. Steps 1-4 concern preparing and performing the literature search, and selecting relevant studies. Steps 5-8 concern the evaluation of the quality of the eligible studies, the measurement properties, and the interpretability and feasibility aspects. Steps 9 and 10 concern formulating recommendations and reporting the systematic review.

CONCLUSIONS

The COSMIN guideline for systematic reviews of PROMs includes methodology to combine the methodological quality of studies on measurement properties with the quality of the PROM itself (i.e., its measurement properties). This enables reviewers to draw transparent conclusions and making evidence-based recommendations on the quality of PROMs, and supports the evidence-based selection of PROMs for use in research and in clinical practice.

COSMIN guideline for systematic reviews of patient-reported outcome measures

Purpose

Systematic reviews of patient-reported outcome measures (PROMs) differ from reviews of interventions and diagnostic test accuracy studies and are complex. In fact, conducting a review of one or more PROMs comprises of multiple reviews (i.e., one review for each measurement property of each PROM). In the absence of guidance specifically designed for reviews on measurement properties, our aim was to develop a guideline for conducting systematic reviews of PROMs.

Methods

Based on literature reviews and expert opinions, and in concordance with existing guidelines, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) steering committee developed a guideline for systematic reviews of PROMs.

Results

A consecutive ten-step procedure for conducting a systematic review of PROMs is proposed. Steps 1–4 concern preparing and performing the literature search, and selecting relevant studies. Steps 5–8 concern the evaluation of the quality of the eligible studies, the measurement properties, and the interpretability and feasibility aspects. Steps 9 and 10 concern formulating recommendations and reporting the systematic review.

Conclusions

The COSMIN guideline for systematic reviews of PROMs includes methodology to combine the methodological quality of studies on measurement properties with the quality of the PROM itself (i.e., its measurement properties). This enables reviewers to draw transparent conclusions and making evidence-based recommendations on the quality of PROMs, and supports the evidence-based selection of PROMs for use in research and in clinical practice.

COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures

PURPOSE

The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties.

METHODS

For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis.

RESULTS

Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system.

CONCLUSIONS

The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.

Long-term trajectories of patients with neck pain and low back pain presenting to chiropractic care: A latent class growth analysis

BACKGROUND

Information on the course of neck pain (NP) and low back pain (LBP) typically relies on data collected at few time intervals during a period of up to 1 year.

METHODS

In this prospective, multicentre practice-based cohort study, patients consulting a chiropractor responded weekly for 52 weeks to text messages on their cell phones. Data from 448 patients (153 NP, 295 LBP) who had returned at least one set of answers in the first 26 weeks were used. Outcome measures were pain intensity (VAS) and functional outcome, assessed using four different questions: pain intensity, limitation in activities of daily living (ADL), number of days with pain in the previous week and number of days limited in ADL. Distinct patterns of pain were analysed with quadratic latent class growth analysis.

RESULTS

The final model was a 4-class model for NP and LBP. The 'recovering from mild baseline pain' is most common (76.3% of NP patients/58.3% of LBP patients) followed by the 'recovering from severe baseline pain' class (16.3% NP/29.8% LBP). They follow similar trajectories when considered over a period of 6 months. Pain at baseline, duration of complaints, functional status, limitations in ADL and the score on psychosocial scales were the variables that most contributed to distinguish between groups.

CONCLUSIONS

Most patients with NP or LBP presenting in chiropractic care show a trajectory of symptoms characterized by persistent or fluctuating pain of low or medium intensity. Only a minority either experience a rapid complete recovery or develop chronic severe pain.

SIGNIFICANCE

Ninety percentage of patients with neck pain or low back pain presenting to chiropractors have a 30% improvement within 6 weeks and then show a trajectory of symptoms characterized by persistent or fluctuating pain of low or medium intensity. Only a minority either experience a rapid complete recovery or develop chronic severe pain.

[Measurement of patient safety: a systematic review of the reliability and validity of adverse event detection with record review]

OBJECTIVES

Record review is the most used method to quantify patient safety. We systematically reviewed the reliability and validity of adverse event detection with record review.

DESIGN

A systematic review of the literature.

METHODS

We searched PubMed, EMBASE, CINAHL, PsycINFO and the Cochrane Library and from their inception through February 2015. We included all studies that aimed to describe the reliability and/or validity of record review. Two reviewers conducted data extraction. We pooled κ values (κ) and analysed the differences in subgroups according to number of reviewers, reviewer experience and training level, adjusted for the prevalence of adverse events.

RESULTS

In 25 studies, the psychometric data of the Global Trigger Tool (GTT) and the Harvard Medical Practice Study (HMPS) were reported and 24 studies were included for statistical pooling. The inter-raterreliability of the GTT and HMPS showed a pooled κ of 0.65 and 0.55, respectively. The inter-rater agreement was statistically significantly higher when the group of reviewers within a study consisted of a maximum five reviewers. We found no studies reporting on the validity of the GTT and HMPS.

CONCLUSIONS

The reliability of record review is moderate to substantial and improved when a small group of reviewers carried out record review. The validity of the record review method has never been evaluated, while clinical data registries, autopsy or direct observations of patient care are methods that can be used to test concurrent validity.

The quality of systematic reviews of health-related outcome measurement instruments

BACKGROUND

Systematic reviews of outcome measurement instruments are important tools for the selection of instruments for research and clinical practice. Our aim was to assess the quality of systematic reviews of health-related outcome measurement instruments and to determine whether the quality has improved since our previous study in 2007.

METHODS

A systematic literature search was performed in MEDLINE and EMBASE between July 1, 2013, and June 19, 2014. The quality of the reviews was rated using a study-specific checklist.

RESULTS

A total of 102 reviews were included. In many reviews the search strategy was considered not comprehensive; in only 59 % of the reviews a search was performed in EMBASE and in about half of the reviews there was doubt about the comprehensiveness of the search terms used for type of measurement instruments and measurement properties. In 41 % of the reviews, compared to 30 % in our previous study, the methodological quality of the included studies was assessed. In 58 %, compared to 55 %, the quality of the included instruments was assessed. In 42 %, compared to 7 %, a data synthesis was performed in which the results from multiple studies on the same instrument were somehow combined.

CONCLUSION

Despite a clear improvement in the quality of systematic reviews of outcome measurement instruments in comparison with our previous study in 2007, there is still room for improvement with regard to the search strategy, and especially the quality assessment of the included studies and the included instruments, and the data synthesis.

The assessment of publication pressure in medical science; validity and reliability of a Publication Pressure Questionnaire (PPQ)

PURPOSE

To determine content validity, structural validity, construct validity and reliability of an internet-based questionnaire designed for assessment of publication pressure experienced by medical scientists.

METHODS

The Publication Pressure Questionnaire (PPQ) was designed to assess psychological pressure to publish scientific papers. Content validity was evaluated by collecting independent comments from external experts (n = 7) on the construct, comprehensiveness and relevance of the PPQ. Structural validity was assessed by factor analysis and item response theory (IRT) using the generalized partial credit model. Pearson's correlation coefficients were calculated to assess potential correlations with the emotional exhaustion and depersonalization subscales of the Maslach Burnout Inventory (MBI). Single test reliability (lambda2) was obtained from the IRT analysis.

RESULTS

Content validity was satisfactory. Confirmatory factor analysis did not support the presence of three initially assumed separate domains of publication pressure (i.e., personally experienced publication pressure, publication pressure in general, pressure on position of scientist). After exclusion of the third domain (six items), we performed exploratory factor analysis and IRT. The goodness-of-fit statistics for the IRT assuming a single dimension were satisfactory when four items were removed, resulting in 14 items of the final PPQ. Correlations with the emotional exhaustion and depersonalization scales of the MBI were 0.34 and 0.31, respectively, supporting construct validity. Single test administration reliability lambda2 was 0.69 and 0.90 on the test scores and expected a posteriori scores, respectively.

CONCLUSION

The PPQ seems a valid and reliable instrument to measure publication pressure among medical scientists.

Validity of nutritional screening with MUST and SNAQ in hospital outpatients

BACKGROUND/OBJECTIVES

The majority of hospital outpatients with undernutrition is unrecognized, and therefore untreated. There is a need for an easy and valid screening tool to detect undernutrition in this setting. The aim of this study was to determine the diagnostic accuracy of the MUST (Malnutrition Universal Screening Tool) and SNAQ (Short Nutritional Assessment Questionnaire) tools for undernutrition screening in hospital outpatients.

METHODS

In a large multicenter-hospital-outpatient population, patients were classified as: severely undernourished (body mass index (BMI) <18.5 (<65 years) or <20 ( ≥ 65 years) and/or unintentional weight loss >5% in the last month or >10% in the last 6 months), moderately undernourished (BMI 18.5-20 (<65 years) or 20-22 ( ≥ 65 years) and/or 5-10% unintentional weight loss in the last 6 months) or not undernourished. Diagnostic accuracy of the screening tools versus the reference method was expressed as sensitivity (Se), specificity (Sp), positive predictive value (PPV) and negative predictive value (NPV).

RESULTS

Out of the 2236 outpatients, 6% were severely and 7% were moderately undernourished according to the reference method. MUST and SNAQ identified 9% and 3% as severely undernourished, respectively. MUST had a low PPV (Se=75, Sp=95, PPV=43, NPV=98), whereas SNAQ had a low Se (Se=43, Sp=99, PPV=78, NPV=96).

CONCLUSIONS

The validity of MUST and SNAQ is insufficient for hospital outpatients. While SNAQ identifies too few patients as undernourished, MUST identifies too many patients as undernourished. We advise to measure body weight, height and weight loss, in order to define undernutrition in hospital outpatients.

Instruments to assess physical activity in patients with osteoarthritis of the hip or knee: a systematic review of measurement properties

OBJECTIVE

There is no consensus on the best approach for measuring physical activity in patients with osteoarthritis (OA) of the hip or knee. The aims of this study were (1) to identify all physical activity measures that have been validated in patients with OA of the hip or knee and to systematically review their measurement properties, and (2) to give recommendations on which instrument is most suitable for what purpose.

DESIGN

A search was performed in PubMed, Embase, and Sportdiscus (complete databases until November 10, 2010). Three reviewers independently evaluated the quality of the included studies, using the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN) checklist. Subsequently, the reviewers independently evaluated the quality of the included physical activity instruments, using the recently developed QAPAQ checklist for appraising the qualitative attributes and measurement properties of physical activity questionnaires.

RESULTS

Nine studies were included, in which 12 measurement instruments were evaluated: five single-item rating scales, six multi-item questionnaires, and one pedometer. In general, the methodological quality of the studies was poor to moderate. Only the Lower-Extremity Activity Scale (LEAS) and the pedometer received positive ratings for content validity. The LEAS and Baecke questionnaire received positive ratings for reliability. The University of California at Los Angeles (UCLA), the Tegner score, and the LEAS received positive ratings for construct validity. The Daily Activity Questionnaire (DAQ) received a positive rating for criterion validity. Responsiveness was not evaluated for any of the included instruments.

CONCLUSION

For monitoring physical activity levels of populations the UCLA or LEAS seem most useful. For studies measuring physical activity as a risk factor for developing OA or as a protective factor against functional decline there is not enough evidence for any instrument to conclude that it has adequate measurement properties. For follow-up studies on wear in joint replacement patients we recommend to use accelerometers. However, more validation studies of adequate quality are needed for all included instruments.

The SNAQ(RC), an easy traffic light system as a first step in the recognition of undernutrition in residential care

OBJECTIVE

Development and validation of a quick and easy screening tool for the early detection of undernourished residents in nursing homes and residential homes.

DESIGN

Multi-center, cross sectional observational study.

SETTING

Nursing homes and residential homes.

PARTICIPANTS

The screening tool was developed in a total of 308 residents (development sample; sample A) and cross validated in a new sample of 720 residents (validation sample) consisting of 476 nursing home residents (Sample B1) and 244 residential home residents (sample B2).

MEASUREMENTS

Patients were defined severely undernourished when they met at least one of the following criteria: BMI or= 5% unintentional weight loss in the past month and/or >or= 10% unintentional weight loss in the past 6 months. Patients were defined as moderately undernourished if they met the following criteria: BMI 20.1-22 kg/m2 and/or 5-10% unintentional weight loss in the past six months. The most predictive questions (originally derived from previously developed screening instruments) of undernourishment were selected in sample A and cross validated in sample B. In a second stage BMI was added to the SNAQRC in sample B. The diagnostic accuracy of the screening tool in the development and validation samples was expressed in sensitivity, specificity, and the negative and positive predictive value.

RESULTS

The four most predictive questions for undernutrition related to: unintentional weight loss more than 6 kg during the past 6 months and more than 3 kg in the past month, capability of eating and drinking with help, and decreased appetite during the past month. The diagnostic accuracy of these questions alone was insufficient (Se=45%, Sp=87%, PPV=50% and NPV=84%). However, combining the questions with measured BMI sufficiently improved the diagnostic accuracy (Se=87%, Sp=82%, PPV=59% and NPV=95%).

CONCLUSION

Early detection of undernourished nursing- and residential home residents is possible using four screening questions and measured BMI.

Validation study of a distress screener

OBJECTIVES

A 3-item screening instrument called the Distress Screener was developed for early identification of distress among employees on sick leave. The Distress Screener consists of three items obtained from the distress subscale of the four-dimensional symptom questionnaire (4DSQ). This study assessed an optimal cut-off point and validated the Distress Screener by relating it to the 4DSQ and to medical diagnoses.

METHODS

171 sick-listed employees filled in the Distress Screener and the 4DSQ (containing four subscales: distress, depression, anxiety and somatisation) and medical diagnoses were obtained from occupational physicians (OPs). The optimal cut-off point was assessed by computing sensitivity and specificity values. Validity was assessed by relating the Distress Screener score to the scores on 4DSQ subscales. In addition scores were compared to mental health medical diagnoses and the degree of similarity between two repeated measurements was obtained.

RESULTS

Using the 4DSQ distress score >10 as reference standard, the optimal cut-off point of the Distress Screener was > or =4. Regarding validity, a high correlation (0.82) existed between the Distress Screener and the 4DSQ distress subscale and it was significantly different from the correlations with the other 4DSQ subscales. Also a high correlation existed for the test-retest reliability (0.83). Furthermore, a high score on the Distress Screener seemed to be related to the medical diagnosis 'Stress-related complaints'. All low scores seemed to be related to the medical diagnosis 'Other complaints'. Sensitivity (0.85) and specificity (0.78) values, and positive and negative predictive values of the screener were comparable to those of the 4DSQ distress subscale.

CONCLUSIONS

The Distress Screener is a valid instrument for use by the OP during consulting time as a quick scan for early identification of distress in employees on sick leave. The cut-off point > or =4 is useful for early identification of distress in employees on sick leave.

The fidelity of treatment delivery can be assessed in treatment outcome studies: a successful illustration from behavioral medicine

OBJECTIVES

Treatment outcome studies ought to assess the fidelity of their treatments, including treatment delivery, but practical guidelines and examples for this are lacking. Based on general recommendations in available literature, this study proposes and illustrates the design and application of a Method of Assessing Treatment Delivery (MATD) in a behavioral medicine trial comparing two treatments for chronic low back pain.

STUDY DESIGN AND SETTING

In designing MATD, two experts identified several feasible treatment elements. Agreement between the experts in classifying these elements into five categories (essential and unique, essential but not unique, unique but not essential, compatible, prohibited) was assessed. In applying MATD, treatment recordings were evaluated by two independent raters, who coded the (non)-occurrence of MATD elements and who categorized each session as belonging to one of the two treatments.

RESULTS

MATDs content validity was supported by adequate agreement between the experts' classifications of the treatment elements. MATDs interrater reliability was good.

CONCLUSION

Comprehensive illustrations of designing and applying MATD may encourage the verification of treatment delivery as a partial reflection of treatment fidelity in forthcoming treatment outcome studies.

Balneotherapy for osteoarthritis

BACKGROUND

Balneotherapy (or spa therapy, mineral baths) for patients with arthritis is one of the oldest forms of therapy. One of the aims of balneotherapy is to soothe the pain and as a consequence to relieve patients' suffering and make them feel well. In this update we included one extra study.

OBJECTIVES

To assess the effectiveness of balneotherapy for patients with osteoarthritis (OA).

SEARCH STRATEGY

We searched the following databases up to October 2006: EMBASE, PubMed, the Cochrane 'Rehabilitation and Related Therapies' Field database, PEDro, CENTRAL (Issue 3, 2006) and performed reference checking and communicated with authors to retrieve eligible studies.

SELECTION CRITERIA

Randomised controlled trials (RCT) comparing balneotherapy with any intervention or no intervention. At least 90% of the patient population had to be diagnosed with OA.

DATA COLLECTION AND ANALYSIS

Two authors independently assessed quality and extracted data. Disagreements were solved by consensus. In the event of clinical heterogeneity or lack of data we refrained from statistical pooling.

MAIN RESULTS

Seven trials (498 patients) were included in this review. Two studies compared spa-treatment with no treatment. One study evaluated baths as an add-on treatment to home exercises and another compared thermal water from Cserkeszölö with tap water (placebo). Three studies evaluated sulphur or Dead Sea baths with no treatment or mineral baths with tap water baths or no treatment. Only one of the trials performed an intention-to-treat analysis and two studies provided data to perform an intention-to-treat analysis ourselves. A 'quality of life' outcome was reported by one trial. We found: silver level evidence concerning the beneficial effects on pain, quality of life and analgesic intake of mineral baths compared to no treatment (SMD between 1.82 and 0.34). a statistically significant difference in pain and function of Dead Sea + sulphur versus no treatment, only at end of treatment (WMD 5.7, 95%CI 3.3 to 8.1), but not at 3 month follow-up (WMD 2.6, 95%CI -1.1 to 6.3). no statistically significant differences in pain or function at one or three months of Dead Sea baths versus no treatment (WMD 0.5, 95%CI -0.6 to 1.6) or at one or three months of sulphur baths versus no treatment (WMD 0.4, 95%CI -0.9 to 1.7). Adverse events were not measured in the included trials.

AUTHORS' CONCLUSIONS

We found silver level evidence (www.cochranemsk.org) concerning the beneficial effects of mineral baths compared to no treatment. Of all other balneological treatments no clear effects were found. However, the scientific evidence is weak because of the poor methodological quality and the absence of an adequate statistical analysis and data presentation. Therefore, the noted "positive findings" should be viewed with caution.

Surgical treatment options for carpal tunnel syndrome

BACKGROUND

Carpal tunnel syndrome is a common disorder for which several surgical treatment options are available.

OBJECTIVES

To compare the efficacy of the various surgical techniques in relieving symptoms and promoting return to work or activities of daily living and to compare the occurrence of side-effects and complications in patients suffering from carpal tunnel syndrome.

SEARCH STRATEGY

We updated the searches in 2006. We conducted computer-aided searches of the Cochrane Neuromuscular Disease Group Trials Register (searched in June 2006), Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library 2006, Issue 2), MEDLINE (January 1966 to June 2006), EMBASE (January 1980 to June 2006) and also tracked references in bibliographies.

SELECTION CRITERIA

Randomised controlled trials comparing various surgical techniques for the treatment of carpal tunnel syndrome.

DATA COLLECTION AND ANALYSIS

Two review authors performed study selection, assessment of methodological quality and data extraction independently of each other.

MAIN RESULTS

Thirty-three studies were included in the review of which 10 were newly identified in this update. The methodological quality of the trials ranged from fair to good; however, the use of allocation concealment was mentioned explicitly in only seven trials. Many studies failed to present the results in sufficient detail to enable statistical pooling. Pooling was also impeded by the vast variety of outcome measures that were applied in the various studies. None of the existing alternatives to standard open carpal tunnel release offered significantly better relief from symptoms in the short- or long-term. In three studies with a total of 294 participants, endoscopic carpal tunnel release resulted in earlier return to work or activities of daily living than open carpal tunnel release, with a weighted mean difference of -6 days (95% CI -9 to -3 days).

AUTHORS' CONCLUSIONS

There is no strong evidence supporting the need for replacement of standard open carpal tunnel release by existing alternative surgical procedures for the treatment of carpal tunnel syndrome. The decision to apply endoscopic carpal tunnel release instead of open carpal tunnel release seems to be guided by the surgeon's and patient's preferences.

WITHDRAWN: Injection therapy for subacute and chronic benign low-back pain

BACKGROUND

Injection with anaesthetics and/or steroids is one of the treatment modalities used in patients with chronic low back pain which needs evaluation with respect to the effectiveness on short and long term pain relief.

OBJECTIVES

To evaluate the effectiveness of injection therapy in patients with low back pain lasting longer than one month. We distinguished between three injection sites: facet joint, epidural or local injections.

SEARCH STRATEGY

We searched the Medline and Embase databases up to 1996 and other search methods as advocated by the Back Review Group search strategy. Abstracts and unpublished studies were not included.

SELECTION CRITERIA

Randomized controlled trials of injection therapy for pain relief (although additional treatments were allowed) in patients with benign low back pain lasting longer than one month and not originating from cancer.

DATA COLLECTION AND ANALYSIS

Two authors independently assessed the trials for methodological quality. Subgroup analyses were made between trials with different control groups (placebo and active injections), with different injection site (facet joint, epidural and local injection), and timing of outcome measurement (short and long term). Within the resulting 12 subcategories of studies (2*3*2), the overall relative risks and corresponding 95% confidence intervals were estimated, using a random effects model (DerSimonian and Laird). In the case of trials in which control groups were active injections, we refrained from pooling the results.

MAIN RESULTS

Twenty-one randomized trials were included in this review. All studies involved patients with low back pain lasting longer than one month. Only 11 studies compared injection therapy with placebo injections (explanatory trials). The methodologic quality of many studies was low: only 8 studies had a methodologic score of 50 or more points. There were only three well designed explanatory clinical trials: one on injections into the facet joints with a short-term RR of 0.89 (95% CI: 0.65-1.21) and a long-term RR of 0.90 (95% CI: 0.69-1.17); one on epidural injections with a short-term RR of 0.94 (95% CI: 0.76-1.15) and a long-term RR of 1.00 (95% CI: 0.71-1.41); and one on local injections with a long-term RR of 0.79 (95% CI: 0.65-0.96). Within the 6 subcategories of explanatory studies the pooled RRs with 95% confidence intervals were: facet joint, short-term: RR=0.89 (0.65-1.21); facet joint, long-term: RR=0.90 (0.69-1.17); epidural, short-term: RR=0.93 (0.79-1.09); epidural, long-term: RR=0.92 (0.76-1.11); local, short-term: RR=0.80 (0.40-1.59); local, long-term: RR=0.79 (0.65-0.96).

AUTHORS' CONCLUSIONS

Convincing evidence is lacking on the effects of injection therapies for low back pain. There is a need for more, well designed explanatory trials in this field.

Traction for low-back pain with or without sciatica

BACKGROUND

Traction is used to treat low-back pain (LBP), often with other treatments.

OBJECTIVES

To determine traction's effectiveness, compared to reference treatments, placebo, sham traction or no treatment for LBP.

SEARCH STRATEGY

We searched CENTRAL (The Cochrane Library 2006, issue 4), MEDLINE, EMBASE, and CINAHL to October 2006, references in relevant reviews and personal files.

SELECTION CRITERIA

Randomized controlled trials (RCTs) involving traction to treat acute (less than four weeks duration), sub-acute (four to 12 weeks) or chronic (more than 12 weeks) non-specific LBP with or without sciatica.

DATA COLLECTION AND ANALYSIS

Study selection, methodological quality assessment and data extraction were done independently by two authors. As there were insufficient data for statistical pooling, we performed a qualitative analysis.

MAIN RESULTS

We included 25 RCTs (2206 patients; 1045 receiving traction). Five trials were considered high quality. For patients with mixed symptom patterns (acute, sub-acute and chronic LBP with and without sciatica) there is: strong evidence of no statistically significant difference in outcomes between traction as a single treatment and placebo, sham or no treatment; moderate evidence that traction as a single treatment is no more effective than other treatments; limited evidence of no significant difference in outcomes between a standard physical therapy program with or without continuous traction. For LBP patients with sciatica (with acute, sub-acute or chronic pain), there is conflicting evidence in several comparisons: autotraction compared to placebo, sham or no treatment; other forms of traction compared to other treatments; different forms of traction. In other comparisons, there were no statistically significant differences; the evidence is moderate for continuous or intermittent traction compared to placebo, sham or no treatment, and limited for light versus normal force traction.

IMPLICATIONS FOR PRACTICE

The results of the available studies involving mixed groups of acute, sub-acute and chronic patients with LBP with and without sciatica were quite consistent, indicating that continuous or intermittent traction as a single treatment for LBP is not likely effective for this group. Traction for patients with sciatica cannot be judged effective at present either, due to inconsistent results and methodological problems in most studies. We conclude that traction as a single treatment for LBP is probably not effective.

IMPLICATIONS FOR RESEARCH

Any future research on traction for patients with LBP should distinguish between symptom pattern and duration, and should be carried out according to the highest methodological standards.

The quality of diagnostic accuracy studies since the STARD statement: has it improved?

OBJECTIVE

To assess whether the quality of reporting of diagnostic accuracy studies has improved since the publication of the Standards for the Reporting of Diagnostic Accuracy studies (STARD statement).

METHODS

The quality of reporting of diagnostic accuracy studies published in 12 medical journals in 2000 (pre-STARD) and 2004 (post-STARD) was evaluated by two reviewers independently. For each article, the number of reported STARD items was counted (range 0 to 25). Differences in completeness of reporting between articles published in 2000 and 2004 were analyzed, using multilevel analyses.

RESULTS

We included 124 articles published in 2000 and 141 articles published in 2004. Mean number of reported STARD items was 11.9 (range 3.5 to 19.5) in 2000 and 13.6 (range 4.0 to 21.0) in 2004, an increase of 1.81 items (95% CI: 0.61 to 3.01). Articles published in 2004 reported the following significantly more often: methods for calculating test reproducibility of the index test (16% vs 35%); distribution of the severity of disease and other diagnoses (23% vs 53%); estimates of variability of diagnostic accuracy between subgroups (39% vs 60%); and a flow diagram (2% vs 12%).

CONCLUSIONS

The quality of reporting of diagnostic accuracy studies has improved slightly over time, without a more pronounced effect in journals that adopted the STARD statement. As there is still room for improvement, editors should mention the use of the STARD statement as a requirement in their guidelines for authors, and instruct reviewers to check the STARD items. Authors should include a flow diagram in their manuscript.

Evaluation of cross-sectional and longitudinal construct validity of two vision-related quality of life questionnaires: the LVQOL and VCM1

The Low Vision Quality of Life (LVQOL) questionnaire and the Vision-related Quality of Life Core Measure (VCM1) are two of the many vision-related quality of life (QOL) questionnaires that have been developed in recent years. Although psychometric properties of the LVQOL and VCM1 compare well with other vision-related QOL questionnaires, construct and longitudinal validity have not been assessed (adequately). The purpose of this study was to examine the cross-sectional and longitudinal construct validity of these questionnaires by testing specific pre-specified hypotheses about the relations of these questionnaires with other measures. The percentage of hypotheses regarding the cross-sectional construct validity that were refuted for the LVQOL was 22% for the basic aspects of vision subscale, 50% for the mobility subscale, 39% for the adjustment subscale and 17% for the reading and fine work subscale. For the VCM1 this percentage was 57%. For the longitudinal construct validity the percentage of hypotheses that were refuted ranged from 33 to 75% for the LVQOL subscales and was 50% for the VCM1. In conclusion, cross-sectional construct validity was satisfactory for the LVQOL subscales, but seemed poor for the VCM1. In addition, the longitudinal validity of these scales was poor to moderate.

Ergonomic and physiotherapeutic interventions for treating work-related complaints of the arm, neck or shoulder in adults

BACKGROUND

Conservative interventions such as physiotherapy and ergonomic adjustments (such as keyboard adjustments or ergonomic advice) play a major role in the treatment of most work-related complaints of the arm, neck or shoulder (CANS).

OBJECTIVES

This systematic review aims to determine whether conservative interventions have a significant impact on outcomes for work-related CANS in adults.

SEARCH STRATEGY

We searched the Cochrane Bone, Joint and Muscle Trauma Group Specialised Register (March 2005) and Cochrane Rehabilitation and Related Therapies Field Specialised Register (March 2005), the Cochrane Controlled Trials Register (The Cochrane Library, Issue 1, 2005), PubMed, EMBASE, CINAHL, AMED and reference lists of articles. The date of the last search was March 2005. No language restrictions were applied.

SELECTION CRITERIA

We included randomised and non-randomised controlled trials studying conservative interventions (e.g. exercises, relaxation, physical applications, biofeedback, myofeedback and work-place adjustments) for adults suffering CANS.

DATA COLLECTION AND ANALYSIS

Two authors independently selected trials from the search yield, assessed the methodological quality using the Delphi list, and extracted relevant data. We pooled data or, in the event of clinical heterogeneity or lack of data, we used a rating system to assess levels of evidence.

MAIN RESULTS

For this update we included six additional studies; 21 trials in total. Seventeen trials included people with chronic non-specific neck or shoulder complaints, or non-specific upper extremity disorders. Over 25 interventions were evaluated; five main subgroups of interventions could be determined: exercises, manual therapy, massage, ergonomics, and energised splint. Overall, the quality of the studies was poor. In 14 studies a form of exercise was evaluated, and contrary to the previous review we now found limited evidence about the effectiveness of exercises when compared to massage and conflicting evidence when exercises are compared to no treatment. In this update there is limited evidence for adding breaks during computer work; massage as add-on treatment on manual therapy, manual therapy as add-on treatment on exercises; and some keyboard designs when compared to other keyboards or placebo in participants with carpal tunnel syndrome.

AUTHORS' CONCLUSIONS

There is limited evidence for the effectiveness of keyboards with an alternative force-displacement of the keys or an alternative geometry, and limited evidence for the effectiveness of exercises compared to massage; breaks during computer work compared to no breaks; massage as an add-on treatment to manual therapy; and manual therapy as an add-on treatment to exercises.

The effectiveness of graded activity for low back pain in occupational healthcare

BACKGROUND

Low back pain is a common medical and social problem associated with disability and absence from work. Knowledge on effective return to work (RTW) interventions is scarce.

OBJECTIVE

To determine the effectiveness of graded activity as part of a multistage RTW programme.

DESIGN

Randomised controlled trial.

SETTING

Occupational healthcare.

SUBJECTS

112 workers absent from work for more than eight weeks due to low back pain were randomised to either graded activity (n = 55) or usual care (n = 57).

INTERVENTION

Graded activity, a physical exercise programme aimed at RTW based on operant-conditioning behavioural principles.

MAIN OUTCOME MEASURES

The number of days off work until first RTW for more then 28 days, total number of days on sick leave during follow up, functional status, and severity of pain. Follow up was 26 weeks.

RESULTS

Graded activity prolonged RTW. Median time until RTW was equal to the total number of days on sick leave and was 139 (IQR = 69) days in the graded activity group and 111 (IQR = 76) days in the usual care group (hazard ratio = 0.52, 95% CI 0.32 to 0.86). An interaction between a prior workplace intervention and graded activity, together with a delay in the start of the graded activity intervention, explained most of the delay in RTW (hazard ratio = 0.86, 95% CI 0.40 to 1.84 without prior intervention and 0.39, 95% CI 0.19 to 0.81 with prior intervention). Graded activity did not improve pain or functional status clinically significantly.

CONCLUSIONS

Graded activity was not effective for any of the outcome measures. Different interventions combined can lead to a delay in RTW. Delay in referral to graded activity delays RTW. In implementing graded activity special attention should be paid to the structure and process of care.

The usefulness of evaluative outcome measures in patients with multiple sclerosis

To select the most useful evaluative outcome measures for early multiple sclerosis, we included 156 recently diagnosed patients in a 3-year follow-up study, and assessed them on 23 outcome measures in the domains of disease-specific outcomes, physical functioning, mental health, social functioning and general health. A global rating scale (GRS) and the Expanded Disability Status Scale (EDSS) were used as external criteria to determine the minimally important change (MIC) for each outcome measure. Subsequently, we determined whether the outcome measures could detect their MIC reliably. From these, per domain the outcome measure that was found to be most sensitive to changes (responsive) was identified. At group level, 11 outcomes of the domains of physical functioning, mental health, social functioning and general health could reliably detect the MIC. Of these 11, the most responsive measures per domain were the Medical Outcome Study 36 Short Form sub-scale physical functioning (SF36pf), the Disability and Impact Profile (DIP) sub-scale psychological, the Rehabilitation Activities Profile sub-scale occupation (RAPocc) and the SF36 sub-scale health, respectively. Overall, the most responsive measures were the SF36pf and the RAPocc. In individual patients, none of the measures could reliably detect the MIC. In sum, in the early stages of multiple sclerosis the most useful evaluative outcome measures for research are the SF36pf (physical functioning) and the RAPocc (social functioning).

[The use of complementary and alternative medicine in children: how to advise the parents?]

Recently it has been shown that complementary or alternative medicine (CAM) is used in about 30% of paediatric patients. Is this a problem? It has also been shown that most parents would like their paediatrician to advise them on the subject. But what can be the basis of such advice? Little is known and less has been published regarding the effectiveness of CAM. Society and the medical profession are notoriously ambiguous on the subject. Sound scientific evidence is advocated to make bold decisions on whether to allow and reimburse the use of CAM.

Neck/shoulder, low back, and arm pain in relation to computer use, physical activity, stress, and depression among Dutch adolescents

OBJECTIVE

To examine among adolescents (1) the prevalence of neck/shoulder, low back, and arm pain within different sociodemographic groups and (2) the association of neck/shoulder, low back, and arm pain with computer use, physical activity, depression, and stress.

METHODS

A school-based questionnaire survey in 2002 and 2003 of neck/shoulder, low back, and arm pain, computer use, physical activity, depression, and stress. The survey was given to 3485 adolescents aged 12 to 16 years who attended secondary schools in Amsterdam, Netherlands.

RESULTS

The overall prevalence of neck/shoulder, low back, and arm pain was 11.5%, 7.5%, and 3.9%, respectively. The prevalence of neck/shoulder pain was higher among girls and adolescents not living with both parents. The prevalence of low back pain also was higher among girls. Depressive symptoms were associated with neck/shoulder pain, low back pain, and arm pain. The stress experienced was associated with neck/shoulder pain and with low back pain.

CONCLUSIONS

This study strengthens the findings that musculoskeletal pain is common among adolescents and is associated with depression and stress but not with computer use and physical activity.

Construct validity of instruments measuring impairments in body structures and function in rheumatic disorders: which constructs are selected for validation? A systematic review

PURPOSE

This paper focuses on the construct validity of instruments measuring impairments in body structures and function in rheumatic disorders. The objective is: 1) to make an inventory of constructs, based on the domains of the International Classification of Functioning, Disabilities and Health problems (ICF), against which instruments measuring impairments in body structures and function were validated; 2) to analyse whether validation against a similar construct resulted in higher correlation coefficients than validation against a dissimilar construct.

METHODS

In a systematic review papers were identified in which instruments measuring impairments in body structures and function for patients with rheumatic disorders were validated. The instruments identified were assessed on their methodological properties and the constructs against which they were validated. Subsequently, pooled (interclass) correlations of similar constructs and dissimilar constructs against which was validated were compared. An instrument was decided to have good construct validity, if the correlation coefficient was 0.50 or higher, and the measurement instrument in question is validated against similar constructs.

RESULTS

In total 216 papers were identified analysing the validity of 42 different instruments. Only 16% of these instruments were validated against instruments that represent the most similar construct. In general, estimates of construct validity were lower when validated against dissimilar constructs, except for instruments measuring impairments in mental functions.

CONCLUSION

There is a trend that validation against a similar construct yields higher correlation coefficients than validation against a dissimilar construct. If an instrument measuring impairments is validated against the most similar construct, and a criterion of r > 0.50 is applied, only 10 out of the 42 identified instruments turned out to be valid.

Traction for low-back pain with or without sciatica

BACKGROUND

Various types of traction are used in the treatment of low-back pain (LBP), often in conjunction with other treatments.

OBJECTIVES

To determine the effectiveness of traction in the management of LBP.

SEARCH STRATEGY

We searched The Cochrane Library 2004, Issue 4, MEDLINE, EMBASE, and CINAHL to November 2004, references in relevant reviews, and our personal files.

SELECTION CRITERIA

Randomized controlled trials (RCTs) examining any type of traction for the treatment of acute (less than four weeks duration), sub-acute (four to 12 weeks) or chronic (more than 12 weeks) non-specific LBP with or without sciatica.

DATA COLLECTION AND ANALYSIS

Study selection, methodological quality assessment and data extraction were done independently by sets of two reviewers. As available studies did not provide sufficient data for statistical pooling, a qualitative analysis was performed.

MAIN RESULTS

Twenty-four RCTs, involving 2177 patients (1016 receiving traction) were included in the review. Five trials were considered high quality. There is strong evidence that there is no significant difference in short or long-term outcomes between either continuous or intermittent traction and placebo, sham, or other treatments for patients with a mixed duration of LBP, with or without sciatica. There is moderate evidence that: autotraction is more effective other forms of traction are no more effective than placebo, sham or no treatment for patients with a mixed duration of LBP with sciatica. There is limited evidence that: there is no significant difference in outcomes between a standard physical therapy program with continuous traction and the same program without traction, for patients with a mixed duration of LBP, with or without sciatica autotraction on its own is more effective than a physical therapy program that includes Tru-Trac traction for patients with a mixed duration of LBP with sciatica. There is conflicting evidence regarding the short-term effectiveness of either continuous or intermittent traction compared to placebo, sham or other treatments, in the management of patients who have either chronic LBP or a mixed duration of LBP with sciatica.

AUTHORS' CONCLUSIONS

The evidence suggests that traction is probably not effective. Neither continuous nor intermittent traction by itself was more effective in improving pain, disability or work absence than placebo, sham or other treatments for patients with a mixed duration of LBP, with or without sciatica. Although trials studying patients with sciatica had methodological limitations and inconsistent results, there was moderate evidence that autotraction was more effective than mechanical traction for global improvement in this population.

Development and validation of a hospital screening tool for malnutrition: the short nutritional assessment questionnaire (SNAQ)

OBJECTIVE

For the early detection and treatment of malnourished hospital patients no valid screening instrument for the Dutch language exists. Calculation of percentage weight loss and body mass index (BMI) by the nurse at admission to the hospital appeared to be not feasible. Therefore, the short, nutritional assessment questionnaire (SNAQ), was developed.

RESEARCH, DESIGN AND METHODS

Two hundred and ninety one patients on the mixed internal and surgery/oncology wards of the VU University medical center were screened on nutritional status and classified as well nourished (<5% weight loss in the last 6 months and BMI>18.5), moderately malnourished (5-10% weight loss in the last 6 months and BMI>18.5) or severely malnourished (>10% weight loss in the last 6 months or >5% in the last month or BMI<18.5). All patients were asked 26 questions related to eating and drinking difficulties, defecation, condition and pain. Odds ratio, binary and multinomial logistic regression were used to determine the set of questions that best predicts the nutritional status. Based on the regression coefficient a score was composed to detect moderately (2 points) and severely (3 points) malnourished patients. The validity, the nurse-nurse reproducibility and nurse-dietitian reproducibility was tested in another but similar population of 297 patients.

RESULTS

The questions 'Did you lose weight unintentionally?'. 'Did you experience a decreased appetite over the last month?' and 'Did you use supplemental drinks or tube feeding over the last month?' were most predictive of malnutrition. The instrument proved to be valid and reproducible.

CONCLUSION

SNAQ is an easy, short, valid and reproducible questionnaire for early detection of hospital malnutrition.

Surgical treatment options for carpal tunnel syndrome

BACKGROUND

Carpal tunnel syndrome is a common disorder, for which several surgical treatment options are available.

OBJECTIVES

To compare the efficacy of the various surgical techniques in relieving symptoms and promoting return to work and/or activities of daily living and to compare the occurrence of side-effects and complications, in patients suffering from carpal tunnel syndrome.

SEARCH STRATEGY

We updated the searches in 2003. We conducted computer-aided searches of the trials register of the Cochrane Neuromuscular Disease Group (searched in July 2003), the Cochrane Central Register of Controlled Trials (The Cochrane Library, Issue 2, 2003), MEDLINE (January 1966 to August 2003), EMBASE (January 1980 to August 2003) and tracked references in bibliographies.

SELECTION CRITERIA

Randomised controlled trials comparing various surgical techniques for the treatment of carpal tunnel syndrome.

DATA COLLECTION AND ANALYSIS

Two reviewers performed study selection, assessment of methodological quality and data abstraction independently of each other.

MAIN RESULTS

Twenty-three studies were included in the review. The methodological quality of the trials was fair to good. However, the application of allocation concealment was mentioned explicitly in only one trial. Many studies failed to present the results in sufficient detail to enable statistical pooling. Pooling was also impeded by the vast variety of outcome measures that were applied in the various studies. None of the existing alternatives to standard open carpal tunnel release seem to offer better relief from symptoms in the short- or long-term. There was conflicting evidence about whether endoscopic carpal tunnel release resulted in earlier return to work and/or activities of daily living than open carpal tunnel release.

REVIEWERS' CONCLUSIONS

There is no strong evidence supporting the need for replacement of standard open carpal tunnel release by existing alternative surgical procedures for the treatment of carpal tunnel syndrome.