High test-retest reliability of the Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) questionnaire completed by primary care patients in routine care

BACKGROUND

The Alcohol Use Disorders Identification Test-Consumption (AUDIT-C) is a three-item screening measure of unhealthy alcohol use that is widely used in healthcare settings. Evidence shows high test-retest reliability of the AUDIT-C in research samples, but most studies had limited external validity and used small samples that could not be used to evaluate reliability across demographic subgroups and/or screening modalities. This study evaluates the test-retest reliability of the AUDIT-C completed in routine care in a large primary care sample, including across demographic subgroups defined by age, sex, race, ethnicity, and screening modality (i.e., completed in-clinic or online).

METHODS

We used electronic health record (EHR) data from Kaiser Permanente Washington. The sample included 18,491 adult primary care patients who completed two AUDIT-C screens 1-21 days apart as part of routine care in 2021. Test-retest reliability was evaluated for AUDIT-C total scores (0-12) and for a binary measure indicating unhealthy alcohol use (scores ≥3 women, ≥4 men). Using previously established cutoffs, we interpreted reliability coefficients >0.75 as indicating "excellent" reliability.

RESULTS

AUDIT-C screens completed in routine care and documented in EHRs had excellent test-retest reliability for total scores (ICC = 0.87, 95% CI: 0.87-0.87) and the binary indicator of unhealthy alcohol use (κ = 0.79, 95% CI: 0.78-0.80). Reliability coefficients were good to excellent across all demographic groups and for in-clinic and online modalities. Higher reliability was seen when both screens were completed through online patient portals (ICC = 0.93, 95% CI: 0.93-0.93) versus in-clinic (ICC = 0.81, 95% CI: 0.79-0.82) or when one screen was completed using each modality (ICC = 0.83, 95% CI: 0.82-0.83). Lower reliability was seen in American Indian/Alaska Native (ICC = 0.82, 95% CI: 0.75-0.87) and multiracial individuals (ICC = 0.82, 95% 0.80-0.84).

CONCLUSIONS

In real-world routine care conditions, AUDIT-C screens have excellent test-retest reliability across demographic subgroups and modalities (online and in-clinic). Future research should examine why reliability varies slightly across modalities and demographic subgroups.

Assessing injury risk in male and female Royal Navy recruits: does the Functional Movement Screen provide understanding to inform effective injury mitigation?

INTRODUCTION

Musculoskeletal injuries (MSKIs) are common during military and other occupational physical training programmes, and employers have a duty of care to mitigate this injury risk. MSKIs account for a high number of working days lost during initial military training, contribute to training attrition and impact training costs. Poorer movement quality may be associated with increased MSKI risk.

METHODS

The present study evaluated the relationship between the Functional Movement Screen (FMS) Score, as a measure of movement quality, and injury risk in Royal Navy (RN) recruits. A cohort of 957 recruits was assessed using the FMS prior to the 10-week phase I training programme. Injury occurrence, time, type and severity were recorded prospectively during the training period.

RESULTS

Total FMS Score was associated with injury risk (p≤0.001), where recruits scoring ≥13 were 2.6 times more likely to sustain an injury during training. However, FMS Score accounted for only 10% of the variance in injury risk (R2=0.1). Sex was the only additional variable to significantly affect the regression model. Mean FMS Scores for men (14.6±2.3) and women (14.4±2.4) were similar, but injury occurrence in women was 1.7 times greater than in men. Examining the influence of individual FMS movement tests on injury prediction did not improve the model, where those movements that significantly contributed to injury prediction only accounted for a small amount of the variance (R2=0.01).

CONCLUSION

There was a weak relationship between FMS and injury risk in RN recruits. Evidence is provided that FMS score alone would not be appropriate to use as an injury prediction tool in military recruits.

Influence of smoking status on acute biomarker responses to successive days of arduous military training

INTRODUCTION

Habitual smoking is highly prevalent in military populations despite its association with poorer training outcomes. Smoking imposes challenges on the immune and endocrine systems which could alter how smokers acutely respond to, and recover from, intensive exercise particularly over multiple days of training.

METHODS

Over a two-day period, 35 male British Army recruits (age 22±3 years; mass 76.9±8.0 kg; height 1.78±0.06 m; 15 smokers) completed a 16.1 km loaded march (19.1 kg additional mass) on the first morning and a best-effort 3.2 km 'log race' (carrying a 60 kg log between six and eight people) on the subsequent morning. Blood samples were obtained on waking and immediately postexercise on both days and analysed for C reactive protein (CRP), interleukin 6 (IL-6), testosterone to cortisol ratio and insulin-like growth factor 1 (IGF-1).

RESULTS

Independent of smoking group, the exercise bouts on both days evoked significant increases in IL-6 (p<0.001) and decreases in testosterone to cortisol ratio (p<0.05). CRP concentrations on day 2 were significantly higher than both time points on day 1 (p<0.001), and a 9% decline in IGF-1 occurred over the two-day period, but was not significant (p=0.063). No significant differences were observed between smokers and non-smokers (p>0.05).

CONCLUSIONS

Military-specific tasks elicited inflammatory and endocrine responses, with systemic CRP and IGF-1 indicating that the physiological stress generated during the first training day was still evident on the second day. Despite the well-established impacts of smoking on resting levels of the markers examined, responses to two days of arduous military-specific training did not differ by smoking status.

Test-Retest Reliability and Translation of the Musculoskeletal Screening Protocol Questionnaire Used in the Swedish Armed Forces

INTRODUCTION

Musculoskeletal disorders (MSDs) in military personnel are common, and it is important to identify those at risk so that appropriate preventive and rehabilitative strategies can be undertaken. The Musculoskeletal Screening Protocol (MSP) questionnaire is part of the implemented prevention strategy to reduce MSDs in the Swedish Armed Forces. The aims of this study were to evaluate the questionnaire's reliability and to translate it into English.

MATERIALS AND METHODS

One-week test-retest reliability of the questionnaire was evaluated in a sample of 35 Swedish military personnel. Reliability was evaluated by calculations of Cohen's kappa or quadratic-weighted kappa. Percent agreement was used as a parameter for measurement error. Translation into English included forward and backward translations and expert committee discussions.

RESULTS

Kappa values relating to physical complaints/injuries were excellent (>0.75) except for knee and lower leg MSDs and for the intensity ratings, where Kappa values were mostly interpreted as fair-to-good (0.4-0.75). Kappa values of items pertaining physical performance, physical activity and exercise, eating and tobacco habits, sleep, and perceived health ranged between 0.72 and 1. Kappa values for feeling mentally or physical prepared were 0.47 and 0.65, respectively. Most percentage agreement values ranged between 90% and 100%. The English version was found to be satisfactorily equivalent to the Swedish MSP questionnaire.

CONCLUSION

The Swedish MSP questionnaire was found to be highly reliable and was satisfactorily translated into English. This provides support for the questionnaire's ability to trustworthily capture the prevalence of MSDs and perceived health in military personnel. Future research is warranted on the psychometric properties of the English MSP questionnaire.

The Development of the College Students' Experience of Family Harmony Questionnaire (CSEFHQ)

The experience of family harmony, as an individual's subjective evaluation of harmonious family relations, has an important influence on the development of their physical and mental health. This study aimed to develop the College Students' Experience of Family Harmony Questionnaire that is fit for college students in China. On the basis of literature analysis and survey with questionnaires, five pairs of opposite assessment indexes were constructed in this paper, namely, Atmosphere of family (getting along vs. conflict), Responsibility to housework (undertaking housework vs. refusing housework), Time-sharing (sharing vs. self-isolatedness), Seeking help (help-seeking vs. avoidance), and Supporting family members (support-providing vs. indifference). Items of this questionnaire were collected from investigation, relevant scales, and discussion with experts. Here, 562 college students were selected for the pre-test and 696 for the formal test. The results showed that, except for the dimension of refusing housework, which has been deleted, other dimensions remain unchanged, and the final nine dimensions accounted for 66.03% of variance variation. Furthermore, the result of confirmatory factor analysis indicates that the model fit well with the data in construct validity [χ²/df = 2.71, Incremental Fit Index (IFI) = 0.90, Tucker–Lewis Index (TLI) = 0.89, Comparative Fit Index (CFI) = 0.90, root mean square error of approximation (RMSEA) = 0.05, standardized root mean square residual (SRMR) = 0.05]. The Cronbach's alpha (α) coefficient of this questionnaire was 0.97. The split-half reliability was 0.92, and the test–retest reliability was 0.75 for the total questionnaire. The total score of the questionnaire was significantly positively correlated with the total score of family function, family cohesion, family adaptability, and well-being (r = 0.73, 0.71, 0.75, 0.51, respectively, all p < 0.01), and it had a significant negative correlation with loneliness (r = −0.56, p < 0.01). The results showed that the final structure was reasonable, and reliability and validity conformed to the requirements of psychometrics. Therefore, the questionnaire developed in this study can be used as a valid instrument for assessing the experience of family harmony among college students in China.

Reliability of Self-Administered Questionnaire on Dietary Supplement Consumption in Malaysian Adolescents

The repeatability of most questionnaires utilized in previous studies related to the consumption of dietary supplements (DS) among youth has not been well documented. Thus, a simple and easy-to-administer questionnaire to capture the habitual use of DS in the past one year known as the dietary supplement questionnaire (DiSQ) was developed and supported with external reliability evaluation. Analyses were done based on a convenience sample of 46 secondary school students. To elicit information regarding the intake of DS, the questionnaire was partitioned into two domains. The first domain was used to identify vitamin/mineral (VM) supplements, while the second domain was utilized to identify non-vitamin/non-mineral (NVNM) supplements. Cohen’s kappa coefficient (k) was used to evaluate the test–retest reliability of the questionnaire. Questionnaire administration to the respondents was done twice whereby a retest was given two weeks after the first test. Between test and retest, the reliability of individual items ranged from moderate to almost perfect for the VM (k = 0.53–1.00) and NVNM (k = 0.63–1.00) domains. None of the items had “fair” or ”poor” agreement. Various correlation coefficients can be obtained for the DiSQ but are generally reliable over time for assessing information on the consumption of supplements among the adolescent population.

The test-retest reliability of the Military Physical Loading Questionnaire (MPLQ)

INTRODUCTION

Despite the high prevalence of musculoskeletal injuries, there is a shortage of data quantifying the risk factors attributable to cumulative occupational demands among UK Military personnel. We developed a new comprehensive questionnaire that examines occupational and operational physical loading during military service. The aim of this study was to examine the test-retest reliability of the Military Physical Loading Questionnaire (MPLQ).

METHODS

Intraclass correlation coefficients (ICC) were used to evaluate the test-retest reliability (4-week interval) of the MPLQ on 18 occupational and 18 operational items in 50 male (mean age: 36 years; SD ±7.9) UK military personnel. A stratified analysis based on duration of Service (0-10 years, 11-20 years and ≥21 years) was conducted to assess whether stability of task items was dependent on participant length of recall. Internal consistency was assessed by Cronbach's alpha (α) coefficients.

RESULTS

Reliability of individual operational items ranged from fair to almost perfect agreement (ICC range: 0.37-0.89; α range: 0.53-0.94) with most items demonstrating moderate to substantial reliability. Overall scores related to occupational items showed substantial to almost perfect agreement between administrations (ICC range: 0.73-0.94; α range: 0.84-0.96). Stratifying by duration of Service showed similar within group reliability to the entire sample and no pattern of decreasing or increasing reliability with length of recall period was observed.

CONCLUSIONS

It is essential that data used in planning UK military policy and health services are as accurate as possible. This study provides preliminary support for the MPLQ as a reliable self-report instrument for assessing the cumulative lifelong effects of occupational loading in UK military personnel. Further validation studies using larger and more demographically diverse military populations will support its interpretation in future epidemiological research.

Lifestyle behaviours and perceived well-being in different fire service roles

Background

Aspects of the work environment influence employee well-being. However, it is unclear how employee lifestyle behaviours, health characteristics and well-being may differ within a broader occupational sector.

Aims

To investigate the health characteristics, lifestyle behaviours and well-being of three Fire and Rescue Service (FRS) occupational groups that differ in shift work and occupational demands: operational firefighters (FF), emergency control (EC) and administrative support (AS) workers.

Methods

Data were obtained via an online survey using previously validated questionnaires to assess health characteristics, lifestyle behaviours and perceived well-being. Differences between groups were explored, controlling for confounding variables, using analysis of covariance (ANCOVA) methods. Effect sizes are reported where appropriate to demonstrate clinical significance.

Results

Four thousand five hundred and sixty-four FRS personnel volunteered, with 3333 (73%) completing the survey out of a total workforce of 60000 (8%). FF reported the lowest prevalence of chronic medical conditions (10%), compared with AS (21%) and EC (19%) workers. Total physical activity (PA) was 66% higher among FF compared with EC and AS workers. Components of sleep and self-rated health were independent predictors of well-being irrespective of FRS role.

Conclusions

FF reported the highest levels of PA and highest perceptions of well-being, and the lowest prevalence of obesity and chronic medical conditions, compared with other FRS occupational groups. These findings may be used to inform FRS workplace intervention strategies.

Tapering Practices of Strongman Athletes: Test-Retest Reliability Study

BACKGROUND

Little is currently known about the tapering practices of strongman athletes. We have developed an Internet-based comprehensive self-report questionnaire examining the training and tapering practices of strongman athletes.

OBJECTIVE

The objective of this study was to document the test-retest reliability of questions associated with the Internet-based comprehensive self-report questionnaire on the tapering practices of strongman athletes. The information will provide insight on the reliability and usefulness of the online questionnaire for use with strongman athletes.

METHODS

Invitations to complete an Internet questionnaire were sent via Facebook Messenger to identified strongman athletes. The survey consisted of four main areas of inquiry, including demographics and background information, training practices, tapering, and tapering practices. Of the 454 athletes that completed the survey over the 8-week period, 130 athletes responded on Facebook Messenger indicating that they intended to complete, or had completed, the survey. These participants were asked if they could complete the online questionnaire a second time for a test-retest reliability analysis. Sixty-four athletes (mean age 33.3 years, standard deviation [SD] 7.7; mean height 178.2 cm, SD 11.0; mean body mass 103.7 kg, SD 24.8) accepted this invitation and completed the survey for the second time after a minimum 7-day period from the date of their first completion. Agreement between athlete responses was measured using intraclass correlation coefficients (ICCs) and kappa statistics. Confidence intervals (at 95%) were reported for all measures and significance was set at P<.05.

RESULTS

Test-retest reliability for demographic and training practices items were significant (P<.001) and showed excellent (ICC range=.84 to .98) and fair to almost perfect agreement (κ range=.37-.85). Moderate to excellent agreements (ICC range=.56-.84; P<.01) were observed for all tapering practice measures except for the number of days athletes started their usual taper before a strongman competition (ICC=.30). When the number of days were categorized with additional analyses, moderate reliability was observed (κ=.43; P<.001). Fair to substantial agreement was observed for the majority of tapering practices measures (κrange=.38-.73; P<.001) except for how training frequency (κ=.26) and the percentage and type of resistance training performed, which changed in the taper (κ=.20). Good to excellent agreement (ICC=.62-.93; P<.05) was observed for items relating to strongman events and traditional exercises performed during the taper. Only the time at which the Farmer's Walk was last performed before competition showed poor reliability (ICC=.27).

CONCLUSIONS

We have developed a low cost, self-reported, online retrospective questionnaire, which provided stable and reliable answers for most of the demographic, training, and tapering practice questions. The results of this study support the inferences drawn from the Tapering Practices of Strongman Athletes Study.

Low fitness, low body mass and prior injury predict injury risk during military recruit training: a prospective cohort study in the British Army

Background

Injuries sustained by military recruits during initial training impede training progression and military readiness while increasing financial costs. This study investigated training-related injuries and injury risk factors among British Army infantry recruits.

Methods

Recruits starting infantry training at the British Army Infantry Training Centre between September 2008 and March 2010 were eligible to take part. Information regarding lifestyle behaviours and injury history was collected using the Military Pre-training Questionnaire. Sociodemographic, anthropometric, physical fitness and injury (lower limb and lower back) data were obtained from Army databases. Univariable and multivariable Cox regression models were used to explore the association between time to first training injury and potential risk factors.

Results

58% (95% CI 55% to 60%) of 1810 recruits sustained at least 1 injury during training. Overuse injuries were more common than traumatic injuries (65% and 35%, respectively). The lower leg accounted for 81% of all injuries, and non-specific soft tissue damage was the leading diagnosis (55% of all injuries). Injuries resulted in 122 (118 to 126) training days lost per 1000 person-days. Slower 2.4 km run time, low body mass, past injury and shin pain were independently associated with higher risk of any injury.

Conclusions

There was a high incidence of overuse injuries in British Army recruits undertaking infantry training. Recruits with lower pretraining fitness levels, low body mass and past injuries were at higher risk. Faster 2.4 km run time performance and minimal body mass standards should be considered for physical entry criteria.

Exercise-Induced Tendon and Bone Injury in Recreational Runners: A Test-Retest Reliability Study

Background

Long-distance runners are prone to injuries including Achilles tendinopathy and medial tibial stress syndrome. We have developed an Internet comprehensive self-report questionnaire examining the medical history, injury history, and running habits of adult recreational runners.

Objective

The objective of the study was to evaluate two alternative forms of test-retest reliability of a comprehensive self-report Internet questionnaire retrospectively examining the medical history, injury history, and running habits among a sample of adult recreational runners. This will contribute to the broad aims of a wider study investigating genetics and running injury.

Methods

Invitations to complete an Internet questionnaire were sent by email to a convenience pilot population (test group 1). Inclusion criteria required participants to be a recreational runner age 18 or over, who ran over 15 km per week on a consistent basis. The survey questions addressed regular running habits and any injuries (including signs, symptoms, and diagnosis) of the lower limbs that resulted in discontinuation of running for a period of 2 consecutive weeks or more, within the last 2 years. Questions also addressed general health, age, sex, height, weight, and ethnic background. Participants were then asked to repeat the survey using the Internet platform again after 10-14 days. Following analysis of test group 1, we soft-launched the survey to a larger population (test group 2), through a local running club of 900 members via email platform. The same inclusion criteria applied, however, participants were asked to complete a repeat of the survey by telephone interview after 7-10 days. Selected key questions, important to clarify inclusion or exclusion from the wider genetics study, were selected to evaluate test-retest reliability. Reliability was quantified using the kappa coefficient for categorical data.

Results

In response to the invitation, 28 participants accessed the survey from test group 1, 23 completed the Internet survey on the first occasion, and 20 completed the Internet retest within 10-21 days. Test-retest reliability scored moderate to almost perfect (kappa=.41 to .99) for 19/19 of the key questions analyzed. Following the invitation, 122 participants accessed the survey from test group 2, 101 completed the Internet survey on the first occasion, and 50 were randomly selected and contacted by email inviting them to repeat the survey by telephone interview. There were 33 participants that consented to the telephone interview and 30 completed the questionnaire within 7-10 days. Test-retest reliability scored moderate to almost perfect for 18/19 (kappa=.41 to .99) and slight for 1/19 of the key questions analyzed.

Development of a questionnaire to evaluate practitioners' confidence and knowledge in primary care in managing chronic kidney disease

Background

In the UK, chronic disease, including chronic kidney disease (CKD) is largely managed in primary care. We developed a tool to assess practitioner confidence and knowledge in managing CKD compared to other chronic diseases. This questionnaire was part of a cluster randomised quality improvement interventions in chronic kidney disease (QICKD; ISRCTN56023731).

Methods

The questionnaire was developed by family physicians, primary care nurses, academics and renal specialists. We conducted three focus groups (n = 7, 6, and 8) to refine the questionnaire using groups of general practitioners, practice nurses and trainees in general practice. We used paper based versions to develop the questionnaire and online surveys to test it. Practitioners in a group of volunteer, trial practices received the questionnaire twice. We measured its reliability using Cohen’s Kappa (K).

Results

The practitioners in the focus groups reached a consensus as to the key elements to include in the instrument. We achieved a 73.1% (n = 57/78) initial response rate for our questionnaire; of these 57, 54 completed the questionnaire a second time. Family physicians made up the largest single group of respondents (47.4%, n = 27). Initial response showed more female (64.9%, n = 37) than male (35.1%, n = 20) respondents. The reliability results from retesting showed that there was moderate agreement (k > 0.4) on all questions; with many showing substantial agreement (k > 0.6). There was substantial agreement in the questions about loop diuretics (k = 0.608, CI 0.432-0.784, p < 0.001), confidence in managing hypertension (k = 0.628, 95%CI 0.452-0.804, p < 0.001), diastolic blood pressure treatment thresholds in CKD (k = 0.608, 95%CI 0.436-0.780, p < 0.001) and the rate of decline of eGFR that would prompt referral (k = 0.764, 95%CI 0.603-0.925, p < 0.001).

Conclusion

The QICKD-CCQ is a reliable instrument for measuring confidence and knowledge among primary care practitioners on CKD management in the context of UK primary care.

Reliability, validity, and significance of assessment of sense of contribution in the workplace

The purpose of this study was to assess the validity and reliability of the Sense of Contribution Scale (SCS), a newly developed, 7-item questionnaire used to measure sense of contribution in the workplace. Workers at 272 organizations answered questionnaires that included the SCS. Because of non-participation or missing data, the number of subjects included in the analyses for internal consistency and validity varied from 1,675 to 2,462 (response rates 54.6%–80.2%). Fifty-four workers were included in the analysis of test–retest reliability (response rate, 77.1%). The SCS showed high internal consistency (Cronbach’s α coefficients in men and women were 0.85 and 0.86, respectively) and test–retest reliability (intraclass correlation coefficient = 0.91). Significant (p < 0.001), positive, moderate correlations were found between the SCS score and scores for organization-based self-esteem and work engagement in both genders, which support the SCS’s convergent and discriminant validity. The criterion validity of the SCS was supported by the finding that in both genders, the SCS scores were significantly (p < 0.05) and inversely associated with psychological distress and sleep disturbance in crude and in multivariable analyses that adjusted for demographics, organization-based self-esteem, work engagement, effort–reward ratio, workplace bullying, and procedural and interactional justice. The SCS is a psychometrically satisfactory measure of sense of contribution in the workplace. The SCS provides a new and useful instrument to measure sense of contribution, which is independently associated with mental health in workers, for studies in organizational science, occupational health psychology and occupational medicine.