Vocational assessment: a review of the literature from an occupation-based perspective

Evaluation of psychometric properties of Turkish version of assessment of work performance among adults with mild intellectual disabilities

BACKGROUND

Assessment of work performance (AWP) is a test for assessing work performance and can be utilised in matching person-work and applying effective interventions. It was aimed to adjust and to evaluate the validity and reliability of AWP in Turkish with individuals with mild intellectual disability (ID).

METHODS

The study was conducted with 243 individuals with mild ID. Of the 243 participants, data gathered from 105 individuals was used to conduct exploratory factor analysis (EFA) and remaining data (138 individuals) was used to conduct confirmatory factor analysis. Internal consistency, construct validity, test-retest reliability, criterion-related validity and inter-rater reliability were analyzed.

RESULTS

The AWP showed average to good reliability for individuals with mild ID. The construct validity and internal consistency of the AWP were confirmed with satisfying results; test-retest reliability (ICC >0.92) and inter-rater reliability (ICC >0.90) were excellent for all domains. The EFA resulted three factors, explaining 66.46% variance; and CFA resulted average fit.

CONCLUSION

AWP is valid and reliable for evaluating the working performances of individuals with mild ID.

Construct validity of the Danish version of the Work Rehabilitation Questionnaire (WORQ)-sensitivity and specificity of the objectively tested physical capacity items

Objective

The aim of this study was to examine the construct validity of the Danish version of the Work Rehabilitation Questionnaire (WORQ) and to compare the physical capacity items of WORQ to objective, standardized measures of physical capacity and selected SF-36 physical items.

Methods

The study took place at a job center in Holbæk municipality, and 40 clients of working age were enrolled. Participants completed the interviewer-administered version of WORQ, selected SF-36 items, and underwent objective, physical capacity testing, including a 30-s sit-to-stand-test, a hand-grip-strength test, and a 6-min walk test to estimate cardiorespiratory fitness. Correlations between variables were assessed using Spearman's correlation. Further, cross tabulations and chi-square tests were conducted, and sensitivity, specificity, positive predictive values (PPVs), and negative predictive values (NPVs) were calculated.

Results

We found a moderate to strong correlation between WORQ and SF-36 items and a weak to moderate correlation between physical capacity items of WORQ and objectively tested physical capacity measures. On the basis of cross tabulations, calculations yielded overall higher NPVs than PPVs, whereas sensitivity and specificity varied more, with not one parameter being overall better than the other.

Conclusion

We found evidence of construct validity of the WORQ-Danish. However, our study might also raise a question as to whether objective physical capacity tests are the gold standard for evaluating functioning. Our results are promising, and we suggest further investigations of the screening capabilities of WORQ, alongside other legacy measures or instruments, both self-reported and objective physical measures, to complement information-where specific answers to specific questions trigger work-related actions or interventions.

Assessing work speed using MODAPTS: A tool for occupational therapists

BACKGROUND

The importance to be able to work is recognised as one of the United Nation's Sustainable Development Goals. Occupational therapists working in vocational rehabilitation require cost effective, valid, and reliable tests that offer flexibility during the work evaluation process. The use of self-developed work samples standardized using Modular Arrangement of Predetermined Time Standards (MODAPTS) as an objective measure of work speed could meet this need.

OBJECTIVE

Evaluate the face, content, and criterion validity of MODAPTS work samples to assess work speed.

METHODS

We conducted a quantitative cross-sectional, descriptive study. Occupational therapists completed electronic surveys to evaluate face and content validity. We evaluated criterion validity during a multiphase process that involved a realist synthesis, online survey, and comparing MODAPTS to Methods Time Management (MTM) time standards using three work samples that measured work speed.

RESULTS

MODAPTS had good face validity for measuring work speed. We also confirmed the content and criterion validity of codes used to analyse basic movement and handling of smaller and larger articles as well as other body actions.

CONCLUSION

MODAPTS demonstrated adequate face, content and criterion validity of tasks involving basic movements, handling of articles and other body actions.

Translation and validation of Assessment of Work Performance (AWP) into the Icelandic language and culture

BACKGROUND

Evidence-based services in vocational rehabilitation call for valid and reliable assessments of work performance for intervention planning and safe return to work for individuals with illness or injuries. Assessments that are easy to use, culturally adapted, and in a language professionals and clients understand is important for their utility. Translation and adaption of assessments to new languages and cultures are of importance to maintain high standards of assessments and the quality of services in the vocational rehabilitation setting.

OBJECTIVE

The aim of this study was to translate and adapt the Assessment of Work Performance (AWP), into the Icelandic language and culture. Furthermore, to investigate the content validity and utility of the evaluation instrument.

METHODS

The study was conducted in two phases. The first phase, based on the Dual-Panel translation method, included two expert panels performing translation and cultural adaption of the original Swedish AWP. In the second phase field testing of the translated instrument was followed by an online survey among vocational rehabilitation professionals for content validation and utility.

RESULTS

The results of the study showed satisfactory opinions from experienced professionals toward the content validity, translation, cultural adaption, and utility of the Icelandic version of the AWP.

CONCLUSIONS

The study indicates that the Dual-Panel translation method is a valid method when translating instruments from one language and culture to another. Furthermore, the results of the study imply that the AWP can be regarded as content valid up to an acceptable degree for implementation in vocational rehabilitation services in Iceland.

Is a structured work task application for the assessment of work performance in a constructed environment, useful for patients with attention deficits?

BACKGROUND

The purpose of our study was to develop a Structured Work Task application for the Assessment of Work Performance for patients with attention deficits.

MATERIAL AND METHODS

We developed a computer-based registration task titled the Attention-demanding Registration Task. It had a structured administrative procedure with additional scoring regarding time and accuracy, also linked to the original scoring of the Assessment of Work Performance. We evaluated the Attention-demanding Registration Task for content validity. Furthermore, we investigated it concerning sensitivity and specificity in patients with attention deficits due to acquired brain injury (n = 65) against a comparison group of healthy people (n = 47).

RESULTS

Our investigation on content validity using the Assessment of Work Characteristics confirmed that the Attention-demanding Registration Task sets high demands on process skills, especially on energy, temporal organization, and adaptation. The Attention-demanding Registration Task showed high sensitivity and specificity in differing between patients with attention deficits and a healthy working group; nine out of ten participants were placed in the correct group.

CONCLUSIONS

To assess work performance, the use of a Structured Work Task application, the Attention-demanding Registration Task, linked with the Assessment of Work Performance, proved to be sensitive to attention deficits.Implications for rehabilitationA Structured Work Task application for the Assessment of Work Performance was developed for use in people with attention deficits and showing a high degree of sensitivity and specificity.Linking performance time and accuracy to the Assessment of Work Performance scoring and providing a guide for linking task performance to the Assessment of Work Performance skills in addition to the usual observations performed, may increase scoring accuracy.Reference data for a comparison group of healthy subjects are provided.The use of the Attention demanding Registration Task, while using the Assessment of Work Performance within clinical practice ensures a more accurate description of process skills in performance.

Examining changes in occupational participation in forensic patients using the Model of Human Occupation Screening Tool

Introduction

In occupational therapy, there has been an increased interest in patients’ occupational participation within forensic settings. This retrospective study involved a longitudinal analysis of occupational participation within six forensic hospitals in England. The aim was to contribute to the understanding of forensic patients’ occupational participation over a two-year period.

Methods

The Model of Human Occupation Screening Tool (MOHOST) was rated by 78 occupational therapists on 489 patients in low and medium secure units who were receiving occupational therapy over two years. The many-faceted Rasch Model was used to convert their MOHOST scores at each time point into interval scales. Regression analysis was used to examine changes in occupational participation over time.

Results

Patients’ overall occupational participation improved over time. Specifically, participation improved in five of the six MOHOST subdomains, which included their motivation for occupation, pattern of occupation, communication/interaction skills, process skills, and environment. Patients did not demonstrate significant change in their motor skills, which varied as expected. In addition, patients in low secure units had better occupational participation than those in medium secure settings.

Conclusion

Our findings indicated improvements in the patients’ occupational participation over the 2-year period. Further investigations are needed to understand factors contributing to change.

An investigation of the inter-rater reliability of the Valpar Joule functional capacity evaluation in healthy adults

BACKGROUND

A functional capacity evaluation (FCE) can provide a comprehensive, objective measure of a worker's ability to meet work demands to support return to work decision making. Research evidence of a FCE's reliability and validity, involving more than one study, and covering all test components with a diverse range of populations, is essential to ensure confidence in any FCE system.

OBJECTIVE

This study aimed to establish the inter-rater reliability of the Valpar Joule FCE functional capacity evaluation (FCE) for which there is currently limited published literature regarding its reliability.

METHODS

Twelve healthy subjects were digitally recorded completing the initial protocol of the Valpar Joule. Assessments were rated separately by 3 raters and the results then compared.

RESULTS

Using Intraclass Correlation Coefficients (ICC), with percentages of agreement and t-tests to determine bias, inter-rater reliability was high for determining last safe weight lifted for forceful tasks with ICC>0.90. Agreement ranged from 97.2% -100% for determining reasons for terminating tests; 97.2% -98.6% for identifying maximum safe capacity, but was only between 8.3% -50% for full agreement for identification of last weight safely lifted in forceful tasks. Differences were identified between raters with different training and experience for identifying poor body mechanics in lifting.

CONCLUSION

Results demonstrated high inter-rater reliability for the Valpar Joule functional capacity evaluation in healthy adults. Further development of criteria identifying poor body mechanics and training in its use is recommended to increase evaluator objectivity.

Clinical utility of the worker role interview: a survey study among Swedish users

BACKGROUND

Work ability assessments are used to guide interventions designed to reduce or eliminate work disability and facilitate work participation. The clinical utility of assessments concerns the extent to which the assessment results support clinical decision-making.

AIM

The aim was to investigate how users perceive the clinical utility of the Swedish version of the Worker Role Interview (WRI-S). An additional aim was to investigate differences in clinical utility related to the users' level of WRI-S experience, attendance at courses on WRI-S application, and knowledge of the Model of Human Occupation.

MATERIAL AND METHODS

Data are based on 187 respondents' answers to a web-based questionnaire and analysed through descriptive and inferential statistics.

RESULTS

In total, the respondents performed 12,000 WRI-S assessments. The respondents were mainly satisfied with the clinical utility of the WRI-S and experienced the assessment as supporting clinical practice. The main hindrance affecting clinical utility was the amount of time needed for conducting a WRI-S assessment. Experienced users and users with very good knowledge of MOHO conduct WRI-S assessments in a more time-effective manner.

CONCLUSION

This study contributes to further development of the clinical utility of the Swedish version of the WRI-S, mainly focusing on time consumption.

Re/creating entrepreneurs of the self: discourses of worker and employee 'value' and current vocational rehabilitation practices

Vocational rehabilitation for people experiencing work disability is a social practice often situated within health services, but the social and political drivers and effects of this practice are rarely critically analysed in health research or policy. In this study we used a Foucauldian theoretical perspective to analyse the ways in which current vocational rehabilitation practices in New Zealand re/produce notions of worker and employee 'value', and how different approaches to vocational rehabilitation deploy current discourses about value. We also consider the subject positions produced through these different approaches and the identities and actions they make possible for people experiencing work disability. The analysis showed that notions about the importance of worker and employee value in a job market are pervasive in vocational rehabilitation, and reflect wider societal discourses. However, the deployment of those discourses in different approaches to vocational rehabilitation practice are diverse, producing different opportunities and constraints for people experiencing disability. We argue that an examination of these various opportunities and constraints at the level of practice approaches is important, as considerable time and resources are allocated to developing solutions to help those who do not thrive in the current systems, yet we rarely critique the premises on which the systems are based.

Evaluating the Psychometric Properties of a Clinical Vocational Rehabilitation Outcome Measurement: The Assessment of Work Performance (AWP)

This study examined the validity and reliability of the Assessment of Work Performance (AWP) using Rasch analysis. The AWP was administered to 365 clients with a variety of work-related problems. Rasch analysis and principal component analysis were used to examine the appropriateness of the rating scales and unidimensionality of AWP items. The person-response validity, internal consistency, targeting appropriateness, and differential item function were also analyzed. The Rasch analysis confirmed the 4-point rating scale, and the item set met the criteria of unidimensionality. The AWP exhibited satisfactory person-response validity and internal consistency. Among the three subdomains, the targeting of item-difficulty was sufficient in the motor skills and process skills subdomains. Differential item functioning was found across gender and diagnoses. This study presented evidence to support that the AWP functioned as a reliable and valid assessment in assessing work performance. [OTJR: Occupation, Participation and Health 2013;33(3):125-133.].

The impact of adjustment latitude on self-assessed work ability in regard to gender and occupational type

OBJECTIVE

Adjustment latitude describes opportunities to change demands at work when ill and may affect work ability. The aim here is to study the association between adjustment latitude and self-assessed work ability among men and women and employees from different occupational sectors.

METHODS

This cross-sectional study used data from a questionnaire sent to 3020 employees in three occupational sectors in Sweden; 1430 responded. Subjects were divided into: full, moderately reduced, and greatly reduced work ability. Presence of nine adjustment opportunities was requested and subjects were divided into three groups. Each specific opportunity was also analyzed in relation to work ability. Multinomial logistic regression was used for analyses.

RESULTS

Number of opportunities to adjust was associated with work ability among men and employees in health care. "Shortening the working day" was associated with work ability in most groups. For men and industrial employees, "postponing work", "going home and working later", and "working without disturbance" were associated with work ability. "To work from home" was associated with work ability among women and employees in insurance.

CONCLUSIONS

The assumption that adjustment latitude affects work ability is supported. Associations differ with regard to gender and occupational sectors. Further studies with longitudinal design and alternative samples are needed.

Examining the Validity of the Model of Human Occupation Screening Tool: Using Classical Test Theory and Item Response Theory

Introduction:

This study examined the psychometric properties of the Model of Human Occupation Screening Tool, using both item response theory and classical test theory.

Method:

One hundred and one people with mental health problems, aged 18–65 years, were recruited. The Chinese version of the Model of Human Occupation Screening Tool, the National Taiwan University Hospital Symptom Checklist, the Volitional Questionnaire, the Assessment of Communication and Interaction Skills, and the Mini Mental State Examination were administered. Rasch analysis and correlational analysis were used to examine the construct, convergent, divergent validity and known group validity.

Results:

Rasch analysis confirmed that there were six subscales within the Chinese version of Model of Human Occupation Screening Tool. The Volitional Questionnaire strongly correlated with the volition subscale (r = 0.583). The Assessment of Communication and Interaction Skills strongly correlated with the communication and interaction subscale (r = 0.815). The Mini Mental State Examination was moderately correlated with the process subscale (ρ = 0.334) and the symptomatology was not associated with any of the subscales as expected. There were significant differences on selected subscale scores across four known groups of participants.

Conclusion:

The Chinese version of the Model of Human Occupation Screening Tool was valid when applied to people with mental health problems.