Clinical Utility and Usability of the Digital Box and Block Test: Mixed Methods Study

Background

The Box and Block Test (BBT) is a clinical tool used to measure hand dexterity, which is often used for tracking disease progression or the effectiveness of therapy, particularly benefiting older adults and those with neurological conditions. Digitizing the measurement of hand function may enhance the quality of data collection. We have developed and validated a prototype that digitizes this test, known as the digital BBT (dBBT), which automatically measures time and determines and displays the test result.

Objective

This study aimed to investigate the clinical utility and usability of the newly developed dBBT and to collect suggestions for future improvements.

Methods

A total of 4 occupational therapists participated in our study. To evaluate the clinical utility, we compared the dBBT to the BBT across dimensions such as acceptance, portability, energy and effort, time, and costs. We observed therapists using the dBBT as a dexterity measurement tool and conducted a quantitative usability questionnaire using the System Usability Scale (SUS), along with a focus group. Evaluative, structured, and qualitative content analysis was used for the qualitative data, whereas quantitative analysis was applied to questionnaire data. The qualitative and quantitative data were merged and analyzed using a convergent mixed methods approach.

Results

Overall, the results of the evaluative content analysis suggested that the dBBT had a better clinical utility than the original BBT, with ratings of all collected participant statements for the dBBT being 45% (45/99) equal to, 48% (48/99) better than, and 6% (6/99) lesser than the BBT. Particularly in the subcategories "acceptance," "time required for evaluation," and "purchase costs," the dBBT was rated as being better than the original BBT. The dBBT achieved a mean SUS score of 83 (95% CI 76-96). Additionally, several suggested changes to the system were identified.

Conclusions

The study demonstrated an overall positive evaluation of the clinical utility and usability of the dBBT. Valuable insights were gathered for future system iterations. These pioneering results highlight the potential of digitizing hand dexterity assessments.

Occupational Therapy in Mental Health: A Review of the Literature

This review of the literature was conducted as part of the development of a position paper on the way ahead for research, education and practice in occupational therapy in mental health. It included publications over the past decade and concentrated on the British Journal of Occupational Therapy. Recognising that experience from other countries would be beneficial, aspects of the literature from the national journals in the USA, Canada and Australia which were most relevant to practice in the United Kingdom were included.

The shortage of articles on mental health was conspicuous and their diverse topics and descriptive nature were notable. In the United Kingdom literature, the clinical subjects featured most often were dementia, work rehabilitation, community mental health and forensic psychiatry, with other subjects either appearing infrequently or not included. The literature from the other countries also explored new roles for occupational therapy, reflecting the changes in the delivery of mental health services. In the American Journal of Occupational Therapy, there was concern about the declining number of occupational therapists working in mental health. The review identified major themes in the literature, posed questions for research, education and practice in occupational therapy and formed the foundation for the position paper.

Assessments of Activities of Daily Living and Instrumental Activities of Daily Living: Their Use by Community-Based Health Service Occupational Therapists Working in Physical Disability

A preliminary study was carried out to evaluate the appropriateness of activities of dally living (ADL) scales as outcome measures for a community-based multidiscipilnary service to physically disabled young people. A literature search was undertaken which raised several questions and concerns. To find out how other teams addressed these issues, telephone interviews were conducted with occupational therapists working in similar NHS teams. They were asked about the nature of their services, their use of ADL and ADL scales and whether these were administered as outcome measures.

A Comparison of Self-Report versus Observation of Performance Using the Assessment of Living Skills and Resources (ALSAR) with an Older Population

This study assessed the capacity of 15 older people to report their instrumental activities of daily living (IADL) function accurately. The Assessment of Living Skills and Resources (ALSAR) (Williams et al 1991) was used to measure IADL function in the clinical and home environments. The subject's self-reported function was also compared with performance-based measurement of three IADL tasks from the ALSAR (use of the telephone, meal preparation and medication management) in the subject's home environment. The sensitivity of the ALSAR 3-point scale was examined through comparison with the Medical Rehabilitation Follow Along scale (Granger et al 1995).

The findings highlighted problems associated with the use of self-reporting for the assessment of IADL in the older population. The ALSAR 3-point scale was shown to be sufficiently sensitive to detect IADL function when used with performance observation; however, further research is required to ascertain sensitivity when it is used as a self-report tool.

The findings suggest that occupational therapists attain more accurate results when carrying out IADL assessment procedures by using performance-based measures.

A Systematic Review of the Canadian Occupational Performance Measure: A Clinical Practice Perspective

Occupational therapists working to a philosophy of client-centred practice will benefit from a reliable and well reviewed outcome measure to provide an evidence base. A systematic review of the literature was undertaken to evaluate worldwide publications on the use of the Canadian Occupational Performance Measure (COPM) to determine its impact on clinical practice in occupational therapy.

A literature search strategy was deployed which resulted in a review of 64 articles. A themed analysis approach was used to integrate the results into linked and recurring themes. The focus of the systematic review was to concentrate on issues affecting clinical practice and the following were selected as key factors for consideration: negative and positive issues arising from the use of the COPM; impact on practice for the therapist, client, process and clinical work; and change initiatives for the therapist, client, process and clinical practice. A profile of all articles, which identifies where these factors are indicated, is presented as a summary guide to assist the clinician.

The findings of the review indicated that the greatest impact of using the COPM was within clinical practice. Change initiatives indicated the need for further research in different clinical areas, as well as the continual development and need for more training of occupational therapists in the use of COPM as an outcome measure.

Assessing the Outcomes of Equipment and Adaptations: Issues and Approaches

Current government policy emphasises professional accountability, Best Value, evidence-based practice and outcomes for users and carers. In this context, being able to demonstrate the effectiveness of routine practice is increasingly important.

This article considers the key issues in collecting information about the outcomes of equipment and adaptations. These include: the assessment of the full range of outcomes; the timing of data collection; the use of standardised and individualised measures; and linking outcomes to interventions. Three approaches to evaluating the outcomes of equipment and adaptations are distinguished: functional status measures; health status or quality of life measures; and indirect approaches. Examples of each approach are provided and their strengths and limitations discussed.

Previous work on the outcomes of equipment and adaptations has neglected the perspectives of carers and the impact of service delivery (service process outcomes); further work on these areas is needed to ensure that all the outcomes of equipment and adaptations are fully evaluated.

The Use of Standardised and Non-Standardised Assessments in a Social Services Setting: Implications for Practice

The increasing demand for occupational therapists to justify their practice and their clinical judgements stimulated consideration of the importance of the psychometric properties of the tools used by professionals in making clinical judgements. In social services settings a variety of procedures are used to establish a client's entitlement to services, some of which are more rigorous than others. In this study, a non-standardised assessment tool to measure severity of disability was compared with a standardised tool to assesswhether there were differences in outcomes and what, if any, were the consequences for service entitlement. The conclusion was that the use of non-standardised tools may unfairly disadvantage some client groups when being assessedfor services.

A Clinical Audit of the Outcome of Occupational Therapy Assessment and Negotiated Patient Goals in the Acute Setting

The increasing emphasis on quality and effectiveness in the National Health Service requires occupational therapists to provide evidence of accurate assessment and of the outcomes of interventions, supported by clear and concise documentation. With the merging of two acute hospitals came the opportunity for the occupational therapy services of Newcastle upon Tyne Hospitals NHS Trust to audit the process of functional assessment and documentation.

A review of the occupational therapy service was undertaken. Standards were set to combine the Modified Barthel Index with negotiated patient goal setting. Practice was then changed to meet the standards. A subsequent audit considered the outcome of occupational therapy.

The audit findings reaffirmed the acute occupational therapy focus on activities of daily living. The service identified that patients had low levels of disability according to Modified Barthel Index scores on discharge. Importantly, it was found that the reliability and validity of goal setting as a basic outcome measure was dependent upon therapists' ability to write specific quantifiable goals. The practice benefits included promoting patient involvement, enhancing the evidence base to practice, promoting consistent and effective functional assessment and documentation and reaffirming the unique contribution of occupational therapy as that of assessing a patient's level of function in activities of daily living objectively.

A comparison of goal attainment scaling and the Canadian Occupational Performance Measure for paediatric rehabilitation research

PURPOSE

To investigate the relative utility of Canadian Occupational Performance Measure (COPM) (adapted for children) and Goal Attainment Scaling (GAS) as outcome measures for paediatric rehabilitation.

METHODS

A two-group pre-post design investigated the impact of a 3-month programme. Forty-one children with spastic hemiplegic cerebral palsy (mean 3.9 years; GMPM level 1; 21 boys, 10 girls) were randomized to occupational therapy only and occupational therapy plus one Botulinum Toxin A injection. The latter was considered a 'proven' intervention for the purpose of this instrumentation study. Intervention impact was investigated using GAS and COPM. Instrument sensitivity, convergent validity, goal/problem profiles and administration were evaluated.

RESULTS

Both instruments were sensitive to within group change and detected significant between group change. Likert scale coding for GAS scores was more sensitive than the traditional weighted GAS or COPM. Different constructs were measured by each instrument. COPM was more time efficient in training, development and administration.

CONCLUSION

Study aim, logistic and resource factors should guide the choice of COPM and/or GAS instruments as both are sensitive to change with a proven intervention and both evaluate different constructs.

Goal attainment scaling allows program evaluation of a home-based occupational therapy program

A problem for occupational therapists involved in home-based rehabilitation is the dichotomy between traditional functional status measures, which are commonly reported by therapists to have insufficient flexibility and poor clinical sensitivity, and individualized methods (which do not allow inter-patient comparison for program evaluation). This study used Goal Attainment Scaling (GAS) to evaluate both individual and program change. The study demonstrates that GAS was a simple, clinically meaningful measure of change which evaluated patient-centered outcomes and program effects concurrently. In addition, this measure was applied to a traditionally challenging service, that of a home-based rehabilitation program. Goal Attainment Scaling was demonstrated to be useful as an outcome measure for patients with multiple, complex rehabilitation needs, such as those served by a home-based rehabilitation occupational therapy service. It was able to accurately detect meaningful change (necessary for demonstrating clinically significant improvement at the level of the individual patient), be administered in a time-effective manner, and also allow the clinician to evaluate program effectiveness.

An inter-rater reliability study of the Functional Assessment Measure (FIM+FAM)

There is an increasing demand for measures of outcome to evaluate the effects of rehabilitation interventions for brain injury from clinicians, research workers and healthcare providers and purchasers. The Functional Assessment Measure (FIM+FAM), an expanded derivative of the Functional Independence Measure (FIM), is designed specifically for this purpose for this patient group. This study examined the interrater reliability of the FIM+FAM between two independent raters, a physician and a nurse, the subjects being 30 in-patients in a neurological rehabilitation unit. The results show that the inter-rater reliability was good (kappa values 0.50 to 0.95) for all but one of the 30 items rated on the FIM+FAM. The exception (with a kappa value of 0.35) was "adjustment to limits'. Higher agreement was found for rating of physical activities than for cognitive, communication and behavioural items.