The development, content validity and inter-rater reliability of the SMART-Goal Evaluation Method: A standardised method for evaluating clinical goals

INTRODUCTION

Goal setting is a complex skill. The use of formal goal writing procedures (including the use of the SMART goal model) has been advocated. However, a standardised method of writing and evaluating SMART goals is currently lacking. This study comprised of two phases. The aims of phase one was to (i) develop the SMART Goal Evaluation Method (SMART-GEM) based on a SMART goal model; and (ii) investigate the content validity of the SMART-GEM. The aim of phase two of the study was to test the inter-rater reliability of the SMART-GEM.

METHODS

Development of the SMART- GEM involved defining and constructing evaluation criteria suitable for auditing goal statements. A content validity assessment was conducted using an expert panel of Occupational Therapists (n = 10). Inter-rater reliability of the SMART-GEM was examined using a purposive sample of multiple raters (n = 24).

RESULTS

The SMART- GEM was rated as having good content validity (individual items CVI ranged from 0.90 to 1.00; total SMART- GEM CVI = 0.99, ρ = 0.05). Agreement between raters on individual items ranged from poor (κ = 0.254) to excellent (κ = 0.965) and agreement of overall grades was fair to good (κ = 0.582). Inter-rater agreement on total scores was found to be very good (ICC = 0.895, 95% CI = 0.743 to 0.986, ρ = 0.001) with excellent internal consistency (α = 0.995).

CONCLUSION

The SMART-GEM demonstrated good construct validity and very good inter-rater reliability on total score and shows promise as a standardised method to writing and evaluating clinical goals.

An evaluation of Therapy Outcome Measures (TOMs) in community rehabilitation

Background/Aims:

This study considers the feasibility of using Therapy Outcome Measure (TOMs) in a community rehabilitation therapy team working with patients over 65 years of age.

Methods:

An action research approach was used. Data collection and analysis included completion of the outcome measure scores over a 22-week period through semi-structured individual interviews with the therapy team participants, group reflections and the researcher's reflections.

Results:

Sixty-seven patients were considered in this study, with 26 patient scores included. Nineteen (73%) of those scored for admission and discharge demonstrated sensitivity, with a difference of one score or more. However, the study participants agreed that the TOMs scales and descriptors did not best describe their patient group. They acknowledged that the study had included a small sample size.

Conclusions:

TOMs was agreed to be an ‘OK fit’ for community rehabilitation patients rather than a ‘best fit’. It should be used as part of a variety of outcome measures to evidence the multidimensional aspects of this speciality, where patients have complex needs and therefore require more complex interventions. This study also highlights the need for further research with a larger sample size.

The Relationship Between Client-Centered Goal-Setting and Treatment Outcomes

Shifting definitions of health and well-being, prompted by the World Health Organization's International Classification of Functioning (2001), have stimulated changes in traditional clinician-client relationships in rehabilitation. Among these changes, in keeping with the concept of client-centered care, is a trend toward more collaborative goal-setting and joint determination of intervention plans. Evidence suggests that supporting clients' autonomy in prioritizing personally meaningful goals leads to increased engagement in intervention, less emotional anxiety about the rehabilitation process, and improved treatment outcomes. Supporting people with aphasia in a process of collaborative goal formulation may also serve to alter treatment priorities so that they address more relevant communication challenges embedded in post-rehabilitation life.