Applying modern measurement approaches to constructs relevant to evidence-based practice among Canadian physical and occupational therapists

BACKGROUND

Evidence-based practice (EBP) is a complex process. To quantify it, one has to also consider individual and contextual factors using multiple measures. Modern measurement approaches are available to optimize the measurement of complex constructs. This study aimed to develop a robust measurement approach for constructs around EBP including practice, individual (e.g. knowledge, attitudes, confidence, behaviours), and contextual factors (e.g. resources).

METHODS

One hundred eighty-one items arising from 5 validated EBP measures were subjected to an item analysis. Nominal group technique was used to arrive at a consensus about the content relevance of each item. Baseline questionnaire responses from a longitudinal study of the evolution of EBP in 128 new graduates of Canadian physical and occupational therapy programmes were analysed. Principles of Rasch Measurement Theory were applied to identify challenges with threshold ordering, item and person fit to the Rasch model, unidimensionality, local independence, and differential item functioning (DIF).

RESULTS

The nominal group technique identified 70/181 items, and modified Delphi approach identified 68 items that fit a formative model (2 related EBP domains: self-use of EBP (9 items) and EBP activities (7 items)) or a reflective model (4 related EBP domains: attitudes towards EBP (17 items), self-efficacy (9 items), knowledge (11 items) and resources (15 items)). Rasch analysis provided a single score for reflective construct. Among attitudes items, 65% (11/17) fit the Rasch model, item difficulties ranged from - 7.51 to logits (least difficult) to + 5.04 logits (most difficult), and person separation index (PSI) = 0.63. Among self-efficacy items, 89% (8/9) fit the Rasch model, item difficulties ranged from - 3.70 to + 4.91, and PSI = 0.80. Among knowledge items, 82% (9/11) fit the Rasch model, item difficulties ranged from - 7.85 to 4.50, and PSI = 0.81. Among resources items, 87% (13/15) fit the Rasch model, item difficulties ranged from - 3.38 to 2.86, and PSI = 0.86. DIF occurred in 2 constructs: attitudes (1 by profession and 2 by language) and knowledge (1 by language and 2 by profession) arising from poor wording in the original version leading to poor translation.

CONCLUSIONS

Rasch Measurement Theory was applied to develop a valid and reliable measure of EBP. Further modifications to the items can be done for subsequent waves of the survey.

The Impact of Community Paediatric Occupational Therapy on Children with Disabilities and Their Carers

The need to demonstrate ‘best value’ in local authority settings is one of the remits of the white paper Modernising Social Services (Department of Health 1997). This study investigated the effectiveness of paediatric occupational therapy in terms of both increasing the independence of disabled children and reducing the strain of caring on the part of the carers. A before-and-after design was used, in which 33 children with a wide range of disabilities were assessed using two measures — the Community Dependency Index and the Canadian Occupational Performance Measure — prior to receiving and then following occupational therapy. The carers were interviewed separately, using three scales to assess the strain associated with their caring tasks.

The findings show a statistically significant shift in the children's scores indicating that the levels of independence improved for most of the children, although many remained dependent on their carers for a number of daily living tasks. While the carers found the occupational therapy service very satisfactory, the evidence suggests that it did not reduce the strain of caring.

Getting Evidence into Practice: Implementing a Behavioural Joint Protection Education Programme for People with Rheumatoid Arthritis

Researchers must disseminate their research findings, but this alone is often insufficient to change practice. To improve research implementation by rheumatology occupational therapists, a 2-day training course for changing joint protection practice was developed. This aimed to improve understanding of joint protection, patient education and health behaviour research; develop skills in delivering a group joint protection programme; and reduce potential barriers to changing practice. To evaluate the course's effectiveness, a postal survey with attendees was conducted.

Of the 48 respondents, 45 subsequently changed individual patient education practice; 13 implemented the group programme, 25 contemplated doing so and 10 did not (precontemplators). The implementers and contemplators rated the joint protection programme as significantly more effective (p = 0.02) and relevant (p = 0.001) in practice than the precontemplators. The barriers to change were limited staffing, access to facilities, time to make practice changes, funding for programme costs and clinical time to deliver the programme. The implementers considered that supportive managers and teams helped to overcome these.

Changing practice needs clinicians, researchers and managers to work together. This research implementation strategy helped most of the participating clinicians to make practice changes that were readily within their control (that is, individual patient education). Group education requires greater organisational change and manager support is important to facilitate this.

Pressure to Lead: What Can We Learn from the Theory?

Older people are major users of health and social services and, in the United Kingdom, the proportion of the population over 65 years is set to rise in the coming decades (Audit Commission 2000) with growing numbers of long-term conditions. In response, the National Health Service (NHS) must also change if it is to meet the needs of the nation. Whether in the public or the private sector, organisations must have leaders with the skills necessary to implement change if the organisation is to be successful (Landrum et al 2000). In the health care environment, leaders with the capacity to influence, shape and deliver services will enable the NHS to realise its organisational vision (Scottish Executive 2006).

Partnership for Care (Scottish Executive 2003) provides direction to achieve this goal by devolving power to frontline staff and promoting a culture of continuous improvement in the NHS. Occupational therapists form part of the frontline power responsible for implementing change and are in an ideal position to do so (Grady 1991). The author describes four theories of leadership — trait theory, motivational theories, transformational leadership and emotional intelligence — and reflects on their strengths and weaknesses and their relevance to occupational therapists currently working in the NHS.

The Efficacy of Neurodevelopmental Treatment in Paediatrics: A Systematic Review

Neurodevelopmental treatment (NDT) is widely used by clinicians when working with children diagnosed with neurological dysfunction, such as cerebral palsy, high-risk/low-birthweight infants or traumatic brain injury. The objective of this project was to determine the efficacy of NDT, used with a group of paediatric subjects diagnosed with a neurological dysfunction, through the completion of a systematic search and review of the relevant published research literature.

Using a computerised literature search strategy of nine databases, expert consultation, and a hand search of 14 relevant journals and reference lists, 147 relevant citations were identified for consideration. Of the 147 articles reviewed, 17 met the inclusion criteria for further review. These studies were assessed for concealment of treatment allocation (Shulz et al 1995), quality assessment of randomised clinical trial using the Jadad scale (Jadad et al 1996) and level of evidence using the Sackett scale (Sackett 1989). A level of agreement between the two independent reviewers was calculated for each step of the systematic review process using the kappa statistic (κ). Tables containing the details and status of the 17 studies are included.

Overall, the results regarding the efficacy of NDT were largely inconclusive since there were a similar number of published research studies supporting the benefit of NDT intervention (n=6) as compared with no benefit (n=9). One study did not specifically address the efficacy of NDT, while another study was unclear as to whether or not NDT was beneficial. The studies that included the use of NDT with paediatric subjects diagnosed with cerebral palsy also had inconsistent results regarding the efficacy of NDT. The studies that included the use of NDT with high-risk/low-birthweight infants did not support the usefulness of NDT with this paediatric client group. A detailed description of the systematic review process used in the completion of this paper is also included.

Occupational Therapy and Clinical Research in Mental Health Rehabilitation

Practitioners working in Australian mental health services are faced with the challenge of providing appropriate evidence-based interventions that lead to measurable improvement and good outcomes. Current government policy is committed to the development of strategic mental health research. One focus has been on under-researched practice areas, which include the development of psychosocial rehabilitation systems and models that facilitate recovery.

To meet this challenge, an Australian rehabilitation service formed a collaborative partnership with a university. The purposes of the collaboration were to implement new forms of service delivery based on consumer need and evidence and to design research projects to evaluate components of the rehabilitation programme. This article examines the process of developing the collaboration and provides examples of how research projects have been used to inform practice and improve the effectiveness of service delivery. Challenges to the sustainability of this kind of collaboration are considered.

A Survey of Australian Mental Health Occupational Therapists

Mental health reform in the western developed world has resulted in new models of care and changed work practices for all mental health professionals. Occupational therapists, as with other mental health professionals, have been required to assume new roles and responsibilities. Literature from the United Kingdom has reflected concern about this new way of working. The aim of this exploratory study was to examine the current work practices of and issues faced by Australian mental health occupational therapists. One hundred and forty-eight respondents (74%) answered an occupational therapy practice in mental health questionnaire.

The results from this survey suggest that there are two quite distinct groups of occupational therapists working in mental health settings in Australia. One group works as rehabilitation therapists in traditional activity-focused work roles. The other group works as case manager therapists and employs a much broader spectrum of clinical and support roles. The issues facing therapists include the development and maintenance of a clearly defined role, generic case management versus discipline-specific roles, recruitment and retention, the need for research and evidence-based practice, professional standing, and education and professional development.

The concerns over the role of occupational therapy in mental health were similar to those in previous British studies. The implications of these findings include a need for education and training at the undergraduate and postgraduate levels to equip mental health occupational therapists with both discipline-specific and generic skills.

The Use of Evidence-Based Practice by Occupational Therapists Who Treat Adult Stroke Patients

The study identified the extent to which members of the National Association of Neurological Occupational Therapists (NANOT) who treated adult stroke patients used evidence-based practice (EBP). A random sample of NANOT members (n = 200) was surveyed by a postal questionnaire on the use of EBP and the results were analysed for the 125 therapists who worked with adult stroke patients.

The principal factor that ‘greatly’ influenced the choice of intervention of three-quarters of the respondents was working with other therapists, followed by attendance at postgraduate courses; gaining postgraduate qualifications had the least influence. Although over 90% of the respondents considered that they used EBP at some time, the most frequently reported levels of evidence used were custom and practice and expert opinion, which were at the lower end of the hierarchy. Therefore, although the respondents who treated adult stroke patients did use EBP, they did not use the higher levels recommended. The respondents who used the higher levels of evidence were those who were most recently qualified. The factors influencing the use of EBP were the relevance of research to practice, time, lack of therapists' knowledge and lack of support.

Challenges to Measuring Outcomes in Occupational Therapy: A Qualitative Focus Group Study

The aim of this study was to explore the process used by occupational therapists to measure the effect of their interventions. A second aim was to explore the challenges associated with this process. Stroke rehabilitation was chosen as the focus for the study.

This qualitative study was conducted in Sydney, Australia. Ten occupational therapists from both hospital and community-based services participated in one of three focus groups. The data were analysed using constant comparison techniques.

The participants struggled when describing the process used to measure outcomes. They focused more on the challenges to measuring outcomes, which were described as focusing on occupation, deconstructing occupation and lacking knowledge, skills and assistance.

The participants in this study had difficulty in articulating and setting measurable goals linked to interventions. As a result, they could not easily select appropriate outcome measures. Education is needed to help such occupational therapists to develop the required skills and to begin to measure clinical outcomes. Further research should determine if similar experiences and challenges are faced in other areas of practice and by occupational therapists in other countries.

The Journal Club: A Method for Occupational Therapists to Bridge the Theory-Practice Gap

There are still major problems being encountered by occupational therapists when they attempt to access relevant research material. One of the main reasons for this is that there are currently relatively few occupational therapists who are involved in any form of research on a regular basis.

This article highlights the potential of the journal club as a means of introducing occupational therapists to research that will both be relevant to their work area and form a substantial part of their continuing professional development. A brief overview of the format and content of journal clubs both in occupational therapy and in other health care professions is intended to encourage occupational therapists to initiate their own journal clubs as a method of bridging the theory-practice gap that still exists in their profession.

Knowledge Translation in Cancer Services: Implementing the Research and Evidence in Practice Model

Occupational therapy lacks intervention effectiveness studies, which are crucial to the profession's survival. Literature also describes problems in the translation of research findings into occupational therapy clinical practice. Despite this, there is limited information regarding specific organisational models of practice to facilitate both the generation and the translation of research evidence. In 2006, occupational therapists at the Princess Alexandra Hospital, Queensland, Australia, developed and implemented the Research and Evidence in Practice model. Over the past 5 years, the model has evolved to enable a more seamless and strategic approach to the generation and translation of evidence into practice. This practice analysis describes the current model, and provides a practical example from the cancer services clinical team to demonstrate its effectiveness.

How Necessary are Hip Restrictions for Avoiding Dislocation following Hemiarthroplasty or Total Hip Arthroplasty in Older Patients with a Hip Fracture?

Purpose:

It is common practice for health care professionals to encourage hip fracture patients to adhere to a set of guidelines to prevent potential hip dislocation, so-called ‘hip restrictions’, following either hemiarthroplasty (partial) or total hip arthroplasty. The purpose of this critical review was to investigate the factors influencing dislocation for hip fracture patients who had undergone hemiarthroplasty or total hip arthroplasty and whether there was evidence to support compliance with hip restrictions following surgery.

Method:

A search of published literature was carried out to collect the most relevant and highest level evidence available. It was considered important to determine what previous research had accomplished and which studies might have an indirect or implied relationship to the topic. This critical review investigates the literature available and whether studies carried out can inform current clinical practice.

Findings:

The evidence indicates that, for hemiarthroplasty, the surgical approach has a strong influence on the dislocation rate and the recommendation is that, if using an anterolateral approach, restricting hip movement following surgery is unnecessary. For total hip arthroplasty, the dislocation rate is higher in those requiring total hip arthroplasty following a hip fracture (trauma) than for those undergoing an elective, planned procedure.

Conclusion:

The limited body of research regarding compliance with hip restrictions and influence on the dislocation rate refers to elective, planned surgery only. Because the traumatic hip fracture population is entirely different from the elective population, the outcomes for the elective population cannot be generalised to those with a hip fracture in a trauma setting.

Using journal clubs to promote skills for evidence-based practice

Occupational therapy students in Pittsburgh, Pennsylvania, participated in a 9-week journal club experience within an evidence-based practice course. Working in groups of seven, students were provided with an overall framework to develop and conduct weekly journal club sessions. They determined the focus and direction for their journal club section, decided on the number, type, and focus of research articles that were discussed each week, created methods for evaluating individual and group performance, and determined the effect of the journal club experience on their knowledge and professional behavior. Students completed a weekly reflective journal and at the conclusion of the course, results were uniformly positive. Review of the reflexive journals indicated substantive changes in the students' professional development, commitment to evidence-based practice, and perceived responsibility to demonstrate leadership in this area.

Utilization of evidence-based practice by registered occupational therapists

Although the use of evidence-based practice (EBP) is presently on the rise, there have been limited studies examining its use by occupational therapists within the US. The purpose of this study was to examine the use of EBP among registered occupational therapists in the occupational therapy intervention planning process. This descriptive study surveyed 500 members of the American Occupational Therapy Association (AOTA), of which 131 participants responded (26%). The results of the study supported the hypothesis that, within the sample studied, a minority of registered occupational therapists in the US utilize EBP in the intervention planning process. Other results included: (1) As level of academic education increased, the view of the importance of research to occupational therapy decreased. (2) As the years of practice increased, the use of research evidence in making clinical decisions decreased. As the occupational therapy profession moves towards utilization of EBP as a professional standard, it is imperative that the profession examines specific strategies to promote the adoption of such practice by its members, including the promotion of competency in evidence utilization, and the valuing of the established clinical reasoning skills of the practitioner while integrating research evidence into intervention planning to support professional practice.

Teaching evidence-based practice: A survey of UK's higher education provision

Evidence-based practice (EBP) ensures that clinical health management decisions are based on valid, reliable and relevant evidence determined by sound research and evaluation. It is a key element of both the clinical governance and clinical effectiveness agendas. Clearly, there is a need to provide adequate educational and professional attributes to meet this increasing demand.

The aim of this research was to identify and map all UK postgraduate courses that focus on EBP within health- and social-care services. Electronic searching of internet databases and all UK postgraduate electronic prospectuses using the broad term ‘evidence based’ was undertaken. Course details and prospectuses were obtained and reviewed, where available.

In total, 12 postgraduate courses were identified. These could be broadly categorized into two distinct groups: five courses focused on health-care sciences designed for multidisciplinary health professionals and seven targeted clinical specialities. UK universities therefore provide relatively few courses in EBP. Two models of EBP education appear to be developing, which either specifically target EBP as a separate component of the curriculum or integrate into existing subjects. Further evaluation is needed to determine the relative merits of each.

Occupational therapists' assessments of adults with long-term pain: the Swedish experience

The purposes of this study were to describe the needs for occupational therapy among people of working age with long-term pain, and to describe treatment interventions based on these assessments. Occupational therapists working in primary health care and/or with special interest in pain management (n=109) assessed 113 people aged 18-58 years with long-term pain with the Occupational Therapy Needs Assessment--Pain (OTNA--P) questionnaire. The occupational therapists recommended treatment interventions where appropriate. The results generated categories of needs that have implications for interventions: 1) need for patient education, 2) needs due to limitations in activity performance, 3) needs due to patient's discouragement, 4) need as a result of patient's dependency and 5) needs related to work. The suggested interventions focused on increased knowledge of how to handle daily occupations, mainly categorized as education and stress management' and behavioural' interventions. Significant correlations were found between the assessed needs and the suggested interventions. The results of this study could assist in developing guidelines for practitioners working in occupational therapy pain management programmes. It is recommended that further research is done on the effectiveness of occupational therapy interventions with patients with long-term pain.

Identifying performance criteria for staff development competencies

Performance criteria for specific staff development competencies need to be developed. The primary investigator (PI) surveyed the literature and developed specific performance criteria for objective staff development competencies. A survey was sent to a random stratified sample of 300 nurses certified in nursing continuing education and staff development asking them to rank the performance criteria for the objective statements and provide suggestions for an additional five subjective competencies. The PI reviewed the data and developed criteria for the subjective competencies. A survey with performance criteria for all 10 competencies was sent to National Nursing Staff Development Organization (NNSDO) affiliate groups for validation. Performance criteria were identified for all 10 competency statements. The mean scores for the national and regional samples were very similar in most areas. The data will provide a mechanism to promote and advance evidence-based practice in staff development. Instructors, individual staff development specialists, or organizations can use these performance criteria to determine if staff development personnel are meeting specified competencies.