Systematic Reviews for Informing Rehabilitation Practice: An Introduction

Applying Appraisal Tools in Aphasia Systematic Reviews: A Tutorial

PURPOSE

A systematic review (SR) represents a rigorous process of identifying and summarizing current research to answer specific clinical questions. Not all SRs present high-quality information, because they do not adhere to established standards of conduct or reporting. This tutorial aims to (a) describe two tools developed in epidemiology for reporting (PRISMA 2020; Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and appraising (AMSTAR 2; A MeaSurement Tool to Assess Systematic Reviews) SRs and (b) exemplify the use of AMSTAR 2. First, we describe the intents of PRISMA 2020 and AMSTAR 2 and compare the items on each checklist. Next, we apply the AMSTAR 2 list of domains to critically appraise methodological quality in two randomly selected SRs, which describe aphasia treatment. Appraisal results are reported, including the AMSTAR 2 ratings for overall confidence in the results of each review. For each SR, the overall rating was critically low, indicating that within the seven critical domains of AMSTAR 2, the SR had more than one critical weakness.

CONCLUSIONS

While PRISMA 2020 is a tool to guide preparation of SRs, to examine SR quality, the AMSTAR 2 is the tool of choice. Applied to two current aphasia treatment SRs, the AMSTAR 2 demonstrates that although both SRs were thorough, thoughtful summaries of a body of aphasia treatment literature, they did not achieve high ratings for methodological quality. Clinicians reading SRs are advised to familiarize themselves with quality assessment tools to assure that an SR meets expected criteria to document internal and external validity of the SR process, so resulting findings can be confidently applied for patients with aphasia.

Narrative Review of Clinical Practice Guidelines for Rehabilitation of People With Spinal Cord Injury: 2010-2020

ABSTRACT

Clinical practice guidelines provide reliable, vetted, and critical information to bring research to practice. Some medical specialties (e.g., physical medicine and rehabilitation) provide multidomain treatment for various conditions. This presents challenges because physical medicine and rehabilitation is a small specialty, a diverse patient base in terms sociodemographics and diagnosis, treatments are difficult to standardize, and rehabilitation research is underfunded. We wished to identify quality and applicability of clinical practice guidelines and searched "Spinal Cord Injury AND Clinical Practice Guidelines AND Rehabilitation" and vetting process.Three hundred fifty-nine articles were identified of which 58 met all criteria for full-text review of which 13 were included in the final selection. Additional publications were accessed from a nondatabase search. Five articles addressed postacute care, community treatment. Nine articles had no recorded vetting process but addressed rehabilitation as an outcome and were included separately. Many of the clinical practice guidelines were developed without evidence from randomized controlled trials, one had input from stakeholders, and some are out of date and do not address important aspects of changes in demographics of the affected population and the use of newer technologies such as sensors and robotics and devices. Identification of these gaps may help stimulate treatment that is clinically relevant, accessible, and current.

Scoping review of clinical rehabilitation research pertaining to traumatic brain injury: 1990-2016

INTRODUCTION

Understanding the impact of disease on function and improving functional outcomes is an important goal of rehabilitation. This scoping review analyzes 25 years of published traumatic brain injury (TBI) clinical rehabilitation research (CRR) to determine the frequency with which function is a research goal.

METHODS

The review protocol was prepared by experienced clinical researchers in the field. We used these search terms: "moderate traumatic brain injury" and "severe traumatic brain injury" and "functional outcomes"; and "rehabilitation" and "educational outcomes" or "cognition" or "community roles" or "community integration" "behavior" or "neuropsychology" or "quality of life", or "vocational outcomes" or "work" or "return to work". Publications from January 1, 1990-December 31, 2016 were included.

RESULTS

1,815 articles met initial criteria, of which 202 were intervention studies, 84 were randomized controlled trials and 353 prospective non-intervention studies. The combination of intervention and prospective non-intervention studies (n = 555) were analyzed for the kind of measurement tools used. Impairment measures (60%) and symptom measures (43%) either singly or in combination were most frequently used. Measures of function were employed in fewer than 30%. For single outcomes, 95 publications used impairment measures, 60 used symptoms and only18 used function. There were 117 studies that used neurocognitive measures. Participation/societal integration evaluation tools were used for community integration(n = 77) employment (n = 50) and independent living status (n = 37).

DISCUSSION

This scoping review identifies a need for additional types of clinical research in the field of TBI CRR. Much of the research is aimed at mitigating impairment and controlling symptoms rather than promoting function. Future efforts by the CRR community studying TBI should address this need.

Critical appraisal of systematic reviews of executive function treatments in TBI

OBJECTIVE

To appraise the quality of systematic reviews (SRs) and meta-analyses (MAs) that summarize the treatment literature for executive function (EF) impairments following traumatic brain injury (TBI).

METHODS

We used five data sources (PubMed; PsycINFO; ANCDS.org; Cochrane Collaboration; American Speech-Language-Hearing Association Compendium; Psychological Database for Brain Impairment Treatment Efficacy) and identified 19 reviews that met eligibility criteria (adults with TBI; behavioural treatments for EF impairments; no pharmacologic treatments). Two reviewers independently and reliably rated each paper using two appraisal tools (Critical Appraisal of Systematic Review or Meta-Analysis and Evidence in Augmentative and Alternative Communication systematic review scale).

RESULTS

Five MAs received highest ratings. Four SRs also addressed the majority of SR criteria. Reviews were better at addressing SR criteria as outcome measures across studies varied considerably, posing a challenge to quantitatively synthesize TBI outcomes. The strongest research evidence favours compensatory treatments that train individuals with TBI to use strategies to circumvent EF impairments in daily activities. Smaller effects are reported for direct training approaches.

CONCLUSIONS

Researchers need to identify core outcome measures to facilitate future rigorous MAs of the EF rehabilitation literature. SRs/MAs need to be conducted with thorough methods that are reported with detail.

Publication trends of study protocols in rehabilitation

INTRODUCTION

Growing evidence points for the need to publish study protocols in the health field. The aim of this paper was to observe whether the growing interest in publishing study protocols in the broader health field has been translated into increased publications of rehabilitation study protocols.

EVIDENCE ACQUISITION

PubMed was searched with appropriate combinations of Medical Subject Headings up to December 2014. The effective presence of study protocols was manually screened. Regression models analyzed the yearly growth of publications. Two-sample Z-tests analyzed whether the proportion of systematic reviews (SRs) and randomized controlled trials (RCTs) among study protocols differed from that of the same designs for the broader rehabilitation research.

EVIDENCE SYNTHESIS

Up to December 2014, 746 publications of rehabilitation study protocols were identified, with an exponential growth since 2005 (r2=0.981; P<0.001). RCT protocols were the most common among rehabilitation study protocols (83%), while RCTs were significantly more prevalent among study protocols than among the broader rehabilitation research (83% vs. 35.8%; P<0.001). For SRs, the picture was reversed: significantly less common among study protocols (2.8% vs. 9.3%; P<0.001). Funding was more often reported by rehabilitation study protocols than the broader rehabilitation research (90% vs. 53.1%; P<0.001). Rehabilitation journals published a significantly lower share of rehabilitation study protocols than they did for the broader rehabilitation research (1.8% vs.16.7%; P<0.001).

CONCLUSIONS

Identifying the reasons for these discrepancies and reverting unwarranted disparities (e.g. low rate of publication for rehabilitation SR protocols) are likely new avenues for rehabilitation research and its publication. SRs, particularly those aggregating RCT results, are considered the best standard of evidence to guide rehabilitation clinical practice; however, that standard can be improved in rigor and/or transparency if the publications of rehabilitation SRs protocols become more common.

A need for an augmented review when reviewing rehabilitation research

There is a need for additional strategies for performing systematic reviews (SRs) to improve translation of findings into practice and to influence health policy. SRs critically appraise research methodology and determine level of evidence of research findings. The standard type of SR identifies randomized controlled trials (RCTs) as providing the most valid data and highest level of evidence. RCTs are not among the most frequently used research design in disability and health research. RCTs usually measure impairments for the primary research outcome rather than improved function, participation or societal integration. It forces a choice between "validity" and "utility/relevance." Other approaches have effectively been used to assess the validity of alternative research designs, whose outcomes focus on function and patient-reported outcomes. We propose that utilizing existing evaluation tools that measure knowledge, dissemination and utility of findings, may help improve the translation of findings into practice and health policy.

Systematic overview of economic evaluations of health-related rehabilitation

BACKGROUND

Health related rehabilitation is instrumental in improving functioning and promoting participation by people with disabilities. To make clinical and policy decisions about health-related rehabilitation, resource allocation and cost issues need to be considered.

OBJECTIVES

To provide an overview of systematic reviews (SRs) on economic evaluations of health-related rehabilitation.

METHODS

We searched multiple databases to identify relevant SRs of economic evaluations of health-related rehabilitation. Review quality was assessed by AMSTAR checklist.

RESULTS

We included 64 SRs, most of which included economic evaluations alongside randomized controlled trials (RCTs). The review quality was low to moderate (AMSTAR score 5-8) in 35, and high (score 9-11) in 29 of the included SRs. The included SRs addressed various health conditions, including spinal or other pain conditions (n = 14), age-related problems (11), stroke (7), musculoskeletal disorders (6), heart diseases (4), pulmonary (3), mental health problems (3), and injury (3). Physiotherapy was the most commonly evaluated rehabilitation intervention in the included SRs (n = 24). Other commonly evaluated interventions included multidisciplinary programmes (14); behavioral, educational or psychological interventions (11); home-based interventions (11); complementary therapy (6); self-management (6); and occupational therapy (4).

CONCLUSIONS

Although the available evidence is often described as limited, inconsistent or inconclusive, some rehabilitation interventions were cost-effective or showed cost-saving in a variety of disability conditions. Available evidence comes predominantly from high income countries, therefore economic evaluations of health-related rehabilitation are urgently required in less resourced settings.

Finding and using best evidence for rehabilitation

With the demands for improved experiences of care, improved outcomes, and greater efficiency/lower costs, the need for an evidence-based approach to care in rehabilitation settings has never been more urgent. This article guides practitioners in how to find the best available evidence for rehabilitation settings. It then discusses the use of evidence from systematic reviews through a high-impact case study: delirium in patients with postoperative hip fracture.

Systematic review of power mobility outcomes for infants, children and adolescents with mobility limitations

OBJECTIVE

To summarize and critically appraise the evidence related to power mobility use in children (18 years or younger) with mobility limitations.

DATA SOURCES

Searches were performed in 12 electronic databases along with hand searching for articles published in English to September 2012 and updated February 2014.

REVIEW METHODS

The search was restricted to quantitative studies including at least one child with a mobility limitation and measuring an outcome related to power mobility device use. Articles were appraised using American Academy of Cerebral Palsy and Developmental Medicine (AACPDM) criteria for group and single-subject designs. The PRISMA statement was followed with inclusion criteria set a priori. Two reviewers independently screened titles, abstracts and full-text articles. AACPDM quality ratings were completed for levels I-III studies.

RESULTS

Of 259 titles, 29 articles met inclusion criteria, describing 28 primary research studies. One study, rated as strong level II evidence, supported positive impact of power mobility on overall development as well as independent mobility. Another study, rated as moderate level III evidence, supported positive impact on self-initiated movement. Remaining studies, rated evidence levels IV and V, provided support for a positive impact on a broad range of outcomes from to International Classification of Functioning (ICF) components of body structure and function, activity and participation. Some studies suggest that environmental factors may be influential in successful power mobility use and skill development.

CONCLUSION

The body of evidence supporting outcomes for children using power mobility is primarily descriptive rather than experimental in nature, suggesting research in this area is in its infancy.

Practice considerations for the introduction and use of power mobility for children

AIM

The aim of the study was to support clinicians in recommending and justifying power mobility for children of different ages and abilities, and with different needs. The study comprised three distinct parts: a literature review; a Delphi consensus; and clinical practice considerations.

METHOD

A scoping review of eight electronic databases and manual searches carried out in February 2011 identified 15 themes or transferable messages among 27 articles meeting initial inclusion criteria and these formed the basis of a draft paper. Informal consensus at two international conference presentations refined and modified the paper to include 10 messages supported by 24 articles. The literature review was updated in May 2012 and a modified Delphi process sought to formalize the consensus process with an international panel of 16 expert clinicians and researchers using a priori criteria of 80% agreement.

RESULTS

Evidence from studies was classified using the American Academy of Cerebral Palsy and Developmental Medicine guidelines, with evidence from most studies being classified as either level IV or level V, apart from one study each with evidence classified as level II and level III. Expert consensus on the content and wording of nine transferable messages may raise evidence overall to level III.

INTERPRETATION

This paper suggests that power mobility may reasonably be considered as an effective and appropriate intervention for children lacking efficient, independent mobility from around 12 months of age including children who may never become competent drivers and children lacking independent mobility only in early childhood.

Employer Practices for Employment of People With Disabilities: A Literature Scoping Review

Purpose:Research on employment for people with disabilities has focused on the perspectives of people with disabilities and service providers. Empirical evidence about employer practices and their outcomes would inform the development of more targeted interventions.

Method:Scoping review was conducted of literature discussing employer-focused research.

Results:Research heavily concerned with employer attitudes and culture. Reports of studies of attitudes and behavior are primarily being published for an audience of service providers and advocates; much less is targeted at employers.

Conclusions:Results point to need for empirical evidence of employer practices and behaviors and publication outlets reaching broader employer audience.

Postural control and fear of falling assessment in people with chronic obstructive pulmonary disease: a systematic review of instruments, international classification of functioning, disability and health linkage, and measurement properties

OBJECTIVES

To systematically review the instruments used to assess postural control and fear of falling in people with chronic obstructive pulmonary disease (COPD), and to synthesize and evaluate their breadth of content and measurement properties.

DATA SOURCES

MEDLINE, EMBASE, Web of Science, CINAHL, CENTRAL, PsycINFO, PEDro, and OTSeeker databases searched in September 2012.

STUDY SELECTION

Two independent reviewers performed the selection of articles, the ICF linking process and quality assessment. Only quantitative studies were included, irrespective of language or publication date.

DATA EXTRACTION

This systematic review comprised two phases. Phase 1 aimed to identify the commonly used instruments to assess postural control and fear of falling in the COPD literature. The breadth of content of each instrument was examined based on the International Classification of Functioning, Disability and Health (ICF). In phase 2, a measurement property search filter was adopted and used in four electronic databases to retrieve properties reported in the COPD population. The COSMIN checklist was used to assess the methodological quality of each measurement property reported.

DATA SYNTHESIS

Seventeen out of 401 publications were eligible in phase 1. Seventeen instruments were identified including 15 for postural control and 2 for fear of falling assessment. The Berg Balance Scale, the Short Physical Performance Battery, and the Activities-specific Balance Confidence (ABC) scale were the most frequently used instruments to assess postural control and fear of falling respectively. The ICF categories covered varied considerably among instruments. The Balance Evaluation Systems test and ABC presented the greatest breadth of content. Measurement properties reported included criterion predictive validity (4 instruments), construct validity (11 instruments) and responsiveness (1 instrument), with inconsistent findings based on 'fair' and 'poor' quality studies.

CONCLUSIONS

Different instruments with heterogeneous content have been used to assess postural control and fear of falling outcomes. Standardized assessment methods and best evidence on measurement properties is required in the COPD literature.