1
|
Sharp MK, Baki DABA, Quigley J, Tyner B, Devane D, Mahtani KR, Smith SM, O'Neill M, Ryan M, Clyne B. The effectiveness and acceptability of evidence synthesis summary formats for clinical guideline development groups: a mixed-methods systematic review. Implement Sci 2022; 17:74. [PMID: 36303142 PMCID: PMC9615384 DOI: 10.1186/s13012-022-01243-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Accepted: 09/23/2022] [Indexed: 11/15/2022] Open
Abstract
INTRODUCTION Clinical guideline development often involves a rigorous synthesis of evidence involving multidisciplinary stakeholders with different priorities and knowledge of evidence synthesis; this makes communicating findings complex. Summary formats are typically used to communicate the results of evidence syntheses; however, there is little consensus on which formats are most effective and acceptable for different stakeholders. METHODS This mixed-methods systematic review (MMSR) aimed to evaluate the effectiveness and acceptability (e.g. preferences and attitudes and preferences towards) of evidence synthesis summary formats for GDG members. We followed the PRISMA 2020 guideline and Joanna Briggs Institute Manual for Evidence Synthesis for MMSRs. We searched six databases (inception to April 20, 2021) for randomised controlled trials (RCTs), RCTs with a qualitative component, and qualitative studies. Screening, data extraction, and quality appraisal were performed in duplicate. Qualitative findings were synthesised using meta-aggregation, and quantitative findings are described narratively. RESULTS We identified 17,240 citations and screened 54 full-text articles, resulting in 22 eligible articles (20 unique studies): 4 articles reported the results of 5 RCTs, one of which also had a qualitative component. The other 18 articles discussed the results of 16 qualitative studies. Therefore, we had 5 trials and 17 qualitative studies to extract data from. Studies were geographically heterogeneous and included a variety of stakeholders and summary formats. All 5 RCTs assessed knowledge or understanding with 3 reporting improvement with newer formats. The qualitative analysis identified 6 categories of recommendations: 'presenting information', 'tailoring information' for end users, 'trust in producers and summary', 'knowledge required' to understand findings, 'quality of evidence', and properly 'contextualising information'. Across these categories, the synthesis resulted in 126 recommendations for practice. Nine recommendations were supported by both quantitative and qualitative evidence and 116 by only qualitative. A majority focused on how to present information (n = 64) and tailor content for different end users (n = 24). CONCLUSIONS This MMSR provides guidance on how to improve evidence summary structure and layout. This can be used by synthesis producers to better communicate to GDGs. Study findings will inform the co-creation of evidence summary format prototypes based on GDG member's needs. Trial registration The protocol for this project was previously published, and the project was preregistered on Open Science Framework (Clyne and Sharp, Evidence synthesis and translation of findings for national clinical guideline development: addressing the needs and preferences of guideline development groups, 2021; Sharp and Clyne, Evidence synthesis summary formats for decision-makers and Clinical Guideline Development Groups: A mixed-methods systematic review protocol, 2021).
Collapse
Affiliation(s)
- Melissa K Sharp
- Department of General Practice, RCSI University of Medicine and Health Sciences, 123 St Stephens Green, Dublin 2, Ireland.
| | | | - Joan Quigley
- Health Information and Quality Authority, George's Court, George's Lane, Dublin 7, Ireland
| | - Barrie Tyner
- Health Information and Quality Authority, George's Court, George's Lane, Dublin 7, Ireland
| | - Declan Devane
- School of Nursing and Midwifery, NUI Galway, Galway, Ireland
- Evidence Synthesis Ireland & Cochrane, Galway, Ireland
| | - Kamal R Mahtani
- Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, England
| | - Susan M Smith
- Department of General Practice, RCSI University of Medicine and Health Sciences, 123 St Stephens Green, Dublin 2, Ireland
- Department of Public Health and Primary Care, School of Medicine, Trinity College Dublin, Dublin 2, Ireland
| | - Michelle O'Neill
- Health Information and Quality Authority, George's Court, George's Lane, Dublin 7, Ireland
| | - Máirín Ryan
- Health Information and Quality Authority, George's Court, George's Lane, Dublin 7, Ireland
- Department of Pharmacology & Therapeutics, Trinity College Dublin, Trinity Health Sciences, James Street, Dublin 8, Ireland
| | - Barbara Clyne
- Department of General Practice, RCSI University of Medicine and Health Sciences, 123 St Stephens Green, Dublin 2, Ireland
- Health Information and Quality Authority, George's Court, George's Lane, Dublin 7, Ireland
| |
Collapse
|
2
|
Garritty C, Hamel C, Hersi M, Butler C, Monfaredi Z, Stevens A, Nussbaumer-Streit B, Cheng W, Moher D. Assessing how information is packaged in rapid reviews for policy-makers and other stakeholders: a cross-sectional study. Health Res Policy Syst 2020; 18:112. [PMID: 32993657 PMCID: PMC7523380 DOI: 10.1186/s12961-020-00624-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2019] [Accepted: 08/30/2020] [Indexed: 01/08/2023] Open
Abstract
Background Rapid reviews (RRs) are useful products to healthcare policy-makers and other stakeholders, who require timely evidence. Therefore, it is important to assess how well RRs convey useful information in a format that is easy to understand so that decision-makers can make best use of evidence to inform policy and practice. Methods We assessed a diverse sample of 103 RRs against the BRIDGE criteria, originally developed for communicating clearly to support healthcare policy-making. We modified the criteria to increase assessability and to align with RRs. We identified RRs from key database searches and through searching organisations known to produce RRs. We assessed each RR on 26 factors (e.g. organisation of information, lay language use). Results were descriptively analysed. Further, we explored differences between RRs published in journals and those published elsewhere. Results Certain criteria were well covered across the RRs (e.g. all aimed to synthesise research evidence and all provided references of included studies). Further, most RRs provided detail on the problem or issue (96%; n = 99) and described methods to conduct the RR (91%; n = 94), while several addressed political or health systems contexts (61%; n = 63). Many RRs targeted policy-makers and key stakeholders as the intended audience (66%; n = 68), yet only 32% (n = 33) involved their tacit knowledge, while fewer (27%; n = 28) directly involved them reviewing the content of the RR. Only six RRs involved patient partners in the process. Only 23% (n = 24) of RRs were prepared in a format considered to make information easy to absorb (i.e. graded entry) and 25% (n = 26) provided specific key messages. Readability assessment indicated that the text of key RR sections would be hard to understand for an average reader (i.e. would require post-secondary education) and would take 42 (± 36) minutes to read. Conclusions Overall, conformity of the RRs with the modified BRIDGE criteria was modest. By assessing RRs against these criteria, we now understand possible ways in which they could be improved to better meet the information needs of healthcare decision-makers and their potential for innovation as an information-packaging mechanism. The utility and validity of these items should be further explored. Protocol availability The protocol, published on the Open Science Framework, is available at: osf.io/68tj7
Collapse
Affiliation(s)
- Chantelle Garritty
- Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, CPCR Building, 501 Smyth Rd, Box 201B, Ottawa, ON, K1H 8L6, Canada. .,TRIBE Graduate Program, University of Split School of Medicine, Split, Croatia.
| | - Candyce Hamel
- Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, CPCR Building, 501 Smyth Rd, Box 201B, Ottawa, ON, K1H 8L6, Canada.,TRIBE Graduate Program, University of Split School of Medicine, Split, Croatia
| | - Mona Hersi
- Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, CPCR Building, 501 Smyth Rd, Box 201B, Ottawa, ON, K1H 8L6, Canada
| | - Claire Butler
- Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, CPCR Building, 501 Smyth Rd, Box 201B, Ottawa, ON, K1H 8L6, Canada
| | - Zarah Monfaredi
- Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, CPCR Building, 501 Smyth Rd, Box 201B, Ottawa, ON, K1H 8L6, Canada
| | - Adrienne Stevens
- Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, CPCR Building, 501 Smyth Rd, Box 201B, Ottawa, ON, K1H 8L6, Canada
| | | | - Wei Cheng
- Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, CPCR Building, 501 Smyth Rd, Box 201B, Ottawa, ON, K1H 8L6, Canada
| | - David Moher
- Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, The Ottawa Hospital, General Campus, CPCR Building, 501 Smyth Rd, Box 201B, Ottawa, ON, K1H 8L6, Canada.,School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada
| |
Collapse
|
3
|
Garritty C, Hersi M, Hamel C, Stevens A, Monfaredi Z, Butler C, Tricco AC, Hartling L, Stewart LA, Welch V, Thavorn K, Cheng W, Moher D. Assessing the format and content of journal published and non-journal published rapid review reports: A comparative study. PLoS One 2020; 15:e0238025. [PMID: 32845906 PMCID: PMC7449464 DOI: 10.1371/journal.pone.0238025] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2019] [Accepted: 08/08/2020] [Indexed: 01/23/2023] Open
Abstract
BACKGROUND As production of rapid reviews (RRs) increases in healthcare, knowing how to efficiently convey RR evidence to various end-users is important given they are often intended to directly inform decision-making. Little is known about how often RRs are produced in the published or unpublished domains, and what and how information is structured. OBJECTIVES To compare and contrast report format and content features of journal-published (JP) and non-journal published (NJP) RRs. METHODS JP RRs were identified from key databases, and NJP RRs were identified from a grey literature search of 148 RR producing organizations and were sampled proportionate to cluster size by organization and product type to match the JP RR group. We extracted and formally compared 'how' (i.e., visual arrangement) and 'what' information was presented. RESULTS We identified 103 RRs (52 JP and 51 NJP) from 2016. A higher percentage of certain features were observed in JP RRs compared to NJP RRs (e.g., reporting authors; use of a traditional journal article structure; section headers including abstract, methods, discussion, conclusions, acknowledgments, conflict of interests, and author contributions; and use of figures (e.g., Study Flow Diagram) in the main document). For NJP RRs, a higher percentage of features were observed (e.g., use non-traditional report structures; bannering of executive summary sections and appendices; use of typographic cues; and including outcome tables). NJP RRs were more than double in length versus JP RRs. Including key messages was uncommon in both groups. CONCLUSIONS This comparative study highlights differences between JP and NJP RRs. Both groups may benefit from better use of plain language, and more clear and concise design. Alternative innovative formats and end-user preferences for content and layout should be studied further with thought given to other considerations to ensure better packaging of RR results to facilitate uptake into policy and practice. STUDY REGISTRATION The full protocol is available at: https://osf.io/29xvk/.
Collapse
Affiliation(s)
- Chantelle Garritty
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- TRIBE Graduate Program, University of Split School of Medicine, Split, Croatia
| | - Mona Hersi
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Candyce Hamel
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- TRIBE Graduate Program, University of Split School of Medicine, Split, Croatia
| | - Adrienne Stevens
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Zarah Monfaredi
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Claire Butler
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Andrea C. Tricco
- Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada
- Dalla Lana School of Public Health, University of Toronto, Toronto, Canada
| | - Lisa Hartling
- Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, Edmonton, Canada
| | - Lesley A. Stewart
- Centre for Reviews and Dissemination, University of York, York, United Kingdom
| | - Vivian Welch
- Methods Centre, Bruyère Research Institute, Ottawa, Canada
| | - Kednapa Thavorn
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada
| | - Wei Cheng
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - David Moher
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada
| |
Collapse
|
4
|
Evidence summary resources may influence clinical decision making: A case-based scenario evaluation of an evidence summary tool. J Crit Care 2019; 55:9-15. [PMID: 31670150 DOI: 10.1016/j.jcrc.2019.10.001] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2019] [Accepted: 10/02/2019] [Indexed: 12/31/2022]
Abstract
PURPOSE Evidence summary resources are popular with clinicians but it is unknown whether they can influence clinical decision making. We evaluated whether an extremely condensed and explicit evidence summary tool could influence clinical decision making. MATERIALS AND METHODS An evidence summary tool was developed using a formal mapping exercise and graphic design principles. An invitation to participate was sent to subscribers of a critical care e-mail discussion list. Participants received a study package (evidence summary tool précising prone positioning in severe ARDS; case-based scenario describing a patient with severe ARDS plus evaluation questionnaire). Influence on clinical decisions was captured regarding six competing interventions, with Belief in benefit measured before and after reading the summary tool. RESULTS Among 93 participants, 87% were male with a mean age of 49.6(SD9.3) years. Mean ICU experience was 20.0(SD9.9) years. The evidence summary tool significantly influenced clinical decision making: belief in benefit of prone positioning increased (P < .001), belief in benefit of higher PEEP decreased (P = .002) and belief in benefit in ECMO decreased (P = .07). CONCLUSIONS Using a before-after evaluation, we demonstrated an extremely condensed and explicit information format can influence clinical decision making. Evidence summary tools may be a useful adjunct to support closing evidence-practice gaps.
Collapse
|
5
|
Smith CJ, Jungbauer RM, Totten AM. Visual Evidence: Increasing Usability of Systematic Reviews in Health Systems Guidelines Development. Appl Clin Inform 2019; 10:743-750. [PMID: 31578047 DOI: 10.1055/s-0039-1697595] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022] Open
Abstract
BACKGROUND Integration of evidence from systematic reviews is an essential step in the development of clinical guidelines. The current practice for reporting uses a static structure that does not allow for dynamic investigation. A need exists for an alternate reporting modality to facilitate dynamic visualization of results to match different end-users' queries. OBJECTIVES We developed a dynamic visualization of data from a systematic review using the commercial product Tableau and assessed its potential to permit customized inquiries. METHODS Data were selected and extracted from a previously completed systematic review. The resulting dataset was then used to develop an interactive, web-based report designed for use by a guidelines development committee. RESULTS A novel example of combining existing reporting standards for systematic review data and modern reporting tools was developed to investigate potential benefits of a dynamic report. Demonstrations of the report to clinicians sitting on previous and future guideline committees received positive feedback for its potential benefit in guidelines development. The report received a runner-up award during the design challenge at the 2018 Workshop on Visual Analytics in Health Care. CONCLUSION The use of interactive, accessible data may increase the use of systematic reviews and aid decision makers in developing evidence-based practice changes.
Collapse
Affiliation(s)
- Connor J Smith
- Department of Medical Informatics and Clinical Epidemiology, Pacific Northwest Evidence-based Practice Center, Oregon Health and Science University, Portland, Oregon, United States
| | - Rebecca M Jungbauer
- Department of Medical Informatics and Clinical Epidemiology, Pacific Northwest Evidence-based Practice Center, Oregon Health and Science University, Portland, Oregon, United States
| | - Annette M Totten
- Department of Medical Informatics and Clinical Epidemiology, Pacific Northwest Evidence-based Practice Center, Oregon Health and Science University, Portland, Oregon, United States
| |
Collapse
|
6
|
Bian J, Weir C, Unni P, Borbolla D, Reese T, Wan YKJ, Del Fiol G. Interactive Visual Displays for Interpreting the Results of Clinical Trials: Formative Evaluation With Case Vignettes. J Med Internet Res 2018; 20:e10507. [PMID: 29941416 PMCID: PMC6037946 DOI: 10.2196/10507] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2018] [Revised: 04/30/2018] [Accepted: 05/03/2018] [Indexed: 11/13/2022] Open
Abstract
Background At the point of care, evidence from randomized controlled trials (RCTs) is underutilized in helping clinicians meet their information needs. Objective To design interactive visual displays to help clinicians interpret and compare the results of relevant RCTs for the management of a specific patient, and to conduct a formative evaluation with physicians comparing interactive visual versus narrative displays. Methods We followed a user-centered and iterative design process succeeded by development of information display prototypes as a Web-based application. We then used a within-subjects design with 20 participants (8 attendings and 12 residents) to evaluate the usability and problem-solving impact of the information displays. We compared subjects’ perceptions of the interactive visual displays versus narrative abstracts. Results The resulting interactive visual displays present RCT results side-by-side according to the Population, Intervention, Comparison, and Outcome (PICO) framework. Study participants completed 19 usability tasks in 3 to 11 seconds with a success rate of 78% to 100%. Participants favored the interactive visual displays over narrative abstracts according to perceived efficiency, effectiveness, effort, user experience and preference (all P values <.001). Conclusions When interpreting and applying RCT findings to case vignettes, physicians preferred interactive graphical and PICO-framework-based information displays that enable direct comparison of the results from multiple RCTs compared to the traditional narrative and study-centered format. Future studies should investigate the use of interactive visual displays to support clinical decision making in care settings and their effect on clinician and patient outcomes.
Collapse
Affiliation(s)
- Jiantao Bian
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States
| | - Charlene Weir
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States.,George E. Whalen Veterans Affairs Medical Center, Salt Lake City, UT, United States
| | - Prasad Unni
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States.,Intelligent Medical Objects, Chicago, IL, United States
| | - Damian Borbolla
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States
| | - Thomas Reese
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States
| | - Yik-Ki Jacob Wan
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States
| | - Guilherme Del Fiol
- Department of Biomedical Informatics, University of Utah, Salt Lake City, UT, United States
| |
Collapse
|
7
|
Tricco AC, Cardoso R, Thomas SM, Motiwala S, Sullivan S, Kealey MR, Hemmelgarn B, Ouimet M, Hillmer MP, Perrier L, Shepperd S, Straus SE. Barriers and facilitators to uptake of systematic reviews by policy makers and health care managers: a scoping review. Implement Sci 2016; 11:4. [PMID: 26753923 PMCID: PMC4709874 DOI: 10.1186/s13012-016-0370-1] [Citation(s) in RCA: 95] [Impact Index Per Article: 11.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2015] [Accepted: 01/06/2016] [Indexed: 11/23/2022] Open
Abstract
BACKGROUND We completed a scoping review on the barriers and facilitators to use of systematic reviews by health care managers and policy makers, including consideration of format and content, to develop recommendations for systematic review authors and to inform research efforts to develop and test formats for systematic reviews that may optimise their uptake. METHODS We used the Arksey and O'Malley approach for our scoping review. Electronic databases (e.g., MEDLINE, EMBASE, PsycInfo) were searched from inception until September 2014. Any study that identified barriers or facilitators (including format and content features) to uptake of systematic reviews by health care managers and policy makers/analysts was eligible for inclusion. Two reviewers independently screened the literature results and abstracted data from the relevant studies. The identified barriers and facilitators were charted using a barriers and facilitators taxonomy for implementing clinical practice guidelines by clinicians. RESULTS We identified useful information for authors of systematic reviews to inform their preparation of reviews including providing one-page summaries with key messages, tailored to the relevant audience. Moreover, partnerships between researchers and policy makers/managers to facilitate the conduct and use of systematic reviews should be considered to enhance relevance of reviews and thereby influence uptake. CONCLUSIONS Systematic review authors can consider our results when publishing their systematic reviews. These strategies should be rigorously evaluated to determine impact on use of reviews in decision-making.
Collapse
Affiliation(s)
- Andrea C Tricco
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, ON, M5B 1W8, Canada.
- Epidemiology Division, Dalla Lana School of Public Health, University of Toronto, 155 College Street, Toronto, ON, M5T 3M7, Canada.
| | - Roberta Cardoso
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, ON, M5B 1W8, Canada.
| | - Sonia M Thomas
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, ON, M5B 1W8, Canada.
| | - Sanober Motiwala
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, ON, M5B 1W8, Canada.
| | - Shannon Sullivan
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, ON, M5B 1W8, Canada.
| | - Michael R Kealey
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, ON, M5B 1W8, Canada.
- Department of Mechanical and Industrial Engineering, University of Toronto, 5 King's College Road, Toronto, ON, M5S 3G8, Canada.
| | - Brenda Hemmelgarn
- Departments of Medicine and Community Health Sciences, University of Calgary, TRW Building, 3rd Floor, 3280 Hospital Drive NW, Calgary, AB, T2N 4Z6, Canada.
| | - Mathieu Ouimet
- Département de science politique, Pavillon Charles-De Koninck, Université Laval, Quebec City, Canada.
| | - Michael P Hillmer
- Institute of Health Policy, Management and Evaluation, University of Toronto, Health Sciences Building, 155 College Street, Suite 425, Toronto, ON, M5T 3M6, Canada.
- Research, Evaluation, and Analysis Branch, Ontario Ministry of Health and Long-Term Care, 80 Grosvenor Street, Toronto, ON, M7A 1R3, Canada.
| | - Laure Perrier
- Institute of Health Policy, Management and Evaluation, University of Toronto, Health Sciences Building, 155 College Street, Suite 425, Toronto, ON, M5T 3M6, Canada.
| | - Sasha Shepperd
- Nuffield Department of Population Health, University of Oxford, Richard Doll Building, Old Rd Campus, Headington, Oxford, Oxfordshire, OX3 7LF, UK.
| | - Sharon E Straus
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, 209 Victoria Street, East Building, Toronto, ON, M5B 1W8, Canada.
- Department of Geriatric Medicine, University of Toronto, 27 Kings College Circle, Toronto, ON, M5S 1A1, Canada.
| |
Collapse
|
8
|
Perrier L, Persaud N, Thorpe KE, Straus SE. Using a systematic review in clinical decision making: a pilot parallel, randomized controlled trial. Implement Sci 2015; 10:118. [PMID: 26276278 PMCID: PMC4542122 DOI: 10.1186/s13012-015-0303-4] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2015] [Accepted: 07/30/2015] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Evidence suggests that systematic reviews are used infrequently by physicians in clinical decision-making. One proposed solution is to create filtered resources so that information is validated and refined in order to be read quickly. Two shortened systematic review formats were developed to enhance their use in clinical decision-making. METHODS To prepare for a full-scale trial, we conducted a pilot study to test methods and procedures in order to refine the processes. A recruitment email was sent to physicians practicing full- or part-time in family medicine or general internal medicine. The pilot study took place in an online environment and eligible physicians were randomized to one of the systematic review formats (shortened or full-length) and instructed to read the document. Participants were asked to provide the clinical bottom line and apply the information presented to a clinical scenario. Participants' answers were evaluated independently by two investigators against "gold standard" answers prepared by an expert panel. RESULTS Fifty-six clinicians completed the pilot study within a 2-month period with a response rate of 4.3 %. Agreement between investigators in assessing participants' answers was determined by calculating a kappa statistic. Two questions were assessed separately, and a kappa statistic was calculated at 1.00 (100 % agreement) for each. CONCLUSIONS Agreement between investigators in assessing participants' answers is satisfactory. Although recruitment for the pilot study was completed in a reasonable time-frame, response rates were low and will require large numbers of contacts. The results indicate that conducting a full-scale trial is feasible. TRIAL REGISTRATION ClinicalTrials.gov NCT02414360 .
Collapse
Affiliation(s)
- Laure Perrier
- Institute of Health Management, Policy and Evaluation, University of Toronto, 155 College Street, Toronto, ON, M5T 3M6, Canada.
| | - Nav Persaud
- Department of Family and Community Medicine, Li Ka Shing Knowledge Institute, University of Toronto, St. Michael's Hospital, 30 Bond Street, Toronto, Canada.
| | - Kevin E Thorpe
- Dalla Lana, School of Public Health, Applied Health Research Centre, Keenan Research Centre, Li Ka Shing Knowledge Institute, University of Toronto, St. Michael's Hospital, 30 Bond Street, Toronto, Canada.
| | - Sharon E Straus
- Faculty of Medicine, Keenan Research Centre, Li Ka Shing Knowledge Institute, University of Toronto, St. Michael's Hospital, 30 Bond Street, Toronto, Canada.
| |
Collapse
|
9
|
Perrier L, Kealey MR, Straus SE. A usability study of two formats of a shortened systematic review for clinicians. BMJ Open 2014; 4:e005919. [PMID: 25537782 PMCID: PMC4275680 DOI: 10.1136/bmjopen-2014-005919] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/16/2014] [Revised: 10/16/2014] [Accepted: 11/26/2014] [Indexed: 11/26/2022] Open
Abstract
OBJECTIVE The aim of this study was to evaluate the usability of two formats of a shortened systematic review for clinicians. MATERIALS AND METHODS Usability of the prototypes was assessed using three cycles of iterative testing. 10 participants were asked to complete tasks of locating information or items within two prototypes and 'think aloud' while being audio taped. Interviews were also audio recorded and participants completed a systematic usability scale. RESULTS Revisions were made between each iteration in order to address issues identified by participants. Finding information relating to the number of studies in the meta-analysis, and locating the number of studies in the entire systematic review were revealed as areas needing attention during the usability evaluation. CONCLUSIONS Iterative testing combined with a multifaceted approach to usability testing offered essential insight into aspects of the prototypes that required modifications. Alterations were made in order to create finalised versions of the two shortened systematic review formats.
Collapse
Affiliation(s)
- Laure Perrier
- Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Canada
| | - M Ryan Kealey
- Faculty of Applied Science and Engineering, University of Toronto, Toronto, Canada
| | - Sharon E Straus
- Faculty of Medicine, University of Toronto, Toronto, Canada
- Keenan Research Centre, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada
| |
Collapse
|