Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement

OBJECTIVE

To develop a reporting guideline for overviews of reviews of healthcare interventions.

DESIGN

Development of the preferred reporting items for overviews of reviews (PRIOR) statement.

PARTICIPANTS

Core team (seven individuals) led day-to-day operations, and an expert advisory group (three individuals) provided methodological advice. A panel of 100 experts (authors, editors, readers including members of the public or patients) was invited to participate in a modified Delphi exercise. 11 expert panellists (chosen on the basis of expertise, and representing relevant stakeholder groups) were invited to take part in a virtual face-to-face meeting to reach agreement (≥70%) on final checklist items. 21 authors of recently published overviews were invited to pilot test the checklist.

SETTING

International consensus.

INTERVENTION

Four stage process established by the EQUATOR Network for developing reporting guidelines in health research: project launch (establish a core team and expert advisory group, register intent), evidence reviews (systematic review of published overviews to describe reporting quality, scoping review of methodological guidance and author reported challenges related to undertaking overviews of reviews), modified Delphi exercise (two online Delphi surveys to reach agreement (≥70%) on relevant reporting items followed by a virtual face-to-face meeting), and development of the reporting guideline.

RESULTS

From the evidence reviews, we drafted an initial list of 47 potentially relevant reporting items. An international group of 52 experts participated in the first Delphi survey (52% participation rate); agreement was reached for inclusion of 43 (91%) items. 44 experts (85% retention rate) completed the second Delphi survey, which included the four items lacking agreement from the first survey and five new items based on respondent comments. During the second round, agreement was not reached for the inclusion or exclusion of the nine remaining items. 19 individuals (6 core team and 3 expert advisory group members, and 10 expert panellists) attended the virtual face-to-face meeting. Among the nine items discussed, high agreement was reached for the inclusion of three and exclusion of six. Six authors participated in pilot testing, resulting in minor wording changes. The final checklist includes 27 main items (with 19 sub-items) across all stages of an overview of reviews.

CONCLUSIONS

PRIOR fills an important gap in reporting guidance for overviews of reviews of healthcare interventions. The checklist, along with rationale and example for each item, provides guidance for authors that will facilitate complete and transparent reporting. This will allow readers to assess the methods used in overviews of reviews of healthcare interventions and understand the trustworthiness and applicability of their findings.

Creating efficiencies in the extraction of data from randomized trials: a prospective evaluation of a machine learning and text mining tool

Background

Machine learning tools that semi-automate data extraction may create efficiencies in systematic review production. We evaluated a machine learning and text mining tool’s ability to (a) automatically extract data elements from randomized trials, and (b) save time compared with manual extraction and verification.

Methods

For 75 randomized trials, we manually extracted and verified data for 21 data elements. We uploaded the randomized trials to an online machine learning and text mining tool, and quantified performance by evaluating its ability to identify the reporting of data elements (reported or not reported), and the relevance of the extracted sentences, fragments, and overall solutions. For each randomized trial, we measured the time to complete manual extraction and verification, and to review and amend the data extracted by the tool. We calculated the median (interquartile range [IQR]) time for manual and semi-automated data extraction, and overall time savings.

Results

The tool identified the reporting (reported or not reported) of data elements with median (IQR) 91% (75% to 99%) accuracy. Among the top five sentences for each data element at least one sentence was relevant in a median (IQR) 88% (83% to 99%) of cases. Among a median (IQR) 90% (86% to 97%) of relevant sentences, pertinent fragments had been highlighted by the tool; exact matches were unreliable (median (IQR) 52% [33% to 73%]). A median 48% of solutions were fully correct, but performance varied greatly across data elements (IQR 21% to 71%). Using ExaCT to assist the first reviewer resulted in a modest time savings compared with manual extraction by a single reviewer (17.9 vs. 21.6 h total extraction time across 75 randomized trials).

Conclusions

Using ExaCT to assist with data extraction resulted in modest gains in efficiency compared with manual extraction. The tool was reliable for identifying the reporting of most data elements. The tool’s ability to identify at least one relevant sentence and highlight pertinent fragments was generally good, but changes to sentence selection and/or highlighting were often required.

Protocol

https://doi.org/10.7939/DVN/RQPJKS

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01354-2.

Screening for Gestational Diabetes: Updated Evidence Report and Systematic Review for the US Preventive Services Task Force

IMPORTANCE

Gestational diabetes is associated with several poor health outcomes.

OBJECTIVE

To update the 2012 review on screening for gestational diabetes to inform the US Preventive Services Task Force.

DATA SOURCES

MEDLINE, EMBASE, and CINAHL (2010 to May 2020), ClinicalTrials.gov, reference lists; surveillance through June 2021.

STUDY SELECTION

English-language intervention studies for screening and treatment; observational studies on screening; prospective studies on screening test accuracy.

DATA EXTRACTION AND SYNTHESIS

Dual review of titles/abstracts, full-text articles, and study quality. Single-reviewer data abstraction with verification. Random-effects meta-analysis or bivariate analysis (accuracy).

MAIN OUTCOMES AND MEASURES

Pregnancy, fetal/neonatal, and long-term health outcomes; harms of screening; accuracy.

RESULTS

A total of 76 studies were included (18 randomized clinical trials [RCTs] [n = 31 241], 2 nonrandomized intervention studies [n = 190], 56 observational studies [n = 261 678]). Direct evidence on benefits of screening vs no screening was limited to 4 observational studies with inconsistent findings and methodological limitations. Screening was not significantly associated with serious or long-term harm. In 5 RCTs (n = 25 772), 1-step (International Association of Diabetes and Pregnancy Study Group) vs 2-step (Carpenter and Coustan) screening was significantly associated with increased likelihood of gestational diabetes (11.5% vs 4.9%) but no improved health outcomes. At or after 24 weeks of gestation, oral glucose challenge tests with 140- and 135-mg/dL cutoffs had sensitivities of 82% and 93%, respectively, and specificities of 82% and 79%, respectively, against Carpenter and Coustan criteria, and a test with a 140-mg/dL cutoff had sensitivity of 85% and specificity of 81% against the National Diabetes Group Data criteria. Fasting plasma glucose tests with cutoffs of 85 and 90 mg/dL had sensitivities of 88% and 81% and specificities of 73% and 82%, respectively, against Carpenter and Coustan criteria. Based on 8 RCTs and 1 nonrandomized study (n = 3982), treatment was significantly associated with decreased risk of primary cesarean deliveries (relative risk [RR], 0.70 [95% CI, 0.54-0.91]; absolute risk difference [ARD], 5.3%), shoulder dystocia (RR, 0.42 [95% CI, 0.23-0.77]; ARD, 1.3%), macrosomia (RR, 0.53 [95% CI, 0.41-0.68]; ARD, 8.9%), large for gestational age (RR, 0.56 [95% CI, 0.47-0.66]; ARD, 8.4%), birth injuries (odds ratio, 0.33 [95% CI, 0.11-0.99]; ARD, 0.2%), and neonatal intensive care unit admissions (RR, 0.73 [95% CI, 0.53-0.99]; ARD, 2.0%). The association with reduction in preterm deliveries was not significant (RR, 0.75 [95% CI, 0.56-1.01]).

CONCLUSIONS AND RELEVANCE

Direct evidence on screening vs no screening remains limited. One- vs 2-step screening was not significantly associated with improved health outcomes. At or after 24 weeks of gestation, treatment of gestational diabetes was significantly associated with improved health outcomes.

Health inequities related to vaccination: An evidence map of potentially influential factors and systematic review of interventions

INTRODUCTION

The National Advisory Committee on Immunization (NACI) makes recommendations for vaccines in Canada. To inform considerations for equity when making recommendations, the NACI Secretariat developed a matrix of factors that may influence vaccine equity. To inform the matrix we mapped the evidence for P²ROGRESS And Other factors potentially associated with unequal levels of illness or death from vaccine-preventable diseases (VPDs) and systematically reviewed the evidence for interventions aimed at reducing inequities.

METHODS

In October 2019 we searched Medline, Embase, and CINAHL. Two reviewers agreed on the included studies. Our primary outcomes were VPD-related hospitalizations and deaths. Secondary outcomes were differential vaccine access, and exposure, susceptibility, severity, and consequences of VPDs. Two reviewers appraised the certainty of evidence. We mapped the evidence for P²ROGRESS And Other factors and summarized the findings descriptively. We summarized the interventions narratively.

RESULTS

We identified 413 studies reporting on P²ROGRESS And Other factors. The most commonly investigated factors included age (n = 374, 89%), pre-existing conditions (n = 179, 42%), and gender identity or sex (n = 144, 34%). We identified 2 trials investigating the effects of interventions. One (n = 1249) provided very low certainty evidence that staff vaccination policies may reduce hospitalizations and deaths from influenza among private care home residents. The other (n not reported) provided very low certainty evidence that universal vaccination by nurses in clinics may reduce hospitalizations for rotavirus gastroenteritis compared with vaccination by physicians or no intervention.

CONCLUSIONS

There is a large body of studies reporting on hospitalizations and deaths from VPDs stratified by P²ROGRESS And Other factors. We found only two trials examining the effects of interventions on hospitalization for or mortality from VPDs. This review has been helpful to NACI and will be helpful to similar organizations aiming to systematically identify and target health inequities through the development of vaccine program recommendations.

P value and Bayesian analysis in randomized-controlled trials in child health research published over 10 years, 2007 to 2017: a methodological review protocol

BACKGROUND

There is an unresolved debate about the reliability of the interpretation of P value. Some investigators have suggested that an alternative Bayesian method is preferred in conducting health research. As randomized-controlled trials (RCTs) are important in generating research evidence, we decided to investigate the extent, if any, the inferential statistical framework in published RCTs in child health research have changed over 10 years. We aim to examine the change in P value and Bayesian analysis in RCTs in child health research papers published from 2007 to 2017.

METHODS

We will search the Cochrane Central Register of Controlled Trials (Wiley) to identify relevant citations. We will leverage a pre-existing sample of child health RCTs published in 2007 (n=300) used in our previous study of reporting quality of pediatric RCTs. Using the same strategy and study selection methods, we will identify a comparable random sample of child health RCTs published in 2017 (n=300). Eligible studies will include RCTs in health research among individuals aged 21 years and below. One reviewer will select studies for inclusion and extract the data and another reviewer will verify these. Disagreements will be resolved by a discussion between reviewers or by involving another reviewer. We will perform a descriptive analysis of 2007 and 2017 samples and analyze the results using both the frequentist and Bayesian methods. We will present specific characteristics of the clinical trials from 2007 and 2017 in tabular and graphical forms. We will report the difference in the proportion of P value and Bayesian analysis between 2007 and 2017 to assess the 10-year change. Clustering around P values of significance, if observed, will be reported.

DISCUSSION

This review will present the difference in the proportion of trials that reported on P value and Bayesian analysis between 2007 and 2017 to assess the 10-year change. The implications for future clinical research will be discussed and this research work will be published in a peer-reviewed journal. This review has the potential to help inform the need for a change in the methodological approach from the null hypothesis significance test to Bayesian methods.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework https://osf.io/aj2df.

Screening for the prevention and early detection of cervical cancer: protocol for systematic reviews to inform Canadian recommendations

PURPOSE

To inform recommendations by the Canadian Task Force on Preventive Health Care on screening in primary care for the prevention and early detection of cervical cancer by systematically reviewing evidence of (a) effectiveness; (b) test accuracy; (c) individuals' values and preferences; and (d) strategies aimed at improving screening rates.

METHODS

De novo reviews will be conducted to evaluate effectiveness and to assess values and preferences. For test accuracy and strategies to improve screening rates, we will integrate studies from existing systematic reviews with search updates to the present. Two Cochrane reviews will provide evidence of adverse pregnancy outcomes from the conservative management of cervical intraepithelial neoplasia. We will search Medline, Embase, and Cochrane Central (except for individuals' values and preferences, where Medline, Scopus, and EconLit will be searched) via peer-reviewed search strategies and the reference lists of included studies and reviews. We will search ClinicalTrials.gov and the World Health Organization International Clinical Trials Registry Platform for ongoing trials. Two reviewers will screen potentially eligible studies and agree on those to include. Data will be extracted by one reviewer with verification by another. Two reviewers will independently assess risk of bias and reach consensus. Where possible and suitable, we will pool studies via meta-analysis. We will compare accuracy data per outcome and per comparison using the Rutter and Gatsonis hierarchical summary receiver operating characteristic model and report relative sensitivities and specificities. Findings on values and preferences will be synthesized using a narrative synthesis approach and thematic analysis, depending on study designs. Two reviewers will appraise the certainty of evidence for all outcomes using GRADE (Grading of Recommendations Assessment, Development and Evaluation) and come to consensus.

DISCUSSION

The publication of guidance on screening in primary care for the prevention and early detection of cervical cancer by the Task Force in 2013 focused on cytology. Since 2013, new studies using human papillomavirus tests for cervical screening have been published that will improve our understanding of screening in primary care settings. This review will inform updated recommendations based on currently available studies and address key evidence gaps noted in our previous review.

Decoding semi-automated title-abstract screening: findings from a convenience sample of reviews

BACKGROUND

We evaluated the benefits and risks of using the Abstrackr machine learning (ML) tool to semi-automate title-abstract screening and explored whether Abstrackr's predictions varied by review or study-level characteristics.

METHODS

For a convenience sample of 16 reviews for which adequate data were available to address our objectives (11 systematic reviews and 5 rapid reviews), we screened a 200-record training set in Abstrackr and downloaded the relevance (relevant or irrelevant) of the remaining records, as predicted by the tool. We retrospectively simulated the liberal-accelerated screening approach. We estimated the time savings and proportion missed compared with dual independent screening. For reviews with pairwise meta-analyses, we evaluated changes to the pooled effects after removing the missed studies. We explored whether the tool's predictions varied by review and study-level characteristics.

RESULTS

Using the ML-assisted liberal-accelerated approach, we wrongly excluded 0 to 3 (0 to 14%) records that were included in the final reports, but saved a median (IQR) 26 (9, 42) h of screening time. One missed study was included in eight pairwise meta-analyses in one systematic review. The pooled effect for just one of those meta-analyses changed considerably (from MD (95% CI) - 1.53 (- 2.92, - 0.15) to - 1.17 (- 2.70, 0.36)). Of 802 records in the final reports, 87% were correctly predicted as relevant. The correctness of the predictions did not differ by review (systematic or rapid, P = 0.37) or intervention type (simple or complex, P = 0.47). The predictions were more often correct in reviews with multiple (89%) vs. single (83%) research questions (P = 0.01), or that included only trials (95%) vs. multiple designs (86%) (P = 0.003). At the study level, trials (91%), mixed methods (100%), and qualitative (93%) studies were more often correctly predicted as relevant compared with observational studies (79%) or reviews (83%) (P = 0.0006). Studies at high or unclear (88%) vs. low risk of bias (80%) (P = 0.039), and those published more recently (mean (SD) 2008 (7) vs. 2006 (10), P = 0.02) were more often correctly predicted as relevant.

CONCLUSION

Our screening approach saved time and may be suitable in conditions where the limited risk of missing relevant records is acceptable. Several of our findings are paradoxical and require further study to fully understand the tasks to which ML-assisted screening is best suited. The findings should be interpreted in light of the fact that the protocol was prepared for the funder, but not published a priori. Because we used a convenience sample, the findings may be prone to selection bias. The results may not be generalizable to other samples of reviews, ML tools, or screening approaches. The small number of missed studies across reviews with pairwise meta-analyses hindered strong conclusions about the effect of missed studies on the results and conclusions of systematic reviews.

Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review

BACKGROUND

Overviews of reviews (overviews) provide an invaluable resource for healthcare decision-making by combining large volumes of systematic review (SR) data into a single synthesis. The production of high-quality overviews hinges on the availability of practical evidence-based guidance for conduct and reporting.

OBJECTIVES

Within the broad purpose of informing the development of a reporting guideline for overviews, we aimed to provide an up-to-date map of existing guidance related to the conduct of overviews, and to identify common challenges that authors face when undertaking overviews.

METHODS

We updated a scoping review published in 2016 using the search methods that had produced the highest yield: ongoing reference tracking (2014 to March 2020 in PubMed, Scopus, and Google Scholar), hand-searching conference proceedings and websites, and contacting authors of published overviews. Using a qualitative meta-summary approach, one reviewer extracted, organized, and summarized the guidance and challenges presented within the included documents. A second reviewer verified the data and synthesis.

RESULTS

We located 28 new guidance documents, for a total of 77 documents produced by 34 research groups. The new guidance helps to resolve some earlier identified challenges in the production of overviews. Important developments include strengthened guidance on handling primary study overlap at the study selection and analysis stages. Despite marked progress, several areas continue to be hampered by inconsistent or lacking guidance. There is ongoing debate about whether, when, and how supplemental primary studies should be included in overviews. Guidance remains scant on how to extract and use appraisals of quality of the primary studies within the included SRs and how to adapt GRADE methodology to overviews. The challenges that overview authors face are often related to the above-described steps in the process where evidence-based guidance is lacking or conflicting.

CONCLUSION

The rising popularity of overviews has been accompanied by a steady accumulation of new, and sometimes conflicting, guidance. While recent guidance has helped to address some of the challenges that overview authors face, areas of uncertainty remain. Practical tools supported by empirical evidence are needed to assist authors with the many methodological decision points that are encountered in the production of overviews.

Effectiveness and safety of interventions to manage childhood overweight and obesity: An Overview of Cochrane systematic reviews

Background

Childhood overweight and obesity are associated with adverse physical, social, and psychological outcomes.

Objectives

We conducted an overview of Cochrane systematic reviews on the effectiveness and risks of interventions to treat overweight and obesity in children and adolescents.

Methods

In June 2019, we searched the Cochrane Database of Systematic Reviews for eligible reviews. The primary outcomes were change in adiposity (body mass and body mass index [BMI] z-score) and adverse events. Two reviewers screened studies and one reviewer extracted and another verified data. Two reviewers assessed methodological quality and reached consensus. Data were synthesized narratively.

Results

We included seven Cochrane reviews published between 2011 and 2017 containing evidence from 167 randomized controlled trials with 21,050 participants. Lifestyle and behavioural interventions more effectively reduced weight compared with no intervention, usual care, or another behavioural treatment (three reviews, low-to-moderate certainty). Parent-child lifestyle and behavioural interventions more effectively reduced BMI z-score compared with no intervention (one review, low certainty). Decision support tools for healthcare providers more effectively limited increases in BMI z-score compared with usual care (one review, moderate certainty). Pharmacologic treatments combined with behavioural modification more effectively reduced adiposity compared with placebo or usual care (one review, low certainty), but the risk of adverse events was greater than non-pharmacologic therapy. Surgical interventions (e.g., LAP-BAND) combined with behavioural modification more effectively reduced adiposity compared with behavioural modification alone (one review, low certainty). Those who underwent surgery reported a higher number of adverse events compared with those treated with lifestyle modification.

Conclusions

There is low-certainty evidence that lifestyle and behavioural interventions, pharmacologic interventions, and surgical interventions are effective in weight management for children with overweight and obesity. Safety data remain lacking across all intervention modalities. Future research should focus on implementation strategies. Further, a focus on overall well-being may be more beneficial than weight management specifically.

The semi-automation of title and abstract screening: a retrospective exploration of ways to leverage Abstrackr's relevance predictions in systematic and rapid reviews

BACKGROUND

We investigated the feasibility of using a machine learning tool's relevance predictions to expedite title and abstract screening.

METHODS

We subjected 11 systematic reviews and six rapid reviews to four retrospective screening simulations (automated and semi-automated approaches to single-reviewer and dual independent screening) in Abstrackr, a freely-available machine learning software. We calculated the proportion missed, workload savings, and time savings compared to single-reviewer and dual independent screening by human reviewers. We performed cited reference searches to determine if missed studies would be identified via reference list scanning.

RESULTS

For systematic reviews, the semi-automated, dual independent screening approach provided the best balance of time savings (median (range) 20 (3-82) hours) and reliability (median (range) proportion missed records, 1 (0-14)%). The cited references search identified 59% (n = 10/17) of the records missed. For the rapid reviews, the fully and semi-automated approaches saved time (median (range) 9 (2-18) hours and 3 (1-10) hours, respectively), but less so than for the systematic reviews. The median (range) proportion missed records for both approaches was 6 (0-22)%.

CONCLUSION

Using Abstrackr to assist one of two reviewers in systematic reviews saves time with little risk of missing relevant records. Many missed records would be identified via other means.

How Perceptions Impact Recovery from Concussion in Childhood and Adolescence: a Systematic Review

Perceptions about the causes and consequences of concussion, and individual representations and interpretations of these factors, can influence the post-concussive recovery process. The goal of this project was to synthesize evidence on perceptions related to concussions as experienced by children, adolescents, and parents, and to evaluate how these perceptions impact post-concussive recovery in physical, behavioural, cognitive, and psychological domains. We undertook a systematic review based on the Cochrane Handbook, conducting a comprehensive search of six databases and Google Scholar. Duplicate, independent screening was employed and the quality of studies was assessed using the Mixed Methods Appraisal Tool (MMAT). A total of 1552 unique records were identified, and six records (5 scientific articles and 1 thesis, published between 1990 and 2018; N = 26 to 412, age range from 2 to 18 years) were included. Perceptions about concussions were assessed differently between studies, with a range in types of measures and respondents. Some evidence suggested that perceptions could negatively impact concussion recovery, mostly post-concussive symptoms. However, results were not consistent between studies and the methodological quality was variable (and often low). There is limited evidence of the impact of perceptions of children, adolescents, and their parents on concussion recovery. Priorities for future research investigating concussion recovery should include recruiting representative samples, accounting for potential confounders, and measuring perceptions in children, adolescents and parents using validated measures. Higher quality studies are needed to better understand the role of perceptions in concussion recovery and to inform clinical care.

Performance and usability of machine learning for screening in systematic reviews: a comparative evaluation of three tools

BACKGROUND

We explored the performance of three machine learning tools designed to facilitate title and abstract screening in systematic reviews (SRs) when used to (a) eliminate irrelevant records (automated simulation) and (b) complement the work of a single reviewer (semi-automated simulation). We evaluated user experiences for each tool.

METHODS

We subjected three SRs to two retrospective screening simulations. In each tool (Abstrackr, DistillerSR, RobotAnalyst), we screened a 200-record training set and downloaded the predicted relevance of the remaining records. We calculated the proportion missed and workload and time savings compared to dual independent screening. To test user experiences, eight research staff tried each tool and completed a survey.

RESULTS

Using Abstrackr, DistillerSR, and RobotAnalyst, respectively, the median (range) proportion missed was 5 (0 to 28) percent, 97 (96 to 100) percent, and 70 (23 to 100) percent for the automated simulation and 1 (0 to 2) percent, 2 (0 to 7) percent, and 2 (0 to 4) percent for the semi-automated simulation. The median (range) workload savings was 90 (82 to 93) percent, 99 (98 to 99) percent, and 85 (85 to 88) percent for the automated simulation and 40 (32 to 43) percent, 49 (48 to 49) percent, and 35 (34 to 38) percent for the semi-automated simulation. The median (range) time savings was 154 (91 to 183), 185 (95 to 201), and 157 (86 to 172) hours for the automated simulation and 61 (42 to 82), 92 (46 to 100), and 64 (37 to 71) hours for the semi-automated simulation. Abstrackr identified 33-90% of records missed by a single reviewer. RobotAnalyst performed less well and DistillerSR provided no relative advantage. User experiences depended on user friendliness, qualities of the user interface, features and functions, trustworthiness, ease and speed of obtaining predictions, and practicality of the export file(s).

CONCLUSIONS

The workload savings afforded in the automated simulation came with increased risk of missing relevant records. Supplementing a single reviewer's decisions with relevance predictions (semi-automated simulation) sometimes reduced the proportion missed, but performance varied by tool and SR. Designing tools based on reviewers' self-identified preferences may improve their compatibility with present workflows.

SYSTEMATIC REVIEW REGISTRATION

Not applicable.

Evidence available for patient-identified priorities in depression research: results of 11 rapid responses

OBJECTIVES

Patient priority setting projects (PPSPs) can reduce research agenda bias. A key element of PPSPs is a review of available literature to determine if the proposed research priorities have been addressed, identify research gaps, recognise opportunities for knowledge translation (KT) and avoid duplication of research efforts. We conducted rapid responses for 11 patient-identified priorities in depression to provide a map of the existing evidence.

DESIGN

Eleven rapid responses.

DATA SOURCES

Single electronic database (PubMed).

ELIGIBILITY CRITERIA

Each rapid response had unique eligibility criteria. For study designs, we used a stepwise inclusion process that started with systematic reviews (SRs) if available, then randomised controlled trials and observational studies as necessary.

RESULTS

For all but one of the rapid responses we identified existing SRs (median 7 SRs per rapid response, range 0-179). There were questions where extensive evidence exists (ie, hundreds of primary studies), yet uncertainties remain. For example, there is evidence supporting the effectiveness of many non-pharmacological interventions (including psychological interventions and exercise) to reduce depressive symptoms. However, targeted research is needed that addresses comparative effectiveness of promising interventions, specific populations of interest (eg, children, minority groups) and adverse effects.

CONCLUSIONS

We identified an extensive body of evidence addressing patient priorities in depression and mapped the results and limitations of existing evidence, areas of uncertainty and general directions for future research. This work can serve as a solid foundation to guide future research in depression and KT activities. Integrated knowledge syntheses bring value to the PPSP process; however, the role of knowledge synthesis in PPSPs and methodological approaches are not well defined at present.

Glucocorticoids for Croup in Children

CLINICAL QUESTION

For children with croup, do glucocorticoids reduce clinical symptoms, minimize return visits and/or (re)admissions to the hospital, shorten lengths of stay, or reduce the need for additional treatments?

BOTTOM LINE

Glucocorticoids reduce symptoms of croup at 2 hours and for at least 24 hours, reduce return visits to care, and shorten hospital stays by about 15 hours. Serious adverse events are infrequent.

Reporting of data monitoring committees and adverse events in paediatric trials: a descriptive analysis

OBJECTIVES

For 300 paediatric trials, we evaluated the reporting of: a data monitoring committee (DMC); interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints.

METHODS

For this cross-sectional evaluation, we randomly selected 300 paediatric trials published in 2012 from the Cochrane Central Register of Controlled Trials. We collected data on the reporting of a DMC; interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints. We reported the findings descriptively and stratified by trial characteristics.

RESULTS

Eighty-five (28%) of the trials investigated drugs, and 18% (n=55/300) reported a DMC. The reporting of a DMC was more common among multicentre than single centre trials (n=41/132, 31% vs n=14/139, 10%, p<0.001) and industry-sponsored trials compared with those sponsored by other sources (n=16/50, 32% vs n=39/250, 16%, p=0.009). Trials that reported a DMC enrolled more participants than those that did not (median [range]): 224 (10-60480) vs 91 (10-9528) (p<0.001). Only 25% of these trials reported interim analyses, and 42% reported stopping rules. Less than half (n=143/300, 48%) of trials reported on adverse events, and 72% (n=215/300) reported on harm-related endpoints. Trials that reported a DMC compared with those that did not were more likely to report adverse events (n=43/55, 78% vs 100/245, 41%, p<0.001) and harm-related endpoints (n=52/55, 95% vs. 163/245, 67%, p<0.001). Only 32% of drug trials reported a DMC; 18% and 19% did not report on adverse events or harm-related endpoints, respectively.

CONCLUSIONS

The reporting of a DMC was infrequent, even among drug trials. Few trials reported stopping rules or interim analyses. Reporting of adverse events and harm-related endpoints was suboptimal.

Glucocorticoids for croup in children

BACKGROUND

Glucocorticoids are commonly used for croup in children. This is an update of a Cochrane Review published in 1999 and previously updated in 2004 and 2011.

OBJECTIVES

To examine the effects of glucocorticoids for the treatment of croup in children aged 0 to 18 years.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials (CENTRAL) (the Cochrane Library, Issue 2, 2018), which includes the Cochrane Acute Respiratory Infections Group's Specialised Register, Ovid MEDLINE Epub Ahead of Print, In-Process & Other Non-Indexed Citations and Ovid MEDLINE (1946 to 3 April 2018), and Embase (Ovid) (1996 to 3 April 2018, week 14), and the trials registers ClinicalTrials.gov (3 April 2018) and the World Health Organization International Clinical Trials Registry Platform (ICTRP, 3 April 2018). We scanned the reference lists of relevant systematic reviews and of the included studies.

SELECTION CRITERIA

We included randomised controlled trials (RCTs) that investigated children aged 0 to 18 years with croup and measured the effects of glucocorticoids, alone or in combination, compared to placebo or another pharmacologic treatment. The studies needed to report at least one of our primary or secondary outcomes: change in croup score; return visits, (re)admissions or both; length of stay; patient improvement; use of additional treatments; and adverse events.

DATA COLLECTION AND ANALYSIS

One author extracted data from each study and another verified the extraction. We entered the data into Review Manager 5 for meta-analysis. Two review authors independently assessed risk of bias for each study using the Cochrane 'Risk of bias' tool and the certainty of the body of evidence for the primary outcomes using the GRADE approach.

MAIN RESULTS

We added five new RCTs with 330 children. This review now includes 43 RCTs with a total of 4565 children. We assessed most (98%) studies as at high or unclear risk of bias. Compared to placebo, glucocorticoids improved symptoms of croup at two hours (standardised mean difference (SMD) -0.65, 95% confidence interval (CI) -1.13 to -0.18; 7 RCTs; 426 children; moderate-certainty evidence), and the effect lasted for at least 24 hours (SMD -0.86, 95% CI -1.40 to -0.31; 8 RCTs; 351 children; low-certainty evidence). Compared to placebo, glucocorticoids reduced the rate of return visits or (re)admissions or both (risk ratio 0.52, 95% CI 0.36 to 0.75; 10 RCTs; 1679 children; moderate-certainty evidence). Glucocorticoid treatment reduced the length of stay in hospital by about 15 hours (mean difference -14.90, 95% CI -23.58 to -6.22; 8 RCTs; 476 children). Serious adverse events were infrequent. Publication bias was not evident. Uncertainty remains with regard to the optimal type, dose, and mode of administration of glucocorticoids for reducing croup symptoms in children.

AUTHORS' CONCLUSIONS

Glucocorticoids reduced symptoms of croup at two hours, shortened hospital stays, and reduced the rate of return visits to care. Our conclusions have changed, as the previous version of this review reported that glucocorticoids reduced symptoms of croup within six hours.

A systematic review of parents' experiences and information needs related to their child's urinary tract infection

OBJECTIVE

As a first step toward the development of an animated video and infographic to increase parents' knowledge of pediatric urinary tract infections (UTIs), we conducted a systematic review of their experiences and information needs.

METHODS

We searched Ovid Medline, Ovid PsycINFO, CINAHL, and ProQuest Dissertations and Theses Global for studies published in 2000 or thereafter. We appraised quality using the Mixed Methods Appraisal Tool. We summarised the quantitative data narratively and the qualitative data thematically.

RESULTS

We identified 1493 records and included four. Sample size ranged from 20 to 2726 parents. The children ranged from <1 to 12 years old and had experienced one to >10 UTIs. Parents were not always aware of UTI symptoms and generally received little information. Parents sought information online, and desired it via other means. Some parents were not confident in healthcare providers' (HCPs') knowledge of UTIs. Inadequate information about diagnostic tests sometimes resulted in fear and non-compliance.

CONCLUSIONS

From the limited literature, it appears that parents would like information about prevention, diagnosis, treatment, and prognosis, but do not always consider HCPs good information sources.

PRACTICE IMPLICATIONS

Care providers should communicate information in ways that suit parents' self-identified needs.

Evaluation of the reliability, usability, and applicability of AMSTAR, AMSTAR 2, and ROBIS: protocol for a descriptive analytic study

BACKGROUND

Systematic reviews (SRs) of randomised controlled trials (RCTs) can provide the best evidence to inform decision-making, but their methodological and reporting quality varies. Tools exist to guide the critical appraisal of quality and risk of bias in SRs, but evaluations of their measurement properties are limited. We will investigate the interrater reliability (IRR), usability, and applicability of A MeaSurement Tool to Assess systematic Reviews (AMSTAR), AMSTAR 2, and Risk Of Bias In Systematic reviews (ROBIS) for SRs in the fields of biomedicine and public health.

METHODS

An international team of researchers at three collaborating centres will undertake the study. We will use a random sample of 30 SRs of RCTs investigating therapeutic interventions indexed in MEDLINE in February 2014. Two reviewers at each centre will appraise the quality and risk of bias in each SR using AMSTAR, AMSTAR 2, and ROBIS. We will record the time to complete each assessment and for the two reviewers to reach consensus for each SR. We will extract the descriptive characteristics of each SR, the included studies, participants, interventions, and comparators. We will also extract the direction and strength of the results and conclusions for the primary outcome. We will summarise the descriptive characteristics of the SRs using means and standard deviations, or frequencies and proportions. To test for interrater reliability between reviewers and between the consensus agreements of reviewer pairs, we will use Gwet's AC₁ statistic. For comparability to previous evaluations, we will also calculate weighted Cohen's kappa and Fleiss' kappa statistics. To estimate usability, we will calculate the mean time to complete the appraisal and to reach consensus for each tool. To inform applications of the tools, we will test for statistical associations between quality scores and risk of bias judgments, and the results and conclusions of the SRs.

DISCUSSION

Appraising the methodological and reporting quality of SRs is necessary to determine the trustworthiness of their conclusions. Which tool may be most reliably applied and how the appraisals should be used is uncertain; the usability of newly developed tools is unknown. This investigation of common (AMSTAR) and newly developed (AMSTAR 2, ROBIS) tools will provide empiric data to inform their application, interpretation, and refinement.

Dissemination of evidence in paediatric emergency medicine: a quantitative descriptive evaluation of a 16-week social media promotion

OBJECTIVES

TRanslating Emergency Knowledge for Kids (TREKK) and Cochrane Child Health collaborate to develop knowledge products on paediatric emergency medicine topics. Via a targeted social media promotion, we aimed to increase user interaction with the TREKK and Cochrane Child Health Twitter accounts and the uptake of TREKK Bottom Line Recommendations (BLRs) and Cochrane systematic reviews (SRs).

DESIGN

Quantitative descriptive evaluation.

SETTING

We undertook this study and collected data via the internet.

PARTICIPANTS

Our target users included online healthcare providers and health consumers.

INTERVENTION

For 16 weeks, we used Twitter accounts (@TREKKca and @Cochrane_Child) and the Cochrane Child Health blog to promote 6 TREKK BLRs and 16 related Cochrane SRs. We published 1 blog post and 98 image-based tweets per week.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome was user interaction with @TREKKca and @Cochrane_Child. Secondary outcomes were visits to TREKK's website and the Cochrane Child Health blog, clicks to and views of the TREKK BLRs, and Altmetric scores and downloads of Cochrane SRs.

RESULTS

Followers to @TREKKca and @Cochrane_Child increased by 24% and 15%, respectively. Monthly users of TREKK's website increased by 29%. Clicks to the TREKK BLRs increased by 22%. The BLRs accrued 59% more views compared with the baseline period. The 16 blog posts accrued 28% more views compared with the 8 previous months when no new posts were published. The Altmetric scores for the Cochrane SRs increased by ≥10 points each. The mean number of full text downloads for the promotion period was higher for nine and lower for seven SRs compared with the 16-week average for the previous year (mean difference (SD), +4.0 (22.0%)).

CONCLUSIONS

There was increased traffic to TREKK knowledge products and Cochrane SRs during the social media promotion. Quantitative evidence supports blogging and tweeting as dissemination strategies for evidence-based knowledge products.

Procedural Pain: Systematic Review of Parent Experiences and Information Needs

Parents wish to reduce their child's pain during medical procedures but may not know how to do so. We systematically reviewed the literature on parents' experiences and information needs related to managing their child's pain for common medical procedures. Of 2678 records retrieved through database searching, 5 were included. Three additional records were identified by scanning reference lists. Five studies were qualitative, and 3 were quantitative. Most took place in North America or Europe (n = 7) and described neonatal intensive care unit experiences (n = 5). Procedures included needle-related medical procedures (eg, venipuncture, phlebotomy, intravenous insertion), sutures, and wound repair and treatment, among others. Generally, parents desired being present during procedures, wanted to remain stoic for their child, and thought that information would be empowering and relieve stress but felt unsupported in taking an active role. Supporting and educating parents may empower them to lessen pain for their children while undergoing medical procedures.

Technology-assisted title and abstract screening for systematic reviews: a retrospective evaluation of the Abstrackr machine learning tool

BACKGROUND

Machine learning tools can expedite systematic review (SR) processes by semi-automating citation screening. Abstrackr semi-automates citation screening by predicting relevant records. We evaluated its performance for four screening projects.

METHODS

We used a convenience sample of screening projects completed at the Alberta Research Centre for Health Evidence, Edmonton, Canada: three SRs and one descriptive analysis for which we had used SR screening methods. The projects were heterogeneous with respect to search yield (median 9328; range 5243 to 47,385 records; interquartile range (IQR) 15,688 records), topic (Antipsychotics, Bronchiolitis, Diabetes, Child Health SRs), and screening complexity. We uploaded the records to Abstrackr and screened until it made predictions about the relevance of the remaining records. Across three trials for each project, we compared the predictions to human reviewer decisions and calculated the sensitivity, specificity, precision, false negative rate, proportion missed, and workload savings.

RESULTS

Abstrackr's sensitivity was > 0.75 for all projects and the mean specificity ranged from 0.69 to 0.90 with the exception of Child Health SRs, for which it was 0.19. The precision (proportion of records correctly predicted as relevant) varied by screening task (median 26.6%; range 14.8 to 64.7%; IQR 29.7%). The median false negative rate (proportion of records incorrectly predicted as irrelevant) was 12.6% (range 3.5 to 21.2%; IQR 12.3%). The workload savings were often large (median 67.2%, range 9.5 to 88.4%; IQR 23.9%). The proportion missed (proportion of records predicted as irrelevant that were included in the final report, out of the total number predicted as irrelevant) was 0.1% for all SRs and 6.4% for the descriptive analysis. This equated to 4.2% (range 0 to 12.2%; IQR 7.8%) of the records in the final reports.

CONCLUSIONS

Abstrackr's reliability and the workload savings varied by screening task. Workload savings came at the expense of potentially missing relevant records. How this might affect the results and conclusions of SRs needs to be evaluated. Studies evaluating Abstrackr as the second reviewer in a pair would be of interest to determine if concerns for reliability would diminish. Further evaluations of Abstrackr's performance and usability will inform its refinement and practical utility.

Consent and recruitment: the reporting of paediatric trials published in 2012

OBJECTIVES

We evaluated 300 paediatric trials to determine: the consent and recruitment strategies used, who trial information was targeted to, how incentives were used and if they achieved their recruitment targets.

METHODS

For this cross-sectional evaluation, we searched the Cochrane Central Register of Controlled Trials for paediatric trials published in 2012 and randomly selected 300 that reported on outcomes for participants aged ≤21 years. We collected data on consent and recruitment procedures for each trial and undertook descriptive analyses in SPSS statistics V.23.

RESULTS

All but one trial (99.7%) used a standard recruitment strategy. Most (92%) trials reported that consent was obtained but only 13% reported who obtained consent. Two-thirds (65%) of trials included school-aged participants, and of these 68% reported obtaining assent. Half (50%) of the trials reported who the trial information was targeted to. Most trials (75%) of school-aged participants targeted information towards children or children and their parents. Fourteen per cent of trials reported using incentives, half (50%) of which were in the form of compensation. Only 48% of trials reported sufficient data to determine if their recruitment targets were achieved. Of these, 70% achieved their targets.

CONCLUSIONS

Notable reporting shortcomings included: how families were recruited into the trial, who obtained consent and/or assent and how, who trial information was directed to, whether incentives were used and sufficient data to determine if the recruitment target was achieved. Forthcoming paediatric-specific reporting standards may improve reporting in this priority area. Our data provide a baseline for ongoing monitoring of the state of the research.

Technology-assisted risk of bias assessment in systematic reviews: a prospective cross-sectional evaluation of the RobotReviewer machine learning tool

OBJECTIVES

To evaluate the reliability of RobotReviewer's risk of bias judgments.

STUDY DESIGN AND SETTING

In this prospective cross-sectional evaluation, we used RobotReviewer to assess risk of bias among 1,180 trials. We computed reliability with human reviewers using Cohen's kappa coefficient and calculated sensitivity and specificity. We investigated differences in reliability by risk of bias domain, topic, and outcome type using the chi-square test in meta-analysis.

RESULTS

Reliability (95% CI) was moderate for random sequence generation (0.48 [0.43, 0.53]), allocation concealment (0.45 [0.40, 0.51]), and blinding of participants and personnel (0.42 [0.36, 0.47]); fair for overall risk of bias (0.34 [0.25, 0.44]); and slight for blinding of outcome assessors (0.10 [0.06, 0.14]), incomplete outcome data (0.14 [0.08, 0.19]), and selective reporting (0.02 [-0.02, 0.05]). Reliability for blinding of participants and personnel (P < 0.001), blinding of outcome assessors (P = 0.005), selective reporting (P < 0.001), and overall risk of bias (P < 0.001) differed by topic. Sensitivity and specificity (95% CI) ranged from 0.20 (0.18, 0.23) to 0.76 (0.72, 0.80) and from 0.61 (0.56, 0.65) to 0.95 (0.93, 0.96), respectively.

CONCLUSION

Risk of bias appraisal is subjective. Compared with reliability between author groups, RobotReviewer's reliability with human reviewers was similar for most domains and better for allocation concealment, blinding of participants and personnel, and overall risk of bias.

Which outcomes are important to patients and families who have experienced paediatric acute respiratory illness? Findings from a mixed methods sequential exploratory study

OBJECTIVES

To identify the outcome priorities of parents of children who had experienced an acute respiratory infection (ARI).

DESIGN

This was a two-phase, mixed methods study with a sequential exploratory design. We used a cross-sectional quantitative web-based survey to elicit parents' priorities for paediatric ARI. We then used a discussion moderated via Facebook to elucidate richer descriptions of parents' priorities.

SETTING

Survey and discussion data were collected via the internet.

PARTICIPANTS

110 parents (90% women, median age 35 years, 92.7% urban dwelling, 94.5% with a postsecondary education) with a child who had experienced an ARI responded to the survey. Four parents participated in the Facebook discussion.

PRIMARY AND SECONDARY OUTCOME MEASURES

The primary outcome was parents' rankings of outcomes related to paediatric ARI. The secondary outcomes were the alignment of parent-reported important outcomes with those commonly reported in Cochrane systematic reviews (SRs).

RESULTS

Commonly reported ARIs included croup (44.5%), wheezing (43.6%) and influenza (38.2%). Parents ranked major complications, illness symptoms and length of stay as the most important outcome categories. With respect to specific outcomes, severe complications, major side effects, doctor's assessment, relapse, oxygen supplementation and results from laboratory measures were reported as most important (75th-99th percentile). Taking time off work, mild complications, interference with daily activities, treatment costs, absenteeism, follow-up visits and other costs were deemed minimally important (<25th percentile). In 35 Cochrane SRs, 29 unique outcomes were reported. Although participants' priorities sometimes aligned with outcomes frequently reported in the literature, this was not always true. Additional priorities from the survey (n=50) and Facebook discussions (n=4) included healthcare access, interacting with healthcare providers, education, impact on daily activities and child well-being.

CONCLUSIONS

In the context of paediatric ARI, parents' priorities did not always align with commonly researched outcomes. Appealing and efficient strategies to engage patients and parents in research should be developed.

Parent experiences and information needs relating to procedural pain in children: a systematic review protocol

BACKGROUND

There exist many evidence-based interventions available to manage procedural pain in children and neonates, yet they are severely underutilized. Parents play an important role in the management of their child's pain; however, many do not possess adequate knowledge of how to effectively do so. The purpose of the planned study is to systematically review and synthesize current knowledge of the experiences and information needs of parents with regard to the management of their child's pain and distress related to medical procedures in the emergency department.

METHODS

We will conduct a systematic review using rigorous methods and reporting based on the PRISMA statement. We will conduct a comprehensive search of literature published between 2000 and 2016 reporting on parents' experiences and information needs with regard to helping their child manage procedural pain and distress. Ovid MEDLINE, Ovid PsycINFO, CINAHL, and PubMed will be searched. We will also search reference lists of key studies and gray literature sources. Two reviewers will screen the articles following inclusion criteria defined a priori. One reviewer will then extract the data from each article following a data extraction form developed by the study team. The second reviewer will check the data extraction for accuracy and completeness. Any disagreements with regard to study inclusion or data extraction will be resolved via discussion. Data from qualitative studies will be summarized thematically, while those from quantitative studies will be summarized narratively. The second reviewer will confirm the overarching themes resulting from the qualitative and quantitative data syntheses. The Critical Appraisal Skills Programme Qualitative Research Checklist and the Quality Assessment Tool for Quantitative Studies will be used to assess the quality of the evidence from each included study.

DISCUSSION

To our knowledge, no published review exists that comprehensively reports on the experiences and information needs of parents related to the management of their child's procedural pain and distress. A systematic review of parents' experiences and information needs will help to inform strategies to empower them with the knowledge necessary to ensure their child's comfort during a painful procedure.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016043698.

Peer-led nutrition education programs for school-aged youth: a systematic review of the literature

To date, the impacts of school-based, peer-led nutrition education initiatives have not been summarized or assessed collectively. This review presents the current evidence, identifies knowledge gaps, and provides recommendations for future research. PubMed, Scopus, ERIC and Google Scholar were searched for refereed Canadian and American primary studies published between January 2000 and November 2013, following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Seventeen articles (11 programs) from Canada (24%) and the United States (76%) were identified. The results were summarized in terms of the study population, program design and main outcomes. Common outcome measures included healthy eating knowledge (n = 5), self-efficacy or attitudes towards healthy eating (n = 13), dietary measures (n = 9) and body mass index (n = 4), all of which tended to improve as a result of the programs. More research is needed to ascertain the effect of improvements in knowledge, self-efficacy and attitudes towards healthy eating on food behaviors. When evaluated, programs were generally well received, while the long-term maintenance of positive impacts was a challenge. Studies of sustainability and feasibility to promote long-term impact are a logical next step.

Body weight perceptions and eating-related weight control behaviors of on-reserve First Nations youth from Ontario, Canada

INTRODUCTION

Research investigating the body weight perceptions and eating-related weight control behaviors of First Nations (FN) youth living on reserve in Canada has been scarce. Knowledge of body weight perceptions may help to improve the relevance of initiatives promoting healthy weights. The purpose of this study was to examine the body weight perceptions and eating-related weight control behaviors of grade 6-8 on-reserve FN youth from seven Ontario communities.

METHODS

Data were collected from December 2003 to June 2010 from a convenience sample of FN youth (aged 10-14 years) using the Waterloo Web-based Eating Behaviour Questionnaire (WEB-Q). Participants were categorized into body mass index (BMI) categories based on International Obesity Task Force (IOTF) cut points. Frequency statistics were computed in the comparison of measured BMI and weight perceptions and weight control behaviors. Differences by BMI category were tested using Pearson &#935;2 tests.

RESULTS

A total of 267 youth from seven Ontario FN communities participated in the study (48.6% male). Overall, 36.3% of youth were overweight and 21.3% were obese (combined total of 57.6%). Similar to non-Aboriginal youth, a greater proportion of FN girls who were at a normal weight were concerned that their weight was too high compared to boys. However, one-third of normal weight boys were currently trying to lose weight. A greater proportion of obese girls were trying to lose weight compared to boys. Overall, a large proportion of both overweight boys and girls were attempting to gain weight.

CONCLUSIONS

The present study provides a unique investigation into the weight perceptions and weight control behaviors of on-reserve FN youth living in isolated communities in Ontario, Canada. Many of the perceptions elucidated in this study are similar to those observed in non-Aboriginal youth, while others differed. The knowledge of these perceptions and further research to investigate what factors influences them will help to customize health promoting initiatives that are relevant to the youth in the participating communities.

ClereMed: Lessons Learned From a Pilot Study of a Mobile Screening Tool to Identify and Support Adults Who Have Difficulty With Medication Labels

BACKGROUND

In order to take medications safely and effectively, individuals need to be able to see, read, and understand the medication labels. However, one-half of medication labels are currently misunderstood, often because of low literacy, low vision, and cognitive impairment. We sought to design a mobile tool termed ClereMed that could rapidly screen for adults who have difficulty reading or understanding their medication labels.

OBJECTIVE

The aim of this study was to build the ClereMed prototype; to determine the usability of the prototype with adults 55 and over; to assess its accuracy for identifying adults with low-functional reading ability, poor ability on a real-life pill-sorting task, and low cognition; and to assess the acceptability of a touchscreen device with older adults with age-related changes to vision and cognition.

METHODS

This pilot study enrolled adults (≥55 years) who were recruited through pharmacies, retirement residences, and a low-vision optometry clinic. ClereMed is a hypertext markup language (HTML)-5 prototype app that simulates medication taking using an iPad, and also provides information on how to improve the accessibility of prescription labels. A paper-based questionnaire included questions on participant demographics, computer literacy, and the Systems Usability Scale (SUS). Cognition was assessed using the Montreal Cognitive Assessment tool, and functional reading ability was measured using the MNRead Acuity Chart. Simulation results were compared with a real-life, medication-taking exercise using prescription vials, tablets, and pillboxes.

RESULTS

The 47 participants had a mean age of 76 (SD 11) years and 60% (28/47) were female. Of the participants, 32% (15/47) did not own a computer or touchscreen device. The mean SUS score was 76/100. ClereMed correctly identified 72% (5/7) of participants with functional reading difficulty, and 63% (5/8) who failed a real-life pill-sorting task, but only 21% (6/28) of participants with cognitive impairment. Participants who owned a computer or touchscreen completed ClereMed in a mean time of 26 (SD 16) seconds, compared with 52 (SD 34) seconds for those who do not own a device (P<.001). Those who had difficulty, struggled with screen glare, button activation, and the "drag and drop" function.

CONCLUSIONS

ClereMed was well accepted by older participants, but it was only moderately accurate for reading ability and not for mild cognitive impairment. Future versions may be most useful as part of a larger medication assessment or as a tool to help family members and caregivers identify individuals with impaired functional reading ability. Future research is needed to improve the sensitivity for measuring cognitive impairment and on the feasibility of implementing a mobile app into pharmacy workflow.

Evaluating user perceptions of mobile medication management applications with older adults: a usability study

BACKGROUND

Medication nonadherence has a significant impact on the health and wellbeing of individuals with chronic disease. Several mobile medication management applications are available to help users track, remember, and read about their medication therapy.

OBJECTIVE

The objective of this study was to explore the usability and usefulness of existing medication management applications for older adults.

METHODS

We recruited 35 participants aged 50 and over to participate in a 2-hour usability session. The average age ranged from 52-78 years (mean 67 years) and 71% (25/35) of participants were female. Each participant was provided with an iPad loaded with four medication management applications: MyMedRec, DrugHub, Pillboxie, and PocketPharmacist. These applications were evaluated using the 10 item System Usability Scale (SUS) and visual analog scale. An investigator-moderated 30-minute discussion followed, and was recorded. We used a grounded theory (GT) approach to analyze qualitative data.

RESULTS

When assessing mobile medication management applications, participants struggled to think of a need for the applications in their own lives. Many were satisfied with their current management system and proposed future use only if cognition and health declined. Most participants felt capable of using the applications after a period of time and training, but were frustrated by their initial experiences with the applications. The early experiences of participants highlighted the benefits of linear navigation and clear wording (eg, "undo" vs "cancel") when designing for older users. While there was no order effect, participants attributed their poor performance to the order in which they tried the applications. They also described being a part of a technology generation that did not encounter the computer until adulthood. Of the four applications, PocketPharmacist was found to be the least usable with a score of 42/100 (P<.0001) though it offered a drug interaction feature that was among the favorite features of participants. The usability scores for MyMedRec (56/100), DrugHub (57/100), and Pillboxie (52/100) were not significantly different and participants preferred MyMedRec and DrugHub for their simple, linear interfaces.

CONCLUSIONS

With training, adults aged 50 and over can be capable and interested in using mHealth applications for their medication management. However, in order to adopt such technology, they must find a need that their current medication management system cannot fill. Interface diversity and multimodal reminder methods should be considered to increase usability for older adults. Lastly, regulation or the involvement of older adults in development may help to alleviate generation bias and mistrust for applications.

Body Mass Index of First Nations youth in Ontario, Canada: influence of sleep and screen time

INTRODUCTION

Prevalence rates of overweight and obesity in Canada have risen rapidly in the past 20 years. Concurrent with the obesity epidemic, sleep time and physical activity levels have decreased among youth. Aboriginal youth experience disproportionately high obesity prevalence but there is inadequate knowledge of contributing factors. This research aimed to examine sleep and screen time behavior and their relationship to Body Mass Index (BMI) in on-reserve First Nations youth from Ontario, Canada.

METHODS

This was an observational population-based study of cross-sectional design. Self-reported physical activity, screen time, and lifestyle information was collected from 348 youth aged 10-18 years residing in five northern, remote First Nations communities and one southern First Nations community in Ontario, Canada, from October 2004 to June 2010. Data were collected in the school setting using the Waterloo Web-based Eating Behaviour Questionnaire. Based on self-reported height and weight, youth were classified normal (including underweight), overweight and obese according to BMI. Descriptive cross-tabulations and Pearson's χ² tests were used to compare screen time, sleep habits, and physical activity across BMI categories.

RESULTS

Participants demonstrated low levels of after-school physical activity, and screen time in excess of national guidelines. Overall, 75.5% reported being active in the evening three or less times per week. Approximately one-quarter of the surveyed youth watched more than 2 hours of television daily and 33.9% spent more than 2 hours on the internet or playing video games. For boys, time using the internet/video games (p=0.022) was positively associated with BMI category, with a greater than expected proportion of obese boys spending more than 2 hours using the internet or video games daily (56.7%). Also for boys, time spent outside after school (p=0.033) was negatively associated with BMI category, with a lesser than expected proportion spending 'most of the time' outside (presumably being active) after school. These relationships were not observed in girls. Adjusted standardized residuals suggest a greater than expected proportion of obese individuals had a television in their bedroom (66.7%) as compared with the rest of the population.

CONCLUSIONS

The current study adds to the limited information about contributors to overweight and obesity in First Nations youth living on-reserve in Canada. Concerns about inadequate sleep, excess screen time, and inadequate physical activity mirror those of the general population. Further investigation is warranted to improve the understanding of how various lifestyle behaviors influence overweight, obesity, and the development of chronic disease among First Nations youth. Initiatives to reduce screen time, increase physical activity, and encourage adequate sleep among on-reserve First Nations youth are recommended.

A pilot comprehensive school nutrition program improves knowledge and intentions for intake of milk and milk alternatives among youth in a remote first nation

OBJECTIVE

To evaluate the impact of a pilot comprehensive school nutrition program modeled on Social Cognitive Theory on knowledge, intentions, self-efficacy, and intake of milk and milk alternatives (MMA) in First Nations youth.

METHODS

A pilot school nutrition program was implemented at Peetabeck Academy in Fort Albany, Ontario in May, 2010. The Knowledge, Self-Efficacy, and Intentions Questionnaire (KSIQ) and Waterloo Web-based Eating Behavior Questionnaire (WEB-Q) were used to assess change in attitudes and behavior from pre- to postprogram.

RESULTS

The KSIQ preprogram (n = 26), postprogram (n = 19); WEB-Q preprogram (n = 30), postprogram (n = 10). Improved knowledge (6.0 ± 1.5 vs. 6.9 ± 1.5, P = .05) and intention scores (9.6 ± 4.4 vs. 11.3 ± 4.1, P = .01) were observed.

CONCLUSIONS AND IMPLICATIONS

A comprehensive school nutrition program can improve knowledge and intentions for intake of MMA in First Nations youth. Environmental constraints beyond the school environment need to be addressed.