Tackling publication bias and selective reporting in health informatics research: register your eHealth trials in the International eHealth Studies Registry

Gray Literature in Evaluating Effectiveness in Digital Health and Health and Welfare Technology: A Source Worth Considering

Background

The need to assess the effectiveness and value of interventions involving digital health and health and welfare technologies is becoming increasingly important due to the rapidly growing development of these technologies and their areas of application. Systematic reviews of scientific literature are a mainstay of such assessment, but publications outside the realm of traditional scientific bibliographic databases—known as gray literature—are often not included. This is a disadvantage, particularly apparent in the health and welfare technology (HWT) domain.

Objective

The aim of this article is to investigate the significance of gray literature in digital health and HWT when reviewing literature. As an example, the impact of including gray literature to the result of two systematic reviews in HWT is examined.

Methods

In this paper, we identify, discuss, and suggest methods for including gray literature sources when evaluating effectiveness and appropriateness for different review types related to HWT. The analysis also includes established sources, search strategies, documentation, and reporting of searches, as well as bias and credibility assessment. The differences in comparison to scientific bibliographic databases are elucidated. We describe the results, challenges, and benefits of including gray literature in 2 examples of systematic reviews of HWT.

Results

In the 2 systematic reviews described in this paper, most included studies came from context-specific gray literature sources. Gray literature contributed to the overall result of the reviews and corresponded well with the reviews’ aims. The assessed risk of bias of the included studies derived from gray literature was similar to the included studies from other types of sources. However, because of less standardized publication formats, assessing and extracting data from gray literature studies were more time-consuming and compiling statistical results was not possible. The search process for gray literature required more time and the reproducibility of gray literature searches were less certain due to more unstable publication platforms.

Conclusions

Gray literature is particularly relevant for digital health and HWT but searches need to be conducted systematically and reported transparently. This way gray literature can broaden the range of studies, highlight context specificity, and decrease the publication bias of reviews of effectiveness of HWT. Thus, researchers conducting systematic reviews related to HWT should consider including gray literature based on a systematic approach.

Prospective trial registration and publication rates of randomized clinical trials in digital health: A cross-sectional analysis of global trial registries

Objectives

We sought to examine the prevalence of prospective registration and publication rates in digital health trials.

Materials and Methods

We included 417 trials that enrolled participants in 2012 and were registered in any of the 17 WHO data provider registries. The evaluation of the prospective trial registration was based on the actual difference between the registration and enrollment dates. We identified existing publications through an automated PubMed search by every trial registration number as well as a pragmatic search in PubMed and Google based on extracted metadata from the trial registries.

Results

The prospective registration and publication rates were at (38.4%) and (65.5%), respectively. We identified a statistically significant ( p < 0.001) “Selective Registration Bias” with 95.7% of trials published within a year after registration, were registered retrospectively. We reported a statistically significant relationship ( p = 0.003) between prospective registration and funding sources, with industry-funded trials having the lowest compliance with prospective registration at (14.3%). The lowest non-publication rates were in the Middle East (26.7%) and Europe (28%), and the highest were in Asia (56.5%) and the U.S. (42.5%). We found statistically significant differences ( p < 0.001) between trial location and funding sources with the highest percentage of industry-funded trials in Asia (17.4%) and the U.S. (3.3%).

Conclusion

The adherence of investigators to the best practices of trial registration and result dissemination is still evolving in digital health trials. Further research is required to identify contributing factors and mitigation strategies to low compliance rate with trial publication and prospective registration in digital health trials.

A systematic review of the effects of implementing clinical pathways supported by health information technologies

OBJECTIVE

Health information technology (HIT) interventions include electronic patient records, prescribing, and ordering systems. Clinical pathways are multidisciplinary plans of care that enable the delivery of evidence-based healthcare. Our objective was to systematically review the effects of implementing HIT-supported clinical pathways.

MATERIALS AND METHODS

A systematic review protocol was developed including Medline, Embase, and CENTRAL database searches. We recorded data relating to study design, participants, intervention, and outcome characteristics and formally assessed risk of bias.

RESULTS

Forty-four studies involving more than 270 000 patients were included. Investigation methodologies included before-after (n = 16, 36.4%), noncomparative (n = 14, 31.8%), interrupted time series (n = 5, 11.4%), retrospective cohort (n = 4, 9.1%), cluster randomized (n = 2, 4.5%), controlled before-after (n = 1, 2.3%), prospective case-control (n = 1, 2.3%), and prospective cohort (n = 1, 2.3%) study designs. Clinical decision support (n = 25, 56.8%), modified electronic documentation (n = 23, 52.3%), and computerized provider order entry (n = 23, 52.3%) were the most frequently utilized HIT interventions. The majority of studies (n = 38, 86.4%) reported benefits associated with HIT-supported pathways. These included reported improvements in objectively measured patient outcomes (n = 15, 34.1%), quality of care (n = 29, 65.9%), and healthcare resource utilization (n = 10, n = 22.7%).

DISCUSSION

Although most studies reported improvements in outcomes, the strength of evidence was limited by the study designs that were utilized.

CONCLUSIONS

Ongoing evaluations of HIT-supported clinical pathways are justified but would benefit from study designs that report key outcomes (including adverse events) and minimize the risk of bias.

Quality assessment of clinical trial registration with traditional Chinese medicine in WHO registries

OBJECTIVE

This study aimed to assess the registration quality of clinical trials (CTs) with traditional Chinese medicine (TCM) in the WHO International Clinical Trials Registry Platform (ICTRP) and identify the common problems if any.

METHODS

The ICTRP database was searched for all TCM CTs that were registered up to 31 December 2017. Registered information of each trial was collected from specific registry involved in ICTRP through hyperlink. The primary analysis was to assess the reporting quality of registered trials with TCM interventions, which is based on the minimum 20 items of WHO Trial Registration Data Set (TRDS, V.1.2.1) plus optional additional three items recommended by ICTRP, and some specific items for TCM information (including TCM intervention, diagnosis, outcome and rationale). Descriptive statistics were additionally used to analyse the baseline characteristics of TCM trial registrations.

RESULTS

A total of 3339 records in 15 registries were examined. The number of TCM registered trials has increased rapidly after the requirement of mandatory trial registration proposed by International Committee of Medical Journal Editors on 1 July 2005, and the top two registries were Chinese Clinical Trial Registry and ClincialTrials.gov. Of 3339 trials, 61% were prospective registration and 12.8% shared resultant publications. There were 2955 interventional trials but none of them had a 100% reporting rate of the minimum 20 items and additional three items. The reporting quality of these 23 items was not optimal due to 11 of them had a lower reporting rate (<65%). For TCM details, 49.2% lacked information on description of TCM intervention(s), 85.9% did not contain TCM diagnosis criteria, 92.6% did not use TCM outcome(s) and 67.1% lacked information on TCM background and rationale.

CONCLUSION

The registration quality of TCM CTs should be improved by prospective registration, full completion of WHO TRDS, full reporting of TCM information and results sharing. Further full set of trial registration items for TCM trials should be developed thus to standardise the content of TCM trial registration.

Nonpublication Rates and Characteristics of Registered Randomized Clinical Trials in Digital Health: Cross-Sectional Analysis

Background

Clinical trials are key to advancing evidence-based medical research. The medical research literature has identified the impact of publication bias in clinical trials. Selective publication for positive outcomes or nonpublication of negative results could misdirect subsequent research and result in literature reviews leaning toward positive outcomes. Digital health trials face specific challenges, including a high attrition rate, usability issues, and insufficient formative research. These challenges may contribute to nonpublication of the trial results. To our knowledge, no study has thus far reported the nonpublication rates of digital health trials.

Objective

The primary research objective was to evaluate the nonpublication rate of digital health randomized clinical trials registered in ClinicalTrials.gov. Our secondary research objective was to determine whether industry funding contributes to nonpublication of digital health trials.

Methods

To identify digital health trials, a list of 47 search terms was developed through an iterative process and applied to the “Title,” “Interventions,” and “Outcome Measures” fields of registered trials with completion dates between April 1, 2010, and April 1, 2013. The search was based on the full dataset exported from the ClinlicalTrials.gov database, with 265,657 trials entries downloaded on February 10, 2018, to allow publication of studies within 5 years of trial completion. We identified publications related to the results of the trials through a comprehensive approach that included an automated and manual publication-identification process.

Results

In total, 6717 articles matched the a priori search terms, of which 803 trials matched our latest completion date criteria. After screening, 556 trials were included in this study. We found that 150 (27%) of all included trials remained unpublished 5 years after their completion date. In bivariate analyses, we observed statistically significant differences in trial characteristics between published and unpublished trials in terms of the intervention target condition, country, trial size, trial phases, recruitment, and prospective trial registration. In multivariate analyses, differences in trial characteristics between published and unpublished trials remained statistically significant for the intervention target condition, country, trial size, trial phases, and recruitment; the odds of publication for non-US–based trials were significant, and these trials were 3.3 (95% CI 1.845-5.964) times more likely to be published than US–based trials. We observed a trend of 1.5 times higher nonpublication rates for industry-funded trials. However, the trend was not statistically significant.

Conclusions

In the domain of digital health, 27% of registered clinical trials results are unpublished, which is lower than nonpublication rates in other fields. There are substantial differences in nonpublication rates between trials funded by industry and nonindustry sponsors. Further research is required to define the determinants and reasons for nonpublication and, more importantly, to articulate the impact and risk of publication bias in the field of digital health trials.

Diabetes and the direct secondary use of electronic health records: Using routinely collected and stored data to drive research and understanding

Introduction

Electronic health records provide an unparalleled opportunity for the use of patient data that is routinely collected and stored, in order to drive research and develop an epidemiological understanding of disease. Diabetes, in particular, stands to benefit, being a data-rich, chronic-disease state. This article aims to provide an understanding of the extent to which the healthcare sector is using routinely collected and stored data to inform research and epidemiological understanding of diabetes mellitus.

Methods

Narrative literature review of articles, published in both the medical- and engineering-based informatics literature.

Results

There has been a significant increase in the number of papers published, which utilise electronic health records as a direct data source for diabetes research. These articles consider a diverse range of research questions. Internationally, the secondary use of electronic health records, as a research tool, is most prominent in the USA. The barriers most commonly described in research studies include missing values and misclassification, alongside challenges of establishing the generalisability of results.

Discussion

Electronic health record research is an important and expanding area of healthcare research. Much of the research output remains in the form of conference abstracts and proceedings, rather than journal articles. There is enormous opportunity within the United Kingdom to develop these research methodologies, due to national patient identifiers. Such a healthcare context may enable UK researchers to overcome many of the barriers encountered elsewhere and thus to truly unlock the potential of electronic health records.

Evidence-based Health Informatics: How Do We Know What We Know?

Background: Health IT is expected to have a positive impact on the quality and efficiency of health care. But reports on negative impact and patient harm continue to emerge. The obligation of health informatics is to make sure that health IT solutions provide as much benefit with as few negative side effects as possible. To achieve this, health informatics as a discipline must be able to learn, both from its successes as well as from its failures.

Objectives: To present motivation, vision, and history of evidence-based health informatics, and to discuss achievements, challenges, and needs for action.

Methods: Reflections on scientific literature and on own experiences.

Results: Eight challenges on the way towards evidence-based health informatics are identified and discussed: quality of studies; publication bias; reporting quality; availability of publications; systematic reviews and meta-analysis; training of health IT evaluation experts; translation of evidence into health practice; and post-market surveil-lance. Identified needs for action comprise: establish health IT study registers; increase the quality of publications; develop a taxonomy for health IT systems; improve indexing of published health IT evaluation papers; move from meta-analysis to meta-summaries; include health IT evaluation competencies in curricula; develop evidence-based implementation frameworks; and establish post-marketing surveillance for health IT.

Conclusions: There has been some progress, but evidence-based health informatics is still in its infancy. Building evidence in health informatics is our obligation if we consider medical informatics a scientific discipline.

Public Health, Population Health, and Epidemiology Informatics: Recent Research and Trends in the United States

Objectives: To survey advances in public and population health and epidemiology informatics over the past 18 months. Methods: We conducted a review of English-language research works conducted in the domain of public and population health informatics and published in MEDLINE or Web of Science between January 2015 and June 2016 where information technology or informatics was a primary subject or main component of the study methodology. Selected articles were presented using a thematic analysis based on the 2011 American Medical Informatics Association (AMIA) Public Health Informatics Agenda tracks as a typology. Results: Results are given within the context developed by Dixon et al., (2015) and key themes from the 2011 AMIA Public Health Informatics Agenda. Advances are presented within a socio-technical infrastructure undergirded by a trained, competent public health workforce, systems development to meet the business needs of the practice field, and research that evaluates whether those needs are adequately met. The ability to support and grow the infrastructure depends on financial sustainability. Conclusions: The fields of public health and population health informatics continue to grow, with the most notable developments focused on surveillance, workforce development, and linking to or providing clinical services, which encompassed population health informatics advances. Very few advances addressed the need to improve communication, coordination, and consistency with the field of informatics itself, as identified in the AMIA agenda. This will likely result in the persistence of the silos of public health information systems that currently exist. Future research activities need to aim toward a holistic approach of informatics across the enterprise.

Magnitude and characteristics of clinical trials in the Kingdom of Saudi Arabia: A cross-sectional analysis

The clinical trial is an important type of research design in the spectrum of translational research. The extent to which clinical trials are conducted is a reflection of the level of advancement that exists within a healthcare system. This study aims at describing the clinical trial activity within the Kingdom of Saudi Arabia since 2000 through reviewing those trials that have been registered with clinicaltrials.gov in that time period. Since February 2000, 405 trials have been registered. These trials fall into one of 22 different ICD-10 codes, and with the top four being neoplasms (92), diseases of the circulatory system (57), endocrine, nutritional and metabolic diseases (46), and diseases of the respiratory system (25). About half (200) were classified as trials with both safety and efficacy endpoints. 52% were phase IV and 28% were phase III. About 64% were randomized, and with about equal numbers of those coming from industry (86) and university sponsors (85), and smaller numbers coming from hospitals (51) and other sponsors. A total of 24 phase III university- or hospital-sponsored trials have been registered during the 15-year time period. With a population approaching 30 million and very large annual healthcare expenses, it would appear that the level of clinical trial activity within the Kingdom during the past 15 years has been rather paltry. The emphasis has been on post-marketing phase IV trials. The academic setting (i.e. universities and hospitals) has seen a new trial registered every 11 months on average.

DEC-net: the development of the European register of clinical trials on medicines for children

Background The need to register a clinical trial in a public database has been widely discussed in the last few years. Several general clinical trial registers have been developed, but none are dedicated to children. A web-based register of trials on drug therapy in children was therefore developed.

Purpose The general objective of the project, called DEC-NET ( European register of clinical trials on medicines for children - drug evaluation in children) and supported by the European Community, is to provide the scientific community with a flexible tool for promoting communication and collaboration among researchers, disseminating clinical trial results and facilitating patient access and recruitment to trials.

Methods We identified a set of trial data to be collected, including general trial information, recruitment information, eligibility criteria, clinical trial characteristics, location and contact information, study drugs, study sponsor and results. The system was set up to allow authorized users (working in Italy, Spain, France and UK) to enter, handle and monitor the trial information. Medical data dictionaries (ie, ICD-9 and ATC) were used in order to make the data-entry and the search and data analysis procedures easier. A multilingual approach in data handling and presentation made accessing trial information in English and in the language of the country where the study is located feasible. Search and browsing tools were set up to allow investigators, researchers, and citizens to freely search and browse the entire register and access the details of trials meeting the search criteria.

Results The DEC-net register (http://www.dec-net.org) was opened for trial notification and consultation.

Limitations DEC-net currently involves four European countries and is a pilot project.

Conclusions It is hoped that what has been achieved so far represents only the beginning of a long-term project, and the next step will be to extend the register’s use to other European countries.

Effectiveness and cost-effectiveness of ehealth interventions in somatic diseases: a systematic review of systematic reviews and meta-analyses

Background

eHealth potentially enhances quality of care and may reduce health care costs. However, a review of systematic reviews published in 2010 concluded that high-quality evidence on the benefits of eHealth interventions was still lacking.

Objective

We conducted a systematic review of systematic reviews and meta-analyses on the effectiveness/cost-effectiveness of eHealth interventions in patients with somatic diseases to analyze whether, and to what possible extent, the outcome of recent research supports or differs from previous conclusions.

Methods

Literature searches were performed in PubMed, EMBASE, The Cochrane Library, and Scopus for systematic reviews and meta-analyses on eHealth interventions published between August 2009 and December 2012. Articles were screened for relevance based on preset inclusion and exclusion criteria. Citations of residual articles were screened for additional literature. Included papers were critically appraised using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Statement before data were extracted. Based on conclusions drawn by the authors of the included articles, reviews and meta-analyses were divided into 1 of 3 groups: suitable, promising, or limited evidence on effectiveness/cost-effectiveness. Cases of uncertainty were resolved by consensus discussion. Effect sizes were extracted from papers that included a meta-analysis. To compare our results with previous findings, a trend analysis was performed.

Results

Our literature searches yielded 31 eligible reviews, of which 20 (65%) reported on costs. Seven papers (23%) concluded that eHealth is effective/cost-effective, 13 (42%) underlined that evidence is promising, and others found limited or inconsistent proof. Methodological quality of the included reviews and meta-analyses was generally considered high. Trend analysis showed a considerable accumulation of literature on eHealth. However, a similar percentage of papers concluded that eHealth is effective/cost-effective or evidence is at least promising (65% vs 62%). Reviews focusing primarily on children or family caregivers still remained scarce. Although a pooled (subgroup) analysis of aggregate data from randomized studies was performed in a higher percentage of more recently published reviews (45% vs 27%), data on economic outcome measures were less frequently reported (65% vs 85%).

Conclusions

The number of reviews and meta-analyses on eHealth interventions in patients with somatic diseases has increased considerably in recent years. Most articles show eHealth is effective/cost-effective or at least suggest evidence is promising, which is consistent with previous findings. Although many researchers advocate larger, well-designed, controlled studies, we believe attention should be given to the development and evaluation of strategies to implement effective/cost-effective eHealth initiatives in daily practice, rather than to further strengthen current evidence.

Many randomized trials of physical therapy interventions are not adequately registered: a survey of 200 published trials

BACKGROUND

Clinical trial registration has several putative benefits: prevention of selective reporting, avoidance of duplication, encouragement of participation, and facilitation of reviews. Previous surveys suggest that most trials are registered. However, these surveys examined only trials in journals with high impact factors, which may bias the results.

PURPOSE

This study examined the completeness of clinical trial registration and the extent of selective reporting of outcomes in a random sample of published randomized trials in physical therapy.

DATA SOURCES

This was a retrospective cohort study in which 200 randomized trials of physical therapy interventions were randomly selected from those published in 2009 and indexed in the Physiotherapy Evidence Database (PEDro), regardless of the publishing journal.

DATA EXTRACTION

Evidence of registration was sought for each trial in the study, on clinical trial registers, and by contacting authors.

DATA SYNTHESIS

The proportion of randomized trials that were registered was 67/200 (34%). This proportion was significantly lower than among the trials in journals with high impact factors, where the proportion was 75% (odds ratio=7.4, 95% confidence interval=2.6-21.4). Unambiguous primary outcomes (ie, method and time points of measurement clearly defined in the trial registry entry) were registered for 32 trials, and registration was adequate (ie, prospective with unambiguous primary outcomes) for 5/200 (2.5%) trials. Selective outcome reporting occurred in 23 (47%) of the 49 trials in which selective reporting was assessable.

LIMITATIONS

The inclusion of only English-language trials prevents generalization of the results to non-English-language trials.

CONCLUSIONS

Registration of randomized trials of physical therapy interventions is rarely adequate. Consequently, the putative benefits of registration are not being fully realized.

The Brustkrebs-Studien.de website for breast cancer patients: User acceptance of a German internet portal offering information on the disease and treatment options, and a clinical trials matching service

Background

The internet portal http://www.brustkrebs-studien.de (BKS) was launched in 2000 by the German Society of Senology (DGS) and the Baden-Württemberg Institute for Women's Health (IFG) to provide expert-written information on breast cancer online and to encourage and facilitate the participation of breast cancer patients in clinical trials. We describe the development of BKS and its applications, and report on website statistics and user acceptance.

Methods

Existing registries, including ClinicalTrials.gov, were analysed before we designed BKS, which combines a trial registry, a knowledge portal, and an online second opinion service. An advisory board guided the process. Log files and patient enquiries for trial participation and second opinions were analysed. A two-week user satisfaction survey was conducted online.

Results

During 10/2005-06/2010, the portal attracted 702,655 visitors, generating 15,507,454 page views. By 06/2010, the website's active scientific community consisted of 189 investigators and physicians, and the registry covered 163 clinical trial protocols. In 2009, 143 patients requested trial enrolment and 119 sought second opinions or individual treatment advice from the expert panel. During the two-week survey in 2008, 5,702 BKS visitors submitted 507 evaluable questionnaires. Portal acceptance was high. Respondents trusted information correctness (80%), welcomed self-matching to clinical trials (79%) and planned to use the portal in the future (76%) and recommend it to others (81%).

Conclusions

BKS is an established and trusted breast cancer information platform offering up-to-date resources and protocols to the growing physician and patient community to encourage participation in clinical trials. Further studies are needed to assess potential increases in trial enrolment by eligibility matching services.

Vision and challenges of Evidence-Based Health Informatics: a case study of a CPOE meta-analysis

OBJECTIVE

To discuss, taking the example of a meta-analysis on computerized physician order entry (CPOE) systems, the special challenges of Evidence-Based Health Informatics, defined as the conscientious, explicit and judicious use of current best evidence when making decisions about introduction and operation of information technology in a given health care setting.

METHODS

We conducted a case study by performing a systematic review and meta-analysis of CPOE studies. We collected and discussed the challenges we addressed and how they could be overcome.

RESULTS

Challenges comprise the correct identification of published health informatics evaluation studies, the low reporting and study quality of studies, the problem of combining evidence from heterogeneous studies, and the problem of publication bias in health informatics.

CONCLUSION

Based on our experiences while conducting the CPOE meta-analysis, we argue that we are still at the beginning of Evidence-Based Health Informatics. To overcome the discussed challenges, health informatics should strive for harmonized terminology, a study registry, reporting standards, financial or legal incentives for conducting studies, methods to combine evidence from quantitative and qualitative studies, and guidelines for conducting and publishing evaluation studies.

Comparing two web-based smoking cessation programs: randomized controlled trial

Background

Smoking cessation remains a significant public health problem. Innovative interventions that use the Internet have begun to emerge that offer great promise in reaching large numbers of participants and encouraging widespread behavior change. To date, the relatively few controlled trials of Web-based smoking cessation programs have been limited by short follow-up intervals.

Objective

We describe the 6-month follow-up results of a randomized controlled trial in which participants recruited online were randomly assigned to either a Web-based smoking cessation program (Quit Smoking Network; QSN) or a Web-based exercise enhancement program (Active Lives) adapted somewhat to encourage smoking cessation.

Methods

The study was a two-arm randomized controlled trial that compared two Web-based smoking cessation programs: (1) the QSN intervention condition presented cognitive-behavioral strategies, and (2) the Active Lives control condition provided participants with guidance in developing a physical activity program to assist them with quitting. The QSN condition provided smoking cessation information and behavior change strategies while the Active Lives condition provided participants with physical activity recommendations and goal setting. The QSN condition was designed to be more engaging (eg, it included multimedia components) and to present much greater content than is typically found in smoking cessation programs.

Results

Contrary to our hypotheses, no between-condition differences in smoking abstinence were found at 3- and 6-month follow-up assessments. While participants in the QSN intervention condition spent more time than controls visiting the online program, the median number of 1.0 visit in each condition and the substantial attrition (60.8% at the 6-month follow-up) indicate that participants were not as engaged as we had expected.

Conclusions

Contrary to our hypothesis, our test of two Web-based smoking cessation conditions, an intervention and an attention placebo control, failed to show differences at 3- and 6-month assessments. We explored possible reasons for this finding, including limited engagement of participants and simplifying program content and architecture. Future research needs to address methods to improve participant engagement in online smoking cessation programs. Possible approaches in this regard can include new informed consent procedures that better explain the roles and responsibilities of being a research participant, new program designs that add more vitality (changing content from visit to visit), and new types of reminders pushed out to participants to encourage return visits. Simplifying program content through a combination of enhanced tailoring and information architecture also merits further research attention.

Exercise therapy and low back pain: insights and proposals to improve the design, conduct, and reporting of clinical trials

STUDY DESIGN

A report on plenary presentations and discussions of an expert panel and workshop ("Exercise as a treatment for subacute low back pain") that was held at the Amsterdam Forum VIII on Low Back Pain Research in Primary Care.

OBJECTIVE

To highlight important issues regarding the design, conduct, and reporting of exercise trials and to provide suggestions for improvement in each domain.

SUMMARY OF BACKGROUND DATA

Despite improvements in the methodologic quality of randomized clinical trials on low back pain (LBP), current trials on exercise therapy still show methodologic flaws in the design and reporting of studies.

METHODS

Important issues in the design, conduct, and reporting of exercise trials for LBP were introduced by 4 presenters, and this was followed by a group discussion among 40 attendees. Conclusions and recommendations were extracted by the workshop panelists and summarized in this paper. RESULTS.: Workshop discussion led to the following recommendations: (1) specify a theoretical framework for exercise therapy for designing the intervention and selecting appropriate treatment efficacy measures; (2) to overcome blinding problems in exercise trials, include sham interventions and assess the success of blinding; (3) evaluate the role of patient-provider interactions through the use of self-report or observational measures; (4) to assure quality of treatment, use exercise interventions that reference existing exercise guidelines; (5) use subgroup analyses to identify subgroups of patients most likely to benefit; (6) in reporting exercise trials, include an extensive description of the study population, a detailed exercise protocol, and some measure of patient compliance; (7) further categorization of exercise interventions is needed, in terms of concept, mode, intensity, duration, frequency, and length; and (8) trial;1>registration and publication of detailed study protocols may help to strengthen the evidence-based concerning exercise therapy for LBP.

CONCLUSION

By paying proper attention to issues concerning the design and reporting of exercise protocols when conducting future exercise trials, research may provide a stronger evidence basis for recommending or not recommending various exercise therapies for LBP.

Methodologic challenges of e-health research

In this article, research-based strategies used in a web-based study to assess the impact of participation in cancer-related mailing lists on cancer survivors and caregivers are highlighted. The paper presents alternative methods of measuring response rates in Internet surveys. Response rate estimation, based on American Association for Public Opinion Research guidelines, ranged between 15% and 24.9% for the study. Several strategies were implemented to increase survey participation and reduce item nonresponse. Few participants opted for a telephone survey as an alternative to a web-administered design (mixed-mode administration). Using time-stamp data to diagnose places in the questionnaire where respondents prematurely exited or broke off survey participation, we were able to decrease premature break offs. A lottery was also implemented to increase response rates. The experience of obtaining institutional review board approval for the lottery is also described. Preliminary data indicate slight increases in response rates subsequent to implementing the lottery.

"Is Cybermedicine Killing You?"--The story of a Cochrane disaster

This editorial briefly reviews the series of unfortunate events that led to the publication, dissemination, and eventual retraction of a flawed Cochrane systematic review on interactive health communication applications (IHCAs), which was widely reported in the media with headlines such as "Internet Makes Us Sick," "Knowledge May Be Hazardous to Web Consumers' Health," "Too Much Advice Can Be Bad for Your Health," "Click to Get Sick?" and even "Is Cybermedicine Killing You?" While the media attention helped to speed up the identification of errors, leading to a retraction of the review after only 13 days, a paper published in this issue of JMIR by Rada shows that the retraction, in contrast to the original review, remained largely unnoticed by the public. We discuss the three flaws of the review, which include (1) data extraction and coding errors, (2) the pooling of heterogeneous studies, and (3) a problematic and ambiguous scope and, possibly, some overlooked studies. We then discuss "retraction ethics" for researchers, editors/publishers, and journalists. Researchers and editors should, in the case of retractions, match the aggressiveness of the original dissemination campaign if errors are detected. It is argued that researchers and their organizations may have an ethical obligation to track down journalists who reported stories on the basis of a flawed study and to specifically ask them to publish an article indicating the error. Journalists should respond to errors or retractions with reports that have the same prominence as the original story. Finally, we look at some of the lessons for the Cochrane Collaboration, which include (1) improving the peer-review system by routinely sending out pre-prints to authors of the original studies, (2) avoiding downplay of the magnitude of errors if they occur, (3) addressing the usability issues of RevMan, and (4) making critical articles such as retraction notices open access.

A consumer health informatics (CHI) toolbox: challenges and implications

Consumer health informatics (CHI) is a rapidly evolving sub-discipline of medical informatics. Such developing fields typically share common needs, such as harmonizing terms and building a common foundation of research methods and instruments. The authors describe a pilot study to conceptualize and develop a "CHI toolbox," a repository of existing methods and instruments across relevant established fields. The challenges encountered in attempting to organize concepts in a nascent, interdisciplinary field are discussed. The authors' experiences in creating a comprehensive CHI toolbox suggest that a larger, concerted effort to develop a similar product by members of the relevant research communities could accelerate the development of common terms, operational definitions, variables, and instruments within the CHI field.

Peer-review and publication of research protocols and proposals: a role for open access journals

Peer-review and publication of research protocols offer several advantages to all parties involved. Among these are the following opportunities for authors: external expert opinion on the methods, demonstration to funding agencies of prior expert review of the protocol, proof of priority of ideas and methods, and solicitation of potential collaborators. We think that review and publication of protocols is an important role for Open Access journals. Because of their electronic form, openness for readers, and author-pays business model, they are better suited than traditional journals to ensure the sustainability and quality of protocol reviews and publications. In this editorial, we describe the workflow for investigators in eHealth research, from protocol submission to a funding agency, to protocol review and (optionally) publication at JMIR, to registration of trials at the International eHealth Study Registry (IESR), and to publication of the report. One innovation at JMIR is that protocol peer reviewers will be paid a honorarium, which will be drawn partly from a new submission fee for protocol reviews. Separating the article processing fee into a submission and a publishing fee will allow authors to opt for “peer-review only” (without subsequent publication) at reduced costs, if they wish to await a funding decision or for other reasons decide not to make the protocol public.