A re-analysis of the Cochrane Library data: the dangers of unobserved heterogeneity in meta-analyses

Background

Heterogeneity has a key role in meta-analysis methods and can greatly affect conclusions. However, true levels of heterogeneity are unknown and often researchers assume homogeneity. We aim to: a) investigate the prevalence of unobserved heterogeneity and the validity of the assumption of homogeneity; b) assess the performance of various meta-analysis methods; c) apply the findings to published meta-analyses.

Methods and Findings

We accessed 57,397 meta-analyses, available in the Cochrane Library in August 2012. Using simulated data we assessed the performance of various meta-analysis methods in different scenarios. The prevalence of a zero heterogeneity estimate in the simulated scenarios was compared with that in the Cochrane data, to estimate the degree of unobserved heterogeneity in the latter. We re-analysed all meta-analyses using all methods and assessed the sensitivity of the statistical conclusions. Levels of unobserved heterogeneity in the Cochrane data appeared to be high, especially for small meta-analyses. A bootstrapped version of the DerSimonian-Laird approach performed best in both detecting heterogeneity and in returning more accurate overall effect estimates. Re-analysing all meta-analyses with this new method we found that in cases where heterogeneity had originally been detected but ignored, 17–20% of the statistical conclusions changed. Rates were much lower where the original analysis did not detect heterogeneity or took it into account, between 1% and 3%.

Conclusions

When evidence for heterogeneity is lacking, standard practice is to assume homogeneity and apply a simpler fixed-effect meta-analysis. We find that assuming homogeneity often results in a misleading analysis, since heterogeneity is very likely present but undetected. Our new method represents a small improvement but the problem largely remains, especially for very small meta-analyses. One solution is to test the sensitivity of the meta-analysis conclusions to assumed moderate and large degrees of heterogeneity. Equally, whenever heterogeneity is detected, it should not be ignored.

Molecular studies of microbial community structure on stained pages of Leonardo da Vinci's Atlantic Codex

In 2006, after a visual inspection of the Leonardo da Vinci's Atlantic Codex by a scholar, active molds were reported to have been present on Codex pages showing areas of staining. In the present paper, molecular methods were used to assess the current microbiological risk to stained pages of the manuscript. Bacterial and fungal communities were sampled by a non-invasive technique employing nitrocellulose membranes. Denaturing gradient gel electrophoresis of 16 S rRNA gene and internal transcribed space regions were carried out to study the structure of the bacterial and fungal communities and band patterns were analyzed by the multivariate technique of principal component analysis. Any relationship between the presence of an active microbial community and staining was excluded. The presence of potential biodeteriogens was evaluated by constructing bacterial and fungal clone libraries, analyzing them by an operational taxonomic unit (OTU) approach. Among the bacteria, some OTUs were associated with species found on floors in clean room while others were identified with human skin contamination. Some fungal OTU representatives were potential biodeteriogens that, under proper thermo-hygrometric conditions, could grow. The retrieval of these potential biodeteriogens and microorganisms related to human skin suggests the need for a continuous and rigorous monitoring of the environmental conditions, and the need to improve handling procedures.

Healthcare librarians and learner support: a review of competences and methods

OBJECTIVES

London Health Libraries (LHL) are undertaking a project in order to develop the role of their library and knowledge services staff in supporting learners within the NHS in the London area (LHL Learner Support Project). This paper reports on the first phase of the project.

METHODOLOGY

A literature analysis was carried out to provide guidance on the skills and competences needed for library and knowledge staff to perform this function.

RESULTS

A variety of competences are identified, and structured in a model incorporating both training skills and general professional competencies. Library and knowledge staff will themselves need to have a high-level of information literacy and to be active lifelong learners.

CONCLUSIONS

A 'blended learning' approach, involving e-learning together with other methods, is identified as the most appropriate way for skills to be acquired.

Community-based organizations' perspective on health information outreach: a panel discussion

OBJECTIVE

A panel was convened to elicit guidance for librarians in initiating and implementing community-based health information outreach.

PARTICIPANTS

Participants included a panel of individuals from communities or community organizations who represented the types of groups with which librarians or information specialists need to interact and an audience who represented health sciences libraries, public libraries, academic institutions, government agencies, funding agencies, and community-based organizations and could contribute to a discussion on community-based health information outreach.

PROGRAM

The panel was presented with a hypothetical community setting and asked to respond to a series of questions: What do librarians need to learn about the community before they make their visits? What methods of outreach have been successful in your work? How would you implement and sustain a health information program in your community? How would health information interventions reduce racial and ethnic disparities in health?

MAIN RESULTS

The panel helped to frame many of the issues that may confront librarians as they initiate information-related programs in communities.

CONCLUSION

There is clear consensus on the need for librarians to make the effort to reach out into the community, to make the contacts, to seek to understand the community, to talk with leaders, and to respect the community as they promote and teach the use of health information resources. It was confirmed that librarians and libraries have an important role in diminishing health disparities by improving access to health information.

Ecologies, outreach, and the evolution of medical libraries

QUESTION

What are some of the forces shaping the evolution of medical libraries, and where might they lead?

DATA SOURCES

Published literature in the fields of library and information sciences, technology, health services research, and business was consulted.

MAIN RESULTS

Medical libraries currently have a modest footprint in most consumers' personal health ecologies, the network of resources and activities they use to improve their health. They also occupy a relatively small space in the health care, information, and business ecologies of which they are a part. Several trends in knowledge discovery, technology, and social organizations point to ways in which the roles of medical libraries might grow and become more complex.

CONCLUSION

As medical libraries evolve and reach out to previously underserved communities, an ecological approach can serve as a useful organizing framework for the forces shaping this evolution.

PubMed enhancements: fulfilling the promise of a great product

There have been many recent changes to PubMed to enhance its usefulness. Those changes include: LinkOut Libraries (local holding field), PubMed Central (full-text articles archived by the National Library of Medicine), and LinkOut (access to full-text articles right from the PubMed citation). Medical librarians should be aware of how these features work to best assist their clients. These new features offer the possibility of true desktop access for library patrons. Not only will patrons appreciate these new features, but their use in libraries will literally change what we do, who does it, and how it is done.

Controlling hospital library theft

At Capital Health System/Fuld Campus (formerly Helene Fuld Medical Center), the Health Sciences Library lost many books and videocassettes. These materials were listed in the catalog but were missing when staff went to the shelves. The hospital had experienced a downsizing of staff, a reorganization, and a merger. When the library staff did an inventory, $10,000 worth of materials were found to be missing. We corrected the situation through a series of steps that we believe will help other libraries control their theft. Through regularly scheduling inventories, monitoring items, advertising, and using specific security measures, we have successfully controlled the library theft. The January 2002 inventory resulted in meeting our goal of zero missing books and videocassettes. We work to maintain that goal.

Introducing HEAL: the Health Education Assets Library

Digital multimedia, such as images and videos, are playing an increasingly important role in health sciences education. Educators, however, often do not have the time or resources to create high-quality materials. The authors describe the development of a new Health Education Assets Library (HEAL), a freely accessible, national library of high-quality digital multimedia to support all levels of health sciences education. HEAL's primary mission is to provide educators with high-quality and free multimedia materials (such as images and videos) to augment health science education. In addition, HEAL is working with other organizations to establish a network of distributed databases of high-quality teaching resources. By using state-of-the-art Internet technologies HEAL enables educators across the country to efficiently search and retrieve teaching materials from a variety of sources.

The use of Ovid local holdings messages: one library's experience

The Ovid local holdings feature can be very beneficial to library patrons. This article describes the use of the feature for one academic health sciences library. Topics covered include how the function works, how it was implemented, how it is updated, the impact of electronic journals, and benefits and drawbacks of use. Also included are suggested enhancements to the system.

Interlibrary loan in primary access libraries: challenging the traditional view

INTRODUCTION

Primary access libraries serve as the foundation of the National Network of Libraries of Medicine (NN/LM) interlibrary loan (ILL) hierarchy, yet few published reports directly address the important role these libraries play in the ILL system. This may reflect the traditional view that small, primary access libraries are largely users of ILL, rather than important contributors to the effectiveness and efficiency of the national ILL system.

OBJECTIVE

This study was undertaken to test several commonly held beliefs regarding ILL system use by primary access libraries.

HYPOTHESES

Three hypotheses were developed. HI: Colorado and Wyoming primary access libraries comply with the recommended ILL guideline of adhering to a hierarchical structure, emphasizing local borrowing. H2: The closures of two Colorado Council of Medical Librarians (CCML) primary access libraries in 1996 resulted in twenty-three Colorado primary access libraries' borrowing more from their state resource library in 1997. H3: The number of subscriptions held by Colorado and Wyoming primary access libraries is positively correlated with the number of items they loan and negatively correlated with the number of items they borrow.

METHODS

The hypotheses were tested using the 1992 and 1997 DOCLINE and OCLC data of fifty-four health sciences libraries, including fifty primary access libraries, two state resource libraries, and two general academic libraries in Colorado and Wyoming. The ILL data were obtained electronically and analyzed using Microsoft Word 98, Microsoft Excel 98, and JMP 3.2.2.

RESULTS

CCML primary access libraries comply with the recommended guideline to emphasize local borrowing by supplying each other with the majority of their ILLs, instead of overburdening libraries located at higher levels in the ILL hierarchy (H1). The closures of two CCML primary access libraries appear to have affected the entire ILL system, resulting in a greater volume of ILL activity for the state resource library and other DOCLINE libraries higher up in the ILL hierarchy and highlighting the contribution made by CCML primary access libraries (H2). CCML primary access libraries borrow and lend in amounts that are proportional to their collection size, rather than overtaxing libraries at higher levels in the ILL hierarchy with large numbers of requests (H3).

LIMITATIONS

The main limitations of this study were the small sample size and the use of data collected for another purpose, the CCML ILL survey.

CONCLUSIONS

The findings suggest that there is little evidence to support several commonly held beliefs regarding ILL system use by primary access libraries. In addition to validating the important contributions made by primary access libraries to the national ILL system, baseline data that can be used to benchmark current practice performance are provided.

Monograph use at an academic health sciences library: the first three years of shelf life

OBJECTIVE

To study the circulation of monographs during the first three years of shelf life at an academic health sciences library.

METHOD

A record was kept of monographs added to the circulating collection from mid-1994 to mid-1995. After three years, each monograph was located and the number of times it circulated during the first, second, and third year of shelf life determined by counting checkout stamps on the circulation slip.

RESULTS

Of the 1,958 monographs studied, 1,674 had complete data for the first three years of shelf life. Of those 1,674 titles, 81.48% circulated at least once. A total of 7,659 circulations were recorded; 38.69% occurred in the first year of shelf life, 32.37% in the second year, and 28.95% in the third year. The data did not fit the well-known 80/20 rule. Instead, approximately 38% of monographs accounted for 80% of circulation. A small percentage, 2.21%, of monographs accounted for a substantial percentage of circulation, 21.84%.

CONCLUSIONS

A large percentage of the monographs circulated and use did not decline sharply with age within the first three years of shelf life, indicating a high demand for monographs at this academic health sciences library. These results, combined with the findings of earlier studies, suggested two possibilities. First, academic health sciences libraries might exhibit use of a higher percentage of monograph acquisitions than other types of libraries; or, second, a low monograph-to-user ratio might result in a higher percentage of monographs being used. Perhaps both factors contributed to the results found in this study. Further investigation would be needed to determine the extent to which library type and monograph-to-user ratio influenced monograph use.