Automatic inference of indexing rules for MEDLINE

MeSH Now: automatic MeSH indexing at PubMed scale via learning to rank

BACKGROUND

MeSH indexing is the task of assigning relevant MeSH terms based on a manual reading of scholarly publications by human indexers. The task is highly important for improving literature retrieval and many other scientific investigations in biomedical research. Unfortunately, given its manual nature, the process of MeSH indexing is both time-consuming (new articles are not immediately indexed until 2 or 3 months later) and costly (approximately ten dollars per article). In response, automatic indexing by computers has been previously proposed and attempted but remains challenging. In order to advance the state of the art in automatic MeSH indexing, a community-wide shared task called BioASQ was recently organized.

METHODS

We propose MeSH Now, an integrated approach that first uses multiple strategies to generate a combined list of candidate MeSH terms for a target article. Through a novel learning-to-rank framework, MeSH Now then ranks the list of candidate terms based on their relevance to the target article. Finally, MeSH Now selects the highest-ranked MeSH terms via a post-processing module.

RESULTS

We assessed MeSH Now on two separate benchmarking datasets using traditional precision, recall and F1-score metrics. In both evaluations, MeSH Now consistently achieved over 0.60 in F-score, ranging from 0.610 to 0.612. Furthermore, additional experiments show that MeSH Now can be optimized by parallel computing in order to process MEDLINE documents on a large scale.

CONCLUSIONS

We conclude that MeSH Now is a robust approach with state-of-the-art performance for automatic MeSH indexing and that MeSH Now is capable of processing PubMed scale documents within a reasonable time frame.

AVAILABILITY

http://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/MeSHNow/ .

Lost in translation: Review of identification bias, translation bias and research waste in dentistry

OBJECTIVE

To review how articles are retrieved from bibliographic databases, what article identification and translation problems have affected research, and how these problems can contribute to research waste and affect clinical practice.

METHODS

This literature review sought and appraised articles regarding identification- and translation-bias in the medical and dental literature, which limit the ability of users to find research articles and to use these in practice.

RESULTS

Articles can be retrieved from bibliographic databases by performing a word or index-term (for example, MeSH for MEDLINE) search. Identification of articles is challenging when it is not clear which words are most relevant, and which terms have been allocated to indexing fields. Poor reporting quality of abstracts and articles has been reported across the medical literature at large. Specifically in dentistry, research regarding time-to-event survival analyses found the allocation of MeSH terms to be inconsistent and inaccurate, important words were omitted from abstracts by authors, and the quality of reporting in the body of articles was generally poor. These shortcomings mean that articles will be difficult to identify, and difficult to understand if found. Use of specialized electronic search strategies can decrease identification bias, and use of tailored reporting guidelines can decrease translation bias. Research that cannot be found, or cannot be used results in research waste, and undermines clinical practice.

SIGNIFICANCE

Identification- and translation-bias have been shown to affect time-to-event dental articles, are likely affect other fields of research, and are largely unrecognized by authors and evidence seekers alike. By understanding that the problems exist, solutions can be sought to improve identification and translation of our research.

Feature engineering for MEDLINE citation categorization with MeSH

BACKGROUND

Research in biomedical text categorization has mostly used the bag-of-words representation. Other more sophisticated representations of text based on syntactic, semantic and argumentative properties have been less studied. In this paper, we evaluate the impact of different text representations of biomedical texts as features for reproducing the MeSH annotations of some of the most frequent MeSH headings. In addition to unigrams and bigrams, these features include noun phrases, citation meta-data, citation structure, and semantic annotation of the citations.

RESULTS

Traditional features like unigrams and bigrams exhibit strong performance compared to other feature sets. Little or no improvement is obtained when using meta-data or citation structure. Noun phrases are too sparse and thus have lower performance compared to more traditional features. Conceptual annotation of the texts by MetaMap shows similar performance compared to unigrams, but adding concepts from the UMLS taxonomy does not improve the performance of using only mapped concepts. The combination of all the features performs largely better than any individual feature set considered. In addition, this combination improves the performance of a state-of-the-art MeSH indexer. Concerning the machine learning algorithms, we find that those that are more resilient to class imbalance largely obtain better performance.

CONCLUSIONS

We conclude that even though traditional features such as unigrams and bigrams have strong performance compared to other features, it is possible to combine them to effectively improve the performance of the bag-of-words representation. We have also found that the combination of the learning algorithm and feature sets has an influence in the overall performance of the system. Moreover, using learning algorithms resilient to class imbalance largely improves performance. However, when using a large set of features, consideration needs to be taken with algorithms due to the risk of over-fitting. Specific combinations of learning algorithms and features for individual MeSH headings could further increase the performance of an indexing system.

Comparison and combination of several MeSH indexing approaches

MeSH indexing of MEDLINE is becoming a more difficult task for the group of highly qualified indexing staff at the US National Library of Medicine, due to the large yearly growth of MEDLINE and the increasing size of MeSH. Since 2002, this task has been assisted by the Medical Text Indexer or MTI program. We extend previous machine learning analysis by adding a more diverse set of MeSH headings targeting examples where MTI has been shown to perform poorly. Machine learning algorithms exceed MTI's performance on MeSH headings that are used very frequently and headings for which the indexing frequency is very low. We find that when we combine the MTI suggestions and the prediction of the learning algorithms, the performance improves compared to any single method for most of the evaluated MeSH headings.

Using cited references to improve the retrieval of related biomedical documents

BACKGROUND

A popular query from scientists reading a biomedical abstract is to search for topic-related documents in bibliographic databases. Such a query is challenging because the amount of information attached to a single abstract is little, whereas classification-based retrieval algorithms are optimally trained with large sets of relevant documents. As a solution to this problem, we propose a query expansion method that extends the information related to a manuscript using its cited references.

RESULTS

Data on cited references and text sections in 249,108 full-text biomedical articles was extracted from the Open Access subset of the PubMed Central® database (PMC-OA). Of the five standard sections of a scientific article, the Introduction and Discussion sections contained most of the citations (mean = 10.2 and 9.9 citations, respectively). A large proportion of articles (98.4%) and their cited references (79.5%) were indexed in the PubMed® database. Using the MedlineRanker abstract classification tool, cited references allowed accurate retrieval of the citing document in a test set of 10,000 documents and also of documents related to six biomedical topics defined by particular MeSH® terms from the entire PMC-OA (p-value<0.01). Classification performance was sensitive to the topic and also to the text sections from which the references were selected. Classifiers trained on the baseline (i.e., only text from the query document and not from the references) were outperformed in almost all the cases. Best performance was often obtained when using all cited references, though using the references from Introduction and Discussion sections led to similarly good results. This query expansion method performed significantly better than pseudo relevance feedback in 4 out of 6 topics.

CONCLUSIONS

The retrieval of documents related to a single document can be significantly improved by using the references cited by this document (p-value<0.01). Using references from Introduction and Discussion performs almost as well as using all references, which might be useful for methods that require reduced datasets due to computational limitations. Cited references from particular sections might not be appropriate for all topics. Our method could be a better alternative to pseudo relevance feedback though it is limited by full text availability.

Histamine: an undercover agent in multiple rare diseases?

Histamine is a biogenic amine performing pleiotropic effects in humans, involving tasks within the immune and neuroendocrine systems, neurotransmission, gastric secretion, cell life and death, and development. It is the product of the histidine decarboxylase activity, and its effects are mainly mediated through four different G-protein coupled receptors. Thus, histamine-related effects are the results of highly interconnected and tissue-specific signalling networks. Consequently, alterations in histamine-related factors could be an important part in the cause of multiple rare/orphan diseases. Bearing this hypothesis in mind, more than 25 rare diseases related to histamine physiopathology have been identified using a computationally assisted text mining approach. These newly integrated data will provide insight to elucidate the molecular causes of these rare diseases. The data can also help in devising new intervention strategies for personalized medicine for multiple rare diseases.

A bottom-up approach to MEDLINE indexing recommendations

MEDLINE indexing performed by the US National Library of Medicine staff describes the essence of a biomedical publication in about 14 Medical Subject Headings (MeSH). Since 2002, this task is assisted by the Medical Text Indexer (MTI) program. We present a bottom-up approach to MEDLINE indexing in which the abstract is searched for indicators for a specific MeSH recommendation in a two-step process. Supervised machine learning combined with triage rules improves sensitivity of recommendations while keeping the number of recommended terms relatively small. Improvement in recommendations observed in this work warrants further exploration of this approach to MTI recommendations on a larger set of MeSH headings.

Orthopaedic literature and MeSH

BACKGROUND

Since 1916 there has been a recognized demand for a method of classification of orthopaedic literature inclusive enough to permit the proper collection and retrieval of all literature on the subject. Today, MEDLINE, available through the PubMed interface, has become the de facto standard for organization and retrieval of medical literature. The Medical Subject Headings (MeSH), used to provide indexing and assist in searching, are partly responsible for this standard. Understanding how MeSH is built and maintained may lead the user to a better understanding of how to use MEDLINE, and what to expect from the indexing of an article.

QUESTIONS/PURPOSES

The purpose of this review is to provide an understanding of the organization of large quantities of indexed material, the indexing process and the considerations involved in developing an indexing vocabulary. WHERE ARE WE NOW?: Successful terminology development and use, a prerequisite for any sharing of information by electronic means, depends on both user (how the user is expected to use the system) and information (how the information is organized) models. MEDLINE has a simple user model and a simpler information model. The user is expected to determine what is relevant and which MeSH descriptors are appropriate. WHERE DO WE NEED TO GO?: While MEDLINE through PubMed is a success as viewed by the number of hits, further improvements will depend on better, faster indexing with a controlled terminology. Terminology development requires careful consideration of the nature of the subject, how users employ the terminology, the overall purpose of the terminology, and the framework of the systems in which it is used. HOW DO WE GET THERE?: For the future, understanding terminology development might enable the user to comprehend some of the issues involved in sharing of other information by electronic means. Further improvements in the availability and accessibility of medical literature will depend on continued maintenance and development of MeSH, as well as on refinement of the indexing process.