Active Learning-based corpus annotation--the PathoJen experience

We report on basic design decisions and novel annotation procedures underlying the development of PathoJen, a corpus of Medline abstracts annotated for pathological phenomena, including diseases as a proper subclass. This named entity type is known to be hard to delineate and capture by annotation guidelines. We here propose a two-category encoding schema where we distinguish short from long mention spans, the first covering standardized terminology (e.g. diseases), the latter accounting for less structured descriptive statements about norm-deviant states, as well as criteria and observations that might signal pathologies. The second design decision relates to the way annotation instances are sampled. Here we subscribe to an Active Learning-based approach which is known to save annotation costs without sacrificing annotation quality by means of a sample bias. By design, Active Learning picks up 'hard' to annotate instances for human annotators, whereas 'easier' ones are passed over to the automatic classifier whose models already incorporate and gradually improve with previous annotation experience.

Towards valid and reusable reference alignments - ten basic quality checks for ontology alignments and their application to three different reference data sets

Identifying relationships between hitherto unrelated entities in different ontologies is the key task of ontology alignment. An alignment is either manually created by domain experts or automatically by an alignment system. In recent years, several alignment systems have been made available, each using its own set of methods for relation detection. To evaluate and compare these systems, typically a manually created alignment is used, the so-called reference alignment. Based on our experience with several of these reference alignments we derived requirements and translated them into simple quality checks to ensure the alignments’ validity and also their reusability. In this article, these quality checks are applied to a standard reference alignment in the biomedical domain, the Ontology Alignment Evaluation Initiative Anatomy track reference alignment, and two more recent data sets covering multiple domains, including but not restricted to anatomy and biology.

The extraction of pharmacogenetic and pharmacogenomic relations--a case study using PharmGKB

In this paper, we report on adapting the JREX relation extraction engine, originally developed For the elicitation of protein-protein interaction relations, to the domains of pharmacogenetics and pharmacogenomics. We propose an intrinsic and an extrinsic evaluation scenario which is based on knowledge contained in the PharmGKB knowledge base. Porting JREX yields favorable results in the range of 80% F-score for Gene-Disease, Gene-Drug, and Drug-Disease relations.

Assessment of NER solutions against the first and second CALBC Silver Standard Corpus

Background

Competitions in text mining have been used to measure the performance of automatic text processing solutions against a manually annotated gold standard corpus (GSC). The preparation of the GSC is time-consuming and costly and the final corpus consists at the most of a few thousand documents annotated with a limited set of semantic groups. To overcome these shortcomings, the CALBC project partners (PPs) have produced a large-scale annotated biomedical corpus with four different semantic groups through the harmonisation of annotations from automatic text mining solutions, the first version of the Silver Standard Corpus (SSC-I). The four semantic groups are chemical entities and drugs (CHED), genes and proteins (PRGE), diseases and disorders (DISO) and species (SPE). This corpus has been used for the First CALBC Challenge asking the participants to annotate the corpus with their text processing solutions.

Results

All four PPs from the CALBC project and in addition, 12 challenge participants (CPs) contributed annotated data sets for an evaluation against the SSC-I. CPs could ignore the training data and deliver the annotations from their genuine annotation system, or could train a machine-learning approach on the provided pre-annotated data. In general, the performances of the annotation solutions were lower for entities from the categories CHED and PRGE in comparison to the identification of entities categorized as DISO and SPE. The best performance over all semantic groups were achieved from two annotation solutions that have been trained on the SSC-I.

The data sets from participants were used to generate the harmonised Silver Standard Corpus II (SSC-II), if the participant did not make use of the annotated data set from the SSC-I for training purposes. The performances of the participants’ solutions were again measured against the SSC-II. The performances of the annotation solutions showed again better results for DISO and SPE in comparison to CHED and PRGE.

Conclusions

The SSC-I delivers a large set of annotations (1,121,705) for a large number of documents (100,000 Medline abstracts). The annotations cover four different semantic groups and are sufficiently homogeneous to be reproduced with a trained classifier leading to an average F-measure of 85%. Benchmarking the annotation solutions against the SSC-II leads to better performance for the CPs’ annotation solutions in comparison to the SSC-I.

CALBC silver standard corpus

The CALBC initiative aims to provide a large-scale biomedical text corpus that contains semantic annotations for named entities of different kinds. The generation of this corpus requires that the annotations from different automatic annotation systems be harmonized. In the first phase, the annotation systems from five participants (EMBL-EBI, EMC Rotterdam, NLM, JULIE Lab Jena, and Linguamatics) were gathered. All annotations were delivered in a common annotation format that included concept identifiers in the boundary assignments and that enabled comparison and alignment of the results. During the harmonization phase, the results produced from those different systems were integrated in a single harmonized corpus ("silver standard" corpus) by applying a voting scheme. We give an overview of the processed data and the principles of harmonization--formal boundary reconciliation and semantic matching of named entities. Finally, all submissions of the participants were evaluated against that silver standard corpus. We found that species and disease annotations are better standardized amongst the partners than the annotations of genes and proteins. The raw corpus is now available for additional named entity annotations. Parts of it will be made available later on for a public challenge. We expect that we can improve corpus building activities both in terms of the numbers of named entity classes being covered, as well as the size of the corpus in terms of annotated documents.

Aligning UniProt and MeSH - a case study on human protein terms

Terminologies which lack semantic connectivity hamper the effective search in biomedical fact databases and document retrieval systems. We here focus on the integration of two such isolated resources, the term lists from the protein fact database UNIPROT and the indexing vocabulary MESH from the bibliographic database MEDLINE. The generated semantic ties result from string matching and term set inclusion. We investigated the implicit terminological overlap between both resources in the domain of human proteins and evaluated our approach on a sample of 550 randomly selected UNIPROT entries that were manually mapped to their corresponding MESH headings. We achieved 90% precision and 79% recall (applying taxonomy-sensitive metrics). Fortunately, those proteins we were able to map to the MESH are ten times as frequently discussed in the literature as those on which we failed.

Alignment of the UMLS semantic network with BioTop: methodology and assessment

MOTIVATION

For many years, the Unified Medical Language System (UMLS) semantic network (SN) has been used as an upper-level semantic framework for the categorization of terms from terminological resources in biomedicine. BioTop has recently been developed as an upper-level ontology for the biomedical domain. In contrast to the SN, it is founded upon strict ontological principles, using OWL DL as a formal representation language, which has become standard in the semantic Web. In order to make logic-based reasoning available for the resources annotated or categorized with the SN, a mapping ontology was developed aligning the SN with BioTop.

METHODS

The theoretical foundations and the practical realization of the alignment are being described, with a focus on the design decisions taken, the problems encountered and the adaptations of BioTop that became necessary. For evaluation purposes, UMLS concept pairs obtained from MEDLINE abstracts by a named entity recognition system were tested for possible semantic relationships. Furthermore, all semantic-type combinations that occur in the UMLS Metathesaurus were checked for satisfiability.

RESULTS

The effort-intensive alignment process required major design changes and enhancements of BioTop and brought up several design errors that could be fixed. A comparison between a human curator and the ontology yielded only a low agreement. Ontology reasoning was also used to successfully identify 133 inconsistent semantic-type combinations.

AVAILABILITY

BioTop, the OWL DL representation of the UMLS SN, and the mapping ontology are available at http://www.purl.org/biotop/.

MaHCO: an ontology of the major histocompatibility complex for immunoinformatic applications and text mining

MOTIVATION

The high level of polymorphism associated with the major histocompatibility complex (MHC) poses a challenge to organizing associated bioinformatic data, particularly in the area of hematopoietic stem cell transplantation. Thus, this area of research has great potential to profit from the ongoing development of biomedical ontologies, which offer structure and definition to MHC-data related communication and portability issues.

RESULTS

We introduce the design considerations, methodological foundations and implementational issues underlying MaHCO, an ontology which represents the alleles and encoded molecules of the major histocompatibility complex. Importantly for human immunogenetics, it includes a detailed level of human leukocyte antigen (HLA) classification. We then present an ontology browser, search interfaces for immunogenetic fact and document retrieval, and the specification of an annotation language for semantic metadata, based on MaHCO. These use cases are intended to demonstrate the utility of ontology-driven bioinformatics in the field of immunogenetics.

AVAILABILITY AND IMPLEMENTATION

The MaHCO Ontology is available via the BioPortal: http://www.bioontology.org/tools/portal/bioportal.html, and at: http://purl.org/stemnet/.

Gene Regulation Ontology (GRO): design principles and use cases

The Gene Regulation Ontology (GRO) is designed as a novel approach to model complex events that are part of the gene regulatory processes. We introduce the design requirements for such a conceptual model and discuss terminological resources suitable to base its construction on. The ontology defines gene regulation events in terms of ontological classes and imposes constraints on them by specifying the participants involved. The logical structure of the ontology is intended to meet the needs of advanced information extraction and text mining systems which target the identification of event representations in scientific literature. The GRO has just been submitted to the OBO library and is currently under review. It is available at http://www.ebi.ac.uk/Rebholz-srv/GRO/GRO.html.

An ontology for major histocompatibility complex (MHC) alleles and molecules

We present a formally coherent and consistent multi-species MHC ontology which includes all human MHC alleles and serological groups. The ontology is part of StemNet, a knowledge management system for hematopoietic stem cell transplantation with an integrated semantic search engine. The Owl-encoded MHC ontology contributes to the system in a threefold manner. First, it supports query formulation and query processing as well as mapping onto external terminological resources, second, it eases the interaction with the search engine when navigating through search results, and finally, it provides a formal language for text annotation, a methodological prerequisite for state-of-the-art natural language text processors which are increasingly based on machine learning methods and hence require annotated text corpora.

An automatically curated MHC ontology for intelligent immunogenetic information retrieval (B42)

Accurate information retrieval of scientific articles is complicated by the intricacies of natural language. Ontologies are useful for formalizing scientific concepts and providing a foundation to mediate between formal, computationally accessible, representations and informal natural language expressions. The terminology associated with the MHC exemplifies the need for a sophisticated translation process between a natural language query and the relevant documents. For example, a researcher searching for “peptides that bind MHC” should be presented with a text containing “peptides were eluted from HLA-A2.” To achieve this goal, we have developed a multi-species MHC ontology which encompasses all human MHC alleles and serologic groups. Because of the rapid rate of new allele discovery, allele incorporation is automated using the IMGT/HLA and IPD-MHC databases. In the context of the StemNet knowledge management project, this MHC ontology contributes to machine-learning-driven text annotation, as well as to search-string interpretation and mapping. In this way, StemNet aims to offer researchers a platform to efficiently access the goldmine of clinically relevant immunogenetic information, which is otherwise inaccessible by standard search engines and databases.

This work was financed by the Federal Ministry of Education and Research, Germany, funding number 01DS001C; the authors are solely responsible for content.

Towards a top-domain ontology for linking biomedical ontologies

In this paper we present the ongoing development and extension work on BioTop--a top-domain ontology for linking biomedical domain ontologies. We start by making the case for the application of a common ontology to interface independent biomedical domain ontologies by introducing a set of more general classes. Then we briefly depict the relation of BioTop to the GENIA ontology as starting point of its initial developement. Afterwards we propose our distinction of ontologies into top, top-domain and domain ones and describe our approach to the integration of the top ontology BFO into BioTop. Then we present our plans to join the OBO and OBO Foundry repository of ontologies and list its admission principles in relation to our ontology. Some actual BioTop interface classes are shown subsequently. We conclude by detailing on some planned BioTop usages in the area of BioNLP and cancer research and show some further intended improvements.

Towards an upper level ontology for molecular biology

There is a growing need for the general-purpose description of the basic conceptual entities in the life sciences. Up until now, upper level models have mainly been purpose-driven, such as the GENIA ontology, originally devised as a vocabulary for corpus annotation. As an alternative,we here present BioTop, a description-logic-based top level ontology for molecular biology, which we consider as an ontologically conscious redesign of the GENIA ontology.