Evaluating the granularity balance of hierarchical relationships within large biomedical terminologies towards quality improvement

Self-prediction of relations in GO facilitates its quality auditing

As applications of the gene ontology (GO) increase rapidly in the biomedical field, quality auditing of it is becoming more and more important. Existing auditing methods are mostly based on rules, observed patterns or hypotheses. In this study, we propose a machine-learning-based framework for GO to audit itself: we first predict the IS-A relations among concepts in GO, then use differences between predicted results and existing relations to uncover potential errors. Specifically, we transfer the taxonomy of GO 2020 January release into a dataset with concept pairs as items and relations between them as labels(pairs with no direct IS-A relation are labeled as ndrs). To fully obtain the representation of each pair, we integrate the embeddings for the concept name, concept definition, as well as concept node in a substring-based topological graph. We divide the dataset into 10 parts, and rotate over all the parts by choosing one part as the testing set and the remaining as the training set each time. After 10 rotations, the prediction model predicted 4,640 existing IS-A pairs as ndrs. In the GO 2022 March release, 340 of these predictions were validated, demonstrating significance with a p-value of 1.60e-46 when compared to the results of randomly selected pairs. On the other hand, the model predicted 2,840 out of 17,079 selected ndrs in GO to be IS-A's relations. After deleting those that caused redundancies and circles, 924 predicted IS-A's relations remained. Among 200 pairs randomly selected, 30 were validated as missing IS-A's by domain experts. In conclusion, this study investigates a novel way of auditing biomedical ontologies by predicting the relations in it, which was shown to be useful for discovering potential errors.

Extending import detection algorithms for concept import from two to three biomedical terminologies

BACKGROUND

While enrichment of terminologies can be achieved in different ways, filling gaps in the IS-A hierarchy backbone of a terminology appears especially promising. To avoid difficult manual inspection, we started a research program in 2014, investigating terminology densities, where the comparison of terminologies leads to the algorithmic discovery of potentially missing concepts in a target terminology. While candidate concepts have to be approved for import by an expert, the human effort is greatly reduced by algorithmic generation of candidates. In previous studies, a single source terminology was used with one target terminology.

METHODS

In this paper, we are extending the algorithmic detection of "candidate concepts for import" from one source terminology to two source terminologies used in tandem. We show that the combination of two source terminologies relative to one target terminology leads to the discovery of candidate concepts for import that could not be found with the same "reliability" when comparing one source terminology alone to the target terminology. We investigate which triples of UMLS terminologies can be gainfully used for the described purpose and how many candidate concepts can be found for each individual triple of terminologies.

RESULTS

The analysis revealed a specific configuration of concepts, overlapping two source and one target terminology, for which we coined the name "fire ladder" pattern. The three terminologies in this pattern are tied together by a kind of "transitivity." We provide a quantitative analysis of the discovered fire ladder patterns and we report on the inter-rater agreement concerning the decision of importing candidate concepts from source terminologies into the target terminology. We algorithmically identified 55 instances of the fire ladder pattern and two domain experts agreed on import for 39 instances. In total, 48 concepts were approved by at least one expert. In addition, 105 import candidate concepts from a single source terminology into the target terminology were also detected, as a "beneficial side-effect" of this method, increasing the cardinality of the result.

CONCLUSION

We showed that pairs of biomedical source terminologies can be transitively chained to suggest possible imports of concepts into a target terminology.

Automatic Structuring of Ontology Terms Based on Lexical Granularity and Machine Learning: Algorithm Development and Validation

Background

As the manual creation and maintenance of biomedical ontologies are labor-intensive, automatic aids are desirable in the lifecycle of ontology development.

Objective

Provided with a set of concept names in the Foundational Model of Anatomy (FMA), we propose an innovative method for automatically generating the taxonomy and the partonomy structures among them, respectively.

Methods

Our approach comprises 2 main tasks: The first task is predicting the direct relation between 2 given concept names by utilizing word embedding methods and training 2 machine learning models, Convolutional Neural Networks (CNN) and Bidirectional Long Short-term Memory Networks (Bi-LSTM). The second task is the introduction of an original granularity-based method to identify the semantic structures among a group of given concept names by leveraging these trained models.

Results

Results show that both CNN and Bi-LSTM perform well on the first task, with F1 measures above 0.91. For the second task, our approach achieves an average F1 measure of 0.79 on 100 case studies in the FMA using Bi-LSTM, which outperforms the primitive pairwise-based method.

Conclusions

We have investigated an automatic way of predicting a hierarchical relationship between 2 concept names; based on this, we have further invented a methodology to structure a group of concept names automatically. This study is an initial investigation that will shed light on further work on the automatic creation and enrichment of biomedical ontologies.

Alternative classification of identical concepts in different terminologies: Different ways to view the world

In previous research, we have studied concepts that occur in pairs of medical terminologies and are known to be identical, because they have the same ID number in the Unified Medical Language System (UMLS). We observed that such concepts rarely have exactly the same sets of children (=subconcepts) in the two terminologies. The number of common children was found to vary widely. A special situation was identified where the children in one terminology relate to the common parent in a very different way than the children in the other terminology. For example, children in one terminology might subdivide a parent concept by anatomical location in one terminology and by disease kind in the other terminology. We coined the term "alternative classification" (of the same parent concept) for such situations. In previous work, only human experts could recognize alternative classifications. In this paper, we present a mathematically expressed criterion for likely cases of alternative classifications. We compare the recommendations of this criterion, expressed by a mathematical quantity called "EFI" becoming zero, with the decisions of a human expert. It is found that the human expert agreed with the criterion in 72% of all cases, which is a big improvement over having no computable criterion at all. Besides alternative classifications, common parent concepts in a pair of terminologies might also indicate a possible import of a child concept missing in one terminology, different granularities, or errors in either one of the two terminologies. In this paper, we further investigate different kinds of alternative classifications.