Supporting the analysis of ontology evolution processes through the combination of static and dynamic scaling functions in OQuaRE

A substring replacement approach for identifying missing IS-A relations in SNOMED CT

Biomedical ontologies provide formalized information and knowledge in the biomedical domain. Over the years, biomedical ontologies have played an important role in facilitating biomedical research and applications. Common quality issues of biomedical ontologies include inconsistent naming of concepts, redundant concepts, redundant relations, incomplete/incorrect concept definitions, and incomplete/incorrect class hierarchies. In this work, we focus on addressing the incompleteness of the class hierarchy in SNOMED CT. We develop a substring replacement approach, leveraging concepts' lexical features and existing IS-A relations to identify potential missing IS-A relations in SNOMED CT. To evaluate the effectiveness of our approach, we performed both automated and manual validation. For the automated evaluation, we leverage relations from external terminologies in the Unified Medical Language System (UMLS) to validate the identified missing IS-A relations. For the manual validation, a randomly selected 100 samples from the results are reviewed by a domain expert. Applying our approach to the March 2022 release of SNOMED CT US Edition, we identified 3,228 potential missing IS-A relations, among which 63 were validated through the UMLS. The evaluation by the domain expert revealed that 89 out of 100 (a precision of 89%) missing IS-A relations are valid cases, showing the effectiveness of this substring replacement approach to facilitate the quality assurance of IS-A relations in SNOMED CT.

A Framework for the Evaluation of the Cultural Heritage Information Ontology

The intelligent management of built cultural heritage, including heritage buildings, requires common semantics in the form of standardized ontologies to achieve semantic interoperability. Foundational ontologies should be reused when building new ontologies, as they provide high-level terms; however, candidate foundational ontologies should be evaluated for quality and pitfalls. Simple metrics (e.g., number of concepts) are easy to obtain with existing tools. Complex metrics such as quality of ontology structure, functional adequacy, transferability, reliability, compatibility, maintainability, and operability, are defined in recent ontology evaluation frameworks; however, these do not evaluate interoperability features. The paper proposes an improved framework for an automated ontology evaluation based on the OQuaRE framework. Our approach improved some of the metrics of the OQuaRE framework and introduced three metrics for assessing the interoperability of the ontology in question (Externes, Composability, and Aggregability). In the experimental section, the framework is validated in an evaluation of cultural heritage information ontology (CIDOC CRM—ISO 12217:2014) with the use of new software for ontology evaluation. The detailed results reveal that the ontology is minimally acceptable and that the improved evaluation framework efficiently integrated interoperability metrics. Recommendations for the improvement of the cultural heritage information ontology are described in the Discussion and Conclusions section.

Evaluation of ontology structural metrics based on public repository data

The development and application of biological ontologies have increased significantly in recent years. These ontologies can be retrieved from different repositories, which do not provide much information about quality aspects of the ontologies. In the past years, some ontology structural metrics have been proposed, but their validity as measurement instrument has not been sufficiently studied to date. In this work, we evaluate a set of reproducible and objective ontology structural metrics. Given the lack of standard methods for this purpose, we have applied an evaluation method based on the stability and goodness of the classifications of ontologies produced by each metric on an ontology corpus. The evaluation has been done using ontology repositories as corpora. More concretely, we have used 119 ontologies from the OBO Foundry repository and 78 ontologies from AgroPortal. First, we study the correlations between the metrics. Second, we study whether the clusters for a given metric are stable and have a good structure. The results show that the existing correlations are not biasing the evaluation, there are no metrics generating unstable clusterings and all the metrics evaluated provide at least reasonable clustering structure. Furthermore, our work permits to review and suggest the most reliable ontology structural metrics in terms of stability and goodness of their classifications. Availability: http://sele.inf.um.es/ontology-metrics.