Integration of relational and textual biomedical sources. A pilot experiment using a semi-automated method for logical schema acquisition

Learning the Preferences of Physicians for the Organization of Result Lists of Medical Evidence Articles

Background: Online medical knowledge repositories such as MEDLINE and The Cochrane Library are increasingly used by physicians to retrieve articles to aid with clinical decision making. The prevailing approach for organizing retrieved articles is in the form of a rank-ordered list, with the assumption that the higher an article is presented on a list, the more relevant it is.

Objectives: Despite this common list-based organization, it is seldom studied how physicians perceive the association between the relevance of articles and the order in which articles are presented. In this paper we describe a case study that captured physician preferences for 3-element lists of medical articles in order to learn how to organize medical knowledge for decision-making.

Methods: Comprehensive relevance evaluations were developed to represent 3-element lists of hypothetical articles that may be retrieved from an online medical knowledge source such as MEDLINE or The Cochrane Library. Comprehensive relevance evalua tions asses not only an article’s relevance for a query, but also whether it has been placed on the correct list position. In other words an article may be relevant and correctly placed on a result list (e.g. the most relevant article appears first in the result list), an article may be relevant for a query but placed on an incorrect list position (e.g. the most relevant article appears second in a result list), or an article may be irrelevant for a query yet still appear in the result list. The relevance evaluations were presented to six senior physi cians who were asked to express their preferences for an article’s relevance and its position on a list by pairwise comparisons representing different combinations of 3-element lists. The elicited preferences were assessed using a novel GRIP (Generalized Regression with Intensities of Preference) method and represented as an additive value function. Value functions were derived for individual physicians as well as the group of physicians.

Results: The results show that physicians assign significant value to the 1st position on a list and they expect that the most relevant article is presented first. Whilst physicians still prefer obtaining a correctly placed article on position 2, they are also quite satisfied with misplaced relevant article. Low consideration of the 3rd position was uniformly confirmed.

Conclusions: Our findings confirm the importance of placing the most relevant article on the 1st position on a list and the importance paid to position on a list significantly diminishes after the 2nd position. The derived value functions may be used by developers of clinical decision support applications to decide how best to organize medical knowledge for decision making and to create personalized evaluation measures that can augment typical measures used to evaluate information retrieval systems.

Natural language processing systems for capturing and standardizing unstructured clinical information: A systematic review

We followed a systematic approach based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses to identify existing clinical natural language processing (NLP) systems that generate structured information from unstructured free text. Seven literature databases were searched with a query combining the concepts of natural language processing and structured data capture. Two reviewers screened all records for relevance during two screening phases, and information about clinical NLP systems was collected from the final set of papers. A total of 7149 records (after removing duplicates) were retrieved and screened, and 86 were determined to fit the review criteria. These papers contained information about 71 different clinical NLP systems, which were then analyzed. The NLP systems address a wide variety of important clinical and research tasks. Certain tasks are well addressed by the existing systems, while others remain as open challenges that only a small number of systems attempt, such as extraction of temporal information or normalization of concepts to standard terminologies. This review has identified many NLP systems capable of processing clinical free text and generating structured output, and the information collected and evaluated here will be important for prioritizing development of new approaches for clinical NLP.

Using nanoinformatics methods for automatically identifying relevant nanotoxicology entities from the literature

Nanoinformatics is an emerging research field that uses informatics techniques to collect, process, store, and retrieve data, information, and knowledge on nanoparticles, nanomaterials, and nanodevices and their potential applications in health care. In this paper, we have focused on the solutions that nanoinformatics can provide to facilitate nanotoxicology research. For this, we have taken a computational approach to automatically recognize and extract nanotoxicology-related entities from the scientific literature. The desired entities belong to four different categories: nanoparticles, routes of exposure, toxic effects, and targets. The entity recognizer was trained using a corpus that we specifically created for this purpose and was validated by two nanomedicine/nanotoxicology experts. We evaluated the performance of our entity recognizer using 10-fold cross-validation. The precisions range from 87.6% (targets) to 93.0% (routes of exposure), while recall values range from 82.6% (routes of exposure) to 87.4% (toxic effects). These results prove the feasibility of using computational approaches to reliably perform different named entity recognition (NER)-dependent tasks, such as for instance augmented reading or semantic searches. This research is a "proof of concept" that can be expanded to stimulate further developments that could assist researchers in managing data, information, and knowledge at the nanolevel, thus accelerating research in nanomedicine.

Biomedical informatics publications: a global perspective. Part II: Journals

BACKGROUND

Biomedical Informatics (BMI) is a broad discipline, having evolved from both Medical Informatics (MI) and Bioinformatics (BI). An analysis of publications in the fieldshould provide an indication about the geographic distribution of BMI research contributions and possible lessons for the future, both for research and professional practice.

OBJECTIVES

In part I of our analysis of biomedical informatics publications we presented results from BMI conferences. In this second part, we analyse BMI journals, which provide a broader perspective and comparison between data from conferences and journals that ought to confirm or suggest alternatives to the original distributional findings from the conferences.

METHODS

We manually collected data about authors and their geographical origin from various MI journals: the International Journal of Medical Informatics (IJMI), the Journal of Biomedical Informatics (JBI), Methods of In formation in Medicine (MIM) and The Journal of the American Medical Informatics Association (JAMIA). Focusing on first authors, we also compared these findings with data from the journal Bioinformatics.

RESULTS

Our results confirm those obtained in our analysis of BMI conferences - that local and regional authors favor their corresponding MI journals just as they do their conferences. Consideration of other factors, such as the increasingly open source nature of data and software tools, is consistent with these findings.

CONCLUSIONS

Our analysis suggests various indicators that could lead to further, deeper analyses, and could provide additional insights for future BMI research and professional activities.

Biomedical informatics publications: a global perspective: part I: conferences

BACKGROUND

In the past decade, Medical Informatics (MI) and Bioinformatics (BI) have converged towards a new discipline, called Biomedical Informatics (BMI) bridging informatics methods across the spectrum from genomic research to personalized medicine and global healthcare. This convergence still raises challenging research questions which are being addressed by researchers internationally, which in turn raises the question of how biomedical informatics publications reflect the contributions from around the world in documenting the research.

OBJECTIVES

To analyse the worldwide participation of biomedical informatics researchers from professional groups and societies in the best-known scientific conferences in the field. The analysis is focused on their geographical affiliation, but also includes other features, such as the impact and recognition of the conferences.

METHODS

We manually collected data about authors of papers presented at three major MI conferences: Medinfo, MIE and the AMIA symposium. In addition, we collected data from a BI conference, ISMB, as a comparison. Finally, we analyzed the impact and recognition of these conferences within their scientific contexts.

RESULTS

Data indicate a predominance of local authors at the regional conferences (AMIA and MIE), whereas other conferences with a world-wide scope (Medinfo and ISMB) had broader participation. Our analysis shows that the influence of these conferences beyond the discipline remains somewhat limited.

CONCLUSIONS

Our results suggest that for BMI to be recognized as a broad discipline, both in the geographical and scientific sense, it will need to extend the scope of collaborations and their interdisciplinary impacts worldwide.

Automated classification of free-text pathology reports for registration of incident cases of cancer

OBJECTIVE

Our study aimed to construct and evaluate functions called "classifiers", produced by supervised machine learning techniques, in order to categorize automatically pathology reports using solely their content.

METHODS

Patients from the Poitou-Charentes Cancer Registry having at least one pathology report and a single non-metastatic invasive neoplasm were included. A descriptor weighting function accounting for the distribution of terms among targeted classes was developed and compared to classic methods based on inverse document frequencies. The classification was performed with support vector machine (SVM) and Naive Bayes classifiers. Two levels of granularity were tested for both the topographical and the morphological axes of the ICD-O3 code. The ability to correctly attribute a precise ICD-O3 code and the ability to attribute the broad category defined by the International Agency for Research on Cancer (IARC) for the multiple primary cancer registration rules were evaluated using F1-measures.

RESULTS

5121 pathology reports produced by 35 pathologists were selected. The best performance was achieved by our class-weighted descriptor, associated with a SVM classifier. Using this method, the pathology reports were properly classified in the IARC categories with F1-measures of 0.967 for both topography and morphology. The ICD-O3 code attribution had lower performance with a 0.715 F1-measure for topography and 0.854 for morphology.

CONCLUSION

These results suggest that free-text pathology reports could be useful as a data source for automated systems in order to identify and notify new cases of cancer. Future work is needed to evaluate the improvement in performance obtained from the use of natural language processing, including the case of multiple tumor description and possible incorporation of other medical documents such as surgical reports.