The WHO ICD-11 Classification of Dermatological Disorders: a new comprehensive online skin disease taxonomy designed by and for dermatologists

In May 2019, the World Health Assembly officially adopted the Eleventh Revision of the International Classification of Diseases (ICD-11)¹ . When the current 10th Revision (ICD-10) was released some three decades ago, the world was at the beginning of the modern information technology era. The World Health Organization (WHO) had long recognised that the "one size fits all" nature of ICD-10 and its inability to adapt to change seriously hampered its usefulness in healthcare research and management. The initial designs for ICD-11 were formulated by the WHO in 2007. Since then, many individuals from around the globe have participated in its development.

Health Concept and Knowledge Management: Twenty-five Years of Evolution

OBJECTIVES

The fields of health terminology, classification, ontology, and related information models have evolved dramatically over the past 25 years. Our objective was to review notable trends, described emerging or enabling technologies, and highlight major terminology systems during the interval.

METHODS

We review the progression in health terminology systems informed by our own experiences as part of the community involved in this work, reinforced with literature review and citation.

RESULTS

The transformation in size, scope, complexity, and adoption of health terminological systems and information models has been tremendous, on the scale of orders of magnitude.

CONCLUSION

The present "big science" era of inference and discovery in biomedicine would not have been possible or scalable absent the growth and maturation of health terminology systems and information models over the past 25 years.

Genetic variation among 82 pharmacogenes: The PGRNseq data from the eMERGE network

Genetic variation can affect drug response in multiple ways, although it remains unclear how rare genetic variants affect drug response. The electronic Medical Records and Genomics (eMERGE) Network, collaborating with the Pharmacogenomics Research Network, began eMERGE‐PGx, a targeted sequencing study to assess genetic variation in 82 pharmacogenes critical for implementation of “precision medicine.” The February 2015 eMERGE‐PGx data release includes sequence‐derived data from ∼5,000 clinical subjects. We present the variant frequency spectrum categorized by variant type, ancestry, and predicted function. We found 95.12% of genes have variants with a scaled Combined Annotation‐Dependent Depletion score above 20, and 96.19% of all samples had one or more Clinical Pharmacogenetics Implementation Consortium Level A actionable variants. These data highlight the distribution and scope of genetic variation in relevant pharmacogenes, identifying challenges associated with implementing clinical sequencing for drug treatment at a broader level, underscoring the importance for multifaceted research in the execution of precision medicine.

Harmonization of detailed clinical models with clinical study data standards

INTRODUCTION

This article is part of the Focus Theme of METHODS of Information in Medicine on "Managing Interoperability and Complexity in Health Systems".

BACKGROUND

Data sharing and integration between the clinical research data management system and the electronic health record system remains a challenging issue. To approach the issue, there is emerging interest in utilizing the Detailed Clinical Model (DCM) approach across a variety of contexts. The Intermountain Healthcare Clinical Element Models (CEMs) have been adopted by the Office of the National Coordinator awarded Strategic Health IT Advanced Research Projects for normalization (SHARPn) project for normalizing patient data from the electronic health records (EHR).

OBJECTIVE

The objective of the present study is to describe our preliminary efforts toward harmonization of the SHARPn CEMs with CDISC (Clinical Data Interchange Standards Consortium) clinical study data standards.

METHODS

We were focused on three generic domains: demographics, lab tests, and medications. We performed a panel review on each data element extracted from the CDISC templates and SHARPn CEMs.

RESULTS

We have identified a set of data elements that are common to the context of both clinical study and broad secondary use of EHR data and discussed outstanding harmonization issues.

CONCLUSIONS

We consider that the outcomes would be useful for defining new requirements for the DCM modeling community and ultimately facilitating the semantic interoperability between systems for both clinical study and broad secondary use domains.

Design and anticipated outcomes of the eMERGE-PGx project: a multicenter pilot for preemptive pharmacogenomics in electronic health record systems

We describe here the design and initial implementation of the eMERGE-PGx project. eMERGE-PGx, a partnership of the eMERGE and PGRN consortia, has three objectives : 1) Deploy PGRNseq, a next-generation sequencing platform assessing sequence variation in 84 proposed pharmacogenes, in nearly 9,000 patients likely to be prescribed drugs of interest in a 1–3 year timeframe across several clinical sites; 2) Integrate well-established clinically-validated pharmacogenetic genotypes into the electronic health record with associated clinical decision support and assess process and clinical outcomes of implementation; and 3) Develop a repository of pharmacogenetic variants of unknown significance linked to a repository of EHR-based clinical phenotype data for ongoing pharmacogenomics discovery. We describe site-specific project implementation and anticipated products, including genetic variant and phenotype data repositories, novel variant association studies, clinical decision support modules, clinical and process outcomes, approaches to manage incidental findings, and patient and clinician education methods.

Effectiveness of lexico-syntactic pattern matching for ontology enrichment with clinical documents

OBJECTIVE

To evaluate the effectiveness of a lexico-syntactic pattern (LSP) matching method for ontology enrichment using clinical documents.

METHODS

Two domains were separately studied using the same methodology. We used radiology documents to enrich RadLex and pathology documents to enrich National Cancer Institute Thesaurus (NCIT). Several known LSPs were used for semantic knowledge extraction. We first retrieved all sentences that contained LSPs across two large clinical repositories, and examined the frequency of the LSPs. From this set, we randomly sampled LSP instances which were examined by human judges. We used a two-step method to determine the utility of these patterns for enrichment. In the first step, domain experts annotated medically meaningful terms (MMTs) from each sentence within the LSP. In the second step, RadLex and NCIT curators evaluated how many of these MMTs could be added to the resource. To quantify the utility of this LSP method, we defined two evaluation metrics: suggestion rate (SR) and acceptance rate (AR). We used these measures to estimate the yield of concepts and relationships, for each of the two domains.

RESULTS

For NCIT, the concept SR was 24%, and the relationship SR was 65%. The concept AR was 21%, and the relationship AR was 14%. For RadLex, the concept SR was 37%, and the relationship SR was 55%. The concept AR was 11%, and the relationship AR was 44%.

CONCLUSION

The LSP matching method is an effective method for concept and concept relationship discovery in biomedical domains.

Distributed biomedical terminology development: from experiments to open process

OBJECTIVE

Can social computing efforts materially alter the distributed creation and maintenance of complex biomedical terminologies and ontologies; a review of distributed authoring history and status.

BACKGROUND

Social computing projects, such as Wikipedia, have dramatically altered the perception and reality of large-scale content projects and the labor required to create and maintain them. Health terminologies have become large, complex, interdependent content artifacts of increasing importance to biomedical research and the communities understanding of biology, medicine, and optimal healthcare practices. The question naturally arises as to whether social computing models and distributed authoring platforms can be applied to the voluntary, distributed authoring of high-quality terminologies and ontologies.

METHODS

An historical review of distributed authoring developments.

RESULTS

The trajectory of description logic-driven authoring tools, group process, and web-based platforms suggests that public distributed authoring is likely feasible and practical; however, no compelling example on the order of Wikipedia is yet extant. Nevertheless, several projects, including the Gene Ontology and the new revision of the International Classification of Disease (ICD-11) hold promise.

Designing and testing computer based screening engine for severe sepsis/septic shock

This study addresses the role of a sepsis "sniffer", an automatic screening tool for the timely identification of patients with severe sepsis/septic shock, based electronic medical records. During the two months prospective implementation in a medical intensive care unit, 37 of 320 consecutive patients developed severe sepsis/septic shock. The sniffer demonstrated a sensitivity of 48% and specificity of 86%, and positive predictive value 32%. Further improvements are needed prior to the implementation of sepsis sniffer in clinical practice and research.

Rule base system for identification of patients with specific critical care syndromes: The "sniffer" for acute lung injury

Early detection of specific critical care syndromes, such as sepsis or acute lung injury (ALI)is essential for timely implementation of evidence based therapies. Using a near-real time copy of the electronic medical records ("ICU data mart") we developed and validated custom electronic alert (ALI"sniffer") in a cohort of 485 critically ill medical patients. Compared with the gold standard of prospective screening, ALI "sniffer" demonstrated good sensitivity, 93% (95% CI 90 to 95) and specificity, 90% (95% CI 87 to 92). It is not known if the bedside implementation of ALI "sniffer" will improve the adherence to evidence-based therapies and outcome of patients with ALI.

A population-based study on the quality of life for cancer patients as reported in a tumor registry follow-up

8594

Background: Between 1/96 and 3/00, 94,516 SF-36 or SF-12 Quality of Life (QOL) forms were sent to every patient on the Mayo Clinic tumor registry not seen at the clinic in the last year. This study is a descriptive summary of each patient’s first reported QOL score. Method: QOL scores were combined with tumor registry information about age, gender, tumor site, tumor stage, residence, and date of diagnosis. Responder versus non-responder characteristics were compared via Fisher’s exact tests and two-sample t-tests. Means and standard deviations were used to summarize the data. Patient reported scores and SF norms from a healthy population were compared using one sample t-tests. Results: Data were available on 28,166 patients (48,463 forms). Men were more likely to respond. More patients were male (56%) with a mean age of 66 and filled out their first SF form an average of 9 years after their initial diagnosis. Overall, mean physical and mental subscale scores were not clinically different from the age- and sex-adjusted SF normative data (Table). Within tumor sites, mental subscale scores were similar to the general population. Physical subscale scores were also approximately normal within each of the tumor sites. The exceptions were that patients with lung or musculoskeletal cancers tended to report lower physical scores than a healthy population. Conclusions: Patients responding to a tumor registry follow-up program reported QOL scores that were not clinically different from that of a healthy population. QOL deficits observed in patients with lung cancer and musculoskeletal cancers may indicate a need for targeted interventions specific for these cancer patient populations.

[Table: see text]

No significant financial relationships to disclose.

Frame semantics and the domain of functioning, disability and health

We explore the use of FrameNet (FN) frames and their frame elements to represent content from the International Classification of Functioning, Disability and Health (ICF) Self-Care chapter. Terms were extracted from the ICF and mapped to FN frames. The mappings were validated by an expert. FN provided relevant and nearly complete coverage of ICF terms, suggesting FN may be an important resource to leverage for semantic language processing and knowledge representation in this domain.

Human cytosolic sulfotransferase database mining: identification of seven novel genes and pseudogenes

A total of 10 SULT genes are presently known to be expressed in human tissues. We performed a comprehensive genome-wide search for novel SULT genes using two different but complementary approaches, and developed a novel graphical display to aid in the annotation of the hits. Seven novel human SULT genes were identified, five of which were predicted to be pseudogenes, including two processed pseudogenes and three pseudogenes that contained introns. Those five pseudogenes represent the first unambiguous SULT pseudogenes described in any species. Expression-profiling studies were conducted for one novel gene, SULT6B1, and a series of alternatively spliced transcripts were identified in the human testis. SULT6B1 was also present in chimpanzee and gorilla, differing at only seven encoded amino-acid residues among the three species. The results of these database mining studies will aid in studies of the regulation of these SULT genes, provide insights into the evolution of this gene family in humans, and serve as a starting point for comparative genomic studies of SULT genes.

Integrating pharmacokinetics knowledge into a drug ontology: as an extension to support pharmacogenomics

The newly developed U.S. Common Medication Information Infrastructure was used as a basis to capture and formally express the properties of drugs relevant to research and the clinical application of pharmacogenomics. Two associated taxonomies within the model, Mechanism of Action and Physiologic Effect, were enriched to accommodate pharmacogenomic use-cases; the 4,000 active ingredients in the VA NDF-RT drug file were related to the enhanced taxonomies. Pharmacokinetics were independently modeled for pharmacogenomics and tested against thirty-one high-profile drugs to demonstrate our approach.

Guideline for health informatics: controlled health vocabularies--vocabulary structure and high-level indicators

Developers and purchasers of controlled health terminologies require valid mechanisms for comparing terminological systems. By Controlled Health Vocabularies we refer to terminologies and terminological systems designed to represent clinical data at a granularity consistent with the practice of today's healthcare delivery. Comprehensive criterion for the evaluation of such systems are lacking and the known criteria are inconsistently applied. Although there are many papers, which describe specific desirable features of a controlled health vocabulary, to date there is not a consistent guide for evaluators of terminologies to reference, which will help them compare implementations of terminological systems on an equal footing 1,2 This guideline serves to fill the gap between academic enumeration of desirable terminological characteristics and the practical implementation or rigorous evaluations which will yield comparable data regarding the quality of one or more controlled health vocabularies.

An assessment of cancer clinical trials vocabulary and IT infrastructure in the U.S

Twenty-three cancer research centers in the U.S. were assessed to determine data standards, vocabularies, and information infrastructure used in support of clinical trials. Eighteen of the 23 responded. Major findings were related to: 1) clinical trials infrastructure information, 2) current systems environment, 3) technical details, and 4) vocabulary and data standards. The size of the facility correlated with the quality, features and functionality of the clinical trials system (CTS). One facility had as many as 22 separate CTS. There were only 2 sites that had integrated clinical information systems (CIS) with CTS. The responses included the major vocabularies and data standards used in CTS. The majority used some automation but many also reported manual data entry. CTS had more manual entry than CIS because of regulatory reporting requirements. The assessment identified opportunities for guidance in defining vocabularies and standards for cancer clinical trial systems in the US.

Expression of a domain ontology model in unified modeling language for the World Health Organization International classification of impairment, disability, and handicap, version 2

The International Classification of Impairment, Disability, and Handicap Version 2(ICIDH-2), an anticipated addition to the World Health Organization suite of terminologies, has been put forth as a means for standardized representation of generic health and/or functional status data. In an attempt to make explicit the ontology upon which ICIDH-2 is based the authors derived a concept model expressed as a Unified Modeling Language static class diagram through abstraction of concept-terms in the documentation provided with the Full Version Pre-Final Draft of ICIDH-2 (December 2000). ICIDH-2's semantic structure is analyzed and evaluated for its semantic consistency. Discussion is presented on the utility of domain ontology models in terminology development and potential roles ICIDH-2 might play, as it undergoes refinement towards a representational standard. It is intended that the proposed UML rendering will stimulate domain discourse and consensus that will lead to enhancement of conceptual clarity in the ICIDH-2 ontological hierarchy and further enable its study and development as a healthcare classification.

Clinical classification and terminology: some history and current observations

The evolution of health terminology has undergone glacial transition over time, although this pace has quickened recently. After a long history of near neglect, unimaginative structure, and factitious development, health terminologies are in an era of unprecedented importance, sophistication, and collaboration. The major highlights of this history are reviewed, together with important intellectual advances in health terminology development. The inescapable conclusion is that we are amidst a major revolution in the role and capabilities of health terminologies, entering an age of large-scale systems for health concept representation with international implications.

A randomized controlled trial of concept based indexing of Web page content

OBJECTIVE

Medical information is increasingly being presented in a web-enabled format. Medical journals, guidelines, and textbooks are all accessible in a web-based format. It would be desirable to link these reference sources to the electronic medical record to provide education, to facilitate guideline implementation and usage and for decision support. In order for these rich information sources to be accessed via the medical record they will need to be indexed by a single comparable underlying reference terminology.

METHODS

We took a random sample of 100 web pages out of the 6,000 web pages on the Mayo Clinic's Health Oasis web site. The web pages were divided into four datasets each containing 25 pages. These were humanly reviewed by four clinicians to identify all of the health concepts present (R1DA, R2DB, R3DC, R4DD). The web pages were simultaneously indexed using the SNOMED-RT beta release. The indexing engine has been previously described and validated. A new clinician reviewed the indexed web pages to determine the accuracy of the automated mappings as compared with the human identified concepts (R4DA, R3DB, R2DC, R1DD).

RESULTS

This review found 13,220 health concepts. Of these 10,383 concepts were identified by the initial human review (78.5% +/- 3.6%). The automated process identified 10,083 concepts correctly (76.3% +/- 4.0%) from within this corpus. The computer identified 2,420 concepts, which were not identified by the clinician's review but were upon further consideration important to include as health concepts. There was on average a 17.1% +/- 3.5% variability in the human reviewers ability to identify the important health concepts within web page content. Concept Based Indexing provided a positive predictive value (PPV) of finding a health concept of 79.3% as compared with keyword indexing which only has a PPV of 33.7% (p < 0.001).

CONCLUSION

SNOMED-RT is a reasonable ontology for web page indexing. Concept based indexing provides a significantly greater accuracy in identifying health concepts when compared with keyword indexing.

Representation by standard terminologies of health status concepts contained in two health status assessment instruments used in rheumatic disease management

Health and functional status data have been shown to have clinical utility in predicting outcome. Various metadata registries in the form of patient self-administered health assessment questionnaires have been incorporated into routine clinical care and clinical research of patients with rheumatic disease. Examples of such health assessment instruments are the Clinical Health Assessment Questionnaire (CLINHAQ) and the Modified Health Assessment Questionnaire (MHAQ). These instruments contain concepts that are an integral part of the health and functional status domain. Using an automated indexing tool we examined the clinical content coverage by SNOMED RT and the Unified Medical Language System (UMLS) Metathesaurus for health and functional status concepts identified in the MHAQ and CLINHAQ. Significant differences existed between the overall representational ability of SNOMED and UMLS for concepts identified in the MHAQ (49%, vs. 77% respectively, p < .005) and for concepts identified in the CLINHAQ (30% vs. 64% respectively p < .005). Representational capability by SNOMED-RT and UMLS for concepts in a given health assessment instrument was carried across four semantic classes of "attitudes", "symptoms", "activities", and "social attributes". The conceptual content coverage of health status assessment concepts contained in the MHAQ and CLINHAQ by SNOMED-RT and UMLS was incomplete but better for UMLS with its panoply of vocabulary sources. This observed overall improved representation by UMLS appeared to be due to better representation of concepts in "activities" and "social attributes" semantic classes. Representation of health or functional status concepts in a computerized medical record should be founded on a universally agreed concept model of that domain. Established functional and health status metadata registries can serve as important sources for concepts and candidate classes within that domain.

A formal approach to integrating synonyms with a reference terminology

Medical terminologies continue to grow in scope, completeness and detail. The emerging generation of terminology systems define concepts in terms of their position within a categorical structure. It is still necessary, however, to access and represent the concepts using everyday spoken and written language, which introduces both lexical and semantic ambiguity. This ambiguity can have a negative impact on both selectivity and recall when it comes to associating free-form textual phrases with their coded equivalent. Lexical ambiguity issues can often be addressed algorithmically, but semantic ambiguity presents a more difficult problem. A common solution to the semantic problem is to associate many different representational permutations with a given target concept. This approach has several drawbacks. An alternate solution is to build separate synonym tables that can serve as permuted indices into the terms representing the underlying concepts. A potential shortcoming of this approach, however, is a further reduction in the lookup selectivity. One possible source of loss of selectivity could be "meaning drift"--the gradual change in meaning that can be introduced when following a chain of nearly synonymous words. We posited that organizing synonyms into separate "meaning clusters" might reduce this loss in precision, but the results of this study did not bear that out.

Embedded structures and representation of nursing knowledge

Nursing Vocabulary Summit participants were challenged to consider whether reference terminology and information models might be a way to move toward better capture of data in electronic medical records. A requirement of such reference models is fidelity to representations of domain knowledge. This article discusses embedded structures in three different approaches to organizing domain knowledge: scientific reasoning, expertise, and standardized nursing languages. The concept of pressure ulcer is presented as an example of the various ways lexical elements used in relation to a specific concept are organized across systems. Different approaches to structuring information-the clinical information system, minimum data sets, and standardized messaging formats-are similarly discussed. Recommendations include identification of the polyhierarchies and categorical structures required within a reference terminology, systematic evaluations of the extent to which structured information accurately and completely represents domain knowledge, and modifications or extensions to existing multidisciplinary efforts.

The content coverage and organizational structure of terminologies: the example of postoperative pain

Concepts such as symptoms present specific representational challenges in the EMR. This is because concepts without clear boundaries and external referents such as physical objects can only be examined against other terminology-based concept representation systems. The truth and falsity of such concept representation is therefore relative to the terminology-based systems. Using the concept of acute postoperative pain as an example, we examined three terminology based approaches to representing the concept. Widely varying coverage across existing clinical terminologies was evident, although the common clinical approach to reporting attributes of symptoms provided a useful organizational structure and should be examined in relation to developing terminology and information models.

The role of compositionality in standardized problem list generation

Compositionality is the ability of a Vocabulary System to record non-atomic strings. In this manuscript we define the types of composition, which can occur. We will then propose methods for both server based and client-based composition. We will differentiate the terms Pre-Coordination, Post-Coordination, and User-Directed Coordination. A simple grammar for the recording of terms with concept level identification will be presented, with examples from the Unified Medical Language System's (UMLS) Metathesaurus. We present an implementation of a Window's NT based client application and a remote Internet Based Vocabulary Server, which makes use of this method of compositionality. Finally we will suggest a research agenda which we believe is necessary to move forward toward a more complete understanding of compositionality. This work has the promise of paving the way toward a robust and complete Problem List Entry Tool.

Incremental costs of enrolling cancer patients in clinical trials: a population-based study

BACKGROUND

Payment for care provided as part of clinical research has become less predictable as a result of managed care. Because little is known at present about how entry into cancer trials affects the cost of care for cancer patients, we conducted a matched case-control comparison of the incremental medical costs attributable to participation in cancer treatment trials.

METHODS

Case patients were residents of Olmsted County, MN, who entered phase II or phase III cancer treatment trials at the Mayo Clinic from 1988 through 1994. Control patients were patients who did not enter trials but who were eligible on the basis of tumor registry matching and medical record review. Sixty-one matched pairs were followed for up to 5 years after the date of trial entry for case patients or from an equivalent date for control patients. Hospital, physician, and ancillary service costs were estimated from a population-based cost database developed at the Mayo Clinic.

RESULTS

Trial enrollees incurred modestly (no more than 10%) higher costs over various follow-up periods. The mean cumulative 5-year cost in 1995 inflation-adjusted U.S. dollars among trial enrollees after adjustment for censoring was $46424 compared with $44 133 for control patients. After 1 year, trial enrollee costs were $24645 compared with $23 964 for control patients.

CONCLUSIONS

This study suggests that cancer chemotherapy trials may not imply budget-breaking costs. Cancer itself is a high-cost illness. Clinical protocols may add relatively little to that cost.

A randomized double-blind controlled trial of automated term dissection

OBJECTIVE

To compare the accuracy of an automated mechanism for term dissection to represent the semantic dependencies within a compositional expression, with the accuracy of a practicing Internist to perform this same task. We also compare the results of four evaluators to determine the inter-observer variability and the variance between term sets, with respect to the accuracy of the mappings and the consistency of the failure analysis.

METHODS

500 terms, which required a compositional expression to effect an exact match, were randomly distributed into two sets of 250 terms (Set A and Set B). Set A was dissected using the Automated Term Dissection (ATD) Algorithm. A physician specializing in Internal Medicine dissected set B. He had no prior knowledge of the dissection algorithm or how it functioned. In this manuscript, the authors use Human Term Dissection (HTD) to refer to this method. Set A was randomized to two sets of 125 terms (Set A1 and Set A2). Set B was randomized to two sets of 125 terms (Set B1 and Set B2). A new set of 250 terms Set C was created from Set A1 and Set B2. A second new set of 250 terms Set D was created from Set A2 and Set B1. Two expert Indexers reviewed Set C and another two expert Indexers reviewed Set D. They were blinded to which terms were dissected by the clinician and which terms were dissected by the automated term dissection algorithm. The person providing the files for review to the Indexers was also unaware of which terms were dissected by ATD vs. the HTD method. The Indexers recorded whether or not the dissection was the best possible representation of the input concept. If not, a failure analysis was conducted. They recorded whether or not the dissection was in error and if so was a modifier not subsumed or was a Kernel concept subsumed when it should not have been. If a concept was missing, the Indexers recorded whether it was a Kernel concept, a modifier, a qualifier or a negative qualifier.

RESULTS

The ATD method was judged to be accurate and readable in 265 out of the 424 terms with adequate content (62.7%). The HTD method was judged to be accurate in 272 out of 414 terms with adequate content (65.7%). There was no statistically significant difference between the rates of acceptability of the ATD and HTD methods (p = 0.33). There was a non-significant trend toward greater acceptability of the ATD method in the subgroup of terms with three or more compositional elements. ATD was acceptable in 53.6% of the terms where the HTD was only acceptable in 43.6% (p = 0.11). The failure analysis showed that both methods misrepresented kernel concepts and modifiers much more commonly than qualifiers (p < 0.001).

CONCLUSIONS

There is no statistically significant difference in the accuracy and readability of terms dissected using the automated term dissection method when compared with human term dissection, as judged by four expert medical indexers. There is a non-significant trend toward improved performance of the ATD method in the subset of more complex terms. The authors submit that this may be due to a tendency for users to be less compulsive when the time to complete the task is long. Automated term dissection is a useful and perhaps preferable method for representing readable and accurate compound terminological expressions.

A large-scale evaluation of terminology integration characteristics

OBJECTIVE

To describe terminology integration characteristics of local specialty specific and general vocabularies in order to facilitate the appropriate inclusion and mapping of these terms into a large-scale terminology.

METHODS

We compared the sensitivity, specificity, positive predictive value, and positive likelihood ratios for Automated Term Composition to correctly map 9050 local specialty specific (dermatology) terms and 4994 local general terms to UMLS using Metaphrase. Results were systematically combined among exact matches, semantic type filtered matches, and non-filtered matches. For the general set, an analysis of semantic type filtering was performed.

RESULTS

Dermatology exact matches defined a sensitivity of 51% (57% for general terms) and a specificity of 86% (92% general terms). Including semantic type filtered matches increased sensitivity (75% dermatology; 88% general); as did inclusion of non-filtered matches (98% and 99%). These inclusions correspondingly decreased specificity (filtered: 82% and 74%; non-filtered: 52% and 32%). Positive predictive values for exact matches (93.0% dermatology, 97.6% general) were improved by small but significant (p < 0.001) margins by including filtered matches (95.1% dermatology, 98.4% general) but decreased with non-filtered matches (89.2% dermatology, 87.8% general). Adding additional semantic types to the filtering algorithm failed to improve the positive predictive value or the positive likelihood ratio of term mapping, in spite of a 2.3% improvement in sensitivity.

CONCLUSIONS

Automated methods for mapping local "colloquial" terminologies to large-scale controlled health vocabulary systems are practical (ppv 95% dermatology, 98% general). Semantic type filtering improves specificity without sacrificing sensitivity and yields high positive predictive values in every set analyzed.

Desiderata for a clinical terminology server

Clinical terminology servers are distinguished from more broadly based terminology servers intended for nomenclature development or mediation across classifications. Focusing upon the consistent and comparable entry of clinical observations, findings, and events, key desiderata are enumerated and expanded. These include 1) word normalization, 2) word completion, 3) target terminology specification, 4) spelling correction, 5) lexical matching, 6) term completion, 7) semantic locality, 8) term composition and 9) decomposition. Comparisons of this functionality to previously published models and specifications are made. Experience with a clinical terminology server, Metaphrase, is described.

Barriers to the clinical implementation of compositionality

BACKGROUND

Compositional mechanisms for the entry of clinically relevant controlled vocabularies have been suggested as a possible solution to providing adequate descriptive precision while keeping term vocabulary redundancy under control. As of yet, there are no widely accepted term navigators that allow physicians to enter problem lists utilizing controlled vocabularies with compositionality.

METHODS

We report on the results of a usability trial of 5 physicians using our most recent attempt at developing the Mayo Problem List Manager. We tested the implementation of an automated term composition, and hierarchical term dissection.

RESULTS

Participants found acceptable terms 96% of the time and found automated term composition helpful in 85% of the case scenarios. There was significant confusion about the terminology used to describe compositional elements (kernel concepts, modifiers, and qualifiers) however participants used the functions appropriately. Speed of entry was universally stated as the limiting factor.

CONCLUSIONS

The variety of methods that our participants used to enter terms highlights the need for multiple ways to accomplish the task of data entry. Successful implementation of user directed compositionality could be accomplished with further improvement of the user interface and the underlying terminology.

Detailed content and terminological properties of DSM-IV

DSM-IV, the Diagnostic and Statistical Manual of Mental Disorders, is the internationally accepted standard for nomenclature and diagnosis in psychiatric practice. The objective of this project is to parse the rubric criteria of the DSM to extract the clinically detailed signs, symptoms, findings, and conditions that are present. These are a "latent terminology" implicit within the DSM, which is highly granular and clinically specific. This manuscript describes the content of these terms that heretofore existed sub rosa, though we recognize that during the authorship of the DSM such terms were constructed deliberately and systematically. Relevant characteristics of the classification system are briefly reviewed. Summary results of parsing the defining criteria for the 400 ICD-9 Codes enumerated in DSM-IV are presented.

Metaphrase: an aid to the clinical conceptualization and formalization of patient problems in healthcare enterprises

Patient descriptors, or "problems," such as "brain metastases of melanoma" are an effective way for caregivers to describe patients. But most problems, e.g., "cubital tunnel syndrome" or "ulnar nerve compression," found in problem lists in an Electronic Medical Record (EMR) are not comparable computationally--in general, a computer cannot determine whether they describe the same or a related problem, or whether the user would have preferred "ulnar nerve compression syndrome." Metaphrase is a scalable, middleware component designed to be accessed from problem-manager applications in EMR systems. In response to caregivers' informal descriptors it suggests potentially equivalent, authoritative, and more formally comparable descriptors. Metaphrase contains a clinical subset of the 1997 UMLS Metathesaurus and some 10,000 "problems" from the Mayo Clinic and Harvard Beth Israel Hospital. Word and term completion, spelling correction, and semantic navigation, all combine to ease the burden of problem conceptualization, entry and formalization.

Scalable methodologies for distributed development of logic-based convergent medical terminology

As the size and complexity of medical terminologies increase, terminology modelers are increasingly hampered by lack of tools and methods to manage the development process. This paper presents our use and ongoing evaluation of a description-logic classifier to support cognitive scalability of the underlying terminology and our enhancements to that classifier to support concurrent development utilizing semantics-based concurrency control methods. Our enhancements, collectively referred to as the Gálapagos, consist of several applications that take locally-developed terminology enhancements from multiple sites, identify conflicting design decisions, support the modelers' reconciliation of the conflicting designs, and efficiently disseminate updates tailored for locally enhanced terminologies. We have tested our ideas through concurrent evolutionary enhancement of SNOMED International at three Kaiser Permanente regions and the Mayo Clinic. We have found that the underlying environment has met our design objectives, and supports semantic-based concurrency control, and identification and resolution of conflicting design decisions.

Secular changes in radical prostatectomy utilization rates in Olmsted County, Minnesota 1980 to 1995

PURPOSE

We estimated the changes in utilization of radical prostatectomy for treatment of prostate cancer and describe the clinical characteristics of men undergoing radical prostatectomy in a population based setting.

MATERIALS AND METHODS

The Rochester Epidemiology Project was used to identify all Olmsted County residents who underwent radical prostatectomy from 1980 to 1995. The community medical records of these men were reviewed to determine the clinical and pathological stage and grade at biopsy and following surgery.

RESULTS

From 1980 to 1995, 311 radical prostatectomies were performed on Olmsted County men. From 1980 to 1987 prostatectomy rates ranged from 6.3 to 31.0/100,000 men but rates increased dramatically to 53.6/100,000 in 1988 and 106.2/100,000 in 1992. The rate after 1992 decreased to 53.0/100,000 and then increased slightly to 80.4/100,000. There was a shift to younger age in more recent times (mean patient age 65.4 years in 1980 to 1986 and 62.4 in 1993 to 1995, p = 0.02), a nonsignificant (p = 0.10) trend toward lower pathological stage in recent years (42% stage pT2 in 1980 to 1986 versus 55% in 1993 to 1995) and a significant decrease in the proportion of cases of disease up staged following surgery (53% in 1980 to 1986 versus 37% in 1993 to 1995, p = 0.03). There was no significant trend in pathological grade with time (63% Mayo grade I or II in 1980 to 1986 versus 52% in 1993 to 1995, p = 0.30).

CONCLUSIONS

These findings demonstrate an increase in radical prostatectomy rates that coincided with increases in prostate cancer incidence. There was a decrease in population prostatectomy rates in 1993 which was followed by modest increases to levels lower than the peak in 1992. However, the clinical characteristics of patients during this period did not change dramatically, suggesting that in a population based setting the selection factors for patients undergoing surgical treatment may not have changed.

A clinical terminology in the post modern era: pragmatic problem list development

A brief review of the rich heritage of classifications and terminologies is the background for a description of the Mayo Clinic's clinical terminology development. Vender specific system constraints prompted the scope and style of an interim problem list vocabulary. We describe the sources and review process which led to a working terminology for use in a Computer-based Patient Record (CPR). Because terminology development is often subjective and metrics against which to measure the quality of individual human judgements are few, we decided to compare the selection of preferred terms made by general internists with those made by sub-specialists. A significant difference between a sub-specialist's assignment of preferred terms and a general internist's (948 vs. 2271, P < 0.001) was observed. Sub-specialists were less than half as likely as a generalist to designate a term as a preferred form. These results emphasize the need for sub-specialty editing when assigning preferred terms to concepts.

The Copernican era of healthcare terminology: a re-centering of health information systems

Health terminology and classifications have been an unseen backwater in healthcare practice and information systems development. Today however, the recognized need for comparable patient data is driving a new discovery about its strategic importance. Consistent patient descriptions and concept-centered data representations are crucial for efficient discovery of optimal treatments, best outcomes, and efficient practice patterns. The fabled linkage of knowledge sources at the time and place of care requires the conceptual intermediary of common terminology. A brief history overviewing the evolution of health classifications will provide the foundation for considering present and evolving health terminology developments. Their roles in health information systems will be characterized. Discussion will focus on the likely influences of the HIPAA legislation nationally and the new ISO Healthcare Informatics Technical Committee internationally, on terminology adaptation and incorporation.

A randomized controlled trial of automated term composition

OBJECTIVE

To compare the ability of an Automated Term Composition (ATC) algorithm with non-compositional mappings to provide coverage (exact mappings to a controlled vocabulary) for a randomly selected set of free text entries which were entered as headings to the Impression section of the clinical notes system at the Mayo Foundation. We also compare the results of four evaluators to determine the inter-observer variability and the variance between term sets, with respect to the accuracy of the mappings and the reliability of the failure analysis.

METHODS

From a corpus of approximately 1,000,000 unique terms entered into the Impression/Report/Plan section of the clinical notes system in the calendar year 1997, we randomly selected 1,000 terms. We then further randomized these 1,000 terms into two groups of 500 (Sets A and B). We constructed two copies of the same term matching interface, one without ATC (alpha) and one with ATC (beta). We took four expert Indexers and assigned them to one of the following tasks. The first reviewer (R1) compared set A using the alpha program and then set B using the beta program (R1(Aalpha + Bbeta)). The second compared set A using the alpha program and then set B using the alpha program (R2(A + B) alpha). The third compared set B using the beta program and then set A using the beta program (R3(B + A) beta). The fourth compared set A using the beta program and then set B using the alpha program (R4(Abeta + Balpha)).

RESULTS

The program with Automated Term Composition mapped 540 out of the 1,000 Concepts correctly (54.0%). The same program without ATC mapped only 276 out of the 1,000 Concepts correctly (27.6%). Therefore the program with ATC was significantly more effective at matching concepts in our problem lists than the same search engine without ATC (p < 0.0001; McNemar Method). These figures result from the comparison of the alpha program with the beta program by reviewers one and four. Failure analysis showed that with the alpha version 425 out of the 724 mismatches were because a base concept was missing from the retrieval set (58.7%) and 299 mismatches were from missing qualifiers or modifiers or both (41.3%). In the beta version of the program (with ATC) 340 out of the 460 mismatches were secondary to there being a missing base concept in the retrieval set (73.9%) and only 120 mismatches due to missing modifiers and or qualifiers (26.1%).

CONCLUSIONS

Automated term composition provided significantly better coverage of a randomly chosen set of patient problems, diagnosed at the Mayo Clinic during the 1997 calendar year, when compared with the same information retrieval system without ATC. We believe that these results speak further to the excellent content coverage provided by the UMLS metathesaurus. These authors believe that increased structure, normalization of UMLS content and semantics, and better tools to make use of the currently available content such as automated term composition, are what is needed to leverage the production of commercially viable tools that provide access to controlled vocabularies for medicine.

A framework for comprehensive health terminology systems in the United States: development guidelines, criteria for selection, and public policy implications. ANSI Healthcare Informatics Standards Board Vocabulary Working Group and the Computer-Based Patient Records Institute Working Group on Codes and Structures

Health care in the United States has become an information-intensive industry, yet electronic health records represent patient data inconsistently for lack of clinical data standards. Classifications that have achieved common acceptance, such as the ICD-9-CM or ICD, aggregate heterogeneous patients into broad categories, which preclude their practical use in decision support, development of refined guidelines, or detailed comparison of patient outcomes or benchmarks. This document proposes a framework for the integration and maturation of clinical terminologies that would have practical applications in patient care, process management, outcome analysis, and decision support. Arising from the two working groups within the standards community--the ANSI (American National Standards Institute) Healthcare Informatics Standards Board Working Group and the Computer-based Patient Records Institute Working Group on Codes and Structures--it outlines policies regarding 1) functional characteristics of practical terminologies, 2) terminology models that can broaden their applications and contribute to their sustainability, 3) maintenance attributes that will enable terminologies to keep pace with rapidly changing health care knowledge and process, and 4) administrative issues that would facilitate their accessibility, adoption, and application to improve the quality and efficiency of American health care.

Standardized problem list generation, utilizing the Mayo canonical vocabulary embedded within the Unified Medical Language System

VOCABULARY: The Mayo problem list vocabulary is a clinically derived lexicon created from the entries made to the Mayo Clinic's Master Sheet Index and the problem list entries made to the Impression/ Report/Plan section of the Clinical Notes System over the last three years. The vocabulary was reduced by eliminating repetition including lexical variants, spelling errors, and qualifiers (Administrative or Operational terms). Qualifiers are re-coordinated with other terms, at run-time, which greatly increased the number of input strings which our system is capable of recognizing.

IMPLEMENTATION

The Problem Manager is implemented using standard windows tools in a Windows NT environment. The interface is designed using Object Pascal. HTTP calls are passed over the World Wide Web to a UNIX based vocabulary server. The server returns a document, which is read into Object Pascal structures, parsed, filtered and displayed.

STUDY

This paper reports the results of a recent Usability Trial focused on assessing the viability of this mechanism for standardized problem entry. Eight clinicians engaged in eleven scenarios and responded as to their satisfaction with the systems performance. These responses were observed, videotaped and tabulated. Clinicians in this study were able to find acceptable diagnoses in 91.1% of the scenarios. The response time was acceptable in 92.5% of the scenarios. The presentation of related terms was stated to be useful in at least one scenario by seven of the eight participants. All clinicians wanted to make use of shortcuts which would minimize the amount of typing necessary to encode the concept they were searching for (e.g. Abbreviations, Word Completion).

CONCLUSIONS

Clinicians are willing to choose a canonical term from a suggested list (as opposed to their own wording). Clinicians want an "intelligent" system, which would suggest terms within a category (e.g. Types of "Migraine"). They are able to make functional use of our system, in its current state of development. Finally, all clinicians appreciate the value of encoding their problems in a standardized vocabulary, toward improved research, education and practice.

Phase II evaluation of clinical coding schemes: completeness, taxonomy, mapping, definitions, and clarity. CPRI Work Group on Codes and Structures

OBJECTIVE

To compare three potential sources of controlled clinical terminology (READ codes version 3.1, SNOMED International, and Unified Medical Language System (UMLS) version 1.6) relative to attributes of completeness, clinical taxonomy, administrative mapping, term definitions and clarity (duplicate coding rate).

METHODS

The authors assembled 1929 source concept records from a variety of clinical information taken from four medical centers across the United States. The source data included medical as well as ample nursing terminology. The source records were coded in each scheme by an investigator and checked by the coding scheme owner. The codings were then scored by an independent panel of clinicians for acceptability. Codes were checked for definitions provided with the scheme. Codes for a random sample of source records were analyzed by an investigator for "parent" and "child" codes within the scheme. Parent and child pairs were scored by an independent panel of medical informatics specialists for clinical acceptability. Administrative and billing code mapping from the published scheme were reviewed for all coded records and analyzed by independent reviewers for accuracy. The investigator for each scheme exhaustively searched a sample of coded records for duplications.

RESULTS

SNOMED was judged to be significantly more complete in coding the source material than the other schemes (SNOMED* 70%; READ 57%; UMLS 50%; *p < .00001). SNOMED also had a richer clinical taxonomy judged by the number of acceptable first-degree relatives per coded concept (SNOMED* 4.56, UMLS 3.17; READ 2.14, *p < .005). Only the UMLS provided any definitions; these were found for 49% of records which had a coding assignment. READ and UMLS had better administrative mappings (composite score: READ* 40.6%; UMLS* 36.1%; SNOMED 20.7%, *p < .00001), and SNOMED had substantially more duplications of coding assignments (duplication rate: READ 0%; UMLS 4.2%; SNOMED* 13.9%, *p < .004) associated with a loss of clarity.

CONCLUSION

No major terminology source can lay claim to being the ideal resource for a computer-based patient record. However, based upon this analysis of releases for April 1995, SNOMED International is considerably more complete, has a compositional nature and a richer taxonomy. Is suffers from less clarity, resulting from a lack of syntax and evolutionary changes in its coding scheme. READ has greater clarity and better mapping to administrative schemes (ICD-10 and OPCS-4), is rapidly changing and is less complete. UMLS is a rich lexical resource, with mappings to many source vocabularies. It provides definitions for many of its terms. However, due to the varying granularities and purposes of its source schemes, it has limitations for representation of clinical concepts within a computer-based patient record.

A clinically derived terminology: qualification to reduction

Mayo Foundation is developing synonym rich entry points for the recording of patient problems by clinicians, which will map to the KP-Mayo Convergent Medical Terminology. We describe the empirical sources for these terminology components, and how the number and complexity of the terms could be substantially reduced by the introduction of a Qualifier axis. The expressive power of these entry points is dramatically enhanced by this axis. This work is being integrated into terminology navigation modules being jointly developed with Lexical Technology, which leverages UMLS content. It will from the basis for structured problem entry into Mayo's Computer-based Electronic Record.

Clinical care management and workflow by episodes

This paper describes the implementation of clinically defined episodes of care and the introduction of an episode-based summary list of patient problems across Mayo Clinic Rochester in 1996 and 1997. Although Mayo's traditional paper-based system has always relied on a type of 'episode of care' (called the "registration") for patient and history management, a new, more clinically relevant definition of episode of care was put into practice in November 1996. This was done to improve care management and operational processes and to provide a basic construct for the electronic medical record. Also since November 1996, a computer-generated summary list of patient problems, the "Master Sheet Summary Report," organized by episode, has been placed in all patient histories. In the third quarter of 1997, the ability to view the episode-based problem summary online was made available to the 3000+ EMR-capable workstations deployed across the Mayo Rochester campus. In addition, the clinically oriented problem summarization process produces an improved basic "package" of clinical information expected to lead to improved analytic decision support, outcomes analysis and epidemiological research.

Gálapagos: computer-based support for evolution of a convergent medical terminology

Current controlled medical terminologies fall short of the needs of informatics application developers. To overcome the limitations of current medical terminologies, many groups are independently enhancing existing terminologies to meet their local needs. With proper computer-based support, local enhancements can be used as evolutionary stepping stones toward a convergent medical terminology. Gálapagos is a collection of applications that can take local enhancements from multiple sites, identify conflicting design decisions, allow developers to reconcile the conflicting designs, and efficiently disseminate updates tailored specifically for compatibility with locally enhanced terminologies. This paper describes an initial proof-of-concept of the Gálapagos programs using data generated during concurrent SNOMED enhancement by Kaiser Permanente and the Mayo Clinic.

The content coverage of clinical classifications. For The Computer-Based Patient Record Institute's Work Group on Codes & Structures

BACKGROUND AND OBJECTIVE

Patient conditions and events are the core of patient record content. Computer-based records will require standard vocabularies to represent these data consistently, thereby facilitating clinical decision support, research, and efficient care delivery. To address whether existing major coding systems can serve this function, the authors evaluated major clinical classifications for their content coverage.

METHODS

Clinical text from four medical centers was sampled from inpatient and outpatient settings. The resultant corpus of 14,247 words was parsed into 3,061 distinct concepts. These concepts were grouped into Diagnoses, Modifiers, Findings, Treatments and Procedures, and Other. Each concept was coded into ICD-9-CM, ICD-10, CPT, SNOMED III, Read V2, UMLS 1.3, and NANDA; a secondary reviewer ensured consistency. While coding, the information was scored: 0 = no match, 1 = fair match, 2 = complete match.

RESULTS

ICD-9-CM had an overall mean score of 0.77 out of 2; its highest subscore was 1.61 for Diagnoses. ICD-10 scored 1.60 for Diagnoses, and 0.62 overall. The overall score of ICD-9-CM augmented by CPT was not materially improved at 0.82. The SNOMED International system demonstrated the highest score in every category, including Diagnoses (1.90), and had an overall score of 1.74.

CONCLUSION

No classification captured all concepts, although SNOMED did notably the most complete job. The systems in major use in the United States, ICD-9-CM and CPT, fail to capture substantial clinical content. ICD-10 does not perform better than ICD-9-CM. The major clinical classifications in use today incompletely cover the clinical content of patient records; thus analytic conclusions that depend on these systems may be suspect.