A New Bibliometric Index: The Top 100 Most Disruptive and Developmental Publications in Colorectal Surgery Journals

BACKGROUND

A new bibliometric index called the disruption score was recently proposed to identify innovative and paradigm-changing publications.

OBJECTIVE

The goal was to apply the disruption score to the colorectal surgery literature to provide the community with a repository of important research articles.

DESIGN

This study is a bibliometric analysis.

SETTINGS

The 100 most disruptive and developmental publications in Diseases of the Colon & Rectum, Colorectal Disease, International Journal of Colorectal Disease, and Techniques in Coloproctology were identified from a validated data set of disruption scores and linked with the iCite National Institutes of Health tool to obtain citation counts.

MAIN OUTCOME MEASURES

The primary outcomes measured were the disruption score and citation count.

RESULTS

We identified 12,127 articles published in Diseases of the Colon & Rectum (n = 8109), International Journal of Colorectal Disease (n = 1912), Colorectal Disease (n = 1751), and Techniques in Coloproctology (n = 355) between 1954 and 2014. Diseases of the Colon & Rectum had the most articles in the top 100 most disruptive and developmental lists. The disruptive articles were in the top 1% of the disruption score distribution in PubMed and were cited between 1 and 671 times. Being highly cited was weakly correlated with high disruption scores (r = 0.09). Developmental articles had disruption scores that were more strongly correlated with citation count (r = 0.18).

LIMITATIONS

This study is subject to the limitations of bibliometric indices, which change over time.

DISCUSSION

The disruption score identified insightful and paradigm-changing studies in colorectal surgery. These studies include a wide range of topics and consistently identified editorials and case reports/case series as important research. This bibliometric analysis provides colorectal surgeons with a unique archive of research that can often be overlooked but that may have scholarly significance. See Video Abstract at http://links.lww.com/DCR/B639.UN NUEVO INDICE BIBLIOMÉTRICO: LAS 100 MAS IMPORTANTES PUBLICACIONES EN INNOVACIONES DESESTABILIZADORAS Y DE DESARROLLO EN LAS REVISTAS DE CIRUGÍA COLORRECTALANTECEDENTES:Un nuevo índice bibliométrico llamado innovación desestabilizadora y de desarrollo ha sido propuesto para identificar publicaciones de vanguardia y que pueden romper paradigmas.OBJETIVO:La meta fué aplicar el índice de desestabilización a la literature en cirugía colorectal para aportar a la comunidad con un acervo importante de artículos de investigación.DISEÑO:Un análisis bibliométrico.PARAMETROS:Las 100 publicaciones mas desestabilizadores y de desarrollo en las revistas: Diseases of the Colon and Rectum, Colorectal Disease, International Journal of Colorectal Disease, y Techniques in Coloproctology se recuperaron de una base de datos validada con puntuaciones de desestabilización y se ligaron con la herramienta iCite NIH para obtener la cuantificación de citas.PRINCIPAL MEDIDA DE RESULTADO:El índice desestabilizador y la cuantificación de citas.RESULTADOS:Se identificaron 12,127 articulos publicados en Diseases of the Colon and Rectum (n = 8,109), International Journal of Colorectal Disease (n = 1,912), Colorectal Disease (n = 1,751), y Techniques in Coloproctology (n = 355) de 1954-2014. Diseases of the Colon and Rectum representó la mayoría de las publicaciones dentro de la lista de los 100 mas desestabilizadores y de desarrollo. Esta literatura desestabilizadora se encuentra en el principal 1% de la distribución de la puntuacón desestabilizadora en PubMed y se citaron de 1 a 671 veces. El ser citado con frecuencia se relacionó vagamente con las puntuaciones de desastibilización (r = 0.09). Los artículos de desarrollo tuvieron puntuaciones de desestabilización que estuvieron muy correlacionados con la cuantificación de las citas (r = 0.18).LIMITACIONES:Las sujetas a las limitaciones de los índices bibliométricos, que se modifican en el tiempo.DISCUSION:La putuación de desestabilicación identificó trabajos perspicaces, pragmáticos y modificadores de paradigmas en cirugía colorrectal. Es de interés identificar que se incluyeron una gran variedad de temas y en forma consistente editoriales, reportes de casos y series de casos que representaron una investigación importante. Este análisis bibliométrico aporta a los cirujanos colorrectales de un acervo de investigación único que puede con frecuencia pasarse por alto, y sin embargo tener una gran importancia académica. Consulte Video Resumen en http://links.lww.com/DCR/B639. (Traducción- Dr. Miguel Esquivel-Herrera).

Publication Rate of Presentations at a National Physical Medicine and Rehabilitation Scientific Conference From 2009 to 2018

ABSTRACT

Conferences are important platforms for sharing research, but full-text publication is necessary for broader dissemination and impact. Few studies have examined the abstract-to-publication rate for physical medicine and rehabilitation. This study aimed to determine the abstract-to-publication rate of research abstracts presented from 2009-2018 at the Canadian Association of Physical Medicine and Rehabilitation annual scientific meetings. Using the official book of abstracts from 2009 to 2018, year of conference, abstract title, author names, and the affiliation/level of training of the first author were extracted. Systematic searches of abstract key words, authors' names, and the abstract title were performed using PubMed and Google Scholar to determine whether an abstract proceeded to full-text publication; if so, the date and journal of publication were extracted. Of the 524 total abstracts presented at the Canadian Association of Physical Medicine and Rehabilitation from 2009 to 2018, 187 went on to full-text publication for an abstract-to-publication rate of 35.8%. The mean time to full-text publication was 23.4 ± 8.63 mos. The abstract-to-publication rate was 18.6% for medical students, 28.7% for residents, 41.1% for physical medicine and rehabilitation consultants, and 49.4% for nonphysician presenters. This study highlights that low abstract-to-publication rates for novel research presented at the Canadian Association of Physical Medicine and Rehabilitation disproportionately affects trainees. Further research is needed to identify and remedy barriers to publication.

Meeting Abstracts - AMCP Managed Care & Specialty Pharmacy Annual Meeting 2018

The AMCP Managed Care & Specialty Pharmacy Annual Meeting 2018 in Boston, Massachusetts, is expected to attract more than 3,800 managed care pharmacists and other health care professionals who manage and evaluate drug therapies, develop and manage networks, and work with medical managers and information specialists to improve the care of all individuals enrolled in managed care programs. The AMCP Abstracts program provides a forum through which authors can share their insights and outcomes of advanced managed care practice. Abstracts are presented as posters on Wednesday, April 25, from 12:30 pm to 2:30 pm. Posters will also be displayed on Tuesday, April 24, from 5:45 pm to 7:30 pm, and on Thursday, April 26, from 9:30 am to 11:00 am. Podium presentations for the Platinum award-winning abstracts are Thursday, April 26, from 8:00 am to 9:15 am. Professional abstracts that have been reviewed are published in the Journal of Managed Care & Specialty Pharmacy's Meeting Abstracts supplement.

Medical concept representation: the years beyond 2000

This work aims at understanding the state of the art in the broad contextual research area of "medical concept representation". Our data support the general understanding that the focus of research has moved toward medical ontologies, which we interpret as a paradigm shift. Both the opinion of socially active groups of researchers and changes in bibliometric data since 1988 support this opinion. Socially active researchers mention the OBO foundry, SNOMED CT, and the UMLS as anchor activities.

Features and publication rates of scientific abstracts presented at a rheumatology congress--EULAR 2008

OBJECTIVE

Various types of scientific abstracts are selected and presented at meetings and listed in abstract books. Recently, a systematic review has shown that 45% of 30,000 abstracts were published in a journal. The aim of this study was to determine the features of abstracts selected to be presented at a EULAR meeting (2008) and the corresponding publication rates.

METHODS

The EULAR 2008 Abstract Book was extracted, presented abstracts were assessed, their publication status was checked, and features related with publication rate were determined. The publication status of abstracts as of January 20, 2011, was verified using PubMed.

RESULTS

A total of 1,732 abstracts were assessed. Median publication duration was 13 (range: 0 to 31) months. Most of the abstracts (N=339) were of multi-national origin. Sixty seven percent of abstracts were clinical and 563 (33%) abstracts were preclinical. We found that 601 of all abstracts (34.7%) had been published in a journal, and most of were published in a rheumatology journal. Fifty-seven percent of published abstracts were in journals with an impact factor higher than 4. The publication rate was correlated with presentation type, number of centers involved, trial design, and number of patients enrolled.

CONCLUSIONS

We found that the publication rate of EULAR 2008 abstracts at 30 months was approximately 35%. This is a high rate compared to a previously published systematic review that investigated the publication rate of studies initially presented as abstracts in medical meetings, which reported the publication rate at 24 and 36 months as 20.7% and 28.1%, respectively. More than half of the published abstracts were accepted by high-impact journals. Presentation type, number of centers, trial design, and enrolled patient numbers were all correlated with the rate of publication.

BOSS: context-enhanced search for biomedical objects

BACKGROUND

There exist many academic search solutions and most of them can be put on either ends of spectrum: general-purpose search and domain-specific "deep" search systems. The general-purpose search systems, such as PubMed, offer flexible query interface, but churn out a list of matching documents that users have to go through the results in order to find the answers to their queries. On the other hand, the "deep" search systems, such as PPI Finder and iHOP, return the precompiled results in a structured way. Their results, however, are often found only within some predefined contexts. In order to alleviate these problems, we introduce a new search engine, BOSS, Biomedical Object Search System.

METHODS

Unlike the conventional search systems, BOSS indexes segments, rather than documents. A segment refers to a Maximal Coherent Semantic Unit (MCSU) such as phrase, clause or sentence that is semantically coherent in the given context (e.g., biomedical objects or their relations). For a user query, BOSS finds all matching segments, identifies the objects appearing in those segments, and aggregates the segments for each object. Finally, it returns the ranked list of the objects along with their matching segments.

RESULTS

The working prototype of BOSS is available at http://boss.korea.ac.kr. The current version of BOSS has indexed abstracts of more than 20 million articles published during last 16 years from 1996 to 2011 across all science disciplines.

CONCLUSION

BOSS fills the gap between either ends of the spectrum by allowing users to pose context-free queries and by returning a structured set of results. Furthermore, BOSS exhibits the characteristic of good scalability, just as with conventional document search engines, because it is designed to use a standard document-indexing model with minimal modifications. Considering the features, BOSS notches up the technological level of traditional solutions for search on biomedical information.

[Growing importance of the Polish Gynaecology--comments of the editorial team]

The Impact Factor is a measure of average citation frequency for articles in a specific scientific journal during a specific period of time. IF takes into account the content, sources, layout and frequency and regularity of publishing. In order to get an impact factor, a journal needs to be first put on the Philadelphia List. In 2001, after the change of the Editor-in-Chief, Polish Gynaecology started the process of trying to get an IF The analysis of contemporary value of the publication was done on the basis of the SJR indicator and the Scopus database. It has revealed that, despite a decrease in the number of publications per year, the percentage of citations has risen to 20%, not to mention an increase in the number of references and articles published within international cooperation. The quality of article summaries in English is of great importance and offers a chance to increase the number of citations of the articles, available online at Medline and Pubmed free of charge. Evaluation of the SJR indicator of all Polish gynecological journals gives reason to believe that Polish Gynaecology is a highly regarded and valued publication.

Comparison of outcomes and other variables between conference abstracts and subsequent peer-reviewed papers involving pre-harvest or abattoir-level interventions against foodborne pathogens

Accuracy in the reporting of studies in conference abstracts is important because the majority of studies in such abstracts are never further detailed in peer-reviewed publications, and data from such abstracts may be used in systematic reviews. Previous research on interventional studies in human biomedicine indicates that there is no guarantee of consistency between a conference abstract and paper in the reporting of results and other key variables. However, no research has been done to determine if this lack of reporting consistency in abstracts and papers extends to interventional studies in pre-harvest/harvest-level food safety. The goal of this study was to compare outcome results and other key variables between conference abstracts and subsequent peer-reviewed publications describing studies of pre-harvest and abattoir-level interventions against foodborne pathogens, and to determine whether the agreement in the results or key variables was associated with the time to full publication. A systematic search identified 59 conference abstracts with matching peer-reviewed papers (matches), and data on variables including outcome measures and results, pathogens, species, interventions, overall efficacy of intervention, sample size and housing were extracted from both the conference abstracts and the papers. The matching of variables between abstracts and papers was described, and logistic regression used to test for associations between variable matching and time to publication. Sample size was only provided for both abstract and paper in 24 matches; the same sample size was reported in 20 of these matches. Most other variables were reported in the majority of abstracts/papers, and with the exception of outcomes and intervention effect, the reporting of variables was relatively consistent. There was no significant difference in the numbers of authors, with the first author the same in 78.3% of matches. Of 231 outcome measures reported in both abstracts and papers, nearly one third (77% or 32.2%) had different results, with 32 changing direction of effect. More than a quarter of matches involved at least one significant change in outcome result. The overall conclusion on the efficacy of the intervention changed in 10.7% of matches. There was a significant association between increased time to publication and differences in the number of authors, and having fewer outcome measures in the abstract reported in the paper. These results suggest that data from conference abstracts should be considered with caution.

Frontiers of biomedical text mining: current progress

It is now almost 15 years since the publication of the first paper on text mining in the genomics domain, and decades since the first paper on text mining in the medical domain. Enormous progress has been made in the areas of information retrieval, evaluation methodologies and resource construction. Some problems, such as abbreviation-handling, can essentially be considered solved problems, and others, such as identification of gene mentions in text, seem likely to be solved soon. However, a number of problems at the frontiers of biomedical text mining continue to present interesting challenges and opportunities for great improvements and interesting research. In this article we review the current state of the art in biomedical text mining or 'BioNLP' in general, focusing primarily on papers published within the past year.

OReFiL: an online resource finder for life sciences

Background

Many online resources for the life sciences have been developed and introduced in peer-reviewed papers recently, ranging from databases and web applications to data-analysis software. Some have been introduced in special journal issues or websites with a search function, but others remain scattered throughout the Internet and in the published literature. The searchable resources on these sites are collected and maintained manually and are therefore of higher quality than automatically updated sites, but also require more time and effort.

Description

We developed an online resource search system called OReFiL to address these issues. We developed a crawler to gather all of the web pages whose URLs appear in MEDLINE abstracts and full-text papers on the BioMed Central open-access journals. The URLs were extracted using regular expressions and rules based on our heuristic knowledge. We then indexed the online resources to facilitate their retrieval and comparison by researchers. Because every online resource has at least one PubMed ID, we can easily acquire its summary with Medical Subject Headings (MeSH) terms and confirm its credibility through reference to the corresponding PubMed entry. In addition, because OReFiL automatically extracts URLs and updates the index, minimal time and effort is needed to maintain the system.

Conclusion

We developed OReFiL, a search system for online life science resources, which is freely available. The system's distinctive features include the ability to return up-to-date query-relevant online resources introduced in peer-reviewed papers; the ability to search using free words, MeSH terms, or author names; easy verification of each hit following links to the corresponding PubMed entry or to papers citing the URL through the search systems of BioMed Central, Scirus, HighWire Press, or Google Scholar; and quick confirmation of the existence of an online resource web page.

Publishing perishing? Towards tomorrow's information architecture

Scientific articles are tailored to present information in human-readable aliquots. Although the Internet has revolutionized the way our society thinks about information, the traditional text-based framework of the scientific article remains largely unchanged. This format imposes sharp constraints upon the type and quantity of biological information published today. Academic journals alone cannot capture the findings of modern genome-scale inquiry.

Like many other disciplines, molecular biology is a science of facts: information inherently suited to database storage. In the past decade, a proliferation of public and private databases has emerged to house genome sequence, protein structure information, functional genomics data and more; these digital repositories are now a vital component of scientific communication. The next challenge is to integrate this vast and ever-growing body of information with academic journals and other media. To truly integrate scientific information we must modernize academic publishing to exploit the power of the Internet. This means more than online access to articles, hyperlinked references and web-based supplemental data; it means making articles fully computer-readable with intelligent markup and Structured Digital Abstracts.

Here, we examine the changing roles of scholarly journals and databases. We present our vision of the optimal information architecture for the biosciences, and close with tangible steps to improve our handling of scientific information today while paving the way for an expansive central index in the future.

Taxonomic indexing--extending the role of taxonomy

Taxonomic indexing refers to a new array of taxonomically intelligent network services that use nomenclatural principles and elements of expert taxonomic knowledge to manage information about organisms. Taxonomic indexing was introduced to help manage the increasing amounts of digital information about biology. It has been designed to form a near basal layer in a layered cyberinfrastructure that deals with biological information. Taxonomic Indexing accommodates the special problems of using names of organisms to index biological material. It links alternative names for the same entity (reconciliation), and distinguishes between uses of the same name for different entities (disambiguation), and names are placed within an indefinite number of hierarchical schemes. In order to access all information on all organisms, Taxonomic indexing must be able to call on a registry of all names in all forms for all organisms. NameBank has been developed to meet that need. Taxonomic indexing is an area of informatics that overlaps with taxonomy, is dependent on the expert input of taxonomists, and reveals the relevance of the discipline to a wide audience.

Recent advances in natural language processing for biomedical applications

We survey a set a recent advances in natural language processing applied to biomedical applications, which were presented in Geneva, Switzerland, in 2004 at an international workshop. While text mining applied to molecular biology and biomedical literature can report several interesting achievements, we observe that studies applied to clinical contents are still rare. In general, we argue that clinical corpora, including electronic patient records, must be made available to fill the gap between bioinformatics and medical informatics.

Impact of FUTON and NAA bias on visibility of research

OBJECTIVE

To determine whether availability of journals on MEDLINE as FUTON (full text on the Net) affects their impact factor.

MATERIAL AND METHODS

A comprehensive search identified 324 cardiology, nephrology, and rheumatology/immunology journals on-line until May 2003. The status of these journals was ascertained in MEDLINE as having FUTON, abstracts only, and NAA (no abstract available). Impact factors for all available journals from the Institute for Scientific Information (ISI) were abstracted.

RESULTS

Of the 324 Journals, 124 (38.3%) were FUTON, 138 (42.6%) had abstracts only, and 62 (19.1%) had NAA. The mean (+/-SEM) impact factor was 3.24 (+/-0.32), 1.64 (+/-0.30), and 0.14 (+/-0.45), respectively. Of the 324 current journals, 159 existed in both the pre- and the post-Internet era. An analysis of the change (ie, delta) in impact factor from the pre- to post-Internet era revealed a trend between journals with FUTON and abstracts only (P=.17, Wilcoxon rank sum test). Similar analyses of the delta of cardiology journals revealed a statistically significant difference between Journals with FUTON and abstracts only (P=.04, Wilcoxon rank sum test).

CONCLUSION

FUTON bias is the tendency to peruse what is more readily available. This is the first study to show that on-line availability of medical literature may increase the impact factor and that such increase tends to be greater in FUTON journals. Failure to consider this bias may affect a journal's impact factor. Also, it could limit consideration of medical literature by ignoring relevant NAA articles and thereby influence medical education akin to publication or language bias.

The publication rate of abstracts from the 4th Park City Pediatric Rheumatology meeting in peer-reviewed journals: what factors influenced publication?

OBJECTIVE

To measure the impact of a rheumatology scientific meeting by examining publication rates of the abstracts in peer-reviewed journals and to investigate factors that influenced publication.

METHODS

We examined the outcome of 257 abstracts submitted to the 1998 4th Park City Pediatric Rheumatology meeting, all of which were published in The Journal of Rheumatology (April 2000). A MEDLINE search of all abstracts, by authors, topics, and keywords was performed. A mail questionnaire regarding publication and factors influencing submission was sent to authors of abstracts not found in this search. Factors analyzed for influence on publication included author geography, disease studied, study topic, study design, positive or negative results, and the novelty of the study.

RESULTS

Ninety-two abstracts (36%) were published as of January 2002 in peer-reviewed journals, most frequently in The Journal of Rheumatology (32 abstracts). Factors with a positive influence on publication included abstracts from the Middle East, analytical studies, positive results, and novel studies. Factors negatively associated with publication include origin from South and Central America, studies of physical therapy, education and psychosocial issues, and studies with negative results. The main causes cited by authors for not submitting abstracts for publication included a lack of time or low priority to write a full paper, or a desire to further expand the study.

CONCLUSION

The 4th Park City meeting had an important clinical scientific impact measured by abstracts published in peer-reviewed journals. In future meetings, abstracts that will eventually be published are likely to have an analytical design, positive results, and a novel topic.

Creating an online dictionary of abbreviations from MEDLINE

OBJECTIVE

The growth of the biomedical literature presents special challenges for both human readers and automatic algorithms. One such challenge derives from the common and uncontrolled use of abbreviations in the literature. Each additional abbreviation increases the effective size of the vocabulary for a field. Therefore, to create an automatically generated and maintained lexicon of abbreviations, we have developed an algorithm to match abbreviations in text with their expansions.

DESIGN

Our method uses a statistical learning algorithm, logistic regression, to score abbreviation expansions based on their resemblance to a training set of human-annotated abbreviations. We applied it to Medstract, a corpus of MEDLINE abstracts in which abbreviations and their expansions have been manually annotated. We then ran the algorithm on all abstracts in MEDLINE, creating a dictionary of biomedical abbreviations. To test the coverage of the database, we used an independently created list of abbreviations from the China Medical Tribune.

MEASUREMENTS

We measured the recall and precision of the algorithm in identifying abbreviations from the Medstract corpus. We also measured the recall when searching for abbreviations from the China Medical Tribune against the database.

RESULTS

On the Medstract corpus, our algorithm achieves up to 83% recall at 80% precision. Applying the algorithm to all of MEDLINE yielded a database of 781,632 high-scoring abbreviations. Of all the abbreviations in the list from the China Medical Tribune, 88% were in the database.

CONCLUSION

We have developed an algorithm to identify abbreviations from text. We are making this available as a public abbreviation server at \url[http://abbreviation.stanford.edu/].

The poor quality of information about laparoscopy on the World Wide Web as indexed by popular search engines

BACKGROUND

This study was undertaken to determine the quality of information on the Internet regarding laparoscopy.

METHODS

Four popular World Wide Web search engines were used with the key word "laparoscopy." Advertisements, patient- or physician-directed information, and controversial material were noted.

RESULTS

A total of 14,030 Web pages were found, but only 104 were unique Web sites. The majority of the sites were duplicate pages, subpages within a main Web page, or dead links. Twenty-eight of the 104 pages had a medical product for sale, 26 were patient-directed, 23 were written by a physician or group of physicians, and six represented corporations. The remaining 21 were "miscellaneous." The 46 pages containing educational material were critically reviewed. At least one of the senior authors found that 32 of the pages contained controversial or misleading statements. All of the three senior authors (LKN, NAO, GAF) independently agreed that 17 of the 46 pages contained controversial information.

CONCLUSION

The World Wide Web is not a reliable source for patient or physician information about laparoscopy. Authenticating medical information on the World Wide Web is a difficult task, and no government or surgical society has taken the lead in regulating what is presented as fact on the World Wide Web.

Determinants of abstract acceptance for the Digestive Diseases Week--a cross sectional study

BACKGROUND

The Digestive Diseases Week (DDW) is the major meeting for presentation of research in gastroenterology. The acceptance of an abstract for presentation at this meeting is the most important determinant of subsequent full publication. We wished to examine the determinants of abstract acceptance for this meeting.

METHODS

A cross-sectional study was performed, based on abstracts submitted to the DDW. All 17,205 abstracts submitted from 1992 to 1995 were reviewed for acceptance, country of origin and research type (controlled clinical trials (CCT), other clinical research (OCR), basic science (BSS)). A random sub-sample (n = 1,000) was further evaluated for formal abstract quality, statistical significance of study results and sample size.

RESULTS

326 CCT, 455 OCR and 219 BSS abstracts were evaluated in detail. Abstracts from N/W Europe (OR 0.4, 95% CI 0.3-0.6), S/E Europe (OR 0.4, 95% CI 0.2-0.6) and non-Western countries (OR 0.3, 95% CI 0.2-0.5) were less likely to be accepted than North-American contributions when controlling for research type. In addition, the OR for the acceptance for studies with negative results as compared to those with positive results was 0.4 (95% CI 0.3-0.7). A high abstract quality score was also weakly associated with acceptance rates (OR 1.4, 95% CI 1.0-2.0).

CONCLUSIONS

North-American contributions and reports with statistically positive results have higher acceptance rates at the AGA. Formal abstract quality was also predictive for acceptance.

Clinical vocabularies and terminologies: impact on the future of health information management

As standardized clinical vocabularies and terminologies are being developed, Health Information Management (HIM) practitioners must increase their knowledge of nomenclature and coding systems beyond the traditional ICD-9-CM and CPT. The limited scope of these classifications prevents them from serving as the standard vocabulary for the computer-based patient record. The American Health Information Management Association (AHIMA) Coding Futures Task Force provided insights into the knowledge and skills that HIM professionals will need as clinical vocabularies and terminologies become more integrated into health care systems. Certain characteristics are important in a clinical vocabulary and several clinical vocabularies are in use today.

Evolution of abstracts presented at the annual scientific meetings of academic emergency medicine

There has been a general trend in medicine towards greater sophistication in research design. In order to assess this trend in emergency medicine we compared the characteristics of abstracts presented at the 1974, 1983, 1989, and 1997 annual scientific meetings of academic emergency medicine. All 870 abstracts were reviewed by 1 of 3 investigators who determined research design attributes using a standardized classification scheme that has good interrater reliability. Over the last 25 years the following trends were noted: more surveys (0% v1% v3% v8%, P = 0.002), more randomized studies (0% v10% v12% v15%, P = 0.05), and more blinded studies (0% v7% v5% v11%, P = 0.01). Tests of statistical significance were reported with increasing frequency (8% v26% v59% v69%, P < 0.001) as were power calculations (0% v0% v1% v3%, P = 0.02). During the study period there were also increases in the median number of authors, proportion of foreign lead authors, and the proportion of studies involving human subjects. These results reflect considerable improvement in the degree of research design sophistication reported in selected abstracts of academic emergency medicine over the study period. Further strategies to assure continued enhancement of emergency medicine research should be explored.

Change in the Medicare case-mix index in the 1980s and the effect of the prospective payment system

Persistent increases in the Medicare case-mix index over the 1980s have been ascribed to changes both in medical treatment ("real changes") and in the way medical information is recorded ("coding changes") in hospitals. These changes have been attributed, in the absence of appropriate data and analyses, to the incentives of the Medicare prospective payment system (PPS). Using data for 1980-1986 from 235 hospitals, we estimate the effect on the Medicare case-mix index of a series of variables that reflect medical treatments and coding practices. Each of these underlying real or coding variables was changing prior to PPS and would likely have continued to change even in the absence of PPS. Furthermore, PPS may have had a distinct effect on these variables. These underlying trends and the PPS effects must each be estimated. Thus, the analysis begins by developing separate estimates for each of these real and coding variables (1) in the absence of PPS (autonomous effects) and (2) as a result of PPS (induced effects). Then, changes in the case-mix index are regressed against all of these variables to determine the degree to which specific autonomous real or coding variables or induced real or coding variables actually influenced measured case mix. Results show that real and coding changes each accounted for about half of the change in the Medicare case-mix index between 1980 and 1986, with the influence of coding starting to wane by 1986. PPS-induced factors explain about 80 percent of the change in measured case mix over time, autonomous factors about 20 percent. Especially powerful determinants of case-mix change included PPS-induced substitution of surgical for medical care and PPS-induced improvements in the accuracy of coding that led to assignment of patients to higher-weighted DRGs. Also, stringent Medicare peer review organizations appeared to restrain rises in case-mix indexes for their hospitals. Outpatient substitution for inpatient treatment, which others attributed to PPS, was well underway before PPS was announced.

The accuracy of Medicare's hospital claims data: progress has been made, but problems remain

BACKGROUND

Health care databases provide a widely used source of data for health care research, but their accuracy remains uncertain. We analyzed data from the 1985 National DRG Validation Study, which carefully reabstracted and reassigned ICD-9-CM diagnosis and procedure codes from a national sample of 7050 medical records, to determine whether coding accuracy had improved since the Institute of Medicine studies of the 1970s and to assess the current coding accuracy of specific diagnoses and procedures.

METHODS

We defined agreement as the proportion of all reabstracted records that had the same principal diagnosis or procedure coded on both the original (hospital) record and on the reabstracted record. We also evaluated coding accuracy in 1985 using the concepts of diagnostic test evaluation.

RESULTS

Overall, the percentage of agreement between the principal diagnosis on the reabstracted record and the original hospital record, when analyzed at the third digit, improved from 73.2% in 1977 to 78.2% in 1985. However, analysis of the 1985 data demonstrated that the accuracy of diagnosis and procedure coding varies substantially across conditions.

CONCLUSIONS

Although some diagnoses and all major surgical procedures that we examined were accurately coded, the variability in the accuracy of diagnosis coding poses a problem that must be overcome if claims-based research is to achieve its full potential.

Answer-passage retrieval by text searching

Passage retrieval (already operational for lawyers) has advantages in output form over reference retrieval and is economically feasible. Previous experiments in passage retrieval for scientists have demonstrated recall and false retrieval rates as good or better than those of present reference retrieval services. The present experiment involved a greater variety of forms of retrieval question. In addition, search words were selected independently by two different people for each retrieval question. The search words selected, in combination with the computer procedures used for passage retrieval, produced average recall ratios of 72 and 67%, respectively, for the two selectors. The false retrieval rates were (except for one predictably difficult question) respectively 13 and 10 falsely retrieved sentences per answer-paper retrieved.