Accuracy of references in five leading medical journals

Exploring the Limits of Artificial Intelligence for Referencing Scientific Articles

OBJECTIVE

 To evaluate the reliability of three artificial intelligence (AI) chatbots (ChatGPT, Google Bard, and Chatsonic) in generating accurate references from existing obstetric literature.

STUDY DESIGN

 Between mid-March and late April 2023, ChatGPT, Google Bard, and Chatsonic were prompted to provide references for specific obstetrical randomized controlled trials (RCTs) published in 2020. RCTs were considered for inclusion if they were mentioned in a previous article that primarily evaluated RCTs published by the top medical and obstetrics and gynecology journals with the highest impact factors in 2020 as well as RCTs published in a new journal focused on publishing obstetric RCTs. The selection of the three AI models was based on their popularity, performance in natural language processing, and public availability. Data collection involved prompting the AI chatbots to provide references according to a standardized protocol. The primary evaluation metric was the accuracy of each AI model in correctly citing references, including authors, publication title, journal name, and digital object identifier (DOI). Statistical analysis was performed using a permutation test to compare the performance of the AI models.

RESULTS

 Among the 44 RCTs analyzed, Google Bard demonstrated the highest accuracy, correctly citing 13.6% of the requested RCTs, whereas ChatGPT and Chatsonic exhibited lower accuracy rates of 2.4 and 0%, respectively. Google Bard often substantially outperformed Chatsonic and ChatGPT in correctly citing the studied reference components. The majority of references from all AI models studied were noted to provide DOIs for unrelated studies or DOIs that do not exist.

CONCLUSION

 To ensure the reliability of scientific information being disseminated, authors must exercise caution when utilizing AI for scientific writing and literature search. However, despite their limitations, collaborative partnerships between AI systems and researchers have the potential to drive synergistic advancements, leading to improved patient care and outcomes.

KEY POINTS

· AI chatbots often cite scientific articles incorrectly.. · AI chatbots can create false references.. · Responsible AI use in research is vital..

High Rates of Fabricated and Inaccurate References in ChatGPT-Generated Medical Content

Background The availability of large language models such as Chat Generative Pre-trained Transformer (ChatGPT, OpenAI) has enabled individuals from diverse backgrounds to access medical information. However, concerns exist about the accuracy of ChatGPT responses and the references used to generate medical content. Methods This observational study investigated the authenticity and accuracy of references in medical articles generated by ChatGPT. ChatGPT-3.5 generated 30 short medical papers, each with at least three references, based on standardized prompts encompassing various topics and therapeutic areas. Reference authenticity and accuracy were verified by searching Medline, Google Scholar, and the Directory of Open Access Journals. The authenticity and accuracy of individual ChatGPT-generated reference elements were also determined. Results Overall, 115 references were generated by ChatGPT, with a mean of 3.8±1.1 per paper. Among these references, 47% were fabricated, 46% were authentic but inaccurate, and only 7% were authentic and accurate. The likelihood of fabricated references significantly differed based on prompt variations; yet the frequency of authentic and accurate references remained low in all cases. Among the seven components evaluated for each reference, an incorrect PMID number was most common, listed in 93% of papers. Incorrect volume (64%), page numbers (64%), and year of publication (60%) were the next most frequent errors. The mean number of inaccurate components was 4.3±2.8 out of seven per reference. Conclusions The findings of this study emphasize the need for caution when seeking medical information on ChatGPT since most of the references provided were found to be fabricated or inaccurate. Individuals are advised to verify medical information from reliable sources and avoid relying solely on artificial intelligence-generated content.

Manuscript Referencing Errors and Their Impact on Shaping Current Evidence

It is imperative that articles published in reputable peer-reviewed journals provide balanced, fair, objective, and accurate references. However, studies on the accuracy of references in various scientific disciplines demonstrate an error rate of 25%-54%. These errors can range from minor errors in citation accuracy to major errors that alter the original content and meaning of the material referenced. This article discusses importance of citation accuracy, reviews principles of good citation practices, and offers recommendations aimed to decrease citation error rates.

Normalization of Deviation: Quotation Error in Human Factors

Objective The objective of this paper is to examine quotation error in human factors. Background Science progresses through building on the work of previous research. This requires accurate quotation. Quotation error has a number of adverse consequences: loss of credibility, loss of confidence in the journal, and a flawed basis for academic debate and scientific progress. Quotation error has been observed in a number of domains, including marine biology and medicine, but there has been little or no previous study of this form of error in human factors, a domain that specializes in the causes and management of error. Methods A study was conducted examining quotation accuracy of 187 extracts from 118 published articles that cited a control article (Vaughan's 1996 book: The Challenger Launch Decision: Risky Technology, Culture, and Deviance at NASA). Results Of extracts studied, 12.8% ( n = 24) were classed as inaccurate, with 87.2% ( n = 163) being classed as accurate. A second dimension of agreement was examined with 96.3% ( n = 180) agreeing with the control article and only 3.7% ( n = 7) disagreeing. The categories of accuracy and agreement form a two by two matrix. Conclusion Rather than simply blaming individuals for quotation error, systemic factors should also be considered. Vaughan's theory, normalization of deviance, is one systemic theory that can account for quotation error. Application Quotation error is occurring in human factors and should receive more attention. According to Vaughan's theory, the normal everyday systems that promote scholarship may also allow mistakes, mishaps, and quotation error to occur.

A retrospective analysis of reported errata in five leading medical journals in 2012

Background

Although medical publications are frequently used as the source of information, the prevalence of errata remains unclear. The objective of this study was to examine peer-review and publication processes of medical journals as well as to determine the occurrence of reported errata in medical journals and timeliness in identifying and correcting errata.

Methods

Five medical journals, New England Journal of Medicine, Annals of Internal Medicine, British Medical Journal, Journal of American Medical Association, and Lancet, were evaluated. The characteristics of these journals were obtained from editors’ survey. All these journals report errata noted in their prior publications. We retrospectively analyzed all errata reported from January 1, 2012, to December 31, 2012. The mean number of reported errata per issue, the most common errata, and the mean time to report errata were calculated.

Results

The journals had high impact factors (14–51), received 3,200 to more than 15,000 submissions in 2012, and utilized two or more external reviewers and usually two or more editors for any accepted articles. All the journals edited the accepted articles, including references, figures, and tables for style. A mean of 1.3 articles with ≥1 errata was reported per issue (a total of 306 articles with errata in 226 issues). Errata in author's information, numeric errata, and errata in the figures and tables were the most common errata. The mean time to report the errata was 122 days.

Conclusion

The high-impact journals, with extensive pre-publication review, reported relatively few errata per issue. The delay in reporting errata needs further exploration.

Decay of references to Web sites in articles published in general medical journals: mainstream vs small journals

BACKGROUND

Over the last decade, Web sites (URLs) have been increasingly cited in scientific articles. However, the contents of the page of interest may change over the time.

OBJECTIVE

To investigate the trend of citation to URLs in five general medical journals since January 2006 to June 2013 and to compare the trends in mainstream journals with small journals.

METHODS

References of all original articles and review articles published between January 2006 and June 2013 in three regional journals - Archives of Iranian Medicine (AIM), Eastern Mediterranean Health Journal (EMHJ), and Journal of Postgraduate Medical Institute (JPMI) - and two mainstream journals - The Lancet and British Medical Journal (BMJ) - were reviewed. The references were checked to determine the frequency of citation to URLs as well as the rate of accessibility of the URLs cited.

RESULTS

A total of 2822 articles was studied. Since January 2006 onward, the number of citations to URLs increased in the journals (doubling time ranged from 4.2 years in EMHJ to 13.9 years in AIM). Overall, the percentage of articles citing at least one URL has increased from 24% in 2006 to 48.5% in 2013. Accessibility to URLs decayed as the references got old (half life ranged from 2.2 years in EMHJ to 5.3 years in BMJ). The ratio of citation to URLs in the studied mainstream journals, as well as the ratio of URLs accessible were significantly (p<0.001) higher than the small medical journals.

CONCLUSION

URLs are increasingly cited, but their contents decay with time. The trend of citing and decaying URLs are different in mainstream journals compared to small medical journals. Decay of URL contents would jeopardize the accuracy of the references and thus, the body of evidence. One way to tackle this important obstacle is to archive URLs permanently.

Reference accuracy in four rehabilitation journals

OBJECTIVE

To investigate the incidence of reference errors in major peer-reviewed general physical therapy and rehabilitation journals (American Journal of Physical Medicine and Rehabilitation (AJPMR), Archives of Physical Medicine and Rehabilitation (APMR), Clinical Rehabilitation (CR) and Physical Therapy (PT)).

DESIGN

Descriptive, comparative.

MAIN OUTCOME MEASURES

All issues of the AJPMR, APMR, CR and PT between 2003 and 2007 were studied. For each journal, references from articles were consecutively numbered, and using a random number generator, 100 references were selected from each journal. For each reference, ease of retrieval on MEDLINE and the presence of citation errors were noted. If discrepancies were identified, the reference was compared with the original publication. Two observers independently evaluated each reference for citation errors.

RESULTS

The total number of citations with errors among all published journals was 123 (30.7%). The reference error rates by journal ranged from 23% to 44%. Most errors (48.0%) occurred in the author element, followed by the title (31.7%), journal (8.9%), page (5.7%), year (4.1%), and volume (1.6%). Only 8 (2%) were likely to make retrieval of the reference difficult.

CONCLUSIONS

Errors in references still appear in current physical therapy and rehabilitation literature, but most are not severe.

Accuracy of references in burns journals

AIMS

To study the incidence and risk factors for citation and quotation errors in two major burns surgery journals.

METHODS

120 references were randomly selected from original articles published in the following two journals - January to December 2006 issues of Burns and Journal of Burn Care & Research. For each reference, the ease of retrieval on PubMed and the presence of citation errors were noted. Two independent observers analysed each reference for quotation errors. The characteristics of the root article, that is, type of study, author numbers, number of references and article word count were noted.

RESULTS

Of the 120 selected references, 117 referred to articles from indexed medical journals published in English. Among these, 4 articles could not be retrieved due to fatal citation errors (3.3%). A further 12 citation errors were noted giving a total citation error rate of 13.3% (95% CI: 6.74-19.93%). Of the 117 references analysed, the quotation error rate was 13.7% (95% CI: 8.6-19.5%) half of which were major errors. There was no significant association between the combined error rate per article and the journal (Kruskal-Wallis test; p=0.861, type of study (Kruskal-Wallis test; p=0.717), author numbers (Spearman's rho=0.197, p=0.423), article length (Spearman's rho=0.118, p=0.705) or references per article (Spearman's rho=0.229, p=0.189).

CONCLUSION

Significant numbers of citation and quotation errors still appear in current burns literature. Incorrect spelling of author names and partial omissions of article titles were the two most common errors. No observable underlying factors were identified in this study. The present results serve as a reminder to authors, editors and peer reviewers for more care of citation accuracy when striving for their common goal of scientific excellence.

Technical editing of research reports in biomedical journals

BACKGROUND

Most journals try to improve their articles by technical editing processes such as proof-reading, editing to conform to 'house styles', grammatical conventions and checking accuracy of cited references. Despite the considerable resources devoted to technical editing, we do not know whether it improves the accessibility of biomedical research findings or the utility of articles. This is an update of a Cochrane methodology review first published in 2003.

OBJECTIVES

To assess the effects of technical editing on research reports in peer-reviewed biomedical journals, and to assess the level of accuracy of references to these reports.

SEARCH STRATEGY

We searched The Cochrane Library Issue 2, 2007; MEDLINE (last searched July 2006); EMBASE (last searched June 2007) and checked relevant articles for further references. We also searched the Internet and contacted researchers and experts in the field.

SELECTION CRITERIA

Prospective or retrospective comparative studies of technical editing processes applied to original research articles in biomedical journals, as well as studies of reference accuracy.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed each study against the selection criteria and assessed the methodological quality of each study. One review author extracted the data, and the second review author repeated this.

MAIN RESULTS

We located 32 studies addressing technical editing and 66 surveys of reference accuracy. Only three of the studies were randomised controlled trials. A 'package' of largely unspecified editorial processes applied between acceptance and publication was associated with improved readability in two studies and improved reporting quality in another two studies, while another study showed mixed results after stricter editorial policies were introduced. More intensive editorial processes were associated with fewer errors in abstracts and references. Providing instructions to authors was associated with improved reporting of ethics requirements in one study and fewer errors in references in two studies, but no difference was seen in the quality of abstracts in one randomised controlled trial. Structuring generally improved the quality of abstracts, but increased their length. The reference accuracy studies showed a median citation error rate of 38% and a median quotation error rate of 20%.

AUTHORS' CONCLUSIONS

Surprisingly few studies have evaluated the effects of technical editing rigorously. However there is some evidence that the 'package' of technical editing used by biomedical journals does improve papers. A substantial number of references in biomedical articles are cited or quoted inaccurately.

Technology in nursing scholarship: use of citation reference managers

Nurses, especially those in academia, feel the pressure to publish but have a limited time to write. One of the more time-consuming and frustrating tasks of research, and subsequent publications, is the collection and organization of accurate citations of sources of information. The purpose of this article is to discuss three types of citation reference managers (personal bibliographic software) and how their use can provide consistency and accuracy in recording all the information needed for the research and writing process. The advantages and disadvantages of three software programs, EndNote, Reference Manager, and ProCite, are discussed. These three software products have a variety of options that can be used in personal data management to assist researchers in becoming published authors.

Technical editing of research reports in biomedical journals

BACKGROUND

Most journals try to improve their articles by technical editing processes such as proof-reading, editing to conform to 'house styles' and grammatical conventions. Despite the considerable resources devoted to technical editing, we do not know whether it improves the accessibility of biomedical research findings or the utility of articles.

OBJECTIVES

To assess the effects of technical editing on research reports in peer-reviewed biomedical journals.

SEARCH STRATEGY

We searched the Cochrane Library Issue 1, 2001, MEDLINE (last searched February 2000), 12 other databases, handsearched 9 journals and checked relevant articles for further references. We also searched the Internet and contacted researchers and experts in the field.

SELECTION CRITERIA

Prospective or retrospective comparative studies of technical editing processes applied to original research articles in biomedical journals.

DATA COLLECTION AND ANALYSIS

Two reviewers independently assessed each study against the selection criteria and assessed the methodological quality of each study. One reviewer extracted the data, and the second reviewer repeated this.

MAIN RESULTS

We located 18 studies addressing technical editing and 35 surveys of reference accuracy. Only two of the studies were randomized controlled trials. A 'package' of largely unspecified editorial processes applied between acceptance and publication was associated with improved readability in two studies and improved reporting quality in another two studies, while another study showed mixed results after stricter editorial policies were introduced. More intensive editorial processes were associated with fewer errors in abstracts and references. Providing instructions to authors was associated with improved reporting of ethics requirements in one study and fewer errors in references in two studies, but no difference was seen in the quality of abstracts in one randomized controlled trial. Structuring generally improved the quality of abstracts, but increased their length. The reference accuracy studies showed a median citation error rate of 39% and a median quotation error rate of 20%.

AUTHORS' CONCLUSIONS

Surprisingly few studies have evaluated the effects of technical editing rigorously. However there is some evidence that the 'package' of technical editing used by biomedical journals does improve papers.

Which articles and which topics in the forensic sciences are most highly cited?

Forensic science is a multidisciplinary field, which covers many branches of the pure, the applied and the biomedical sciences. Writing-up and publishing research findings helps to enhance the reputation of the investigators and the laboratories where the work was done. The number of times an article is cited in the reference lists of other articles is generally accepted as a mark of distinction. Indeed, citation analysis has become widely used in research assessment of individual scientists, university departments and entire nations. This article concerns the most highly cited papers published in the Journal of Forensic Sciences (JFS) between 1956 and 2005. These were identified with the help of Web-of-Science, which is the on-line version of Science Citation Index, produced by Thomson Institute for Scientific Information (Thomson ISI) with head offices in Philadelphia, USA. This database tracks, among other things, the annual citation records of articles published in several thousand scientific journals worldwide. Those JFS articles accumulating 50 or more citations were identified and rank-ordered according to the total number of citations. These articles were also evaluated according to the name of first author, the subject category of the article, the country of origin and the pattern of co-authorship. This search strategy located 46 articles cited between 50 and 292 times since they first appeared in print. The most highly cited paper by far was by Kasai, Nakamura and White (USA and Japan) concerning DNA profiling and the application of the polymerase chain reaction (PCR) in forensic science. Some forensic scientists appeared as first author on two to three highly cited articles, namely Wetli (USA), Budowle (USA) and Comey (USA). When the highly cited articles were sub-divided into subject category, 15 were identified as coming from toxicology, closely followed by criminalistics (14 articles), pathology (nine articles), physical anthropology (five articles), forensic psychiatry (two articles) and one from odontology. The number of co-authors on these highly cited articles ranged from one to nine and the names of some investigators appeared on as many as four highly cited papers. The vast majority of papers originated from US laboratories although five came from Japan, two each from Sweden and Canada and there was also a joint USA-Swiss collaboration. The Thompson ISI citation databases provide unique tools for tracking citations to individual articles and impact and citation records of scholarly journals.

The prevalence and inaccessibility of Internet references in the biomedical literature at the time of publication

OBJECTIVES

To determine the prevalence and inaccessibility of Internet references in the bibliography of biomedical publications when first released in PubMed.

METHODS

During a one-month observational study period (Feb 21 to Mar 21, 2006) the Internet citations from a 20% random sample of all forthcoming publications released in PubMed during the previous day were identified. Attempts to access the referenced Internet citations were completed within one day and inaccessible Internet citations were recorded.

RESULTS

The study included 4,699 publications from 844 different journals. Among the 141,845 references there were 840 (0.6%) Internet citations. One or more Internet references were cited in 403 (8.6%) articles. From the 840 Internet references, 11.9% were already inaccessible within two days after an article's release to the public.

CONCLUSION

The prevalence of Internet citations in journals included in PubMed is small (<1%); however, the inaccessibility rate at the time of publication is considered substantial. Authors, editors, and publishers need to take responsibility for providing accurate and accessible Internet references.

Accuracy of referencing in the ophthalmic literature

PURPOSE

To investigate the frequency of citation and quotation errors in the ophthalmic literature.

DESIGN

Analysis of 200 references from 100 papers published in 10 ophthalmic journals.

METHODS

A sample of 20 references was randomly selected from each of the 10 journals and each reference was checked for accuracy. Quotations were categorized as totally, partially, or not accurate.

RESULTS

There were 35 citation errors in 32 references, only four of which were errors in PubMed. Thirty quotations of references were not accurate; 20 were partially accurate.

CONCLUSIONS

Citation and quotation errors are relatively common within the ophthalmic literature. This may be improved through technical editing.

Comment on “Adsorption of naphthalene on zeolite from aqueous solution” by C.F. Chang, C.Y. Chang, K.H. Chen, W.T. Tsai, J.L. Shie, Y.H. Chen

A paper contributes not only by its originality and creativity, but also by its continuity and development toward subsequent research. Referencing and quotation accuracy are an important part of a scientific article. This study presents a literature review concerning the precision of 50 first authors' publications, which originally cited Ho's pseudo-second-order kinetic expression paper in kinetics model for solute sorption on various sorbents. This model applies to a range of solid-liquid systems such as metal ions, dyestuffs, herbicides, oil, and organic substances in aqueous systems onto various sorbents. In addition, citations of Lagergren and Elovich rate equations are also discussed. This comment offers information for citing the original idea of Ho's pseudo-second-order kinetic expression and Lagergren's pseudo-first-order kinetic equation. It is also suggested that the cited paper should be accurately quoted.

Citation and quotation accuracy in three anatomy journals

Citation and quotation errors are common in medical journals. We assessed the prevalence of those errors in gross anatomy journals, where articles often cite old anatomical studies. The study included 199 randomly selected references from articles published in the first 2001 issue of three major gross anatomy journals: Annals of Anatomy, Clinical Anatomy, and Surgical and Radiologic Anatomy. The selected references were checked for accuracy against the original articles. Citation errors were classified as major, intermediate, and minor. Quotation errors were classified as major and minor. Citations errors were found in 27% (54/199) of the references and 38% of them were major errors. Errors occurred in 19% (52/272) of quotations and nearly all (94%) were major. Furthermore, 24% of the quotations were indirect references to a secondary, instead of original, source. There was no statistically significant difference in the rates of citation or quotation errors between the references published before or after the introduction of MEDLINE (chi2 test, P > 0.05) in 1963, and the prevalence of these errors in gross anatomy journals was similar to that found in other medical fields. A high proportion of major citation errors, a very high proportion of major quotation errors, and the substantial number of indirect quotations call for serious editorial action in anatomy journals.

Accuracy of references in five biomedical informatics journals

OBJECTIVE

To determine the rate and type of errors in biomedical informatics journal article references.

METHODS

References in articles from the first 2004 issues of five biomedical informatics journals, Journal of the American Medical Informatics Association, Journal of Biomedical Informatics, International Journal of Medical Informatics, Methods of Information in Medicine, and Artificial Intelligence in Medicine were compared with MEDLINE for journal, authors, title, year, volume, and page number accuracy. If discrepancies were identified, the reference was compared with the original publication. Two reviewers independently evaluated each reference.

RESULTS

The five journal issues contained 37 articles. Among the 656 eligible references, 225 (34.3%) included at least one error. Among the 225 references, 311 errors were identified. One or more errors were found in the bibliography of 31 (84%) of the 37 articles. The reference error rates by journal ranged from 22.1% to 40.7%. Most errors (39.0%) occurred in the author element, followed by the journal (31.2%), title (17.7%), page (7.4%), year (3.5%), and volume (1.3%) information.

CONCLUSION

The study identified a considerable error rate in the references of five biomedical informatics journals. Authors are responsible for the accuracy of references and should more carefully check them, possibly using informatics-based assistance.

WHO systematic review of maternal mortality and morbidity: methodological issues and challenges

Background

Reducing maternal mortality and morbidity are among the key international development goals. A prerequisite for monitoring the progress towards attainment of these goals is accurate assessment of the levels of mortality and morbidity. In order to contribute to mapping the global burden of reproductive ill-health, we are conducting a systematic review of incidence and prevalence of maternal mortality and morbidity.

Methods

We followed the standard methodology for systematic reviews. We prepared a protocol and a form for data extraction that identify key characteristics on study and reporting quality. An extensive search was conducted for the years 1997–2002 including electronic and hand searching.

Results

We screened the titles and abstracts of about 65,000 citations identified through 11 electronic databases as well as various other sources. Four thousand six hundred and twenty-six full-text reports were critically appraised and 2443 are included in the review so far. Approximately one third of the studies were conducted in Asia and Africa. The reporting quality was generally low with definitions for conditions and the diagnostic methods often not reported.

Conclusions

There are unique challenges and issues regarding the search, critical appraisal and summarizing epidemiological data in this systematic review of prevalence/incidence studies. More methodological studies and discussion to advance the field will be useful. Considerable efforts including leadership, consensus building and resources are required to improve the standards of monitoring burden of disease.

Referencing and quotation accuracy in four manual therapy journals

The aim of the study was to investigate the reference and quotation accuracy in four peer-reviewed manual therapy journals. A stratified random sample of original research (n=7) was collected from each of the journals spanning the years January 2000 to December 2001. A further random selection of 80 references from each journal paper sampled was then reviewed (Total N=320) for citation and quotation accuracy. Numbers of citations with errors were determined, then classified as either major or minor and categorized by bibliographic headings (author, title, journal, year, volume, page and irretrievable). Each quotation was individually assessed for accuracy and judged to be either correct or incorrect. A quotation was deemed correct if it accurately substantiated and reported the original authors assertions. One hundred and fifteen citations across all journals contained errors (35.9%). Some citations exhibited multiple major and minor errors. Bibliographically classified errors for all journals showed 61 author, 51 title, 6 journal, 4 year, 12 volume and 25 page errors. JMPT showed the lowest referencing error rate (20%) while JBWMT recorded the highest (58.8%). The total number of quotation errors across all journals was 69 (12.3%). JMPT showed the lowest quotation error rate of 6 (4.7%), MT had 12 errors (7.3%), JOM produced 21 errors (13.3%), while JBWMT recorded the highest error rate with 32 (27.6%). Poor citation and quotation is a reflection on the scholarly work of the authors and the journal. The trend for errors in quotation is more worrying than citation errors as it reflects poor diligence on the part of the investigators.

Caffeine as a promoter of analgesic-associated nephropathy--where is the evidence?

Individual groups of nephrologists - in their responsibility for their patients - initiated a most controversial discussion whether or not caffeine - coformulated to analgesics - might initiate or sustain analgesic overdosing. The original sources (data) of such suspicion have got lost during the debate of the last two decades. Therefore, it seemed to be appropriate to investigate the original data background and the reasons why nephrologists started to suspect caffeine as a stimulant of analgesic overdosing by employing a systematic and exhaustive review of primary nephrological publications. Their selection followed a precise selection plan, including all epidemiological studies on analgesic-associated nephropathy, the original papers of all groups having been involved in those studies, further originals from the mainly involved countries (academically, politically), and any literature thereof cited as a proof. The following results emerged from the investigation: (i) The epidemiological studies warranted no conclusion about a role of caffeine in prompting excessive analgesic use. (ii) The identified groups of nephrologists provided not substantial data to advocate the said suspicion, except for the observation of a preferential choice of phenacetin-containing combinations, especially powder preparations. (iii) Only two cited original data sources revealed drug-seeking behaviour with phenacetin-containing preparations which subsided, after phenacetin was banned from the respective markets. Conclusively, it appears that there is no substantial data to support a pivotal role of caffeine in initiating or sustaining analgesic overdosing. However, there is strong data that phenacetin, by its psychotropic properties, may have caused drug-seeking behaviour and thus led to analgesic overdosing. This conclusion is convincingly supported by thorough pharmacokinetic investigations. Note: All caffeine-related statements within the reviewed literature have been collected in tables (referred to as Table SX) which are provided in full text for check on the following website: http://www.blackwellpublishing.com/products/journals/suppmat/FCP/FCP174/FCP174sm.htm

Impact factors of forensic science and toxicology journals: what do the numbers really mean?

This article presents review and opinion about the use and abuse of journal impact factors for judging the importance and prestige of scientific journals in the field of forensic science and toxicology. The application of impact factors for evaluating the published work of individual scientists is also discussed. The impact factor of a particular journal is calculated by dividing the number of current year citations to a journal's articles that were published in the previous 2 years by the total number of citable items (articles and reviews) published in the same 2-year period. Journal impact factors differ from discipline to discipline and range from 0 for a journal whose articles are not cited in the previous 2 years to 46 for a journal where the average recent article is cited 46 times per year. The impact factor reflects the citation rate of the average article in a journal and not a specific article. Many parameters influence the citation rate of a particular journal's articles and, therefore, its impact factor. These include the visibility and size of the circulation of the journal including availability of electronic formats and options for on-line search and retrieval. Other things to consider are editorial standards especially rapid and effective peer-reviewing and a short time lag between acceptance and appearance in print. The number of self-citations and citation density (the ratio of references to articles) and also the inclusion of many review articles containing hundreds of references to recently published articles will boost the impact factor. Judging the importance of a scientist's work based on the average or median impact factor of the journals used to publish articles is not recommended. Instead an article-by-article citation count should be done, but this involves much more time and effort. Moreover, some weighting factor is necessary to allow for the number of co-authors on each article and the relative positioning of the individual names should also be considered. Authors should submit their research results and manuscripts to journals that are easily available and are read by their peers (the most interested audience) and pay less attention to journal impact factors. To assess the true usefulness of a person's contributions to forensic science and toxicology one needs to look beyond impact factor and citation counts. For example, one might consider whether the articles contained new ideas or innovations that proved useful in routine forensic casework or are widely relied upon in courts of law as proof source.

A review of the accuracy of references in the journal Emergency Medicine

OBJECTIVE

To assess the accuracy of references in articles published in Emergency Medicine, and to categorize these errors.

METHODS

All the references in Volume 12 of Emergency Medicine were listed and numbered consecutively. A sample of 100 references was then selected. Each reference was then checked, initially on an electronic database, with the original article being used as the gold standard.

RESULTS

1469 citations were included in the study. A random sample of 100 was taken and examined in detail. 35 papers were shown to have at least one error, and a total of 41 errors were found. The maximum number of errors in one paper was 3. These errors are analysed in detail.

CONCLUSION

Poor reference accuracy is a common problem in medical literature, and Emergency Medicine is not immune to these failings. Authors need to take more care to ensure that the accuracy of citations improves.

The appropriate use of references in a scientific research paper

References have an important and varied role in any scientific paper. Unfortunately, many authors do not appreciate this importance and errors within reference lists are frequently encountered. Most reference errors involve spelling, numerical and punctuation mistakes, although the use of too many, too few or even inappropriate references is often seen. The consequences of reference errors include difficulty in reference retrieval, limitation for the reader to read more widely, failure to credit the cited authors, and inaccuracies in citation indexes. This paper discusses the value of accurate reference lists and provides guidelines for their preparation.

The accuracy of references in paediatric journals

We analysed the reference error rate of four paediatric journals. The overall rate was 29.7%. Individual rates were as follows: Acta Paediatr 36%, Arch Dis Child 22%, J Pediatr 29%, Pediatrics 32%; the rate of major errors was 1%, 1%, 2%, and 4%, respectively.

Writing manuscripts describing clinical trials: a guide for pharmacotherapeutic researchers

Once researchers have completed a clinical trial and analyzed the data, they have a duty to make the results known to their peers by writing a manuscript suitable for publication in a biomedical journal. Before beginning any writing, those involved in the clinical drug trial first need to decide among themselves who is to be the first author and secondary authors, the readership they want to influence, and the most appropriate target journal to reach this readership. For content and formatting of the manuscript, writers need to carefully follow the instructions for authors as stipulated by the target journal. In general, a manuscript based on a clinical trial is presented sequentially in four sections: Introduction, Methods, Results, and Discussion (IMRAD). The introduction concisely presents background information about the topic that was researched in the clinical trial, provides the study hypothesis, and describes the study rationale, purpose, and objectives. The Methods section delineates the study design, subject/patient selection process, procedures, end points, and types of analysis conducted in the clinical trial to investigate the study hypothesis. The Results section presents the findings of the clinical trial, and the Discussion section interprets the meaning of these findings, assesses how the findings answer the study hypothesis, and compares the study data with findings reported in earlier studies. Writers should select a title for the clinical trial manuscript that is fewer than 10 words and which embodies the essence of the study findings clearly and specifically without being sensationalistic. Writers should create the abstract last; the abstract should provide an overview of the manuscript and stimulate readers' interest. Overall, following the above principles in writing a manuscript based on a clinical drug trial will make the actual writing easier and more enjoyable, as well as increase the possibility of the manuscript being accepted for publication.