Content-based citation analysis: The next generation of citation analysis

A Bibliometric Analysis of Publications on Cadavers: A Study Based on Web of Science Data From 1978 to 2023

This study aimed to investigate the significance of publications examining the effectiveness of cadaver studies in the field of medicine with the method of bibliometric analysis, which emerged in the 1950s, offers the opportunity to conduct a detailed analysis of a specific subject, just like a systematic literature review or meta-analysis. Also, it aimed to enlighten the content of cadaver studies in the last half-century and to present a perspective for the future. In the study, an advanced search was conducted on the Web of Science Core Collection database on August 1, 2023, using the keywords "cadaver," "cadaver study," "cadaveric dissection." Review articles were excluded from the study. There were determined 34554 documents. The documents were transferred to the VOSviewer software program. In this way, the authors made detailed analyses of authors, keywords, journals, organizations, institutions, and countries and created scientific maps. The United States was one of the most important countries in terms of research. Generally, the use of cadaver terms in documents belongs to surgery, anatomy, and transplantation journals. Anatomy, cadaver, biomechanics, ultrasound, and computed tomography were the top 5 most frequently used keywords with cadaver term. The findings of our study showed that cadaver studies are used in many fields of medical science. However, clinical studies, including advanced imaging techniques, draw attention to the developing technology. Along with the powerful institutions of the United States, its great contributions to publications stand out.

Data, measurement and empirical methods in the science of science

The advent of large-scale datasets that trace the workings of science has encouraged researchers from many different disciplinary backgrounds to turn scientific methods into science itself, cultivating a rapidly expanding 'science of science'. This Review considers this growing, multidisciplinary literature through the lens of data, measurement and empirical methods. We discuss the purposes, strengths and limitations of major empirical approaches, seeking to increase understanding of the field's diverse methodologies and expand researchers' toolkits. Overall, new empirical developments provide enormous capacity to test traditional beliefs and conceptual frameworks about science, discover factors associated with scientific productivity, predict scientific outcomes and design policies that facilitate scientific progress.

OpCitance: Citation contexts identified from the PubMed Central open access articles

OpCitance contains all the sentences from 2 million PubMed Central open-access (PMCOA) articles, with 137 million inline citations annotated (i.e., the "citation contexts"). Parsing out the references and citation contexts from the PMCOA XML files was non-trivial due to the diversity of referencing style. Only 0.5% citation contexts remain unidentified due to technical or human issues, e.g., references unmentioned by the authors in the text or improper XML nesting, which is more common among older articles (pre-2000). PubMed IDs (PMIDs) linked to inline citations in the XML files compared to citations harvested using the NCBI E-Utilities differed for 70.96% of the articles. Using an in-house citation matcher, called Patci, 6.84% of the referenced PMIDs were supplemented and corrected. OpCitance includes fewer total number of articles than the Semantic Scholar Open Research Corpus, but OpCitance has 160 thousand unique articles, a higher inline citation identification rate, and a more accurate reference mapping to PMIDs. We hope that OpCitance will facilitate citation context studies in particular and benefit text-mining research more broadly.

The effect of co-opinion on the cocitation-based information retrieval systems’ effectiveness evaluated by semantic similarity

The co-opinionatedness measure, that is, the similarity of cociting documents in their opinions about their cocited articles, has been recently proposed. The present study uses a wider range of baselines and benchmarks to investigate the measure’s effectiveness in retrieval ranking that was previously confirmed in a pilot study. A test collection was built including 30 seed documents and their 4702 cocited articles. Their citances and full-texts were analysed using natural language processing (NLP) and opinion mining techniques. Cocitation values, syntactical similarity and contexts similarity were used as baselines. The distributional semantic similarity and the linear and hierarchical Medical Subject Headings (MeSH) similarities served as benchmarks to evaluate the effect of the co-opinionatedness as a boosting factor on the performance of the baselines. The improvements in the rankings were measured by normalised discounted cumulative gain (nDCG). According to the findings, there existed significant differences between the nDCG mean values obtained before and after weighting the baselines by the co-opinionatedness measures. The results of the generalisability study corroborated the reliability and generalisability of the systems. Accordingly, the similarity in the opinions of the cociting papers towards their cocited articles can explain the cocitation relation in the scientific papers network and can be effectively utilised for improving the results of the cocitation-based retrieval systems.

Virtual Laboratories- A historical review and bibliometric analysis of the past three decades

Online and virtual teaching-learning has been a panacea that most educational institutions adopted from the dire need created by COVID-19. We provide a comprehensive bibliometric study of 9523 publications on virtual laboratories in higher education covering the years 1991 to 2021. Influential bibliometrics such as publications and citations, productive countries, contributing institutions, funders, journals, authors, and bibliographic couplings were studied using the Scientific Procedures and Rationales for Systematic Literature Reviews (SPAR-4-SLR) protocol. A new metric to complement citations called Field Weighted Citation Impact was introduced that considers the differences in research behavior across disciplines. Findings show that 72% of the research work was published between 2011-and 2021, most likely due to digitalization, with the highest number of publications in 2020-2021 highlighting the impact of the pandemic. Top contributing institutions were from the developed economies of Spain, Germany, and the United States. The citation impact from publications with international co-authors is the highest, highlighting the importance of co-authoring papers with different countries. For the first time, Altmetrics in the context of virtual labs were studied though a very low correlation was observed between citations and Altmetrics Attention Score. Still, the overall percentage of publications with attention showed linear growth. Our work also highlights that virtual laboratory could play a significant role in achieving the United Nations Sustainable Development Goals, specifically SDG4-Quality Education, which largely remains under-addressed.

See further upon the giants: Quantifying intellectual lineage in science

Newton’s centuries-old wisdom of standing on the shoulders of giants raises a crucial yet underexplored question: Out of all the prior works cited by a discovery, which one is its giant? Here, we develop a novel, discipline-independent method to identify the giant for any individual paper, allowing us to systematically examine the role and characteristics of giants in science. We find that across disciplines, about 95% of papers stand on the shoulders of giants, yet the weight of scientific progress rests on relatively few shoulders. Defining a new measure of giant index, we find that, while papers with high citations are more likely to be giants, for papers with the same citations, their giant index sharply predicts a paper’s future impact and prize-winning probabilities. Giants tend to originate from both small and large teams, being either highly disruptive or highly developmental. And papers that did not have a giant but later became a giant tend to be home-run papers that are highly disruptive to science. Given the crucial importance of citation-based measures in science, the developed concept of giants may offer a useful new dimension in assessing scientific impact that goes beyond sheer citation counts.

Peer Review

https://publons.com/publon/10.1162/qss_a_00186

The Difference in Open Innovation between Open Access and Closed Access, According to the Change of Collective Intelligence and Knowledge Amount

This study explored the differences in the effects of collective intelligence and references on open innovation between open and closed access journals. This study analyzed the moderating effect of references on the motivation of collective intelligence on open innovation from 2003 to 2006 and 2013 to 2016, considered to be the digital transformation era. The Scopus database on open and closed access journals was used for ordinary regression analysis. During the 2003–2006 period, only papers in closed access journals demonstrated sufficient effect of collective intelligence and reference on open innovation and the effective moderating role of reference. However, between 2013 and 2016, papers in open and closed access journals demonstrated the incentive effects of collective intelligence and references on citation and the moderating role of references on the correlation between collective intelligence and citation. The increase in digital transformation strengthens the collective intelligence and references of open access journals, and citations of open access journals nearly surpass those of closed access journals.

Tracking changes between preprint posting and journal publication during a pandemic

Amid the Coronavirus Disease 2019 (COVID-19) pandemic, preprints in the biomedical sciences are being posted and accessed at unprecedented rates, drawing widespread attention from the general public, press, and policymakers for the first time. This phenomenon has sharpened long-standing questions about the reliability of information shared prior to journal peer review. Does the information shared in preprints typically withstand the scrutiny of peer review, or are conclusions likely to change in the version of record? We assessed preprints from bioRxiv and medRxiv that had been posted and subsequently published in a journal through April 30, 2020, representing the initial phase of the pandemic response. We utilised a combination of automatic and manual annotations to quantify how an article changed between the preprinted and published version. We found that the total number of figure panels and tables changed little between preprint and published articles. Moreover, the conclusions of 7.2% of non-COVID-19-related and 17.2% of COVID-19-related abstracts undergo a discrete change by the time of publication, but the majority of these changes do not qualitatively change the conclusions of the paper.

Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine

We present the first database-wide study on the citation contexts of retracted papers, which covers 7,813 retracted papers indexed in PubMed, 169,434 citations collected from iCite, and 48,134 citation contexts identified from the XML version of the PubMed Central Open Access Subset. Compared with previous citation studies that focused on comparing citation counts using two time frames (i.e., preretraction and postretraction), our analyses show the longitudinal trends of citations to retracted papers in the past 60 years (1960-2020). Our temporal analyses show that retracted papers continued to be cited, but that old retracted papers stopped being cited as time progressed. Analysis of the text progression of pre- and postretraction citation contexts shows that retraction did not change the way the retracted papers were cited. Furthermore, among the 13,252 postretraction citation contexts, only 722 (5.4%) citation contexts acknowledged the retraction. In these 722 citation contexts, the retracted papers were most commonly cited as related work or as an example of problematic science. Our findings deepen the understanding of why retraction does not stop citation and demonstrate that the vast majority of postretraction citations in biomedicine do not document the retraction.

Suresh Kumar, Ch. Vidyadhari, R. Ganeshan. Infgraph: Influential Researcher and Cited Research Analysis Using Citation Network

The understanding of references in research articles is essential for performing effectual research. This paper devises a hybrid model to find the influential cited paper and influential researchers from Web of Science (WOS) data. For determining the influential researcher, a series of steps is performed. Then the co-citation is performed for providing author-author co-relation that predicts the next co-author. Thereafter, visualization of the network is performed for research communication amongst different authors. Then, the network density is computed. Finally, the cluster coefficient is adapted for finding the influential researcher. Concurrently, for discovering influential cited papers, the pre-processing is performed using the stop word removal and stemming process. Then, the word2vec model is utilized for training the model to forecast the suitable word that comes next. Finally, the modified word mover's distance (MWMD) is utilized for determining the semantic similarity in order to discover influential cited papers.

The explanatory power of citations: a new approach to unpacking impact in science

Citation analysis has been applied to map the landscape of scientific disciplines and to assess the impact of publications. However, it is limited in that it assumes all citations to be of equal weight. Doing away with this assumption could make such studies even more insightful. Current developments in this regard focus on the evaluation of the syntactic and semantic qualities of the text that surrounds citations. Still lacking, however, are computational techniques to unpack the thematic context in which citations appear. It is against this backdrop that we propose a text clustering approach to derive contextual aspects of individual citations and the relationship between cited and citing work in an automated and scalable fashion. The method reveals a focal publication’s absorption and use within the scientific community. It can also facilitate impact assessments at all levels. In addition to analyzing individual publications, the method can also be extended to creating impact profiles for authors, institutions, disciplines, and regions. We illustrate our results based on a large corpus of full-text articles from the field of Information systems (IS) with the help of exemplary visualizations. In addition, we provide a case study, the scientific impact of the Technology acceptance model. This way, we not only show the usefulness of our method in comparison to existing techniques but also enhance the understanding of the field by providing an in-depth analysis of the absorption of a key IS theoretical base.

Rotten web citations cited in scholarly journals: use of time travel for retrieval

Purpose

The present study compares the use of web citations as references in leading scholarly journals in Library and Information Science (LIS) and Communication and Media Studies (CMS). A total of 20 journals (each 10 from LIS and CMS) were selected based on the publishing history and reputation published between 2008 and 2017.

Design/methodology/approach

The present study compares the use of web citations as references in leading scholarly journals in LIS and CMS. A PHP script was used to crawl the Uniform Resource Locators (URLs) collected from the reference list. A total of 12,251 articles were downloaded and 555,428 references were extracted. Of the 555,428 references, 102,718 web citations were checked for their accessibility.

Findings

The research findings indicated that 76.90% URLs from LIS journals and 84.32% URLs from Communication and Media Studies journals were accessible and others were rotten. The majority of errors were due to HTTP 404 error code (not found) in both the disciplines. The study also tried to retrieve the rotten URLs through Time Travel, which revived 61.76% rotten URLs in LIS journal articles and 65.46% in CMS journal articles.

Originality/value

This is an in-depth and comprehensive comparative study on the availability of web citations in LIS and CMS journals articles spanning a period of 10 years. The findings of the study will be helpful to authors, publishers, and editorial staff to ensure that web citations will be accessible in the future.

Construction over operation? A study of the usage of digital humanities databases in China

Purpose

Digital humanities database is one of the essential tools in digital humanities research area. Therefore, examining the usage of digital humanities database in academic papers is conducive to assessing the value of digital humanities database for scientific research activities and improving the construction of digital humanities infrastructure.

Design/methodology/approach

This paper constructs an evaluation system of digital humanities database from the perspective of academic influence and social influence, with mention frequency, usage motivation, platform access data, usage region and usage discipline as indicators and takes China Biographical Database Project as the empirical object to explore the usage of digital humanities database in China.

Findings

The data analysis result demonstrates that digital humanities databases are widely used and recognized in China. However, the problem of low actual usage remains.

Originality/value

This paper constructs the digital humanistic database's evaluation system and discusses applying the digital humanistic database in China, which provides a new perspective and method for the influence evaluation study of the digital humanistic database.

Determinants of societal and academic recognition: Evidence from randomised controlled trials

Given the increasing importance of recognition in academia and the vital role of randomised controlled trials (RCTs) in medical research and clinical decisions, this study verifies how RCTs’ academic and societal impacts are affected by visibility factors, subjects and methodological validity. This study concentrated on a sample of 446 RCTs indexed in Scopus and evaluated by Cochrane reviewers in terms of their methodological validity. The altmetrics, bibliometric and bibliographical information were extracted from Altmetric.com and Scopus, and the contributing countries’ development ranks were obtained from the United Nations Development report. The linear regression analyses revealed that citations and altmetrics depend on some subjects. They are also affected by publication year and journals’ previous reputation. Citations are also affected by keyword counts and reference counts. Keyword counts and contributing countries’ developmental rank also predict the tweet counts. While none of the methodological validity dimensions were found to predict citations, ‘Incomplete Outcome Data’ and ‘Random Sequence Generation’ significantly, though slightly, affect Mendeley Readership and tweets, respectively. By confirming the dependence of RCTs’ recognition on some methodological validity features and attention-inducing characteristics, the study provides further evidence on the interaction of quality and visibility dynamisms in the recognition network and the complementary role of societal mentions for academic citation.

Trajectories of influential conceptual articles in service research

Purpose

The purpose of this study is to diagnose the trajectory of influential conceptual articles in developing a research stream. The authors uncover the knowledge diffusion through influential conceptual articles and identify characteristics that make conceptual articles influential in their field.

Design/methodology/approach

This study draws on scientometrics, specifically an integrated approach combining quantitative citation counts with qualitative citation practices analysis that offers a comprehensive understanding of the nature and context of citations. The authors use the case of customer engagement – a prominent contemporary marketing and service research stream – to explore the trajectory of influential articles in shaping a new research stream.

Findings

This research shows that influential articles contribute to the reciprocal knowledge diffusion within and outside their home discipline. They provide anchor points for conceptual framing, conceptual refining and conceptual reconciliation – three application patterns of citations that are pivotal to navigate theory discovery and theory justification in a research field.

Research limitations/implications

The study analyzes the early impact period of two influential customer engagement articles to understand the developments leading to the establishment of a new research stream. Future research drawing on automated citation and bibliometric methods may consider extended time periods.

Originality/value

This study traces the trajectory of influential articles in marketing and service research. The authors identify characteristics of influential conceptual articles, and recommend practices to develop a conceptual paper with the potential for an influential trajectory. It shows that while marketing and service research has a tradition of “borrowing” theories from other fields, seminal articles “lend” theories to other fields.

Modeling the co-citation dependence on semantic layers of co-cited documents

Purpose

Co-citation frequency, defined as the number of documents co-citing two articles, is considered as a quantitative, and thus, an efficient proxy of subject relatedness or prestige of the co-cited articles. Despite its quantitative nature, it is found effective in retrieving and evaluating documents, signifying its linkage with the related documents' contents. To better understand the dynamism of the citation network, the present study aims to investigate various content features giving rise to the measure.

Design/methodology/approach

The present study examined the interaction of different co-citation features in explaining the co-citation frequency. The features include the co-cited works' similarities in their full-texts, Medical Subject Headings (MeSH) terms, co-citation proximity, opinions and co-citances. A test collection is built using the CITREC dataset. The data were analyzed using natural language processing (NLP) and opinion mining techniques. A linear model was developed to regress the objective and subjective content-based co-citation measures against the natural log of the co-citation frequency.

Findings

The dimensions of co-citation similarity, either subjective or objective, play significant roles in predicting co-citation frequency. The model can predict about half of the co-citation variance. The interaction of co-opinionatedness and non-co-opinionatedness is the strongest factor in the model.

Originality/value

It is the first study in revealing that both the objective and subjective similarities could significantly predict the co-citation frequency. The findings re-confirm the citation analysis assumption claiming the connection between the cognitive layers of cited documents and citation measures in general and the co-citation frequency in particular.

Peer review

The peer review history for this article is available at https://publons.com/publon/10.1108/OIR-04-2020-0126.

An approach for bibliographic citation sentiment analysis using deep learning

Sentiment analysis of scientific citations is a novel and remarkable research area. Most of the work on opinion or sentiment analysis has been suggested on social platforms such as Blogs, Twitter, and Facebook. Nevertheless, when it comes to recognizing sentiments from scientific citation papers, investigators used to face difficulties due to the implied and unseen natures of sentiments or opinions. As the citation references are reflected implicitly positive in opinion, famous ranking and indexing prototypes frequently disregard the sentiment existence while citing. Hence, in the proposed framework the paper emphasizes the issue of classifying positive and negative polarity of reference sentiments in scientific research papers. First, the paper scraps the PDF articles from arxiv.org under the computer science group consisting of articles that are comprised of ‘autism’ in their title, then the paper extracted cited references and assigns polarity scores to each cited reference. The paper uses a supervised classifier with a combination of significant feature sets and compared the performance of the models. Experimental results show that a combined CNN-LSTM deep neural network model results in 85% of accuracy while traditional models result in less accuracy.

The evolutionary pattern of language in scientific writings: A case study of Philosophical Transactions of Royal Society (1665–1869)

Scientific writings, as one essential part of human culture, have evolved over centuries into their current form. Knowing how scientific writings evolved is particularly helpful in understanding how trends in scientific culture developed. It also allows us to better understand how scientific culture was interwoven with human culture generally. The availability of massive digitized texts and the progress in computational technologies today provide us with a convenient and credible way to discern the evolutionary patterns in scientific writings by examining the diachronic linguistic changes. The linguistic changes in scientific writings reflect the genre shifts that took place with historical changes in science and scientific writings. This study investigates a general evolutionary linguistic pattern in scientific writings. It does so by merging two credible computational methods: relative entropy; word-embedding concreteness and imageability. It thus creates a novel quantitative methodology and applies this to the examination of diachronic changes in the Philosophical Transactions of Royal Society (PTRS, 1665–1869). The data from two computational approaches can be well mapped to support the argument that this journal followed the evolutionary trend of increasing professionalization and specialization. But it also shows that language use in this journal was greatly influenced by historical events and other socio-cultural factors. This study, as a “culturomic” approach, demonstrates that the linguistic evolutionary patterns in scientific discourse have been interrupted by external factors even though this scientific discourse would likely have cumulatively developed into a professional and specialized genre. The approaches proposed by this study can make a great contribution to full-text analysis in scientometrics.

Citation Context Analysis as a Method for Conducting Rigorous and Impactful Literature Reviews

Citation context analysis is a detailed and rigorous form of literature review that goes beyond traditional narrative and systematic reviews to better understand the impact of seminal works and influential authors. We discuss the types of questions citation context analyses can answer and provide a set of guidelines for how to effectively conduct them. Citation context analysis holds promise for enabling a more systematic assessment of how theories are used, empirically tested, and critiqued by subsequent citing authors. This has implications for both theory development and testing, and for the improvement of citation practices within the field of organizational studies and the social and physical sciences more broadly.

A detailed open access model of the PubMed literature

Portfolio analysis is a fundamental practice of organizational leadership and is a necessary precursor of strategic planning. Successful application requires a highly detailed model of research options. We have constructed a model, the first of its kind, that accurately characterizes these options for the biomedical literature. The model comprises over 18 million PubMed documents from 1996-2019. Document relatedness was measured using a hybrid citation analysis + text similarity approach. The resulting 606.6 million document-to-document links were used to create 28,743 document clusters and an associated visual map. Clusters are characterized using metadata (e.g., phrases, MeSH) and over 20 indicators (e.g., funding, patent activity). The map and cluster-level data are embedded in Tableau to provide an interactive model enabling in-depth exploration of a research portfolio. Two example usage cases are provided, one to identify specific research opportunities related to coronavirus, and the second to identify research strengths of a large cohort of African American and Native American researchers at the University of Michigan Medical School.

Citation recommendation: approaches and datasets

Citation recommendation describes the task of recommending citations for a given text. Due to the overload of published scientific works in recent years on the one hand, and the need to cite the most appropriate publications when writing scientific texts on the other hand, citation recommendation has emerged as an important research topic. In recent years, several approaches and evaluation data sets have been presented. However, to the best of our knowledge, no literature survey has been conducted explicitly on citation recommendation. In this article, we give a thorough introduction to automatic citation recommendation research. We then present an overview of the approaches and data sets for citation recommendation and identify differences and commonalities using various dimensions. Last but not least, we shed light on the evaluation methods and outline general challenges in the evaluation and how to meet them. We restrict ourselves to citation recommendation for scientific publications, as this document type has been studied the most in this area. However, many of the observations and discussions included in this survey are also applicable to other types of text, such as news articles and encyclopedic articles.

Understanding Drug Repurposing From the Perspective of Biomedical Entities and Their Evolution: Bibliographic Research Using Aspirin

BACKGROUND

Drug development is still a costly and time-consuming process with a low rate of success. Drug repurposing (DR) has attracted significant attention because of its significant advantages over traditional approaches in terms of development time, cost, and safety. Entitymetrics, defined as bibliometric indicators based on biomedical entities (eg, diseases, drugs, and genes) studied in the biomedical literature, make it possible for researchers to measure knowledge evolution and the transfer of drug research.

OBJECTIVE

The purpose of this study was to understand DR from the perspective of biomedical entities (diseases, drugs, and genes) and their evolution.

METHODS

In the work reported in this paper, we extended the bibliometric indicators of biomedical entities mentioned in PubMed to detect potential patterns of biomedical entities in various phases of drug research and investigate the factors driving DR. We used aspirin (acetylsalicylic acid) as the subject of the study since it can be repurposed for many applications. We propose 4 easy, transparent measures based on entitymetrics to investigate DR for aspirin: Popularity Index (P₁), Promising Index (P₂), Prestige Index (P₃), and Collaboration Index (CI).

RESULTS

We found that the maxima of P₁, P₃, and CI are closely associated with the different repurposing phases of aspirin. These metrics enabled us to observe the way in which biomedical entities interacted with the drug during the various phases of DR and to analyze the potential driving factors for DR at the entity level. P₁ and CI were indicative of the dynamic trends of a specific biomedical entity over a long time period, while P₂ was more sensitive to immediate changes. P₃ reflected the early signs of the practical value of biomedical entities and could be valuable for tracking the research frontiers of a drug.

CONCLUSIONS

In-depth studies of side effects and mechanisms, fierce market competition, and advanced life science technologies are driving factors for DR. This study showcases the way in which researchers can examine the evolution of DR using entitymetrics, an approach that can be valuable for enhancing decision making in the field of drug discovery and development.

Examining influential factors for acknowledgements classification using supervised learning

Acknowledgements have been examined as important elements in measuring the contributions to and intellectual debts of a scientific publication. Unlike previous studies that were limited in the scope of analysis and manual examination. The present study aimed to conduct the automatic classification of acknowledgements on a large scale of data. To this end, we first created a training dataset for acknowledgements classification by sampling the acknowledgements sections from the entire PubMed Central database. Second, we adopted various supervised learning algorithms to examine which algorithm performed best in what condition. In addition, we observed the factors affecting classification performance. We investigated the effects of the following three main aspects: classification algorithms, categories, and text representations. The CNN+Doc2Vec algorithm achieved the highest performance of 93.58% accuracy in the original dataset and 87.93% in the converted dataset. The experimental results indicated that the characteristics of categories and sentence patterns influenced the performance of classification. Most of the classifiers performed better on the categories of financial, peer interactive communication, and technical support compared to other classes.

Examining similarities and differences of citation patterns between monographs and papers: a case in biology and computer science

Purpose

Citation content in academic papers and academic monographs promotes the knowledge flow among different publications. However, existing citation content analysis (CCA) focuses on academic papers and monographs have not received much research attention. We want to know if monographs are appropriate objects of CCA and whether existing methods of analyzing citation in papers are suitable for citation in monographs. Therefore, this paper aims to learn more about features of cited references and citation content in monographs and compare the characteristic of citation pattern between monographs and papers.

Design/methodology/approach

The authors manually annotate the references and syntactic citation content in academic monographs published by Morgan & Claypool and automatically extracted the references and citation content from academic papers published by Public Library of Science. Five features in two types citation pattern, namely, pattern of cited reference (including year, source and mention frequency of reference) and pattern of citation content (including location, length of citation content) are used to examine similarities and differences between monographs and papers.

Findings

The results indicate that between monographs and papers, differences are shown in location, length of citation content and year, source of reference, whereas frequency of mention of reference is similar.

Originality/value

Previous studies have explored the patter of citation content in academic papers. However, none of the existing literature, as far as the authors know, has considered the citation content in academic monographs and the similarities or differences among academic documents when studying the citation pattern.

Loops in publication citation networks

Traditionally, publication citation networks are regarded as acyclic, that is, no loops in the network as an earlier published article cannot cite a later published article. However, due to the accessibility of pre-print versions of articles, there might be some loops in a publication citation network. This article presents a descriptive statistic on loops in publication citation networks of computer science and physics by employing a network-based indicator, namely, strongly connected component (SCC). By employing computer science and physics disciplines publications from the Web of Science database as examples, this article examines the count of loops, how the count changes over time and how the count relates to the published year difference between publications within the loop in the citation network. Some common structural patterns are also extracted and analysed; we observe that the two disciplines share the most frequent patterns though there exist some minor differences. Moreover, we find that self-citations in terms of authors, authors’ institutions and journals contribute to the formation of loops in publication citation networks.