Web scraping technologies in an API world

Characteristics and Quality of Online Searches for Direct Anterior Versus Posterior Approach for Total Hip Arthroplasty

BACKGROUND

Online resources are important for patient self-education and reflect public interest. We described commonly asked questions regarding the direct anterior versus posterior approach (DAA, PA) to total hip arthroplasty (THA) and the quality of associated websites.

METHODS

We extracted the top 200 questions and websites in Google's "People Also Ask" section for 8 queries on January 8, 2023, and grouped websites and questions into DAA, PA, or comparison. Questions were categorized using Rothwell's classification (fact, policy, value) and THA-relevant subtopics. Websites were evaluated by information source, Journal of the American Medical Association Benchmark Criteria (credibility), DISCERN survey (information quality), and readability.

RESULTS

We included 429 question/website combinations (questions: 52.2% DAA, 21.2% PA, 26.6% comparison; websites: 39.0% DAA, 11.0% PA, 9.6% comparison). Per Rothwell's classification, 56.2% of questions were fact, 31.7% value, 10.0% policy, and 2.1% unrelated. The THA-specific question subtopics differed between DAA and PA (P < .001), specifically for recovery timeline (DAA 20.5%, PA 37.4%), indications/management (DAA 13.4%, PA 1.1%), and technical details (DAA 13.8%, PA 5.5%). Information sources differed between DAA (61.7% medical practice/surgeon) and PA websites (44.7% government; P < .001). The median Journal of the American Medical Association Benchmark score was 1 (limited credibility, interquartile range 1 to 2), with the lowest scores for DAA websites (P < .001). The median DISCERN score was 55 ("good" quality, interquartile range 43 to 65), with the highest scores for comparison websites (P < .001). Median Flesch-Kincaid Grade Level scores were 12th grade level for both DAA and PA (P = .94).

CONCLUSIONS

Patients' informational interests can guide counseling. Internet searches that explicitly compare THA approaches yielded websites that provide higher-quality information. Providers may also advise patients that physician websites and websites only describing the DAA may have less balanced perspectives, and limited information regarding surgical approaches is available from social media resources.

Attention and sentiment of Chinese public toward rural landscape based on Sina Weibo

Rural landscapes, as products of the interaction between humans and nature, not only reflect the history and culture of rural areas but also symbolize economic and social progress. This study proposes a deep learning-based model for Weibo data analysis aimed at exploring the development direction of rural landscapes from the perspective of the Chinese public. The research reveals that the Chinese public's attention to rural landscapes has significantly increased with the evolution of government governance concepts. Most people express a high level of satisfaction and happiness with the existing rural landscapes, while a minority harbor negative emotions towards unreasonable new rural construction. Through the analysis of public opinion regarding rural landscapes, this study will assist decision-makers in understanding the mechanisms of public discourse on social media. It will also aid relevant scholars and designers in providing targeted solutions, which hold significant importance for policy formulation and the exploration of specific development patterns.

Adulteration and substitution of drugs purchased in Australia from cryptomarkets: An analysis of Test4Pay

INTRODUCTION

Prohibited drugs in unregulated markets may be adulterated, resulting in increased risks for people who use drugs. This study investigated levels of drug adulteration and substitution of drugs purchased in Australia from cryptomarkets.

METHODS

Data were collected from a darknet forum called Test4Pay from 1 September 2022 to 23 August 2023. Posts were included if they reported the results of drug samples submitted by post to the Vancouver-based Get Your Drugs Tested service, which uses Fourier-transform infrared spectroscopy with immunoassay strip tests (fentanyl and benzodiazepines).

RESULTS

Of 103 samples, 65% contained only the advertised substance, 14% contained the advertised substance in combination with other psychoactive and/or potentially harmful substances and for 21%, the advertised substance was absent. Substances sold as MDMA, methamphetamine or heroin were consistently found to contain only the advertised substance, while substances sold as 2C-B, alprazolam or ketamine were the most likely to be completely substituted. Only 4 samples sold as cocaine contained solely the advertised substance, with 13 containing cocaine with adulterants like lidocaine, creatine, levamisole and boric acid (n = 19). No fentanyl contamination was detected. Novel dissociatives and novel benzodiazepines were detected, as well as a nitazene compound.

DISCUSSION AND CONCLUSIONS

Drug markets under prohibition continue to contain numerous unexpected substances, some of which can elevate risk of harm. Cryptomarkets are not immune to this problem, despite review systems, which should, in theory, make vendors more accountable for the quality of their stock. These findings demonstrate a need for expansion of local drug checking services in Australia.

Network analysis of the social media activities around the #TeleCheckAF project

Aims

TeleCheck-AF is a novel remote on-demand mobile health infrastructure around teleconsultations for patients with atrial fibrillation. Social media (SoMe) communication on Twitter contributed to the dissemination of this healthcare innovation by using the hashtag #TeleCheckAF. This study aims to analyse the SoMe network behind #TeleCheckAF and determine the key opinion leaders.

Methods and results

Publicly available data on actors and interactions around the hashtag #TeleCheckAF were collected by web scraping from the platform Twitter. With tools based on social network analysis, a social network was created, different communities were identified, and key opinion leaders were determined by their centrality in the network. The SoMe network consisted of 413 086 accounts with 636 502 ties in 22 different communities. A total of 287 accounts that diffused information and/or used the TeleCheck-AF infrastructure were analysed in depth. Those accounts involved users from >15 different countries and multidisciplinary professions. Further, 20 opinion leaders were identified, including four official accounts of societies and associated journals among key opinion leaders. Peaks in #TeleCheckAF tweets were seen after (virtual) conferences and other activities involving national and international cardiology societies. Social network analysis of the TeleCheck-AF Twitter hashtag revealed a wide, multidisciplinary potential reach for the diffusion of a healthcare innovation.

Conclusion

Official society SoMe accounts can amplify the dissemination of research findings by featuring abstract presentations during conferences and published manuscripts. This underlines the synergistic effects between traditional and SoMe-based research dissemination strategies for novel healthcare approaches, such as the TeleCheck-AF project.

[Judicialization of cannabidiol-based products in Brazil: an analysis from 2019 to 2022]

The study analyzed the lawsuits of patients who requested cannabidiol (CBD)-based products from the Brazilian Unified National Health System during the period from 2019 to 2022, describing sociodemographic, clinical, and legal characteristics. This is a cross-sectional study composed of the evaluation of the technical notes issued by the Center for Technical Support of the Judiciary (NatJus), which supports judicial decisions. The data were obtained from the e-NatJus system, of the Brazilian Ministry of Justice, using web scraping techniques. Logistic regression was used to estimate odds ratios with 95% confidence intervals. We analyzed 1,115 technical notes of the CBD plaintiffs, of which 54.7% of the male patients, with a mean age of 18.4 years, mostly from the South Region of the country (38.8%), and 49.6% sought treatment for epilepsy. Regarding the actions with favorable opinions, 28.8% had no scientific evidence, 26.5% pleaded for products without registration with the Brazilian Health Regulatory Agency, and 25.3% of those that had registration were not in compliance with the therapeutic indication. Patients from the Northeast Region had a chance of a favorable opinion increased by 3.0 times and those diagnosed with epilepsy by 2.3. The expert opinions that supported the magistrates for the judicial decisions regarding the demands of patients for cannabidiol-based products in Brazil were mostly in accordance with scientific evidence, denoting the importance of NatJus in the qualification of access to medicinal products in the country.

Monitoring the impact of climate extremes and COVID-19 on statewise sentiment alterations in water pollution complaints

The COVID-19 pandemic and associated prevention policies can directly or indirectly alter the sentiment of individuals while registering water pollution complaints, but observational evidence remains limited. Here, we conducted a sentiment analysis on over 10,000 water pollution complaints from residents in Alabama, USA (2012-2021) to better understand how and to what extent COVID-19 has altered emotion (polarity score-based) and attitude (subjectivity) of water pollution complaints. We found that the 2017 state-wise drought significantly increased the percentage of negative water pollution complaints by +35%, with no significant alternation in attitude before the COVID-19 pandemic. Since COVID-19, the percentage of negative and subjective water pollution complaints significantly decreased and increased by -30 and +20%, respectively, and these sentiment alternations were maintained by 2021. This study provides a new direction for environmental governance and management, requiring a timely response to changes in the public's emotions and attitudes during the next climate extremes and pandemics.

Were ride-hailing fares affected by the COVID-19 pandemic? Empirical analyses in Atlanta and Boston

Ride-hailing services such as Lyft, Uber, and Cabify operate through smartphone apps and are a popular and growing mobility option in cities around the world. These companies can adjust their fares in real time using dynamic algorithms to balance the needs of drivers and riders, but it is still scarcely known how prices evolve at any given time. This research analyzes ride-hailing fares before and during the COVID-19 pandemic, focusing on applications of time series forecasting and machine learning models that may be useful for transport policy purposes. The Lyft Application Programming Interface was used to collect data on Lyft ride supply in Atlanta and Boston over 2 years (2019 and 2020). The Facebook Prophet model was used for long-term prediction to analyze the trends and global evolution of Lyft fares, while the Random Forest model was used for short-term prediction of ride-hailing fares. The results indicate that ride-hailing fares are affected during the COVID-19 pandemic, with values in the year 2020 being lower than those predicted by the models. The effects of fare peaks, uncontrollable events, and the impact of COVID-19 cases are also investigated. This study comes up with crucial policy recommendations for the ride-hailing market to better understand, regulate and integrate these services.

Big Data ETL Process and Its Impact on Text Mining Analysis for Employees’ Reviews

Big data analysis is challenging in the current context for enterprises that would like to apply these capabilities in the human resource sector. This paper will show how an organization can take advantage of the current or former employees’ reviews that are provided on a constant basis on different sites, so that the management can adjust or change business decisions based on employees’ wishes, dissatisfaction or needs. Considering the previously mentioned challenge on big data analysis, this research will first provide the best practice for the collection and transformation of the data proposed for analysis. The second part of this paper presents the extraction of two datasets containing employee reviews using data scraping techniques, the analysis of data by using text mining techniques to retrieve business insights and the comparison of the results for these algorithms. Experimental results with Naïve Bayes, Logistic Regression, K-Nearest Neighbor and Support Vector Machine for employee sentiment prediction showed much better performances for Logistic Regression. Three out of the four analyzed algorithms performed better for the second, triple-size dataset. The final aim of the paper is to provide an end-to-end solution with high performance and reduced costs.

The Use of the Target Trial Approach in Perinatal Pharmacoepidemiology: A Scoping Review Protocol

Background: Pregnant and postpartum women have been historically excluded from clinical trials, with data on the safety of drugs relying on observational research. Methodological concerns regarding the timing and dosing of medications, data sources, study designs, and methods used for estimating associations are still problematic in observational studies. Answering causal questions is even more complex. Despite the increased interest in emulating target trials using observational data, little is known about this approach in perinatal pharmacoepidemiology.

Objective: This scoping review protocol aims to describe the methodology for assessing the available literature concerning emulating target trials for studying outcomes in women exposed to medications in the preconception, perinatal, or postpartum periods.

Methods and Analysis: We will follow the methods detailed in the Joanna Briggs Institute reviewer’s manual. We will adopt the six-stage framework recommended by Arksey and O'Malley and Levac and others. Web scraping techniques will be used for identifying relevant studies. Two authors will select articles based on the title and abstract, with discrepancies resolved by consensus, by a third reviewer. Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews flow diagram will be presented to reflect the search process. We will use existing statements to identify quality gaps in the current literature. Variables related to the content for perinatal pharmacoepidemiologic research will be included. The Risk Of Bias In Non-randomised Studies - of Interventions (ROBINS-I) will guide the assessment of the target trial emulation (i.e., treatment strategies compared, assignment procedures, follow-up period, outcome, and causal contrasts).

Discussion: Data regarding the safety of drugs taken, prior to and during pregnancy and while lactating are lacking and it is necessary to understand how we can answer these questions using rigorous methods in observational research. Through this scoping review, we intend to understand to what extent the target trial approach is being used in perinatal pharmacoepidemiology and provide recommendations to improve its use in this field.

Ethics and Dissemination: Secondary data from published scientific articles will be used, not requiring approval by the Research Ethics Committee with human beings. Findings will be submitted to a peer-reviewed journal.

Developing a Multicriteria Decision-Making Model Based on a Three-Layer Virtual Internet of Things Algorithm Model to Rank Players’ Value

This paper proposes a multicriteria decision-making model based on a three-layer virtual internet of things (IoT) algorithm to automatically track and evaluate professional football players’ performance over the Internet. The three layers were respectively related to (1) automated data reading, (2) the players’ comprehensive grey relational degree calculation, and (3) the players’ classification. The methodology was applied in the context of the COVID-19 pandemic to investigate the performance of the top 10 defenders (according to The Sun, an internationally renowned sports website) in the European leagues, participating in the knockout phase of the 2019–20 UEFA Champions League. The results indicate that Virgil van Dijk of Liverpool FC was the best defender, followed by Harry Maguire of Manchester United, and Sergio Ramos of Real Madrid in the second and third positions, respectively. However, this ranking contradicted that of The Sun’s, which ranked these defenders in the seventh, tenth, and eighth positions, respectively. These results can help club management, coaches, and teams negotiate price positioning and future contract renewals or player transfers.

Technical Job Recommendation System Using APIs and Web Crawling

There has been a sudden boom in the technical industry and an increase in the number of good startups. Keeping track of various appropriate job openings in top industry names has become increasingly troublesome. This leads to deadlines and hence important opportunities being missed. Through this research paper, the aim is to automate this process to eliminate this problem. To achieve this, Puppeteer and Representational State Transfer (REST) APIs for web crawling have been used. A hybrid system of Content-Based Filtering and Collaborative Filtering is implemented to recommend these jobs. The intention is to aggregate and recommend appropriate jobs to job seekers, especially in the engineering domain. The entire process of accessing numerous company websites hoping to find a relevant job opening listed on their career portals is simplified. The proposed recommendation system is tested on an array of test cases with a fully functioning user interface in the form of a web application. It has shown satisfactory results, outperforming the existing systems. It thus testifies to the agenda of quality over quantity.

Multimedia Knowledge Translation Tools for Parents About Childhood Heart Failure: Environmental Scan

BACKGROUND

Childhood heart failure is a factor in many hospital admissions each year. It can impose a steep learning curve for parents who need to learn the key information to care for their child at home. In this study, we conducted an environmental scan to identify and assess web-based knowledge translation tools about childhood heart failure for parent audiences developed within North America.

OBJECTIVE

The aim of this study is to inventory tools publicly available to parents about childhood heart failure from popular web-based venues, assess how each tool communicates health information, and explore how they were developed.

METHODS

Our search strategy included two commonly used multimedia-based platforms: two app stores (Google Play and Apple App Store) and one search engine (Advanced Google Search). Common search terms were used, and results were uploaded to Microsoft Excel for screening between 2 reviewers. The inclusion criteria for the tools were as follows: content focused on educating parents about their child's heart failure, developed in the English language, and originating within Canada and the United States. A total of 2 reviewers screened the app store and internet search results for relevant tools. Each tool was assessed using the Suitability Assessment of Materials (SAM), a validated tool that objectively assesses the suitability of how health information is communicated to a particular audience. Key informants who were involved in tool development were identified and invited for a qualitative interview using a semistructured format to provide data about the development process. Key themes were identified in the semistructured interview process.

RESULTS

Frequencies and SAM percent ratings of eligible tools were reported. No apps exist for parents relating to pediatric heart failure. Overall, 17 relevant internet tools were identified, and their suitability was assessed for the parent audience. Most tools scored well in layout and type, but they scored lower in readability and graphics. Qualitative interviews with key informants revealed three key themes: timely and introductory knowledge, credible and trustworthy knowledge, and challenges and evolution in knowledge.

CONCLUSIONS

This is the first environmental scan looking for parent tools relating to childhood heart failure in Canada and the United States. Findings from this study reveal that there are no apps on this topic and there is a small number of tools for parents on the internet (n=17). Using the SAM, no tools scored in the superior range, and further work in knowledge translation strategies needs to be done in this area to improve more effective education to parents and caregivers who have a child with heart failure. These findings will inform the development of a new resource on children's heart failure that targets parents and caregiver audiences.

What Affects an Orthopaedic Surgeon's Online Rating? A Large-Scale, Retrospective Analysis

Introduction:

In the past decade, online physician review websites have become an important source of information for patients, with the largest and most popular being Healthgrades.com. Our study aims to investigate demographic and volume-based trends for online reviews of every Healthgrades-listed orthopaedic surgeon through a nationwide, retrospective analysis.

Methods:

All available demographic and rating information for orthopaedic surgeons (n = 28,713; Healthgrades.com) was analyzed using one-way Analysis of Variance, Tukey Studentized Range (Honestly Significant Difference), linear regression, and Pearson correlation coefficient.

Results:

The mean rating for all surgeons was 3.99 (SD 0.92), and the mean number of ratings was 13.43 (SD 20.4). Men had a greater mean rating at 4.02 compared with women at 3.91 (P < 0.0001), and DO surgeons had greater mean rating at 4.11 compared with MD surgeons at 3.90 (P < 0.0001). The correlation between rating and age had a significant negative correlation (P < 0.0001). The correlation between average online rating and number of reviews had a significant positive correlation (P < 0.0001).

Discussion:

Our analysis suggests that greater online ratings are associated with the male sex and DO degrees. In addition, our study discovered that the number of ratings was positively correlated with greater mean online ratings, whereas older age was negatively correlated with greater mean online ratings.

Trends in web data extraction using machine learning

Web data extraction has seen significant development in the last decade since its inception in the early nineties. It has evolved from a simple manual way of extracting data from web page and documents to automated extraction to an intelligent extraction using machine learning algorithms, tools and techniques. Data extraction is one of the key components of end-to-end life cycle in web data extraction process that includes navigation, extraction, data enrichment and visualization. This paper presents the journey of web data extraction over the last many years highlighting evolution of tools, techniques, frameworks and algorithms for building intelligent web data extraction systems. The paper also throws light into challenges, opportunities for future research and emerging trends over the years in web data extraction with specific focus on machine learning techniques. Both traditional and machine learning approaches to manual and automated web data extraction are experimented and results published with few use cases demonstrating the challenges in web data extraction in the event of changes in the website layout. This paper introduces novel ideas such as self-healing capability in web data extraction and proactive error detection in the event of changes in website layout as an area of future research. This unique perspective will help readers to get deeper insights in to the present and future of web data extraction.

Social media mining in drug development-Fundamentals and use cases

The incorporation of patients' perspectives into drug discovery and development has become critically important from the viewpoint of accounting for modern-day business dynamics. There is a trend among patients to narrate their disease experiences on social media. The insights gained by analyzing the data pertaining to such social-media posts could be leveraged to support patient-centered drug development. Manual analysis of these data is nearly impossible, but artificial intelligence enables automated and cost-effective processing, also referred as social media mining (SMM). This paper discusses the fundamental SMM methods along with several relevant drug-development use cases.

Decision Support System Applications for Scheduling in Professional Team Sport. The Team's Perspective

Background: Periodization implies the systematic planning of training and competition with the goal of reaching the best possible performance in the most important competition. In team sports, this consists of finding a flight-and-practice schedule that maximizes the opportunities to perform the periodized contents (e.g., trips, practices, games, and days off). This process is conducted whilst considering known constraints (e.g., competitive schedule, roster availability, weather, especial events, holidays, or emotional effect of days away). The way a scheduling decision support system (DSS) leads users to make a decision should allow for flexibility, whilst minimizing users' confusion and facilitating the understanding of the recommendation given by the scheduling decision support system. Traditional approaches to solving scheduling problems use either simulation models, analytical models, heuristic approaches or a combination of these methods. When it comes to evaluate how the scheduling DSS is performing, three overarching aspects need to be reviewed: context satisfaction, process efficiency, and output quality. Appropriate training periodization and scheduling of trips and training sessions are critical for teams to optimize training and recovery processes in order to maximize health and performance. This article presents a methodological framework for designing decision-support systems for scheduling in professional team sports.

Publication outcome of research presented at the European Congress of Endocrinology: a web scraping-based analysis and critical appraisal

PURPOSE

The current study aimed to determine the publication outcome of abstracts presented at the 16th European Congress of Endocrinology (ECE 2014).

METHODS

All presentations were collected with the web scraping - Python coding from the official website and converted into Google Scholar and PubMed search links with coding. A particular interface was coded to evaluate the results. An online survey was sent to the authors to assess the impact of congress on their publication.

RESULTS

A total of 1205 abstracts from 71 countries were featured at the congress of which, 1145 (95%) were poster presentations (PP), and 60 (5%) were oral presentations. Subsequently, 341 abstracts (28.3%) were published as a full paper. There was no major change from the abstract in 73.3% of full articles whereas 68.9% had at least one minor change. OP had higher conversion rates to publication than PP (65% vs 26.4; p = 0.01) and a higher median number of citations than PP (12 vs 6; p = 0.01). The median time to publication was 12 months (IQR: 2-24 months). OP was published in journals with a higher median impact factor (IF) than PP (5 vs 2.94; p = 0.01). Multi-country collaborative studies turned into more publications than single-country studies (OR: 3.91 95% CI: 2.52-6.06; p < 0.01). The congress's potential IF was calculated as 3.18. Among the authors responded to survey, 95% indicated that presenting at the congress was valuable for preparation of their publication.

CONCLUSIONS

This first study evaluating the publication outcome of an international endocrinology congress suggests a 28.3% publication ratio with low discrepancy and 3.18 IF for ECE 2014.

Web Scraping Scientific Repositories for Augmented Relevant Literature Search Using CRISP-DM

Scientific web repositories are central cyber locations where academic papers are stored and maintained. With the nature of the unstructured and semi-structured information/metadata within these repositories, literature analysis for scholar writing becomes a challenge. Correspondingly, applying CRISP-DM poses a stance to address this challenge through formulating a rather augmented process for a relevant literature search. However, almost all repositories do not have a straight forward method where metadata could be extracted for preliminary data processing being applied as part of the CRISP-DM process. Additionally, most repositories do not follow open access standards. Until the time this paper was published, the topic of the augmented, relevant literature search had seen a methodological progress only, with the inability to apply the underlying methods on a larger scale, given data access constraints to open access repositories. The aim of this paper is to propose CRISP-DM as an augmented research methodology with a focus on web scraping as part of the data processing step. To substantiate the proposed methodology, a play role case study is conducted. This then works on alleviating these restrictions, as well as encouraging the wider adoption of the augmented analysis process for a relevant literature search within the research community.

Digital Trespass: Ethical and Terms-of-Use Violations by Researchers Accessing Data From an Online Patient Community

With the expansion and popularity of research on websites such as Facebook and Twitter, there has been increasing concern about investigator conduct and social media ethics. The availability of large data sets has attracted researchers who are not traditionally associated with health data and its associated ethical considerations, such as computer and data scientists. Reliance on oversight by ethics review boards is inadequate and, due to the public availability of social media data, there is often confusion between public and private spaces. In addition, social media participants and researchers may pay little attention to traditional terms of use. In this paper, we review four cases involving ethical and terms-of-use violations by researchers seeking to conduct social media studies in an online patient research network. These violations involved unauthorized scraping of social media data, entry of false information, misrepresentation of researcher identities of participants on forums, lack of ethical approval and informed consent, use of member quotations, and presentation of findings at conferences and in journals without verifying accurate potential biases and limitations of the data. The correction of these ethical lapses often involves much effort in detecting and responding to violators, addressing these lapses with members of an online community, and correcting inaccuracies in the literature (including retraction of publications and conference presentations). Despite these corrective actions, we do not regard these episodes solely as violations. Instead, they represent broader ethical issues that may arise from potential sources of confusion, misinformation, inadequacies in applying traditional informed consent procedures to social media research, and differences in ethics training and scientific methodology across research disciplines. Social media research stakeholders need to assure participants that their studies will not compromise anonymity or lead to harmful outcomes while preserving the societal value of their health-related studies. Based on our experience and published recommendations by social media researchers, we offer potential directions for future prevention-oriented measures that can be applied by data producers, computer/data scientists, institutional review boards, research ethics committees, and publishers.

Webcrawling and machine learning as a new approach for the spatial distribution of atmospheric emissions

In this study we apply two methods for data collection that are relatively new in the field of atmospheric science. The two developed methods are designed to collect essential geo-localized information to be used as input data for a high resolution emission inventory for residential wood combustion (RWC). The first method is a webcrawler that extracts openly online available real estate data in a systematic way, and thereafter structures them for analysis. The webcrawler reads online Norwegian real estate advertisements and it collects the geo-position of the dwellings. Dwellings are classified according to the type (e.g., apartment, detached house) they belong to and the heating systems they are equipped with. The second method is a model trained for image recognition and classification based on machine learning techniques. The images from the real estate advertisements are collected and processed to identify wood burning installations, which are automatically classified according to the three classes used in official statistics, i.e., open fireplaces, stoves produced before 1998 and stoves produced after 1998. The model recognizes and classifies the wood appliances with a precision of 81%, 85% and 91% for open fireplaces, old stoves and new stoves, respectively. Emission factors are heavily dependent on technology and this information is therefore essential for determining accurate emissions. The collected data are compared with existing information from the statistical register at county and national level in Norway. The comparison shows good agreement for the proportion of residential heating systems between the webcrawled data and the official statistics. The high resolution and level of detail of the extracted data show the value of open data to improve emission inventories. With the increased amount and availability of data, the techniques presented here add significant value to emission accuracy and potential applications should also be considered across all emission sectors.

The benefits of in silico modeling to identify possible small-molecule drugs and their off-target interactions

The research into the use of small molecules as drugs continues to be a key driver in the development of molecular databases, computer-aided drug design software and collaborative platforms. The evolution of computational approaches is driven by the essential criteria that a drug molecule has to fulfill, from the affinity to targets to minimal side effects while having adequate absorption, distribution, metabolism, and excretion (ADME) properties. A combination of ligand- and structure-based drug development approaches is already used to obtain consensus predictions of small molecule activities and their off-target interactions. Further integration of these methods into easy-to-use workflows informed by systems biology could realize the full potential of available data in the drug discovery and reduce the attrition of drug candidates.

Vigi4Med Scraper: A Framework for Web Forum Structured Data Extraction and Semantic Representation

The extraction of information from social media is an essential yet complicated step for data analysis in multiple domains. In this paper, we present Vigi4Med Scraper, a generic open source framework for extracting structured data from web forums. Our framework is highly configurable; using a configuration file, the user can freely choose the data to extract from any web forum. The extracted data are anonymized and represented in a semantic structure using Resource Description Framework (RDF) graphs. This representation enables efficient manipulation by data analysis algorithms and allows the collected data to be directly linked to any existing semantic resource. To avoid server overload, an integrated proxy with caching functionality imposes a minimal delay between sequential requests. Vigi4Med Scraper represents the first step of Vigi4Med, a project to detect adverse drug reactions (ADRs) from social networks founded by the French drug safety agency Agence Nationale de Sécurité du Médicament (ANSM). Vigi4Med Scraper has successfully extracted greater than 200 gigabytes of data from the web forums of over 20 different websites.