1
|
Mishra V, Dexter JP. Response of Unvaccinated US Adults to Official Information About the Pause in Use of the Johnson & Johnson-Janssen COVID-19 Vaccine: Cross-Sectional Survey Study. J Med Internet Res 2024; 26:e41559. [PMID: 38557597 PMCID: PMC11019423 DOI: 10.2196/41559] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Revised: 05/26/2023] [Accepted: 12/29/2023] [Indexed: 04/04/2024] Open
Abstract
Using a rapid response web-based survey, we identified gaps in public understanding of the Centers for Disease Control and Prevention's messaging about the pause in use of the Johnson & Johnson-Janssen COVID-19 vaccine and estimated changes in vaccine hesitancy using counterfactual questions.
Collapse
Affiliation(s)
- Vishala Mishra
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC, United States
| | - Joseph P Dexter
- Data Science Initiative, Harvard University, Allston, MA, United States
- Department of Human Evolutionary Biology, Harvard University, Cambridge, MA, United States
- Institute of Collaborative Innovation, University of Macau, Taipa, Macao
| |
Collapse
|
2
|
Chakhunashvili K, Chakhunashvili DG, Kvirkvelia E, Toria T, Basilaia L, Gorjomeladze T. Correlation of geopolitics, education, democracy with COVID-19 vaccination rate. BMC Public Health 2024; 24:694. [PMID: 38438965 PMCID: PMC10913214 DOI: 10.1186/s12889-024-18215-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2023] [Accepted: 02/26/2024] [Indexed: 03/06/2024] Open
Abstract
INTRODUCTION Vaccine hesitancy is an ongoing problem and determining the factors that increase the vaccination rate in various countries of the world might be useful for further implementation of efficient public health policies and negating anti-vaccination campaigns. MATERIALS AND METHODS Human Development Index (HDI), Education Index (EI), Democracy Index (DI), COVID-19 vaccination rates, COVID-19 data were collected from public sources such as UNDP - Human Development Reports, UNESCO - Education Index, Economist Intelligence, WHO- COVID-19 Dashboard, Our World In Data, The Financial Times COVID-19 Dashboard. Statistical analysis such as Pearson correlation, and linear regression analyses were done to determine a relation between the above-mentioned indices and COVID-19 vaccination rates (1-dose, 2-dose, booster, and combined). RESULTS HDI had the strongest positive correlation with the vaccination rates (1-dose- r (181) = 0.632, p < 0.001, 2-dose- r (181) = 0.671, p < 0.001, booster- r (181) = 0.718, p < 0.001, combined- 0.703, p < 0.001). EI (1-dose- r (177) = 0.560, p < 0.001, 2-dose- r (177) = 0.599, p < 0.001, booster- r (177) = 0.642, p < 0.001, combined- 0.626, p < 0.001), DI (1-dose- r (163) = 0.445, p < 0.001, 2-dose- r (163) = 0.479, p < 0.001, booster- r (163) = 0.534, p < 0.001, combined- 0.508, p < 0.001), as well as Geographic location (1-dose- η (Eta) = 0.610 p < 0.001, 2-dose- η (Eta) = 0.633 p < 0.001, booster- η (Eta) = 0.657, p < 0.001, combined- η (Eta) = 0.645, p < 0.001) had positive correlation with vaccination rates. CONCLUSION There is a strong positive correlation of COVID-19 vaccination rates with HDI and EI.
Collapse
Affiliation(s)
| | | | - Eka Kvirkvelia
- Reproductive Education Hub, Tbilisi, Georgia
- Department of Gynecology, Caucasus University, Tbilisi, Georgia
| | - Tornike Toria
- Reproductive Education Hub, Tbilisi, Georgia
- Business and Technology University, Tbilisi, Georgia
| | - Liza Basilaia
- Reproductive Education Hub, Tbilisi, Georgia
- Ilia State University, Tbilisi, Georgia
| | - Tsira Gorjomeladze
- Reproductive Education Hub, Tbilisi, Georgia
- Ilia State University, Tbilisi, Georgia
| |
Collapse
|
3
|
Chandrasekaran R, Konaraddi K, Sharma SS, Moustakas E. Text-Mining and Video Analytics of COVID-19 Narratives Shared by Patients on YouTube. J Med Syst 2024; 48:21. [PMID: 38358554 DOI: 10.1007/s10916-024-02047-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2023] [Accepted: 02/13/2024] [Indexed: 02/16/2024]
Abstract
This study explores how individuals who have experienced COVID-19 share their stories on YouTube, focusing on the nature of information disclosure, public engagement, and emotional impact pertaining to consumer health. Using a dataset of 186 YouTube videos, we used text mining and video analytics techniques to analyze textual transcripts and visual frames to identify themes, emotions, and their relationship with viewer engagement metrics. Findings reveal eight key themes: infection origins, symptoms, treatment, mental well-being, isolation, prevention, government directives, and vaccination. While viewers engaged most with videos about infection origins, treatment, and vaccination, fear and sadness in the text consistently drove views, likes, and comments. Visuals primarily conveyed happiness and sadness, but their influence on engagement varied. This research highlights the crucial role YouTube plays in disseminating COVID-19 patient narratives and suggests its potential for improving health communication strategies. By understanding how emotions and content influence viewer engagement, healthcare professionals and public health officials can tailor their messaging to better connect with the public and address pandemic-related anxieties.
Collapse
Affiliation(s)
| | - Karthik Konaraddi
- Department of Information & Decision Sciences, University of Illinois at Chicago, Chicago, IL, USA
| | - Sakshi S Sharma
- Department of Information & Decision Sciences, University of Illinois at Chicago, Chicago, IL, USA
| | | |
Collapse
|
4
|
Wang Y, Yang S, An X. Opinion leaders and crisis communication during the COVID-19 pandemic: A study of theme evolution and emotional impact on Twitter. Digit Health 2024; 10:20552076241234619. [PMID: 38476974 PMCID: PMC10929030 DOI: 10.1177/20552076241234619] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 01/25/2024] [Indexed: 03/14/2024] Open
Abstract
This study uses Twitter data from the early stages of the pandemic to analyze the evolution of topics during different time periods and attempts to investigate the content and emotional impact of opinion leaders on public opinion evolution under different themes, in order to understand their role in shaping public discourse and emotions. Divide the life cycle into three stages; NLTK emotional analysis and dynamic topic models (DTMs) are employed to extract and analyze topic words. The results showed that there were significant differences between opinion leaders and followers in terms of hot topics and their evolution trends: (1) In terms of hot topics, opinion leaders have always been paying attention to measures and methods aimed at the public, while followers usually have persist in seeking information and dissatisfaction. (2) In terms of identifying and evolving hot topics, opinion leaders have shifted from the impact of the epidemic on individuals and resources to government responses and policies, while followers are more inclined to express people's growing concerns and dissatisfaction with crisis management. The content of opinion leaders has a significant relationship with evolving public opinion, highlighting the importance of understanding their role in crisis communication. Opinion leaders are also categorized into five types, each with different audience sizes, contents, emotions, and network structures, and they impact public opinion differently. This study identifies and analyzes the characteristics and impact mechanisms of opinion leaders in crisis communication. It hopes to contribute to understanding crisis communication dynamics in the digital era and provide insights into effective communication strategies during crises.
Collapse
Affiliation(s)
- Yan Wang
- Department of Medical Science and Technology Evaluation. Institute of Medical Information, Chinese Academy of Medical Sciences/Peking Union Medical College, Beijing, China
| | - Shuang Yang
- Department of Medical Science and Technology Evaluation. Institute of Medical Information, Chinese Academy of Medical Sciences/Peking Union Medical College, Beijing, China
| | - Xinying An
- Department of Medical Science and Technology Evaluation. Institute of Medical Information, Chinese Academy of Medical Sciences/Peking Union Medical College, Beijing, China
| |
Collapse
|
5
|
Yang S, Tao R, Bhattar M, Shen L, Jones M, Garbacz A, Passmore SR. Designing and testing social media campaign messages to promote COVID-19 vaccine confidence among rural adults: A community-engaged approach featuring rural community leader and clinician testimonials. Prev Med Rep 2023; 36:102508. [PMID: 38116276 PMCID: PMC10728439 DOI: 10.1016/j.pmedr.2023.102508] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2023] [Revised: 11/07/2023] [Accepted: 11/08/2023] [Indexed: 12/21/2023] Open
Abstract
Despite the growing availability of effective COVID-19 vaccines in rural communities in the United States, widespread vaccine hesitancy delays COVID-19 vaccine coverage in rural communities and threatens to worsen pre-pandemic rural-urban disparities in other vaccination rates, including influenza and routine pediatric immunizations. Therefore, there is an urgent need to develop communication-based interventions to improve vaccine confidence in rural America. This study demonstrates the efficacy of a community-engaged approach to developing social media campaign messages in promoting COVID-19 vaccine uptake and pro-vaccine social diffusion among rural adults. Using a community-engaged approach, we developed social media campaign videos varying in (a) featured messengers (clinicians versus community leaders) and (b) the presence of personal testimonials. We conducted a national online experiment (N = 1,364 rural adults) in spring 2022. We found that videos featuring clinicians serving rural communities and their testimonials increased (a) vaccination intentions in the unvaccinated group (4-point scale, b = 0.23, p =.015) and (b) intention to discuss the messages with others (4-point scale, b = 0.14, p =.037), share the message (4-point scale, b = 0.15, p =.026), and promote the vaccines to others (9-point scale, b = 0.48, p =.013). Results suggest that vaccine promotional social media campaigns targeting rural populations can benefit from including clinician testimonials.
Collapse
Affiliation(s)
- Sijia Yang
- School of Journalism and Mass Communication, University of Wisconsin-Madison, Madison, WI, United States
| | - Ran Tao
- School of Journalism and Mass Communication, University of Wisconsin-Madison, Madison, WI, United States
| | - Mahima Bhattar
- School of Medicine and Public Health, University of Wisconsin-Madison, Madison, WI, United States
| | - Liwei Shen
- Department of Communication Arts, University of Wisconsin-Madison, Madison, WI, United States
| | - Malia Jones
- Department of Community & Environmental Sociology, University of Wisconsin-Madison, Madison WI
| | - Andy Garbacz
- Department of Educational Psychology, School of Education, University of Wisconsin-Madison, United States
| | - Susan Racine Passmore
- Collaborative Center for Health Equity, Institute for Clinical and Translational Research, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, WI, United States
| |
Collapse
|
6
|
Negrea-Busuioc E. 'Vaccine as a cheat sheet': a metaphor gone awry on Facebook. Front Psychol 2023; 14:1198172. [PMID: 38054170 PMCID: PMC10694614 DOI: 10.3389/fpsyg.2023.1198172] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 10/06/2023] [Indexed: 12/07/2023] Open
Abstract
COVID-19 vaccine-related conspiracy narratives skyrocketed in social media in the first year of the pandemic. Science communicators have tried to debunk false information as did Vlad Mixich, a Romanian public health expert, who tried to explain on Facebook why the vaccine cannot modify the human DNA. Drawing on the literature on metaphor as a discourse phenomenon, this paper uses a discourse-led approach to metaphor analysis to identify and analyze the metaphors used by Mixich to explain how vaccines work and the mRNA technology underlying the COVID-19 vaccine. A particular metaphor is then given special attention: 'vaccine as a cheat sheet'. The author of the Facebook post seems to use this metaphor deliberately both to clarify vaccine-related information and to mock people susceptible to false information about the vaccine. This paper shows that while the 'cheating' metaphor simplifies abstract knowledge and allows the audience to engage with a complex topic, it also has potential to amplify vaccine-related polarization.
Collapse
Affiliation(s)
- Elena Negrea-Busuioc
- National University of Political Studies and Public Administration, Bucharest, Romania
| |
Collapse
|
7
|
Shah YB, Kieran NW, Klasko SK. Empathy in the age of science disinformation: implications for healthcare quality. BMJ LEADER 2023:leader-2022-000716. [PMID: 37192118 DOI: 10.1136/leader-2022-000716] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/15/2022] [Accepted: 04/25/2023] [Indexed: 05/18/2023]
Affiliation(s)
- Yash B Shah
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Nicholas W Kieran
- Sidney Kimmel Medical College, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| | - Stephen K Klasko
- Office of the President, Thomas Jefferson University, Philadelphia, Pennsylvania, USA
| |
Collapse
|
8
|
Narayanan S, Basch CH. YouTube and COVID-19 vaccines: A mini scoping review. Hum Vaccin Immunother 2023:2202091. [PMID: 37129230 DOI: 10.1080/21645515.2023.2202091] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/03/2023] Open
Abstract
YouTube is a highly popular social media platform capable of widespread information dissemination about COVID-19 vaccines. The aim of this mini scoping review was to summarize the content, quality, and methodology of studies that analyze YouTube videos related to COVID-19 vaccines. COVIDENCE was used to screen search results based on inclusion and exclusion criteria. PRISMA was used for data organization, and the final list of 9 articles used in the mini review were summarized and synthesized. YouTube videos included in each study, total number of cumulative views, results, and limitations were described. Overall, most of the videos were uploaded by television and internet news media and healthcare professionals. A variety of coding schemas were used in the studies. Videos with misleading, inaccurate, or anti-vaccination sentiment were more often uploaded by consumers. Officials seeking to encourage vaccination may utilize YouTube for widespread reach and to debunk misinformation and disinformation.
Collapse
Affiliation(s)
- Sandhya Narayanan
- Department of Public Health, William Paterson University, Wayne, NJ, USA
| | - Corey H Basch
- Department of Public Health, William Paterson University, Wayne, NJ, USA
| |
Collapse
|
9
|
Lokmanoglu AD, Nisbet EC, Osborne MT, Tien J, Malloy S, Cueva Chacón L, Villa Turek E, Abhari R. Social Media Sentiment about COVID-19 Vaccination Predicts Vaccine Acceptance among Peruvian Social Media Users the Next Day. Vaccines (Basel) 2023; 11:vaccines11040817. [PMID: 37112729 PMCID: PMC10146388 DOI: 10.3390/vaccines11040817] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Revised: 03/30/2023] [Accepted: 04/04/2023] [Indexed: 04/29/2023] Open
Abstract
Drawing upon theories of risk and decision making, we present a theoretical framework for how the emotional attributes of social media content influence risk behaviors. We apply our framework to understanding how COVID-19 vaccination Twitter posts influence acceptance of the vaccine in Peru, the country with the highest relative number of COVID-19 excess deaths. By employing computational methods, topic modeling, and vector autoregressive time series analysis, we show that the prominence of expressed emotions about COVID-19 vaccination in social media content is associated with the daily percentage of Peruvian social media survey respondents who are vaccine-accepting over 231 days. Our findings show that net (positive) sentiment and trust emotions expressed in tweets about COVID-19 are positively associated with vaccine acceptance among survey respondents one day after the post occurs. This study demonstrates that the emotional attributes of social media content, besides veracity or informational attributes, may influence vaccine acceptance for better or worse based on its valence.
Collapse
Affiliation(s)
- Ayse D Lokmanoglu
- Department of Communication Studies, Northwestern University, Evanston, IL 60208, USA
| | - Erik C Nisbet
- Department of Communication Studies, Northwestern University, Evanston, IL 60208, USA
| | - Matthew T Osborne
- Department of Mathematics, The Ohio State University, Columbus, OH 43210, USA
| | - Joseph Tien
- Department of Mathematics, The Ohio State University, Columbus, OH 43210, USA
| | | | - Lourdes Cueva Chacón
- School of Journalism and Media Studies, San Diego State University, San Diego, CA 92182, USA
| | - Esteban Villa Turek
- Department of Communication Studies, Northwestern University, Evanston, IL 60208, USA
| | - Rod Abhari
- Department of Communication Studies, Northwestern University, Evanston, IL 60208, USA
| |
Collapse
|
10
|
Falcon M, Rodríguez-Blázquez C, Romay-Barja M, Ayala A, Burgos A, De Tena-Dávila MJ, Forjaz MJ. COVID-19 vaccine hesitancy in Spain and associated factors. Front Public Health 2023; 11:1129079. [PMID: 37006532 PMCID: PMC10061089 DOI: 10.3389/fpubh.2023.1129079] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 02/21/2023] [Indexed: 03/18/2023] Open
Abstract
IntroductionThe present study explores the reasons of those who have not been vaccinated in the later stage of the vaccine rollout in Spain and its associated determinants.MethodsCluster and logistic regression analyses were used to assess differences in claimed reasons for vaccine hesitancy in Spain using two samples of unvaccinated people (18–40 years old) gathered by an online cross-sectional survey from social networks (n = 910) and from a representative panel (n = 963) in October-November 2021.ResultsThe main reasons for not being vaccinated were believing that the COVID-19 vaccines had been developed too fast, they were experimental, and they were not safe, endorsed by 68.7% participants in the social network sample and 55.4% in the panel sample. The cluster analysis classified the participants into two groups. Logistic regression showed that Cluster 2 (individuals who reported structural constraints and health-related reasons such as pregnancy or medical recommendation) presented a lower trust in information from health professionals, had a lower willingness to get vaccinated in the future, and avoided less social/family events than those in Cluster 1 (reasons centered in distrust on COVID-19 vaccines, conspiracy thoughts and complacency).ConclusionsIt is important to promote information campaigns that provide reliable information and fight fake news and myths. Future vaccination intention differs in both clusters, so these results are important for developing strategies target to increase vaccination uptake for those who do not reject the COVID-19 vaccine completely.
Collapse
Affiliation(s)
- Maria Falcon
- Legal Medicine Department, Biomedical Research Institute (IMIB), University of Murcia, Murcia, Spain
| | | | - María Romay-Barja
- National Center of Tropical Diseases, Instituto de Salud Carlos III, Madrid, Spain
- Centro de Investigación Biomédica en Red de Enfermedades Infecciosas (CIBERINFEC), Madrid, Spain
- *Correspondence: María Romay-Barja
| | - Alba Ayala
- University Institute on Gender Studies, University Carlos III, Getafe, Spain
- Research Network on Chronic Diseases, Primary Care, and Health Promotion (RICAPPS), Madrid, Spain
| | - Alfredo Burgos
- Digital Health Research Unit, Instituto de Salud Carlos III, Madrid, Spain
| | | | - Maria João Forjaz
- National Epidemiology Center, Instituto de Salud Carlos III, Madrid, Spain
- Research Network on Chronic Diseases, Primary Care, and Health Promotion (RICAPPS), Madrid, Spain
| |
Collapse
|
11
|
Béres F, Michaletzky TV, Csoma R, Benczúr AA. Network embedding aided vaccine skepticism detection. APPLIED NETWORK SCIENCE 2023; 8:11. [PMID: 36811026 PMCID: PMC9933796 DOI: 10.1007/s41109-023-00534-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Accepted: 01/16/2023] [Indexed: 06/18/2023]
Abstract
We investigate automatic methods to assess COVID vaccination views in Twitter content. Vaccine skepticism has been a controversial topic of long history that has become more important than ever with the COVID-19 pandemic. Our main goal is to demonstrate the importance of network effects in detecting vaccination skeptic content. Towards this end, we collected and manually labeled vaccination-related Twitter content in the first half of 2021. Our experiments confirm that the network carries information that can be exploited to improve the accuracy of classifying attitudes towards vaccination over content classification as baseline. We evaluate a variety of network embedding algorithms, which we combine with text embedding to obtain classifiers for vaccination skeptic content. In our experiments, by using Walklets, we improve the AUC of the best classifier with no network information by. We publicly release our labels, Tweet IDs and source codes on GitHub.
Collapse
Affiliation(s)
- Ferenc Béres
- ELKH Institute for Computer Science and Control (SZTAKI), Kende u. 13-17, Budapest, 1111 Hungary
| | | | - Rita Csoma
- Eötvös Loránd University, Pázmány Péter s. 1, Budapest, 1117 Hungary
| | - András A. Benczúr
- ELKH Institute for Computer Science and Control (SZTAKI), Kende u. 13-17, Budapest, 1111 Hungary
| |
Collapse
|
12
|
Carvalho EDM, Santos Junior MAD, Neves LFF, Oliveira TMD, Massarani L, Carvalho MS. [Vaccines and social media: a debate around vaccines on Instagram and Facebook during the COVID-19 pandemic (2020-2021)]. CAD SAUDE PUBLICA 2022; 38:e00054722. [PMID: 36541963 DOI: 10.1590/0102-311xpt054722] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Accepted: 10/14/2022] [Indexed: 12/23/2022] Open
Abstract
Considering that the Internet and especially social media work as a locus for the circulation of information on COVID-19, this study aimed to assess the attention given to the vaccine theme on Instagram and Facebook in posts throughout two years of pandemic, identifying the temporality in which discussion about the different immunizing agents in social media and highlighting the actors who permeated the discussions on the subject. Data were collected using the CrowdTangle graphical interface, based on search terms in Portuguese related to vaccines approved for use in Brazil and it included posts from public Facebook pages and open Instagram profiles made from January 1st, 2020, to December 31st, 2021. The database included 3,876,408 posts (2,901,457 on Facebook and 974,952 on Instagram). The results showed an alternation of Pfizer, CoronaVac, AstraZeneca, and Janssen vaccines as the focus of discussion, following the public debate established in the country around the dilemmas and advances related to the development, production, distribution, and application of immunizing agents. References to institutions involved in the acquisition and production of immunizing agents were also identified, such as the Brazilian Ministry of Health, the Butantan Institute, and the Oswaldo Cruz Foundation, present among the thousand social actors that most generated comments from users in both networks. It was evidenced that the debate on immunizing agents, in the analyzed channels and period, was permeated by social media related to journalism and politicians and celebrities' pages and profiles.
Collapse
Affiliation(s)
| | | | | | | | - Luisa Massarani
- Casa de Oswaldo Cruz, Fundação Oswaldo Cruz, Rio de Janeiro, Brasil.,Instituto Nacional de Comunicação Pública da Ciência e Tecnologia, Rio de Janeiro, Brasil
| | - Marilia Sá Carvalho
- Programa de Computação Científica, Fundação Oswaldo Cruz, Rio de Janeiro, Brasil
| |
Collapse
|
13
|
Weng Z, Lin A. Public Opinion Manipulation on Social Media: Social Network Analysis of Twitter Bots during the COVID-19 Pandemic. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:16376. [PMID: 36554258 PMCID: PMC9779151 DOI: 10.3390/ijerph192416376] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/12/2022] [Revised: 11/27/2022] [Accepted: 12/02/2022] [Indexed: 06/17/2023]
Abstract
Social media is not only an essential platform for the dissemination of public health-related information, but also an important channel for people to communicate during the COVID-19 pandemic. However, social bots can interfere with the social media topics that humans follow. We analyzed and visualized Twitter data during the prevalence of the Wuhan lab leak theory and discovered that 29% of the accounts participating in the discussion were social bots. We found evidence that social bots play an essential mediating role in communication networks. Although human accounts have a more direct influence on the information diffusion network, social bots have a more indirect influence. Unverified social bot accounts retweet more, and through multiple levels of diffusion, humans are vulnerable to messages manipulated by bots, driving the spread of unverified messages across social media. These findings show that limiting the use of social bots might be an effective method to minimize the spread of conspiracy theories and hate speech online.
Collapse
|
14
|
Mitilian E, Gosselin V, Casanova L, Fressard L, Berthiaume P, Verger P, Gagneur A. Assessment of training of general practice interns in motivational interviews about vaccination. Hum Vaccin Immunother 2022; 18:2114253. [PMID: 36494997 PMCID: PMC9746523 DOI: 10.1080/21645515.2022.2114253] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/14/2023] Open
Abstract
The effectiveness of motivational interviewing (MI) for reducing vaccine hesitancy (VH) has been demonstrated in Quebec. We conducted a study to evaluate the acquisition of MI skills after MI training via videoconferencing for interns training as general practitioners (GPs) in southeastern France. A vaccination-specific MI training workshop was offered to interns in 2021, consisting of two separate Zoom videoconference sessions. Participants completed the Motivational Interviewing Skills in Immunization questionnaire before and after the training to measure skills acquisition. We used pairwise exact Wilcoxon-Pratt signed rank tests for the analysis. Among 45 GP interns enrolled in the first MI session, 34 (75.6%) attended both sessions and completed the questionnaire at 3 different time points. After the first session, MI knowledge scores improved significantly (+21.1 ± 21.6; P < .0001), as did application of MI skills (+36.8 ± 36.7; P < .0001), and MI practice confidence (+21.2 ± 11.1; P < .0001). The second MI session maintained the skills developed after the first session without further improvement. Participant satisfaction was high. This is the first study in France assessing the impact of a vaccination-specific MI training for GP interns. It shows a substantial improvement in knowledge, application of MI skills, and self-confidence in the practice of MI. GP interns were highly satisfied with the training despite the videoconference format. These promising results will allow the integration of MI training in GP medical curricula in order to prepare future GPs for communication in the field of vaccination.
Collapse
Affiliation(s)
- Eva Mitilian
- Aix Marseille Univ, DUMG, département universitaire de médecine générale, Marseille, France,ORS PACA. Southeastern Health Regional Observatory, Faculty of Medicine, Marseille, France,CONTACT Eva Mitilian Aix Marseille Univ, DUMG, département universitaire de médecine générale, 40, rue Saint-Bruno, Marseille13004
| | - Virginie Gosselin
- Centre de Recherche du Centre hospitalier universitaire de Sherbrooke, Sherbrooke, Quebec, Canada
| | - Ludovic Casanova
- Aix Marseille Univ, DUMG, département universitaire de médecine générale, Marseille, France,ORS PACA. Southeastern Health Regional Observatory, Faculty of Medicine, Marseille, France
| | - Lisa Fressard
- ORS PACA. Southeastern Health Regional Observatory, Faculty of Medicine, Marseille, France
| | | | - Pierre Verger
- ORS PACA. Southeastern Health Regional Observatory, Faculty of Medicine, Marseille, France
| | - Arnaud Gagneur
- Centre de Recherche du Centre hospitalier universitaire de Sherbrooke, Sherbrooke, Quebec, Canada,Department of Pediatrics, Faculty of Medicine and Health Sciences, Université de Sherbrooke, Sherbrooke, Quebec, Canada
| |
Collapse
|
15
|
Schäfer M, Stark B, Werner AM, Mülder LM, Heller S, Reichel JL, Schwab L, Rigotti T, Beutel ME, Simon P, Letzel S, Dietz P. Determinants of university students' COVID-19 vaccination intentions and behavior. Sci Rep 2022; 12:18067. [PMID: 36302883 PMCID: PMC9610342 DOI: 10.1038/s41598-022-23044-9] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2022] [Accepted: 10/25/2022] [Indexed: 01/24/2023] Open
Abstract
Vaccination hesitancy has been a major challenge for health authorities and societies during the COVID-19 pandemic. The general determinants of vaccination intentions and behavior include sociodemographic and health-related, psychological, and communication-related factors, with varying relevance for different types of vaccinations, countries, and target groups. The predictors of university students' COVID-19 vaccination behavior have not been sufficiently studied. Using a German university as an example and based on an online survey of 1398 university students, we investigated factors related to (a) the likelihood of vaccination against COVID-19 and (b) vaccination intention among those who had not been vaccinated by summer of 2021. Psychological factors showed high relevance. Field of study, trust in, and use of certain media and information channels were found to be important determinants of students' vaccination decision. Our findings can help better understand university students' vaccination behavior to develop and implement tailored strategies and campaigns.
Collapse
Affiliation(s)
- Markus Schäfer
- Department of Communication, Johannes Gutenberg University, Mainz, Germany.
| | - Birgit Stark
- Department of Communication, Johannes Gutenberg University, Mainz, Germany
| | - Antonia M Werner
- Department of Psychosomatic Medicine and Psychotherapy, University Medical Center of the Johannes Gutenberg University, Mainz, Germany
| | - Lina M Mülder
- Department of Work, Organizational, and Business Psychology, Institute for Psychology, Johannes Gutenberg University, Mainz, Germany
| | - Sebastian Heller
- Institute of Occupational, Social, and Environmental Medicine, University Medical Center of the Johannes Gutenberg University, Mainz, Germany
| | - Jennifer L Reichel
- Institute of Occupational, Social, and Environmental Medicine, University Medical Center of the Johannes Gutenberg University, Mainz, Germany
| | - Lisa Schwab
- Department of Sports Medicine, Rehabilitation and Disease Prevention, Institute of Sport Science, Johannes Gutenberg University, Mainz, Germany
| | - Thomas Rigotti
- Department of Work, Organizational, and Business Psychology, Institute for Psychology, Johannes Gutenberg University, Mainz, Germany
- Leibniz Institute for Resilience Research, Mainz, Germany
| | - Manfred E Beutel
- Department of Psychosomatic Medicine and Psychotherapy, University Medical Center of the Johannes Gutenberg University, Mainz, Germany
| | - Perikles Simon
- Department of Sports Medicine, Rehabilitation and Disease Prevention, Institute of Sport Science, Johannes Gutenberg University, Mainz, Germany
| | - Stephan Letzel
- Institute of Occupational, Social, and Environmental Medicine, University Medical Center of the Johannes Gutenberg University, Mainz, Germany
| | - Pavel Dietz
- Institute of Occupational, Social, and Environmental Medicine, University Medical Center of the Johannes Gutenberg University, Mainz, Germany
| |
Collapse
|
16
|
Dang Q, Li S. Exploring Public Discussions Regarding COVID-19 Vaccinations on Microblogs in China: Findings from Machine Learning Algorithms. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:13476. [PMID: 36294061 PMCID: PMC9603472 DOI: 10.3390/ijerph192013476] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 08/31/2022] [Revised: 10/14/2022] [Accepted: 10/16/2022] [Indexed: 06/16/2023]
Abstract
Large-scale, widespread COVID-19 vaccination is the most effective means of cutting off the spread of the novel coronavirus and establishing an immune barrier. Due to the large population base in China, it has been a very difficult task to establish such an immune barrier. Therefore, this study aims to explore the public's discussions related to COVID-19 vaccinations on microblogs and to detect their sentiments toward COVID-19 vaccination so as to improve the vaccination rate in China. This study employed machine learning methods in the field of artificial intelligence to analyze mass data obtained from SinaWeibo. A total of 1,478,875 valid microblog texts were collected between December 2020 and June 2022, the results of which indicated that: (1) overall, negative texts (38.7%) slightly outweighed positive texts (36.1%); "Good" (63%) dominated positive texts, while "disgust" (44.6%) and "fear" (35.8%) dominated negative texts; (2) six overarching themes related to COVID-19 vaccination were identified: public trust in the Chinese government, changes in daily work and study, vaccine economy, international COVID-19 vaccination, the COVID-19 vaccine's R&D, and COVID-19 vaccination for special groups. These themes and sentiments can clarify the public's reactions to COVID-19 vaccination and help Chinese officials' response to vaccine hesitancy. Furthermore, this study seeks to make up for the lack of focus on big data in public health and epidemiology research, and to provide novel insights for future studies.
Collapse
|
17
|
Zhu J, Weng F, Zhuang M, Lu X, Tan X, Lin S, Zhang R. Revealing Public Opinion towards the COVID-19 Vaccine with Weibo Data in China: BertFDA-Based Model. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2022; 19:ijerph192013248. [PMID: 36293828 PMCID: PMC9602858 DOI: 10.3390/ijerph192013248] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/17/2022] [Revised: 10/09/2022] [Accepted: 10/11/2022] [Indexed: 05/27/2023]
Abstract
The COVID-19 pandemic has created unprecedented burdens on people's health and subjective well-being. While countries around the world have established models to track and predict the affective states of COVID-19, identifying the topics of public discussion and sentiment evolution of the vaccine, particularly the differences in topics of concern between vaccine-support and vaccine-hesitant groups, remains scarce. Using social media data from the two years following the outbreak of COVID-19 (23 January 2020 to 23 January 2022), coupled with state-of-the-art natural language processing (NLP) techniques, we developed a public opinion analysis framework (BertFDA). First, using dynamic topic clustering on Weibo through the latent Dirichlet allocation (LDA) model, a total of 118 topics were generated in 24 months using 2,211,806 microblog posts. Second, by building an improved Bert pre-training model for sentiment classification, we provide evidence that public negative sentiment continued to decline in the early stages of COVID-19 vaccination. Third, by modeling and analyzing the microblog posts from the vaccine-support group and the vaccine-hesitant group, we discover that the vaccine-support group was more concerned about vaccine effectiveness and the reporting of news, reflecting greater group cohesion, whereas the vaccine-hesitant group was particularly concerned about the spread of coronavirus variants and vaccine side effects. Finally, we deployed different machine learning models to predict public opinion. Moreover, functional data analysis (FDA) is developed to build the functional sentiment curve, which can effectively capture the dynamic changes with the explicit function. This study can aid governments in developing effective interventions and education campaigns to boost vaccination rates.
Collapse
Affiliation(s)
- Jianping Zhu
- National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen 361005, China
- Data Mining Research Center, Xiamen University, Xiamen 361005, China
- School of Management, Xiamen University, Xiamen 361005, China
| | - Futian Weng
- National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen 361005, China
- Data Mining Research Center, Xiamen University, Xiamen 361005, China
- School of Medicine, Xiamen University, Xiamen 361005, China
| | - Muni Zhuang
- National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen 361005, China
- Data Mining Research Center, Xiamen University, Xiamen 361005, China
- School of Medicine, Xiamen University, Xiamen 361005, China
| | - Xin Lu
- College of Systems Engineering, National University of Defense Technology, Changsha 410073, China
| | - Xu Tan
- Career-Oriented Multidisciplinary Education Center, Shenzhen Institiute of Information Technology, Shenzhen 518172, China
| | - Songjie Lin
- Career-Oriented Multidisciplinary Education Center, Shenzhen Institiute of Information Technology, Shenzhen 518172, China
| | - Ruoyi Zhang
- Columbia College of Art and Science, George Washington University, Washington, DC 20052, USA
| |
Collapse
|
18
|
Gao H, Yin H, Peng L, Wang H. Effectiveness of Social Video Platforms in Promoting COVID-19 Vaccination Among Youth: A Content-Specific Analysis of COVID-19 Vaccination Topic Videos on Bilibili. Risk Manag Healthc Policy 2022; 15:1621-1639. [PMID: 36071816 PMCID: PMC9444025 DOI: 10.2147/rmhp.s374420] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 08/25/2022] [Indexed: 12/15/2022] Open
Abstract
Background With the widespread promotion of the COVID-19 vaccination in China, videos about the vaccination have become increasingly available on social video platforms. With the User Generated Content model, different creators’ interpretations of COVID-19 vaccines may influence the attitudes towards the vaccines and vaccination. Objective To explore the overview of COVID-19 vaccine-related videos on Bilibili, discussing the communication effects of COVID-19 topic videos and its influencing factors. Methods A content analysis was applied to the 202 video samples obtained through data mining regarding the creator’s information, video presentation, and COVID-19 vaccine-related content. Results Individuals and medical professionals preferred VLOG videos, media chose to upload informational videos, and enterprises preferred to post showcase videos. Individuals were more likely to discuss the adverse reactions in their videos, while medical professionals were more likely to discuss the vaccination process for the COVID-19 vaccine. Videos with core issues positively influenced the video’s dissemination breadth. The attitudes toward the COVID-19 vaccine in the videos positively influence the recognition of the videos. The richness of knowledge points related to the COVID-19 vaccine negatively affected the recognition and participation. Conclusion Social video platforms could play an active role in the vaccination promotion for the youth. Health promotion-related departments and individuals could strengthen agenda setting, grasp the characteristics of young groups, and express positive attitudes toward health issues to achieve better health (vaccine) promotion.
Collapse
Affiliation(s)
- Hao Gao
- School of Journalism and Communication, Nanjing Normal University, Nanjing, 210097, People’s Republic of China
| | - Hao Yin
- School of Journalism and Communication, Nanjing Normal University, Nanjing, 210097, People’s Republic of China
| | - Li Peng
- School of Journalism and Communication, Nanjing Normal University, Nanjing, 210097, People’s Republic of China
| | - Han Wang
- School of Journalism and Communication, Jinan University, Guangzhou, Guangdong, 510632, People’s Republic of China
- Correspondence: Han Wang, School of Journalism and Communication, Jinan University, No. 601, West Huangpu Avenue, Guangzhou, Guangdong, 510632, People’s Republic of China, Email
| |
Collapse
|
19
|
Cascini F, Pantovic A, Al-Ajlouni YA, Failla G, Puleo V, Melnyk A, Lontano A, Ricciardi W. Social media and attitudes towards a COVID-19 vaccination: A systematic review of the literature. EClinicalMedicine 2022; 48:101454. [PMID: 35611343 PMCID: PMC9120591 DOI: 10.1016/j.eclinm.2022.101454] [Citation(s) in RCA: 88] [Impact Index Per Article: 44.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/27/2022] [Revised: 04/24/2022] [Accepted: 04/27/2022] [Indexed: 12/24/2022] Open
Abstract
BACKGROUND Vaccine hesitancy continues to limit global efforts in combatting the COVID-19 pandemic. Emerging research demonstrates the role of social media in disseminating information and potentially influencing people's attitudes towards public health campaigns. This systematic review sought to synthesize the current evidence regarding the potential role of social media in shaping COVID-19 vaccination attitudes, and to explore its potential for shaping public health interventions to address the issue of vaccine hesitancy. METHODS We performed a systematic review of the studies published from inception to 13 of March2022 by searching PubMed, Web of Science, Embase, PsychNET, Scopus, CINAHL, and MEDLINE. Studies that reported outcomes related to coronavirus disease 2019 (COVID-19) vaccine (attitudes, opinion, etc.) gathered from the social media platforms, and those analyzing the relationship between social media use and COVID-19 hesitancy/acceptance were included. Studies that reported no outcome of interest or analyzed data from sources other than social media (websites, newspapers, etc.) will be excluded. The Newcastle Ottawa Scale (NOS) was used to assess the quality of all cross-sectional studies included in this review. This study is registered with PROSPERO (CRD42021283219). FINDINGS Of the 2539 records identified, a total of 156 articles fully met the inclusion criteria. Overall, the quality of the cross-sectional studies was moderate - 2 studies received 10 stars, 5 studies received 9 stars, 9 studies were evaluated with 8, 12 studies with 7,16 studies with 6, 11 studies with 5, and 6 studies with 4 stars. The included studies were categorized into four categories. Cross-sectional studies reporting the association between reliance on social media and vaccine intentions mainly observed a negative relationship. Studies that performed thematic analyses of extracted social media data, mainly observed a domination of vaccine hesitant topics. Studies that explored the degree of polarization of specific social media contents related to COVID-19 vaccines observed a similar degree of content for both positive and negative tone posted on different social media platforms. Finally, studies that explored the fluctuations of vaccination attitudes/opinions gathered from social media identified specific events as significant cofactors that affect and shape vaccination intentions of individuals. INTERPRETATION This thorough examination of the various roles social media can play in disseminating information to the public, as well as how individuals behave on social media in the context of public health events, articulates the potential of social media as a platform of public health intervention to address vaccine hesitancy. FUNDING None.
Collapse
Affiliation(s)
- Fidelia Cascini
- Department of Life Sciences and Public Health, Section of Hygiene and Public Health, Università Cattolica del Sacro Cuore, L.go Francesco Vito 1, Rome 00168, Italy
- Corresponding author.
| | - Ana Pantovic
- Faculty of Biology, University of Belgrade, Belgrade, Serbia
| | | | - Giovanna Failla
- Department of Life Sciences and Public Health, Section of Hygiene and Public Health, Università Cattolica del Sacro Cuore, L.go Francesco Vito 1, Rome 00168, Italy
| | - Valeria Puleo
- Department of Life Sciences and Public Health, Section of Hygiene and Public Health, Università Cattolica del Sacro Cuore, L.go Francesco Vito 1, Rome 00168, Italy
| | - Andriy Melnyk
- Department of Life Sciences and Public Health, Section of Hygiene and Public Health, Università Cattolica del Sacro Cuore, L.go Francesco Vito 1, Rome 00168, Italy
| | - Alberto Lontano
- Department of Life Sciences and Public Health, Section of Hygiene and Public Health, Università Cattolica del Sacro Cuore, L.go Francesco Vito 1, Rome 00168, Italy
| | - Walter Ricciardi
- Department of Life Sciences and Public Health, Section of Hygiene and Public Health, Università Cattolica del Sacro Cuore, L.go Francesco Vito 1, Rome 00168, Italy
| |
Collapse
|
20
|
Pourkarim M, Nayebzadeh S, Alavian SM, Hataminasab SH. Digital Marketing: A Unique Multidisciplinary Approach towards the Elimination of Viral Hepatitis. Pathogens 2022; 11:pathogens11060626. [PMID: 35745480 PMCID: PMC9228079 DOI: 10.3390/pathogens11060626] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Revised: 05/17/2022] [Accepted: 05/27/2022] [Indexed: 12/16/2022] Open
Abstract
New technologies are supported by the global implementation of the internet. These improvements have deeply affected various disciplines of sciences and consequently changed services such as daily business, particularly health sectors. Innovative digital marketing strategies utilize the channels of social media and retrieved user data to analyze and improve relevant services. These multidisciplinary innovations can assist specialists, physicians and researchers in diagnostic, prophylaxis and treatment issues in the health sector. Accordingly, compared to recent decades, health decision makers are more accurate and trustful in defining new strategies. Interestingly, using social media and mobile health apps in current pandemics of SARS-CoV-2 could be an important instance of the key role of these platforms at the local and global level of health policies. These digital technologies provide platforms to connect public health sectors and health politicians for communicating and spreading relevant information. Adding influencers and campaigns to this toolbox strengthens the implementation of public health programs. In 2016, the WHO adopted a global program to eliminate viral hepatitis by 2030. Recent constructive measures that have been used in the battle against COVID-19 could be adopted for the elimination of viral hepatitis program. The presented evidence in our narrative review demonstrates that the application of digital marketing tools to create campaigns on social media, armed with professional influencers, can efficiently consolidate this program. The application of different strategies in using these popular tools will raise the public awareness about viral hepatitis. Subsequently, the availability of an effective vaccine for HBV and antiviral medication for HCV can motivate the audience to take steps towards prophylaxis and screening methods against these infectious illnesses. The encouragement of health policy makers to apply digital communication technologies and comprehensive roadmaps to implement this global program will certainly decrease the burden of viral hepatitis worldwide.
Collapse
Affiliation(s)
- Mohammadreza Pourkarim
- Department of Management, Yazd Branch, Islamic Azad University, Yazd P.O. Box 89195/155, Iran; (M.P.); (S.H.H.)
| | - Shahnaz Nayebzadeh
- Department of Management, Yazd Branch, Islamic Azad University, Yazd P.O. Box 89195/155, Iran; (M.P.); (S.H.H.)
- Correspondence: ; Tel.: +98-351-8211391; Fax: +98-351-8214810
| | | | - Seyyed Hassan Hataminasab
- Department of Management, Yazd Branch, Islamic Azad University, Yazd P.O. Box 89195/155, Iran; (M.P.); (S.H.H.)
| |
Collapse
|
21
|
Boguslavsky DV, Sharov KS, Sharova NP. Counteracting conspiracy ideas as a measure of increasing propensity for COVID-19 vaccine uptake in Russian society. J Glob Health 2022; 2:03013. [PMID: 35356758 PMCID: PMC8942296 DOI: 10.7189/jogh.12.03013] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Affiliation(s)
- Dmitry V Boguslavsky
- Koltzov Institute of Developmental Biology of Russian Academy of Sciences, Moscow, Russia
| | - Konstantin S Sharov
- Koltzov Institute of Developmental Biology of Russian Academy of Sciences, Moscow, Russia
| | - Natalia P Sharova
- Koltzov Institute of Developmental Biology of Russian Academy of Sciences, Moscow, Russia
| |
Collapse
|