Information retrieval in an infodemic: the case of COVID-19 publications

Did high frequency phone surveys during the COVID-19 pandemic include disability questions? An assessment of COVID-19 surveys from March 2020 to December 2022

OBJECTIVES

In the midst of the pandemic, face-to-face data collection for national censuses and surveys was suspended due to limitations on mobility and social distancing, limiting the collection of already scarce disability data. Responses to these constraints were met with a surge of high-frequency phone surveys (HFPSs) that aimed to provide timely data for understanding the socioeconomic impacts of and responses to the pandemic. This paper provides an assessment of HFPS datasets and their inclusion of disability questions to evaluate the visibility of persons with disabilities during the COVID-19 pandemic.

DESIGN

We collected HFPS questionnaires conducted globally from the onset of the pandemic emergency in March 2020 until December 2022 from various online survey repositories. Each HFPS questionnaire was searched using a set of keywords for inclusion of different types of disability questions. Results were recorded in an Excel review log, which was manually reviewed by two researchers.

METHODS

The review of HFPS datasets involved two stages: (1) a main review of 294 HFPS dataset-waves and (2) a semiautomated review of the same dataset-waves using a search engine-powered questionnaire review tool developed by our team. The results from the main review were compared with those of a sensitivity analysis using and testing the tool as an alternative to manual search.

RESULTS

Roughly half of HFPS datasets reviewed and 60% of the countries included in this study had some type of question on disability. While disability questions were not widely absent from HFPS datasets, only 3% of HFPS datasets included functional difficulty questions that meet international standards. The search engine-powered questionnaire review tool proved to be able to streamline the search process for future research on inclusive data.

CONCLUSIONS

The dearth of functional difficulty questions and the Washington-Group Short Set in particular in HFPS has contributed to the relative invisibility of persons with disabilities during the pandemic emergency, the lingering effects of which could impede policy-making, monitoring and advocacy on behalf of persons with disabilities.

Online Health Search Via Multidimensional Information Quality Assessment Based on Deep Language Models: Algorithm Development and Validation

BACKGROUND

Widespread misinformation in web resources can lead to serious implications for individuals seeking health advice. Despite that, information retrieval models are often focused only on the query-document relevance dimension to rank results.

OBJECTIVE

We investigate a multidimensional information quality retrieval model based on deep learning to enhance the effectiveness of online health care information search results.

METHODS

In this study, we simulated online health information search scenarios with a topic set of 32 different health-related inquiries and a corpus containing 1 billion web documents from the April 2019 snapshot of Common Crawl. Using state-of-the-art pretrained language models, we assessed the quality of the retrieved documents according to their usefulness, supportiveness, and credibility dimensions for a given search query on 6030 human-annotated, query-document pairs. We evaluated this approach using transfer learning and more specific domain adaptation techniques.

RESULTS

In the transfer learning setting, the usefulness model provided the largest distinction between help- and harm-compatible documents, with a difference of +5.6%, leading to a majority of helpful documents in the top 10 retrieved. The supportiveness model achieved the best harm compatibility (+2.4%), while the combination of usefulness, supportiveness, and credibility models achieved the largest distinction between help- and harm-compatibility on helpful topics (+16.9%). In the domain adaptation setting, the linear combination of different models showed robust performance, with help-harm compatibility above +4.4% for all dimensions and going as high as +6.8%.

CONCLUSIONS

These results suggest that integrating automatic ranking models created for specific information quality dimensions can increase the effectiveness of health-related information retrieval. Thus, our approach could be used to enhance searches made by individuals seeking online health information.

A large-scale dataset of patient summaries for retrieval-based clinical decision support systems

Retrieval-based Clinical Decision Support (ReCDS) can aid clinical workflow by providing relevant literature and similar patients for a given patient. However, the development of ReCDS systems has been severely obstructed by the lack of diverse patient collections and publicly available large-scale patient-level annotation datasets. In this paper, we collect a novel dataset of patient summaries and relations called PMC-Patients to benchmark two ReCDS tasks: Patient-to-Article Retrieval (ReCDS-PAR) and Patient-to-Patient Retrieval (ReCDS-PPR). Specifically, we extract patient summaries from PubMed Central articles using simple heuristics and utilize the PubMed citation graph to define patient-article relevance and patient-patient similarity. PMC-Patients contains 167k patient summaries with 3.1 M patient-article relevance annotations and 293k patient-patient similarity annotations, which is the largest-scale resource for ReCDS and also one of the largest patient collections. Human evaluation and analysis show that PMC-Patients is a diverse dataset with high-quality annotations. We also implement and evaluate several ReCDS systems on the PMC-Patients benchmarks to show its challenges and conduct several case studies to show the clinical utility of PMC-Patients.

Vaccine Development in the Time of COVID-19: The Relevance of the Risklick AI to Assist in Risk Assessment and Optimize Performance

The 2019 coronavirus (COVID-19) pandemic revealed the urgent need for the acceleration of vaccine development worldwide. Rapid vaccine development poses numerous risks for each category of vaccine technology. By using the Risklick artificial intelligence (AI), we estimated the risks associated with all types of COVID-19 vaccine during the early phase of vaccine development. We then performed a postmortem analysis of the probability and the impact matrix calculations by comparing the 2020 prognosis to the contemporary situation. We used the Risklick AI to evaluate the risks and their incidence associated with vaccine development in the early stage of the COVID-19 pandemic. Our analysis revealed the diversity of risks among vaccine technologies currently used by pharmaceutical companies providing vaccines. This analysis highlighted the current and future potential pitfalls connected to vaccine production during the COVID-19 pandemic. Hence, the Risklick AI appears as an essential tool in vaccine development for the treatment of COVID-19 in order to formally anticipate the risks, and increases the overall performance from the production to the distribution of the vaccines. The Risklick AI could, therefore, be extended to other fields of research and development and represent a novel opportunity in the calculation of production-associated risks.