aPROMISE: A Novel Automated-PROMISE platform to Standardize Evaluation of Tumor Burden in 18F-DCFPyL (PSMA) images of Veterans with Prostate Cancer

Rationale: Standardized staging and quantitative reporting is necessary to demonstrate the association of ¹⁸F-DCFPyL PET/CT (PSMA) imaging with clinical outcome. This work introduces an automated platform to implement and extend the Prostate Cancer Molecular Imaging Standardized Evaluation (PROMISE) criteria - aPROMISE. The objective is to validate the performance of aPROMISE in staging and quantifying disease burden in patients with prostate cancer who undergo PSMA Imaging. Methods: This was a retrospective analysis of 109 Veterans with intermediate and high-risk prostate cancer, who underwent PSMA imaging. To validate the performance of aPROMISE, two independent nuclear-medicine physicians conducted aPROMISE-assisted reads, resulting in standardized reports that quantify individual lesions and stage the patients. Patients were staged as having local only disease (miN0M0); regional lymph node only (miN1M0), metastatic disease only (miN0M1), and with both regional and distant metastatic disease (miN1M1). The staging obtained from aPROMISE-assisted reads was compared with the staging by conventional imaging. Cohen's pairwise kappa agreement was used to evaluate the inter-reader variability. Correlation coefficient and ICC was used to evaluate the inter-reader variability of the quantitative assessment (miPSMA-index) in each stage. Kendall Tau and t-test was used to evaluate the association of miPSMA-index with PSA and Gleason Score. Results: All PSMA images of 109 veterans met the DICOM conformity and the requirements for the aPROMISE analysis. Both independent aPROMISE-assisted analyses demonstrated significant upstaging in patients with localized (23%; N = 20/87) and regional tumor burden (25%; N = 2/8). However, a significant number of patients with bone metastases identified on conventional imaging (NaF PET/CT) were downstaged (29%; N = 4/14). The comparison of the two independent aPROMISE-assisted reads demonstrated a high kappa agreement - 0.82 (miN0M0), 0.90 (miN1M0), and 0.77 (miN0M1). The Spearman correlation of quantitative miPSMA-index was 0.93, 0.96 and 0.97, respectively. As a continuous variable, miPSMA index in the prostate (miT) was associated with risk groups defined by the PSA and Gleason.. Conclusion: Here we demonstrate consistency of the aPROMISE platform between readers and observed substantial upstaging in PSMA imaging compared to the conventional imaging. aPROMISE may contribute to the broader standardization of PSMA imaging assessment and to its clinical utility in management of prostate cancer patients.

Garbage in, Garbage Out: Data Collection, Quality Assessment and Reporting Standards for Social Media Data Use in Health Research, Infodemiology and Digital Disease Detection

Background

Social media have transformed the communications landscape. People increasingly obtain news and health information online and via social media. Social media platforms also serve as novel sources of rich observational data for health research (including infodemiology, infoveillance, and digital disease detection detection). While the number of studies using social data is growing rapidly, very few of these studies transparently outline their methods for collecting, filtering, and reporting those data. Keywords and search filters applied to social data form the lens through which researchers may observe what and how people communicate about a given topic. Without a properly focused lens, research conclusions may be biased or misleading. Standards of reporting data sources and quality are needed so that data scientists and consumers of social media research can evaluate and compare methods and findings across studies.

Objective

We aimed to develop and apply a framework of social media data collection and quality assessment and to propose a reporting standard, which researchers and reviewers may use to evaluate and compare the quality of social data across studies.

Methods

We propose a conceptual framework consisting of three major steps in collecting social media data: develop, apply, and validate search filters. This framework is based on two criteria: retrieval precision (how much of retrieved data is relevant) and retrieval recall (how much of the relevant data is retrieved). We then discuss two conditions that estimation of retrieval precision and recall rely on—accurate human coding and full data collection—and how to calculate these statistics in cases that deviate from the two ideal conditions. We then apply the framework on a real-world example using approximately 4 million tobacco-related tweets collected from the Twitter firehose.

Results

We developed and applied a search filter to retrieve e-cigarette–related tweets from the archive based on three keyword categories: devices, brands, and behavior. The search filter retrieved 82,205 e-cigarette–related tweets from the archive and was validated. Retrieval precision was calculated above 95% in all cases. Retrieval recall was 86% assuming ideal conditions (no human coding errors and full data collection), 75% when unretrieved messages could not be archived, 86% assuming no false negative errors by coders, and 93% allowing both false negative and false positive errors by human coders.

Conclusions

This paper sets forth a conceptual framework for the filtering and quality evaluation of social data that addresses several common challenges and moves toward establishing a standard of reporting social data. Researchers should clearly delineate data sources, how data were accessed and collected, and the search filter building process and how retrieval precision and recall were calculated. The proposed framework can be adapted to other public social media platforms.