Assessing and Improving Data Integrity in Web-Based Surveys: Comparison of Fraud Detection Systems in a COVID-19 Study

BACKGROUND

Web-based surveys increase access to study participation and improve opportunities to reach diverse populations. However, web-based surveys are vulnerable to data quality threats, including fraudulent entries from automated bots and duplicative submissions. Widely used proprietary tools to identify fraud offer little transparency about the methods used, effectiveness, or representativeness of resulting data sets. Robust, reproducible, and context-specific methods of accurately detecting fraudulent responses are needed to ensure integrity and maximize the value of web-based survey research.

OBJECTIVE

This study aims to describe a multilayered fraud detection system implemented in a large web-based survey about COVID-19 attitudes, beliefs, and behaviors; examine the agreement between this fraud detection system and a proprietary fraud detection system; and compare the resulting study samples from each of the 2 fraud detection methods.

METHODS

The PhillyCEAL Common Survey is a cross-sectional web-based survey that remotely enrolled residents ages 13 years and older to assess how the COVID-19 pandemic impacted individuals, neighborhoods, and communities in Philadelphia, Pennsylvania. Two fraud detection methods are described and compared: (1) a multilayer fraud detection strategy developed by the research team that combined automated validation of response data and real-time verification of study entries by study personnel and (2) the proprietary fraud detection system used by the Qualtrics (Qualtrics) survey platform. Descriptive statistics were computed for the full sample and for responses classified as valid by 2 different fraud detection methods, and classification tables were created to assess agreement between the methods. The impact of fraud detection methods on the distribution of vaccine confidence by racial or ethnic group was assessed.

RESULTS

Of 7950 completed surveys, our multilayer fraud detection system identified 3228 (40.60%) cases as valid, while the Qualtrics fraud detection system identified 4389 (55.21%) cases as valid. The 2 methods showed only "fair" or "minimal" agreement in their classifications (κ=0.25; 95% CI 0.23-0.27). The choice of fraud detection method impacted the distribution of vaccine confidence by racial or ethnic group.

CONCLUSIONS

The selection of a fraud detection method can affect the study's sample composition. The findings of this study, while not conclusive, suggest that a multilayered approach to fraud detection that includes conservative use of automated fraud detection and integration of human review of entries tailored to the study's specific context and its participants may be warranted for future survey research.

Parents ASSIST: Acceptability and Feasibility of a Video-Based Educational Series for Sexuality-Inclusive Communication between Parents and Gay, Bisexual, and Queer Sons

Young men who have sex with men (YMSM) between the ages of 13 and 24 are a key population for HIV prevention. The parents of gay, bisexual, and queer (GBQ) adolescent males and the sex communication they have with their sons has yet to be explored as an HIV prevention intervention. We developed an online video series called Parents ASSIST (Advancing Supportive and Sexuality-Inclusive Sex Talks) to educate parents about sexual health topics pertinent to GBQ males. We pilot tested the series with a sample of 54 parents of GBQ males using a single-group post-test design. Participants viewed the videos and completed a survey measuring the acceptability and feasibility of an online video series to educate parents. Most of the parents (70.4%) believed that the videos would make parents more likely to initiate talking about sex with their sons. The results suggest that online videos are an acceptable way for parents to learn about GBQ sexual health topics.

Invitational summit: Re-envisioning research focused PHD programs of the future

Ten years ago, in response to the growing discussions, a seminal report, The Research-Focused Doctoral Program in Nursing -Pathways to Excellence was endorsed by AACN members. A re-envisioning of PhD programs is urgently needed to ensure the advancement of nursing science and to situate research focused nursing PhD graduates for success - in academia and beyond. To address these issues, the University of Pennsylvania School of Nursing held a summit in October 2019, Re-envisioning Research Focused PhD Programs of the Future, with stakeholders from 41 educational, governmental, professional and philanthropic institutions. Results from a pre-summit poll shaped the Summit agenda, which consisted of panels and roundtables. Prior to and throughout the Summit, attendees identified and discussed challenges facing research intensive nursing PhD programs, emerging innovations within programs, and suggested changes to re-envision the future training of nurse scientists. These changes include better preparing PhD program graduates to bridge the research and practice divide, redesign funding and support for accelerated PhD students, and to reconceptualize outcome measures and evaluations of graduates. A synthesis of roundtable discussions will be used to frame other manuscripts in this issue as well as to prepare a roadmap for next steps.

[ACCESS TO AN ONLINE EDUCATIONAL MODULE ON SEXUAL HEALTH FOR LATINO/A ADOLESCENTS]

Project Cuídalos aimed to test the efficacy of a web-based intervention aiming to increase communication on sexual health topics between parents and adolescents in the Puerto Rican population. To achieve this goal, we implemented a longitudinal experimental design which examined whether the intervention would impact communication about sexuality between the participants (N=1,320). Parents completed a web-based interactive module in recruitment centers throughout Puerto Rico. Afterwards, parents could revisit the web-based workshop on their own. The process of accessing the workshop proved to be challenging. To explore why, we expanded our design and incorporated a qualitative component that included semi-structured telephone interviews to 23 randomly selected participants. After transcribing the interviews, we conducted a content analysis from which six thematic categories emerged: 1) perceived impact of the module in the parent-adolescent relationship, 2) communication between research team and parents, 3) platform to access the module, 4) assistance to complete the module, 5) remote access to the module, and 6) changes to the module for future implementations. Our findings will allow us to refine our intervention and offer recommendations to similar efforts using the Internet to increase their impact.

[ATTITUDES TOWARDS SEXUAL COMMUNICATION BETWEEN PARENTS AND ADOLESCENTS IN PUERTO RICO]

Communication between parents and adolescents about sexuality faces challenges due in part to sociocultural factors. This study aimed to document the attitudes of parents and adolescents about communication regarding sexuality. The results emanate from the baseline measurement of the Cuídalos Project. The presented data is part of a larger study that aimed to evaluate a web-based module to foster communications between parents and adolescents (13-17 years of age) on health-related issues. In this article, we present the baseline data in which participants answered questions on their comfort when talking about sexual health issues. The sample was composed of 458 parent/adolescent dyads (n=916). We carried out a frequency and central tendency analysis of the gathered data. The average age for adolescents was 15 years of age, and 15% were sexually active. Adolescents reported better attitudes towards sexual communication than parents. Still, parents understood that they shared sufficient information on sexuality related topics. Parents and adolescents reported some level of difficulty and lack of comfort when talking about specific prevention methods. Results highlight the need to incorporate parents into interventions geared for adolescents and their sexual health. In Puerto Rico, it is necessary to develop programs that aim to reduce high-risk sexual activity among adolescents.

Using a Virtual Environment to Deliver Evidence-Based Interventions: The Facilitator's Experience

Background

Evidence-based interventions (EBIs) have the potential to maximize positive impact on communities. However, despite the quantity and quality of EBIs for prevention, the need for formalized training and associated training-related expenses, such as travel costs, program materials, and input of personnel hours, pose implementation challenges for many community-based organizations. In this study, the community of inquiry (CoI) framework was used to develop the virtual learning environment to support the adaptation of the ¡Cuídate! (Take Care of Yourself!) Training of Facilitators curriculum (an EBI) to train facilitators from community-based organizations.

Objective

The purpose of this study was to examine the feasibility of adapting a traditional face-to-face facilitator training program for ¡Cuídate!, a sexual risk reduction EBI for Latino youth, for use in a multi-user virtual environment (MUVE). Additionally, two aims of the study were explored: the acceptability of the facilitator training and the level of the facilitators’ knowledge and self-efficacy to implement the training.

Methods

A total of 35 facilitators were trained in the virtual environment. We evaluated the facilitators' experience in the virtual training environment and determined if the learning environment was acceptable and supported the acquisition of learning outcomes. To this end, the facilitators were surveyed using a modified community of inquiry survey, with questions specific to the Second Life environment and an open-ended questionnaire. In addition, a comparison to face-to-face training was conducted using survey methods.

Results

Results of the community of inquiry survey demonstrated a subscale mean of 23.11 (SD 4.12) out of a possible 30 on social presence, a subscale mean of 8.74 (SD 1.01) out of a possible 10 on teaching presence, and a subscale mean of 16.69 (SD 1.97) out of a possible 20 on cognitive presence. The comparison to face-to-face training showed no significant differences in participants' ability to respond to challenging or sensitive questions (P=.50) or their ability to help participants recognize how Latino culture supports safer sex (P=.32). There was a significant difference in their knowledge of core elements and modules (P<.001). A total of 74% (26/35) of the Second Life participants did agree/strongly agree that they had the skills to deliver the ¡Cuídate! program.

Conclusions

The results showed that participants found the Second Life environment to be acceptable to the learners and supported an experience in which learners were able to acquire the knowledge and skills needed to deliver the curriculum.

The relationship of alcohol use and physical activity from an ecologic perspective

PURPOSE

To determine the predictors of alcohol use (AU) and their relationship to physical activity (PA) among adolescents and to examine differences among groups using an ecologic model.

DESIGN AND METHODS

Cross sectional secondary analysis of 11,432 adolescents (51% 8th and 49% 10th graders).

RESULTS

Attitude, perception of risk, and academic performance were the strongest predictors of AU. PA did not moderate, but it had a positive influence on AU in 10th graders. Differences were observed among groups.

PRACTICE IMPLICATIONS

Nurses can address individual and school factors in an effort to decrease AU and guide nurses in creating an individualized plan.

Research agenda for Hispanics in the United States: a nursing perspective

Heterogeneity is perhaps the most salient characteristic that defines Hispanic populations of the United States. Hispanic populations include native-born, migrant, and immigrant peoples with distinctive national origins and regional settlement patterns. This multigenerational migratory and social adjustment process has produced important cultural variations within and among the respective Hispanic ethnic groups. Moreover, the demographic structure of Hispanic populations is also varied and complex. These historical, demographic, and sociocultural features shape the health and disease experience of Hispanics. As expected, respective Hispanic ethnic groups vary in health status and have differing needs for health services. This article provides demographic background, a historical perspective, a synthesis of Hispanic issues identified in major and authoritative government documents, summary on the state-of-the-science for nursing research with Hispanics, and recommendations for future nursing research directions.