An Electronic Teen Questionnaire, the eTeenQ, for Risk Behavior Screening During Adolescent Well Visits in an Integrated Health System: Development and Pilot Implementation

Background

Screening for risk behaviors is a routine and essential component of adolescent preventive health visits. Early identification of risks can inform targeted counseling and care. If stored in discrete fields in the electronic health record (EHR), adolescent screening data can also be used to understand risk behaviors across a clinic or health system or to support quality improvement projects.

Objective

Goals of this pilot study were to adapt and implement an existing paper adolescent risk behavior screening tool for use as an electronic data capture tool (the eTeenQ), to evaluate acceptance of the eTeenQ, and to describe the prevalence of the selected risk behaviors reported through the eTeenQ.

Methods

The multidisciplinary project team applied an iterative process to develop the 29-item eTeenQ. Two unique data entry forms were created with attention to (1) user interface and user experience, (2) the need to maintain patient privacy, and (3) the potential to transmit and store data for future use in clinical care and research. Three primary care clinics within a large health system piloted the eTeenQ from August 17, 2020, to August 27, 2021. During preventive health visits for adolescents aged 12 to 18 years, the eTeenQ was completed on tablets and responses were converted to a provider display for teens and providers to review together. Responses to the eTeenQ were stored in a REDCap (Research Electronic Data Capture; Vanderbilt University) database, and for patients who agreed, responses were transferred to an EHR flowsheet. Responses to selected eTeenQ questions are reported for those consenting to research. At the conclusion of the pilot, the study team conducted semistructured interviews with providers and staff regarding their experience using the eTeenQ.

Results

Among 2816 adolescents with well visits, 2098 (74.5%) completed the eTeenQ. Of these, 1811 (86.3%) agreed to store responses in the EHR. Of 1632 adolescents (77.8% of those completing the eTeenQ) who consented for research and remained eligible, 1472 (90.2%) reported having an adult they can really talk to and 1510 (92.5%) reported feeling safe in their community, yet 401 (24.6%) reported someone they lived with had a gun and 172 (10.5%) reported having had a stressful or scary event that still bothered them. In addition, 157 (9.6%) adolescents reported they were or wondered if they were gay, lesbian, bisexual, pansexual, asexual, or other, and 43 (2.6%) reported they were or wondered if they were transgender or gender diverse. Of 11 staff and 7 providers completing interviews, all felt that the eTeenQ improved confidentiality and willingness among adolescents to answer sensitive questions. All 7 providers preferred the eTeenQ over the paper screening tool.

Conclusions

Electronic capture of adolescent risk behaviors is feasible in a busy clinic setting and well accepted among staff and clinicians. Most adolescents agreed for their responses to risk behavior screening to be stored in the EHR.

Unit Response and Costs in Web Versus Face-To-Face Data Collection: Comparison of Two Cross-sectional Health Surveys

Background

Potential is seen in web data collection for population health surveys due to its combined cost-effectiveness, implementation ease, and increased internet penetration. Nonetheless, web modes may lead to lower and more selective unit response than traditional modes, and this may increase bias in the measured indicators.

Objective

This research assesses the unit response and costs of a web study versus face-to-face (F2F) study.

Methods

Alongside the Belgian Health Interview Survey by F2F edition 2018 (BHISF2F; net sample used: 3316), a web survey (Belgian Health Interview Survey by Web [BHISWEB]; net sample used: 1010) was organized. Sociodemographic data on invited individuals was obtained from the national register and census linkages. Unit response rates considering the different sampling probabilities of both surveys were calculated. Logistic regression analyses examined the association between mode system and sociodemographic characteristics for unit nonresponse. The costs per completed web questionnaire were compared with the costs for a completed F2F questionnaire.

Results

The unit response rate is lower in BHISWEB (18.0%) versus BHISF2F (43.1%). A lower response rate was observed for the web survey among all sociodemographic groups, but the difference was higher among people aged 65 years and older (15.4% vs 45.1%), lower educated people (10.9% vs 38.0%), people with a non-Belgian European nationality (11.4% vs 40.7%), people with a non-European nationality (7.2% vs 38.0%), people living alone (12.6% vs 40.5%), and people living in the Brussels-Capital (12.2% vs 41.8%) region. The sociodemographic characteristics associated with nonresponse are not the same in the 2 studies. Having another European (OR 1.60, 95% CI 1.20-2.13) or non-European nationality (OR 2.57, 95% CI 1.79-3.70) compared to a Belgian nationality and living in the Brussels-Capital (OR 1.72, 95% CI 1.41-2.10) or Walloon (OR 1.47, 95% CI 1.15-1.87) regions compared to the Flemish region are associated with a higher nonresponse only in the BHISWEB study. In BHISF2F, younger people (OR 1.31, 95% CI 1.11-1.54) are more likely to be nonrespondents than older people, and this was not the case in BHISWEB. In both studies, lower educated people have a higher probability of being nonrespondent, but this effect is more pronounced in BHISWEB (low vs high education level: Web, OR 2.71, 95% CI 2.21-3.39 and F2F OR 1.70, 95% CI 1.48-1.95). The BHISWEB study had a considerable advantage; the cost per completed questionnaire was almost 3 times lower (€41 [US $48]) compared with F2F data collection (€111 [US $131]).

Conclusions

The F2F unit response rate was generally higher, yet for certain groups the difference between web and F2F was more limited. Web data collection has a considerable cost advantage. It is therefore worth experimenting with adaptive mixed-mode designs to optimize financial resources without increasing selection bias (eg, only inviting sociodemographic groups who are keener to participate online for web surveys while continuing to focus on increasing F2F response rates for other groups).

Life as an emergent phenomenon: studies from a large-scale boid simulation and web data

A large group with a special structure can become the mother of emergence. We discuss this hypothesis in relation to large-scale boid simulations and web data. In the boid swarm simulations, the nucleation, organization and collapse dynamics were found to be more diverse in larger flocks than in smaller flocks. In the second analysis, large web data, consisting of shared photos with descriptive tags, tended to group together users with similar tendencies, allowing the network to develop a core-periphery structure. We show that the generation rate of novel tags and their usage frequencies are high in the higher-order cliques. In this case, novelty is not considered to arise randomly; rather, it is generated as a result of a large and structured network. We contextualize these results in terms of adjacent possible theory and as a new way to understand collective intelligence. We argue that excessive information and material flow can become a source of innovation.This article is part of the themed issue 'Reconceptualizing the origins of life'.

Suicide note sentiment classification: a supervised approach augmented by web data

Objective:

To create a sentiment classification system for the Fifth i2b2/VA Challenge Track 2, which can identify thirteen subjective categories and two objective categories.

Design:

We developed a hybrid system using Support Vector Machine (SVM) classifiers with augmented training data from the Internet. Our system consists of three types of classification-based systems: the first system uses spanning n-gram features for subjective categories, the second one uses bag-of-n-gram features for objective categories, and the third one uses pattern matching for infrequent or subtle emotion categories. The spanning n-gram features are selected by a feature selection algorithm that leverages emotional corpus from weblogs. Special normalization of objective sentences is generalized with shallow parsing and external web knowledge. We utilize three sources of web data: the weblog of LiveJournal which helps to improve the feature selection, the eBay List which assists in special normalization of information and instructions categories, and the suicide project web which provides unlabeled data with similar properties as suicide notes.

Measurements:

The performance is evaluated by the overall micro-averaged precision, recall and F-measure.

Result:

Our system achieved an overall micro-averaged F-measure of 0.59. Happiness_peacefulness had the highest F-measure of 0.81. We were ranked as the second best out of 26 competing teams.

Conclusion:

Our results indicated that classifying fine-grained sentiments at sentence level is a non-trivial task. It is effective to divide categories into different groups according to their semantic properties. In addition, our system performance benefits from external knowledge extracted from publically available web data of other purposes; performance can be further enhanced when more training data is available.