Conducting a Community "Street Survey" to Inform an Obesity Intervention: The WE Project

Community-Partnered Research appraisal tool for conducting, reporting and assessing community-based research

Objective The aim of this study was to develop an appraisal tool to support and promote clear, accurate and transparent standards and consistency when conducting, reporting and assessing community-based research. Design Current recommendations for developing reporting guidelines was used with three key differences: (1) an analysis of existing guides, principles and published literature about community engagement, involvement and participation in research using situational and relational maps; (2) feedback and pilot-testing by a community-based research team; and (3) testing the utility and usability of the appraisal tool. Results After a series of iterative revisions, the resulting Community-Partnered Research (CPR) appraisal tool emerged into three products: an elaborate prospective format, a basic retrospective format, and a supplemental checklist format. All three versions of the CPR appraisal tool consist of 11 main question items with corresponding prompts aimed to facilitate awareness, accountability, and transparency about processes and practices employed by professional researchers and community co-researchers throughout four phases of research: (1) partnership and planning, (2) methods, (3) results and (4) sustainment. Conclusion We hope that introducing this tool will contribute to shifting individual and systematic processes and practices towards equitable partnerships, mutual trustworthiness and empowerment among professional researchers and community co-researchers and, in turn, improving the quality of co-created knowledge that benefits communities and creates social change.

Attention-Based Models for Classifying Small Data Sets Using Community-Engaged Research Protocols: Classification System Development and Validation Pilot Study

Background

Community-engaged research (CEnR) is a research approach in which scholars partner with community organizations or individuals with whom they share an interest in the study topic, typically with the goal of supporting that community’s well-being. CEnR is well-established in numerous disciplines including the clinical and social sciences. However, universities experience challenges reporting comprehensive CEnR metrics, limiting the development of appropriate CEnR infrastructure and the advancement of relationships with communities, funders, and stakeholders.

Objective

We propose a novel approach to identifying and categorizing community-engaged studies by applying attention-based deep learning models to human participants protocols that have been submitted to the university’s institutional review board (IRB).

Methods

We manually classified a sample of 280 protocols submitted to the IRB using a 3- and 6-level CEnR heuristic. We then trained an attention-based bidirectional long short-term memory unit (Bi-LSTM) on the classified protocols and compared it to transformer models such as Bidirectional Encoder Representations From Transformers (BERT), Bio + Clinical BERT, and Cross-lingual Language Model–Robustly Optimized BERT Pre-training Approach (XLM-RoBERTa). We applied the best-performing models to the full sample of unlabeled IRB protocols submitted in the years 2013-2019 (n>6000).

Results

Although transfer learning is superior, receiving a 0.9952 evaluation F1 score for all transformer models implemented compared to the attention-based Bi-LSTM (between 48%-80%), there were key issues with overfitting. This finding is consistent across several methodological adjustments: an augmented data set with and without cross-validation, an unaugmented data set with and without cross-validation, a 6-class CEnR spectrum, and a 3-class one.

Conclusions

Transfer learning is a more viable method than the attention-based bidirectional-LSTM for differentiating small data sets characterized by the idiosyncrasies and variability of CEnR descriptions used by principal investigators in research protocols. Despite these issues involving overfitting, BERT and the other transformer models remarkably showed an understanding of our data unlike the attention-based Bi-LSTM model, promising a more realistic path toward solving this real-world application.