Priority skills for equity-focused, evidence-based cancer control in community-based organizations: A group concept mapping analysis with academics and practitioners

Introduction

Community-based organizations (CBOs) are important equity-promoting delivery channels for evidence-based interventions (EBIs). However, CBO practitioners often cannot access needed support to build EBI skills. Additionally, the capacity-building literature is hindered by inconsistent definitions, limited use of validated measures, and an emphasis on the perspectives of EBI developers versus implementers. To address these gaps, we explored commonalities and differences between CBO practitioners and academics in conceptualizing and prioritizing core EBI skills.

Methods

We utilized Group Concept Mapping, a mixed-methods approach connecting qualitative data (e.g., regarding the range of critical EBI skills) and quantitative data (e.g., sorting and ranking data regarding unique skills) to create conceptual maps integrating perspectives from diverse participants. A total of 34 practitioners and 30 academics working with cancer inequities participated in the study.

Results

Participants nominated 581 core skills for EBI use, and our team (including practitioners and academics) identified 98 unique skills from this list. Participants sorted them into conceptual groups, yielding five clusters: (1) using data and evaluation, (2) selecting and adapting EBIs, (3) connecting with community members, (4) building diverse and equitable partnerships, and (5) managing EBI implementation. The ordering of importance and presence of skill clusters were similar across groups. Overall, importance was rated higher than presence, suggesting capacity gaps.

Conclusions

There are helpful commonalities between practitioners' and academics' views of core EBI skills in CBOs and apparent capacity gaps. However, underlying patterns suggest that differences between the groups' perceptions warrant further exploration.

Exploring Characteristics of Academic General Surgery Residency Applicants: A Group Concept-Mapping Approach

OBJECTIVE

Holistic review, which emphasizes qualitative attributes over objective measures, has been proposed as a method for selecting candidates for surgical residency in order to improve diversity in graduate medical education, and, ultimately, the field of surgery. This study seeks to articulate desirable traits of applicants as a first-step in standardizing the holistic review process.

DESIGN

Using Group Concept Mapping, a web-based mixed-methods participatory research methodology, residency selection committee members were asked to 1) list desirable characteristics of applicants, 2) group these into categories, 3) rate their importance to academic/clinical success on a 5-point Likert scale (1 = not at all important, 5 = extremely important), and 4) rate the degree to which each characteristic is feasible to assess on a 3-point Likert scale (1 = not at all feasible, 3 = very feasible). Grouped characteristics submitted to hierarchical cluster analysis depicted committee's consensus about desirable qualities/criteria for applicants. Bivariate scatter-plots and pattern-matching graphics demonstrated which of these criteria were most important and reliably assessed.

SETTING

A single academic general surgery residency training program in Western Pennsylvania.

PARTICIPANTS

Members of the selection committee for the UPMC General Surgery Residency program who had participated in at least 1 prior cycle of applicant selection.

RESULTS

Desirable characteristics of highly qualified applicants into an academic general surgery residency were clustered into domains of 1) scholarly work and research, 2) grades/formal assessments, 3) program fit, 4) behavioral assets, and 5) aspiration. Behavioral assets, which was felt to be the most important to clinical and academic success were considered to be the least feasible to reliably assess. Within this domain, initiative, being self-motivated, intellectual curiosity, work ethic, communication skills, maturity and self-awareness, and thoughtfulness were viewed as most frequently reliably assessed from the application and interview process.

CONCLUSIONS

High quality applicants possess several behavioral assets that faculty deem are important to academic and clinical success. Adapting validated metrics for assessing these assets, may provide a solution for addressing subjectivity and other challenges scrutinized by critics of holistic review.

A large-scale, geographical approach to using group concept mapping for planning: The Teen Opinions Count (TOC) project

The use of group concept mapping (GCM) for large-scale, multi-site planning has been limited. The Teens Opinions Count (TOC) project utilized the group concept mapping methodology to develop a framework for after school time program development for teens in 16 counties in the Southeast Michigan and Western New York regions. This extensive planning effort demonstrated a large-scale, geographical approach to the implementation of GCM, where multiple sites (in this case counties) simultaneously engaged in independent processes on the same topic, with the purpose of producing separate locale-specific conceptualizations. TOC was the largest GCM project ever conducted in terms of the number of participants and volume of data submitted. This report details GCM recruitment, data collection, data compilation, and analysis of ideas from over 20,000 teens and 2,000 adults. Although implementation was conducted at the county level, many of the separate county-level GCM processes were larger than what is typically encountered in the literature. Operationalizing GCM at such a scale required significant planning, design, communication, and data management solutions. The extensive data collection effort required the need for creative and innovative procedures to engage teens and process the sheer volume of data. The lessons learned are discussed and suggestions for future applications offered.

Using Concept Mapping within a Community-Academic Partnership to Examine Obesity among Mexican Immigrants

Background

Weight gain is common following migration to a new country. Mexican immigrants have a disparate prevalence of overweight/obesity and food insecurity. Social stressors, such as unemployment, discrimination, and the threat of deportation, may fuel both food insecurity and weight gain in this population.

Objectives

We sought to (1) examine community-defined causes and correlates of obesity among Mexican-Americans; (2) determine how current social stressors, policies, and programs impact food insecurity and obesity; and (3) identify community-defined priorities for preventive interventions.

Methods

Group concept mapping (GCM) was used in a community-academic partnership (CAP) to describe the factors contributing to weight gain and obesity among Mexican immigrant families. Activities included community brainstorming, sorting and rating, multivariate statistical analysis, and community interpretation of results.

Results

Eighty statements were generated in the brainstorming sessions. These statements were sorted into nine clusters, which were organized into three regions: (1) intrapersonal factors; (2) community-level factors; and (3) social policy-related barriers. Statements reflecting the impact of immigration-related stressors were found in all three regions, addressing participants' fears of deportation, and the prioritization of resources away from healthy eating, resulting in food insecurity. Community members identified five priority areas for intervention planning: (1) lack of exercise; (2) lack of knowledge of a healthy diet; (3) expense of healthy foods; (4) "junk" food; and (5) stress management.

Conclusions

Results suggest high levels of social stress are contributing to food insecurity and obesity among Mexican immigrant families. Areas identified for intervention planning reflect the need for a multifaceted approach toward obesity prevention.

Concept mapping-An effective method for identifying diversity and congruity in cognitive style

This paper investigates the effects of cognitive style for decision making on the behaviour of participants in different phases of the group concept mapping process (GCM). It is argued that cognitive style should be included directly in the coordination of the GCM process and not simply considered as yet another demographic variable. The cognitive styles were identified using the Kirton Adaption-Innovation Inventory, which locates each person's style on a continuum ranging from very adaptive to very innovative. Cognitive style could explain diversity in the participants' behaviour in different phases of the GCM process. At the same time, the concept map as a group's common cognitive construct can consolidate individual differences and serves as a tool for managing diversity in groups of participants. Some of the results were that: (a) the more adaptive participants generated ideas that fit to a particular, well-established and consensually agreed paradigm, frame of reference, theory or practice; (b) the more innovative participants produced ideas that were more general in scope and required changing a settled structure (paradigm, frame of reference, theory or practice); and (c) the empirical comparison of the map configurations through Procrustes analysis indicated a strong dissimilarity between cognitive styles.

The use of concept mapping in measurement development and evaluation: Application and future directions

The past decade has seen an increase of measurement development research in social and health sciences that featured the use of concept mapping as a core technique. The purpose, application, and utility of concept mapping have varied across this emerging literature. Despite the variety of uses and range of outputs, little has been done to critically review how researchers have approached the application of concept mapping in the measurement development and evaluation process. This article focuses on a review of the current state of practice regarding the use of concept mapping as methodological tool in this process. We systematically reviewed 23 scale or measure development and evaluation studies, and detail the application of concept mapping in the context of traditional measurement development and psychometric testing processes. Although several limitations surfaced, we found several strengths in the contemporary application of the method. We determined concept mapping provides (a) a solid method for establishing content validity, (b) facilitates researcher decision-making, (c) insight into target population perspectives that are integrated a priori, and (d) a foundation for analytical and interpretative choices. Based on these results, we outline how concept mapping can be situated in the measurement development and evaluation processes for new instrumentation.

Evaluating protocol lifecycle time intervals in HIV/AIDS clinical trials

BACKGROUND

Identifying efficacious interventions for the prevention and treatment of human diseases depends on the efficient development and implementation of controlled clinical trials. Essential to reducing the time and burden of completing the clinical trial lifecycle is determining which aspects take the longest, delay other stages, and may lead to better resource utilization without diminishing scientific quality, safety, or the protection of human subjects.

PURPOSE

In this study, we modeled time-to-event data to explore relationships between clinical trial protocol development and implementation times, as well as to identify potential correlates of prolonged development and implementation.

METHODS

We obtained time interval and participant accrual data from 111 interventional clinical trials initiated between 2006 and 2011 by National Institutes of Health's HIV/AIDS Clinical Trials Networks. We determined the time (in days) required to complete defined phases of clinical trial protocol development and implementation. Kaplan-Meier estimates were used to assess the rates at which protocols reached specified terminal events, stratified by study purpose (therapeutic, prevention) and phase group (pilot/phase I, phase II, and phase III/IV). We also examined several potential correlates to prolonged development and implementation intervals.

RESULTS

Even though phase grouping did not determine development or implementation times of either therapeutic or prevention studies, overall we observed wide variation in protocol development times. Moreover, we detected a trend toward phase III/IV therapeutic protocols exhibiting longer developmental (median 2½ years) and implementation times (>3 years). We also found that protocols exceeding the median number of days for completing the development interval had significantly longer implementation.

LIMITATIONS

The use of a relatively small set of protocols may have limited our ability to detect differences across phase groupings. Some timing effects present for a specific study phase may have been masked by combining protocols into phase groupings. Presence of informative censoring, such as withdrawal of some protocols from development if they began showing signs of lost interest among investigators, complicates interpretation of Kaplan-Meier estimates. Because this study constitutes a retrospective examination over an extended period of time, it does not allow for the precise identification of relative factors impacting timing.

CONCLUSION

Delays not only increase the time and cost to complete clinical trials but they also diminish their usefulness by failing to answer research questions in time. We believe that research analyzing the time spent traversing defined intervals across the clinical trial protocol development and implementation continuum can stimulate business process analyses and re-engineering efforts that could lead to reductions in the time from clinical trial concept to results, thereby accelerating progress in clinical research.

Assessing the challenges of multi-scope clinical research sites: an example from NIH HIV/AIDS clinical trials networks

RATIONALE, AIMS, AND OBJECTIVES

Large-scale, multi-network clinical trials are seen as a means for efficient and effective utilization of resources with greater responsiveness to new discoveries. Formal structures instituted within the National Institutes of Health (NIH) HIV/AIDS Clinical Trials facilitate collaboration and coordination across networks and emphasize an integrated approach to HIV/AIDS vaccine, prevention and therapeutics clinical trials. This study examines the joint usage of clinical research sites as means of gaining efficiency, extending capacity, and adding scientific value to the networks.

METHODS

A semi-structured questionnaire covering eight clinical management domains was administered to 74 (62% of sites) clinical site coordinators at single- and multi-network sites to identify challenges and efficiencies related to clinical trials management activities and coordination with multi-network units.

RESULTS

Overall, respondents at multi-network sites did not report more challenges than single-network sites, but did report unique challenges to overcome including in the areas of study prioritization, community engagement, staff education and training, and policies and procedures. The majority of multi-network sites reported that such affiliations do allow for the consolidation and cost-sharing of research functions. Suggestions for increasing the efficiency or performance of multi-network sites included streamlining standards and requirements, consolidating protocol activation methods, using a single cross-network coordinating centre, and creating common budget and payment mechanisms.

CONCLUSIONS

The results of this assessment provide important information to consider in the design and management of multi-network configurations for the NIH HIV/AIDS Clinical Trials Networks, as well as others contemplating and promoting the concept of multi-network settings.

Modeling the dissemination and uptake of clinical trials results

A select set of highly cited publications from the National Institutes of Health (NIH) HIV/AIDS Clinical Trials Networks was used to illustrate the integration of time interval and citation data, modeling the progression, dissemination, and uptake of primary research findings. Following a process marker approach, the pace of initial utilization of this research was measured as the time from trial conceptualization, development and implementation, through results dissemination and uptake. Compared to earlier studies of clinical research, findings suggest that select HIV/AIDS trial results are disseminated and utilized relatively rapidly. Time-based modeling of publication results as they meet specific citation milestones enabled the observation of points at which study results were present in the literature summarizing the evidence in the field. Evaluating the pace of clinical research, results dissemination, and knowledge uptake in synthesized literature can help establish realistic expectations for the time course of clinical trials research and their relative impact toward influencing clinical practice.

Community-researcher partnerships at NIAID HIV/AIDS clinical trials sites: insights for evaluation and enhancement

BACKGROUND

Community engagement has been a cornerstone of National Institute of Allergy and Infectious Diseases (NIAID)'s HIV/AIDS clinical trials programs since 1990. Stakeholders now consider this critical to success, hence the impetus to develop evaluation approaches.

OBJECTIVES

The purpose was to assess the extent to which community advisory boards (CABs) at HIV/AIDS trials sites are being integrated into research activities.

METHODS

CABs and research staff (RS) at NIAID research sites were surveyed for how each viewed (a) the frequency of activities indicative of community involvement, (b) the means for identifying, prioritizing, and supporting CAB needs, and (c) mission and operational challenges.

RESULTS

Overall, CABs and RS share similar views about the frequency of community involvement activities. Cluster analysis reveals three groups of sites based on activity frequency ratings, including a group notable for CAB-RS discordance.

CONCLUSIONS

Assessing differences between community and researcher perceptions about the frequency of and challenges posed by specific engagement activities may prove useful in developing evaluation tools for assessing community engagement in collaborative research settings.

Quality and rigor of the concept mapping methodology: a pooled study analysis

The use of concept mapping in research and evaluation has expanded dramatically over the past 20 years. Researchers in academic, organizational, and community-based settings have applied concept mapping successfully without the benefit of systematic analyses across studies to identify the features of a methodologically sound study. Quantitative characteristics and estimates of quality and rigor that may guide for future studies are lacking. To address this gap, we conducted a pooled analysis of 69 concept mapping studies to describe characteristics across study phases, generate specific indicators of validity and reliability, and examine the relationship between select study characteristics and quality indicators. Individual study characteristics and estimates were pooled and quantitatively summarized, describing the distribution, variation and parameters for each. In addition, variation in the concept mapping data collection in relation to characteristics and estimates was examined. Overall, results suggest concept mapping yields strong internal representational validity and very strong sorting and rating reliability estimates. Validity and reliability were consistently high despite variation in participation and task completion percentages across data collection modes. The implications of these findings as a practical reference to assess the quality and rigor for future concept mapping studies are discussed.

Opportunity for collaboration: a conceptual model of success in tobacco control and cancer prevention

INTRODUCTION

Collaborations between cancer prevention and tobacco control programs can leverage scarce resources to address noncommunicable diseases globally, but barriers to cooperation and actual collaboration are substantial. To foster collaboration between cancer prevention and tobacco control programs, the Global Health Partnership conducted research to identify similarities and differences in how the 2 programs viewed program success.

METHODS

Using concept mapping, cancer prevention and tobacco control experts generated statements describing the components of a successful cancer prevention or tobacco control program and 33 participants sorted and rated the final 99 statements. Multidimensional scaling analysis with a 2-dimensional solution was used to identify an 8-cluster conceptual map of program success. We calculated Pearson correlation coefficients for all 99 statements to compare the item-level ratings of both groups and used t tests to compare the mean importance of ratings assigned to each cluster.

RESULTS

Eight major clusters of success were identified: 1) advocacy and persuasion, 2) building sustainability, 3) partnerships, 4) readiness and support, 5) program management fundamentals, 6) monitoring and evaluation, 7) utilization of evidence, and 8) implementation. We found no significant difference between the maps created by the 2 groups and only 1 mean difference for the importance ratings for 1 of the clusters: cancer prevention experts rated partnerships as more important to program success than did tobacco control experts.

CONCLUSION

Our findings are consistent with those of research documenting the necessary components of successful programs and the similarities between cancer prevention and tobacco control. Both programs value the same strategies to address a common risk factor: tobacco use. Identifying common ground between these 2 research and practice communities can benefit future collaborations at the local, state, tribal, national, and international levels, and inform the broader discussion on resource sharing among other organizations whose mission focuses on noncommunicable diseases.

Using concept mapping to describe sources of information for public health and school nursing practice

Concept-mapping methodology was used to construct a conceptual model of information access for public health and school nursing practice. Concept mapping is an integrated mixed-methods approach that combines familiar group processes with multivariate statistical analyses to represent a group’s understanding of a phenomenon. In this project, 36 public health and school nurses brainstormed 207 unique information sources that were collated and reduced to 57 sources to allow sorting of the statements. A subgroup of 25 public health ( n = 16) and school nurses ( n = 9) sorted and rated the 57 statements to generate an eight-cluster concept map. The two-dimensional concept map revealed a sophisticated multivariate framework of information access occurring within an intricate, interrelated network of human and data-driven sources that intersect with community and professionally focused systems. Clusters of information sources included those from community groups, healthcare providers, professional organisations, educational tools, state resources, online resources, surveillance data and government-based resources. Ratings of importance, credibility and usefulness by public health and school nurses revealed overall agreement; however, some important differences of specific sources were noted. More complex than previously described, the map represents sources, means of access and systems of health information available to community-based nurses that influence their practice.

Evaluating research and impact: a bibliometric analysis of research by the NIH/NIAID HIV/AIDS clinical trials networks

Evaluative bibliometrics uses advanced techniques to assess the impact of scholarly work in the context of other scientific work and usually compares the relative scientific contributions of research groups or institutions. Using publications from the National Institute of Allergy and Infectious Diseases (NIAID) HIV/AIDS extramural clinical trials networks, we assessed the presence, performance, and impact of papers published in 2006–2008. Through this approach, we sought to expand traditional bibliometric analyses beyond citation counts to include normative comparisons across journals and fields, visualization of co-authorship across the networks, and assess the inclusion of publications in reviews and syntheses. Specifically, we examined the research output of the networks in terms of the a) presence of papers in the scientific journal hierarchy ranked on the basis of journal influence measures, b) performance of publications on traditional bibliometric measures, and c) impact of publications in comparisons with similar publications worldwide, adjusted for journals and fields. We also examined collaboration and interdisciplinarity across the initiative, through network analysis and modeling of co-authorship patterns. Finally, we explored the uptake of network produced publications in research reviews and syntheses. Overall, the results suggest the networks are producing highly recognized work, engaging in extensive interdisciplinary collaborations, and having an impact across several areas of HIV-related science. The strengths and limitations of the approach for evaluation and monitoring research initiatives are discussed.

The use of concept mapping for scale development and validation in evaluation

Evaluators often make key decisions about what content to include when designing new scales. However, without clear conceptual grounding, there is a risk these decisions may compromise the scale's validity. Techniques such as concept mapping are available to evaluators for the specification of conceptual frameworks, but have not been used as a fully integrated part of scale development. As part of a multi-site evaluation of family support programs, we integrated concept mapping with traditional scale-development processes to strengthen the creation of a scale for inclusion in an evaluation instrument. Using concept mapping, we engaged staff and managers in the development of a framework of intended benefits of program participation and used the information to systematically select the scale's content. The psychometric characteristics of the scale were then formally assessed using a sample of program participants. The implications of the approach for supporting construct validity, inclusion of staff and managers, and theory-driven evaluation are discussed.