Collective intelligence for translational medicine: Crowdsourcing insights and innovation from an interdisciplinary biomedical research community

Exploring Novel Innovation Strategies to Close a Technology Gap in Neurosurgery: The HORAO Crowdsourcing Campaign (Preprint)

BACKGROUND

Scientific research is typically performed by expert individuals or groups who investigate potential solutions in a sequential manner. Given the current worldwide exponential increase in technical innovations, potential solutions for any new problem might already exist, even though they were developed to solve a different problem. Therefore, in crowdsourcing ideation, a research question is explained to a much larger group of individuals beyond the specialist community to obtain a multitude of diverse, outside-the-box solutions. These are then assessed in parallel by a group of experts for their capacity to solve the new problem. The 2 key problems in brain tumor surgery are the difficulty of discerning the exact border between a tumor and the surrounding brain, and the difficulty of identifying the function of a specific area of the brain. Both problems could be solved by a method that visualizes the highly organized fiber tracts within the brain; the absence of fibers would reveal the tumor, whereas the spatial orientation of the tracts would reveal the area's function. To raise awareness about our challenge of developing a means of intraoperative, real-time, noninvasive identification of fiber tracts and tumor borders to improve neurosurgical oncology, we turned to the crowd with a crowdsourcing ideation challenge.

OBJECTIVE

Our objective was to evaluate the feasibility of a crowdsourcing ideation campaign for finding novel solutions to challenges in neuroscience. The purpose of this paper is to introduce our chosen crowdsourcing method and discuss it in the context of the current literature.

METHODS

We ran a prize-based crowdsourcing ideation competition called HORAO on the commercial platform HeroX. Prize money previously collected through a crowdfunding campaign was offered as an incentive. Using a multistage approach, an expert jury first selected promising technical solutions based on broad, predefined criteria, coached the respective solvers in the second stage, and finally selected the winners in a conference setting. We performed a postchallenge web-based survey among the solvers crowd to find out about their backgrounds and demographics.

RESULTS

Our web-based campaign reached more than 20,000 people (views). We received 45 proposals from 32 individuals and 7 teams, working in 26 countries on 4 continents. The postchallenge survey revealed that most of the submissions came from single solvers or teams working in engineering or the natural sciences, with additional submissions from other nonmedical fields. We engaged in further exchanges with 3 out of the 5 finalists and finally initiated a successful scientific collaboration with the winner of the challenge.

CONCLUSIONS

This open innovation competition is the first of its kind in medical technology research. A prize-based crowdsourcing ideation campaign is a promising strategy for raising awareness about a specific problem, finding innovative solutions, and establishing new scientific collaborations beyond strictly disciplinary domains.

Translational researchers' training and development needs, preferences, and barriers: A survey in a National Institute for Health Research Biomedical Research Centre in the United Kingdom

The objective was to identify translational researchers’ training and development needs, preferences, and barriers to attending training. This cross‐sectional study involved an online questionnaire survey. The research population comprised a convenience sample of translational researchers and support staff (N = 798) affiliated with the National Institute for Health Research Oxford Biomedical Research Centre. The response rate was 24%. Of 189 respondents, 114 were women (60%) and 75 were men (40%). The respondents were mainly research scientists (31%), medical doctors and dentists (17%), and research nurses and midwives (16%). Many of the respondents had attended at least one training course in the last year (68%). Training in statistics and data analysis was the most common training received (20%). Leadership training was the most wanted training (25%). Morning was the most preferred time of training (60%). Half a day was the ideal duration of a training course (41%). The main teaching hospital site was the most preferred location of training (46%). An interactive workshop was the most favored delivery style of training (52%). Most common barriers to attending training were the lack of time (31%), work (21%) and clinical commitments (19%), and family and childcare responsibilities (14%). Some differences in training needs, preferences, and barriers were found by gender and role, though these were not statistically significant. Translational researchers want short, easily accessible, and interactive training sessions during the working day. The training needs, preferences, and barriers to attending training need to be considered while developing inclusive training programs in biomedical research settings.

Collaboration between Government and Research Community to Respond to COVID-19: Israel’s Case

Triggered by the COVID-19 crisis, Israel’s Ministry of Health (MoH) held a virtual datathon based on deidentified governmental data. Organized by a multidisciplinary committee, Israel’s research community was invited to offer insights to help solve COVID-19 policy challenges. The Datathon was designed to develop operationalizable data-driven models to address COVID-19 health policy challenges. Specific relevant challenges were defined and diverse, reliable, up-to-date, deidentified governmental datasets were extracted and tested. Secure remote-access research environments were established. Registration was open to all citizens. Around a third of the applicants were accepted, and they were teamed to balance areas of expertise and represent all sectors of the community. Anonymous surveys for participants and mentors were distributed to assess usefulness and points for improvement and retention for future datathons. The Datathon included 18 multidisciplinary teams, mentored by 20 data scientists, 6 epidemiologists, 5 presentation mentors, and 12 judges. The insights developed by the three winning teams are currently considered by the MoH as potential data science methods relevant for national policies. Based on participants’ feedback, the process for future data-driven regulatory responses for health crises was improved. Participants expressed increased trust in the MoH and readiness to work with the government on these or future projects.

Facilitators and Barriers of Teachers’ Use of Effective Classroom Management Strategies for Students with ADHD: A Model Analysis Based on Teachers’ Perspectives

Students with attention-deficit/hyperactivity disorder (ADHD) often experience school-related difficulties. Although evidence-based classroom management strategies (CMS) are known to alleviate such problems, they are rarely implemented. The current study examined whether a path model including variables influencing the use of effective CMS developed by top-down methods can be replicated utilizing an open science method. An extended model including class size and experience with children with ADHD was also calculated. We further explored prominent implementation barriers. N = 336 in-service teachers completed an online survey. Perceived effectiveness, training on ADHD, perceived disruption, and affiliation with primary/special educational needs schools were important variables associated with the use of CMS. While class size was not correlated with the use of CMS, experience mediated by training revealed an indirect association with it. Class size, lack of time, and many students with disabilities were the most frequently reported implementation barriers. The implementation of effective CMS could thus be mainly enhanced by improving how effectiveness is perceived and by engaging teachers in ADHD-specialized training. Preparing teachers in how to cope with potential barriers should also be considered.

A Hybrid In Silico and Tumor-on-a-Chip Approach to Model Targeted Protein Behavior in 3D Microenvironments

Simple Summary

Engineered proteins possess a great therapeutic potential, but the development of such therapies is impeded during preclinical studies by the lack of in vitro models that accurately simulate the human physiology. Animal models, on the other hand, also have difficulties predicting human responses, and are ethically concerning. In this study, we employed a hybrid approach where we combined mathematical modeling with 3D in vitro models that mimic aspects of the tumor microenvironment, in order to simulate the delivery of therapeutic proteins targeting cancer cells and to predict the biological activity. By cross-comparing simulated and experimental data from 3D models, we were able to correctly predict the best dose needed to deliver toxic proteins specifically to tumor cells, while leaving the surrounding non-tumor cells untouched. This study shows the potential of combining computational approaches with novel in vitro models to advance the development of protein therapeutics.

Abstract

To rationally improve targeted drug delivery to tumor cells, new methods combining in silico and physiologically relevant in vitro models are needed. This study combines mathematical modeling with 3D in vitro co-culture models to study the delivery of engineered proteins, called designed ankyrin repeat proteins (DARPins), in biomimetic tumor microenvironments containing fibroblasts and tumor cells overexpressing epithelial cell adhesion molecule (EpCAM) or human epithelial growth factor receptor (HER2). In multicellular tumor spheroids, we observed strong binding-site barriers in combination with low apparent diffusion coefficients of 1 µm²·s⁻¹ and 2 µm² ·s⁻¹ for EpCAM- and HER2-binding DARPin, respectively. Contrasting this, in a tumor-on-a-chip model for investigating delivery in real-time, transport was characterized by hindered diffusion as a consequence of the lower local tumor cell density. Finally, simulations of the diffusion of an EpCAM-targeting DARPin fused to a fragment of Pseudomonas aeruginosa exotoxin A, which specifically kills tumor cells while leaving fibroblasts untouched, correctly predicted the need for concentrations of 10 nM or higher for extensive tumor cell killing on-chip, whereas in 2D models picomolar concentrations were sufficient. These results illustrate the power of combining in vitro models with mathematical modeling to study and predict the protein activity in complex 3D models.

The clinical and translational research activities at the INT - IRCCS "Fondazione Pascale" cancer center (Naples, Italy) during the COVID-19 pandemic

COVID-19 pandemic following the outbreak in China and Western Europe, where it finally lost the momentum, is now devastating North and South America. It has not been identified the reason and the molecular mechanisms of the two different patterns of the pulmonary host responses to the virus from a minimal disease in young subjects to a severe distress syndrome (ARDS) in older subjects, particularly those with previous chronic diseases (including diabetes) and cancer. The Management of the Istituto Nazionale Tumori - IRCCS "Fondazione Pascale" in Naples (INT-Pascale), along with all Health professionals decided not to interrupt the treatment of those hospitalized and to continue, even if after a careful triage in order not to allow SARS-CoV-2 positive subjects to access, to take care of cancer patients with serious conditions. Although very few (n = 3) patients developed a symptomatic COVID-19 and required the transfer to a COVID-19 area of the Institute, no patients died during the hospitalization and completed their oncology treatment. Besides monitoring of the patients, all employees of the Institute (physicians, nurses, researchers, lawyers, accountants, gatekeepers, guardians, janitors) have been tested for a possible exposure. Personnel identified as positive, has been promptly subjected to home quarantine and subdued to health surveillance. One severe case of respiratory distress has been reported in a positive employees and one death of a family member. Further steps to home monitoring of COVID-19 clinical course have been taken with the development of remote Wi-Fi connected digital devices for the detection of early signs of respiratory distress, including heart rate and oxygen saturation.In conclusion cancer care has been performed and continued safely also during COVID-19 pandemic and further remote home strategies are in progress to ensure the appropriate monitoring of cancer patients.

Crowdsourcing in health and medical research: a systematic review

Background

Crowdsourcing is used increasingly in health and medical research. Crowdsourcing is the process of aggregating crowd wisdom to solve a problem. The purpose of this systematic review is to summarize quantitative evidence on crowdsourcing to improve health.

Methods

We followed Cochrane systematic review guidance and systematically searched seven databases up to September 4th 2019. Studies were included if they reported on crowdsourcing and related to health or medicine. Studies were excluded if recruitment was the only use of crowdsourcing. We determined the level of evidence associated with review findings using the GRADE approach.

Results

We screened 3508 citations, accessed 362 articles, and included 188 studies. Ninety-six studies examined effectiveness, 127 examined feasibility, and 37 examined cost. The most common purposes were to evaluate surgical skills (17 studies), to create sexual health messages (seven studies), and to provide layperson cardio-pulmonary resuscitation (CPR) out-of-hospital (six studies). Seventeen observational studies used crowdsourcing to evaluate surgical skills, finding that crowdsourcing evaluation was as effective as expert evaluation (low quality). Four studies used a challenge contest to solicit human immunodeficiency virus (HIV) testing promotion materials and increase HIV testing rates (moderate quality), and two of the four studies found this approach saved money. Three studies suggested that an interactive technology system increased rates of layperson initiated CPR out-of-hospital (moderate quality). However, studies analyzing crowdsourcing to evaluate surgical skills and layperson-initiated CPR were only from high-income countries. Five studies examined crowdsourcing to inform artificial intelligence projects, most often related to annotation of medical data. Crowdsourcing was evaluated using different outcomes, limiting the extent to which studies could be pooled.

Conclusions

Crowdsourcing has been used to improve health in many settings. Although crowdsourcing is effective at improving behavioral outcomes, more research is needed to understand effects on clinical outcomes and costs. More research is needed on crowdsourcing as a tool to develop artificial intelligence systems in medicine.

Trial registration

PROSPERO: CRD42017052835. December 27, 2016.

Global Consensus From Clinicians Regarding Low Back Pain Outcome Indicators for Older Adults: Pairwise Wiki Survey Using Crowdsourcing

BACKGROUND

Low back pain (LBP) is one of the most debilitating conditions among older adults. Unfortunately, existing LBP outcome questionnaires are not adapted for specific circumstances related to old age, which may make these measures less than ideal for evaluating LBP in older adults.

OBJECTIVE

To explore the necessity of developing age-specific outcome measures, crowdsourcing was conducted to solicit opinions from clinicians globally.

METHODS

Clinicians around the world voted and/or prioritized various LBP outcome indicators for older adults on a pairwise wiki survey website. Seven seed outcome indicators were posted for voting while respondents were encouraged to suggest new indicators for others to vote/prioritize. The website was promoted on the social media of various health care professional organizations. An established algorithm calculated the mean scores of all ideas. A score >50 points means that the idea has >50% probability of beating another randomly presented indicator.

RESULTS

Within 42 days, 128 respondents from 6 continents cast 2466 votes and proposed 14 ideas. Indicators pertinent to improvements of physical functioning and age-related social functioning scored >50 while self-perceived reduction of LBP scored 32.

CONCLUSIONS

This is the first crowdsourcing study to address LBP outcome indicators for older adults. The study noted that age-specific outcome indicators should be integrated into future LBP outcome measures for older adults. Future research should solicit opinions from older patients with LBP to develop age-specific back pain outcome measures that suit clinicians and patients alike.

What Do Our Patients Truly Want? Conjoint Analysis of an Aesthetic Plastic Surgery Practice Using Internet Crowdsourcing

BACKGROUND

What do patients want when looking for an aesthetic surgeon? When faced with attributes like reputation, years in practice, testimonials, photos, and pricing, which is more valuable? Moreover, are attributes procedure-specific? Currently, inadequate evidence exists on which attributes are most important to patients, and to our knowledge, none on procedure-specific preferences.

OBJECTIVES

First, to determine the most important attributes to breast augmentation, combined breast/abdominal surgery, and facelift patients using conjoint analysis. Second, to test the conjoint using an internet crowdsourcing service (Amazon Mechanical Turk [MTurk]).

METHODS

Anonymous university members were asked, via mass electronic survey, to pick a surgeon for facelift surgery based on five attributes. Attribute importance and preference was calculated. Once pre-tested, the facelift, breast augmentation and combined breast/abdominal surgery surveys were administered worldwide to MTurk.

RESULTS

The university facelift cohort valued testimonials (33.9%) as the most important, followed by photos (31.6%), reputation (18.2%), pricing (14.4%), and practice years (1.9%). MTurk breast augmentation participants valued photos (35.3%), then testimonials (33.9%), reputation (15.7%), pricing (12.2%), and practice years (3%). MTurk combined breast/abdominal surgery and facelift participants valued testimonials (38.3% and 38.1%, respectively), then photos (27.9%, 29.4%), reputation (17.5%, 15.8%), pricing (13.9%, 13.9%), practice years (2.4%, 2.8%).

CONCLUSIONS

Breast augmentation patients placed higher importance on photos; combined breast/abdominal surgery and facelift patients valued testimonials. Conjoint analysis has had limited application in plastic surgery. To our knowledge, internet crowdsourcing is a novel participant recruitment method in plastic surgery. Its unique benefits include broad, diverse and anonymous participant pools, low-cost, rapid data collection, and high completion rate.

Interactive machine learning for health informatics: when do we need the human-in-the-loop?

Machine learning (ML) is the fastest growing field in computer science, and health informatics is among the greatest challenges. The goal of ML is to develop algorithms which can learn and improve over time and can be used for predictions. Most ML researchers concentrate on automatic machine learning (aML), where great advances have been made, for example, in speech recognition, recommender systems, or autonomous vehicles. Automatic approaches greatly benefit from big data with many training sets. However, in the health domain, sometimes we are confronted with a small number of data sets or rare events, where aML-approaches suffer of insufficient training samples. Here interactive machine learning (iML) may be of help, having its roots in reinforcement learning, preference learning, and active learning. The term iML is not yet well used, so we define it as “algorithms that can interact with agents and can optimize their learning behavior through these interactions, where the agents can also be human.” This “human-in-the-loop” can be beneficial in solving computationally hard problems, e.g., subspace clustering, protein folding, or k-anonymization of health data, where human expertise can help to reduce an exponential search space through heuristic selection of samples. Therefore, what would otherwise be an NP-hard problem, reduces greatly in complexity through the input and the assistance of a human agent involved in the learning phase.