The Promise and Pitfalls of Using Crowdsourcing in Research Prioritization for Back Pain: Cross-Sectional Surveys

Assessing Interventions on Crowdsourcing Platforms to Nudge Patients for Engagement Behaviors in Primary Care Settings: Randomized Controlled Trial

BACKGROUND

Engaging patients in health behaviors is critical for better outcomes, yet many patient partnership behaviors are not widely adopted. Behavioral economics-based interventions offer potential solutions, but it is challenging to assess the time and cost needed for different options. Crowdsourcing platforms can efficiently and rapidly assess the efficacy of such interventions, but it is unclear if web-based participants respond to simulated incentives in the same way as they would to actual incentives.

OBJECTIVE

The goals of this study were (1) to assess the feasibility of using crowdsourced surveys to evaluate behavioral economics interventions for patient partnerships by examining whether web-based participants responded to simulated incentives in the same way they would have responded to actual incentives, and (2) to assess the impact of 2 behavioral economics-based intervention designs, psychological rewards and loss of framing, on simulated medication reconciliation behaviors in a simulated primary care setting.

METHODS

We conducted a randomized controlled trial using a between-subject design on a crowdsourcing platform (Amazon Mechanical Turk) to evaluate the effectiveness of behavioral interventions designed to improve medication adherence in primary care visits. The study included a control group that represented the participants' baseline behavior and 3 simulated interventions, namely monetary compensation, a status effect as a psychological reward, and a loss frame as a modification of the status effect. Participants' willingness to bring medicines to a primary care visit was measured on a 5-point Likert scale. A reverse-coding question was included to ensure response intentionality.

RESULTS

A total of 569 study participants were recruited. There were 132 in the baseline group, 187 in the monetary compensation group, 149 in the psychological reward group, and 101 in the loss frame group. All 3 nudge interventions increased participants' willingness to bring medicines significantly when compared to the baseline scenario. The monetary compensation intervention caused an increase of 17.51% (P<.001), psychological rewards on status increased willingness by 11.85% (P<.001), and a loss frame on psychological rewards increased willingness by 24.35% (P<.001). Responses to the reverse-coding question were consistent with the willingness questions.

CONCLUSIONS

In primary care, bringing medications to office visits is a frequently advocated patient partnership behavior that is nonetheless not widely adopted. Crowdsourcing platforms such as Amazon Mechanical Turk support efforts to efficiently and rapidly reach large groups of individuals to assess the efficacy of behavioral interventions. We found that crowdsourced survey-based experiments with simulated incentives can produce valid simulated behavioral responses. The use of psychological status design, particularly with a loss framing approach, can effectively enhance patient engagement in primary care. These results support the use of crowdsourcing platforms to augment and complement traditional approaches to learning about behavioral economics for patient engagement.

Strong Public Desire for Quality and Price Transparency in Shoulder Arthroplasty

INTRODUCTION

Concerted efforts to optimize outcomes and data transparency in shoulder arthroplasty have led to the creation of the American Academy of Orthopaedic Surgeons (AAOS) Shoulder and Elbow Registry, the first nationwide registry of its kind. We used online crowdsourcing to explore the general public's perceptions and beliefs toward the disclosure of quality and price data in shoulder arthroplasty.

METHODS

A total of 498 participants recruited using Amazon Mechanical Turk (MTurk) completed a survey regarding beliefs about public disclosure of quality and price data in shoulder arthroplasty. The MTurk is an online marketplace for crowdsourcing tasks (e.g., answering surveys) to a pool of over 500,000 registered workers in exchange for financial compensation. Requesters post human-intelligence tasks, and workers can respond to those that they are interested in completing. This web-based platform is an efficient survey tool for medical research, with comparable national representativeness to traditional convenience samples.

RESULTS

The majority (95%) of respondents believed surgeons and hospitals should share their data with national registries such as the AAOS Shoulder and Elbow Registry. Most believed that patients considering shoulder arthroplasty should have public access to surgeons' outcomes and complication rates (96%), years of experience (95%), and case volume (92%). Most respondents desired price transparency in implant costs (95%), surgeon reimbursement (80%), and hospital reimbursement (84%). In decreasing order of importance, the top three factors guiding surgeon choice were: (1) surgeon included in the insurer's network, (2) annual case volume, and (3) publicly available outcomes.

CONCLUSION

Increased quality and price transparency in shoulder arthroplasty may empower patients to make better-informed decisions about their care and ultimately enhance value. Given the strong public desire for data transparency and the notion that public disclosure of data is intrinsically associated with performance improvement, surgeons and hospitals should strongly consider submitting their data to national registries such as the AAOS Shoulder and Elbow Registry.

Essential elements, conceptual foundations and workflow design in crowd-powered projects

Crowdsourcing arose as a problem-solving strategy that uses a large number of workers to achieve tasks and solve specific problems. Although there are many studies that explore crowdsourcing platforms and systems, little attention has been paid to define what a crowd-powered project is. To address this issue, this article introduces a general-purpose conceptual model that represents the essential elements involved in this kind of project and how they relate to each other. We consider that the workflow in crowdsourcing projects is context-oriented and should represent the planning and coordination by the crowdsourcer in the project, instead of only facilitating decomposing a complex task into subtask sets. Since structural models are limited to cannot properly represent the execution flow, we also introduce the use of behavioural conceptual models, specifically Unified Modeling Language (UML) activity diagrams, to represent the user, tasks, assets, control activities and products involved in a specific project.

Fitness Tracker Information and Privacy Management: Empirical Study

Background

Fitness trackers allow users to collect, manage, track, and monitor fitness-related activities, such as distance walked, calorie intake, sleep quality, and heart rate. Fitness trackers have become increasingly popular in the past decade. One in five Americans use a device or an app to track their fitness-related activities. These devices generate massive and important data that could help physicians make better assessments of their patients’ health if shared with health providers. This ultimately could lead to better health outcomes and perhaps even lower costs for patients. However, sharing personal fitness information with health care providers has drawbacks, mainly related to the risk of privacy loss and information misuse.

Objective

This study investigates the influence of granting users granular privacy control on their willingness to share fitness information.

Methods

The study used 270 valid responses collected from Mtrurkers through Amazon Mechanical Turk (MTurk). Participants were randomly assigned to one of two groups. The conceptual model was tested using structural equation modeling (SEM). The dependent variable was the intention to share fitness information. The independent variables were perceived risk, perceived benefits, and trust in the system.

Results

SEM explained about 60% of the variance in the dependent variable. Three of the four hypotheses were supported. Perceived risk and trust in the system had a significant relationship with the dependent variable, while trust in the system was not significant.

Conclusions

The findings show that people are willing to share their fitness information if they have granular privacy control. This study has practical and theoretical implications. It integrates communication privacy management (CPM) theory with the privacy calculus model.

Open innovation facilitates department-wide engagement in quality improvement: experience from the Massachusetts General Hospital

BACKGROUND

Quality improvement (QI) initiatives commonly originate 'top-down' from senior leadership, as staff engagement is often sporadic. We describe our experience with a technology-enabled open innovation contest to encourage participation from multiple stakeholders in a Department of Surgery (DoS) to solicit ideas for QI. We aimed to stimulate engagement and to assist DoS leadership in prioritizing QI initiatives.

METHODS

Observational study of a process improvement. The process had five phases: anonymous online submission of ideas by frontline staff; anonymous online crowd-voting to rank ideas on a scale whether the DoS should implement each idea (1 = No, 3 = Maybe, 5 = Yes); ideas with scores ≥ 95th percentile were invited to submit implementation plans; plans were reviewed by a multi-disciplinary panel to select a winning idea; an award ceremony celebrated the completion of the contest.

RESULTS

152 ideas were submitted from 95 staff (n = 850, 11.2%). All Divisions (n = 12) and all staff roles (n = 12) submitted ideas. The greatest number of ideas were submitted by faculty (27.6%), patient service coordinators (18.4%), and residents (17.8%). The most common QI category was access to care (20%). 195 staff (22.9%) cast 3559 votes. The mean score was 3.5 ± 0.5. 10 Ideas were objectively invited to submit implementation plans. One idea was awarded a grand prize of funding, project management, and leadership buy-in.

CONCLUSION

A web-enabled open innovation contest was successful in engaging faculty, residents, and other critical role groups in QI. It also enabled the leadership to re-affirm a positive culture of inclusivity, maintain an open-door policy, and also democratically vet and prioritize solutions for quality improvement.

Evaluating Knowledge, Attitudes, and Beliefs About Lung Cancer Screening Using Crowdsourcing

BACKGROUND

Lung cancer screening, despite its proven mortality benefit, remains vastly underutilized. Previous studies examined knowledge, attitudes, and beliefs to better understand the reasons underlying the low screening rates. These investigations may have limited generalizability because of traditional participant recruitment strategies and examining only subpopulations eligible for screening. The current study used crowdsourcing to recruit a broader population to assess these factors in a potentially more general population.

METHODS

A 31-item survey was developed to assess knowledge, attitudes, and beliefs regarding screening among individuals considered high risk for lung cancer by the United States Preventive Services Task Force. Amazon's crowdsourcing platform (Mechanical Turk) was used to recruit subjects.

RESULTS

Among the 240 respondents who qualified for the study, 106 (44%) reported knowledge of a screening test for lung cancer. However, only 36 (35%) correctly identified low-dose CT scanning as the appropriate test. A total of 222 respondents (93%) reported believing that early detection of lung cancer has the potential to save lives, and 165 (69%) were willing to undergo lung cancer screening if it was recommended by their physician. Multivariable regression analysis found that knowledge of lung cancer screening, smoking status, chronic pulmonary disease, and belief in the efficacy of early detection of lung cancer were associated with willingness to screen.

CONCLUSIONS

Although a minority of individuals at high risk for lung cancer are aware of screening, the majority believe that early detection saves lives and would pursue screening if recommended by their primary care physician. Health systems may increase screening rates by improving patient and physician awareness of lung cancer screening.

Influencing Pain Inferences Using Random Numerical Anchoring: Randomized Controlled Trial

Background

Numerical anchoring occurs when exposure to a numeric quantity influences a person’s subsequent judgment involving other quantities. This could be applicable to the evaluation of pain, where exposure to an unrelated number before the evaluation of pain could influence pain ratings.

Objective

This study aimed to determine whether exposure to a random numeric anchor influences subsequent pain intensity ratings of a hypothetical patient.

Methods

In this study, 385 participants read a vignette describing a patient with chronic pain before being randomly assigned to one of four groups. Groups 1 and 2 spun an 11-wedge number wheel (0-10), which was, unbeknown to the participants, programmed to stop on a high number (8) or a low number (2), respectively. Group 3 spun a similar letter wheel (A-K), which was programmed to stop on either the letter C or I (control 1). Group 4 did not spin a wheel (control 2). Participants were then asked to rate the patient’s pain intensity using a 0 to 10 numeric rating scale.

Results

The high-number group rated the patient’s pain (median 8, IQR 2) significantly higher than the letter wheel control (median 7, IQR 2; P=.02) and the low-number group (median 6, IQR 2; P<.001). The low-number group rated the pain significantly lower than controls 1 and 2 (median 7, IQR 2; both P=.045).

Conclusions

Pain ratings were influenced by prior exposure to a random number with no relevant information about the patient’s pain, indicating anchoring had occurred. However, contrary to the traditional definition of anchoring where anchoring occurs even when participants are unaware of the anchor’s influence, in this study, the anchoring effect was seen only in participants who believed that the anchor had influenced them. This suggests that anchoring effects could potentially occur among health care providers tasked with evaluating a patient’s pain and should be evaluated further.

Comparing three approaches for involving patients in research prioritization: a qualitative study of participant experiences

BACKGROUND

By participating in priority-setting activities in research, patients and members of the public help ensure that important questions are incorporated into future research agendas. Surveys, focus groups, and online crowdsourcing are increasingly used to obtain input, yet little is known about how they compare for prioritizing research topics. To address this gap, the Study of Methods for Assessing Research Topic Elicitation and pRioritization (SMARTER) evaluated participant satisfaction with the engagement experience across three prioritization activities.

METHODS

Respondents from Back pain Outcomes using Longitudinal Data (BOLD), a registry of patients 65 years and older with low back pain (LBP), were randomly assigned to one of three interactive prioritization activities: online crowd-voting, in-person focus groups using nominal group technique, and two rounds of a mailed survey (Delphi). To assess quality of experience, participants completed a brief survey; a subset were subsequently interviewed. We used descriptive statistics to characterize participants, and we analyzed responses to the evaluation using a mixed-methods approach, tabulating responses to Likert-scale questions and using thematic analysis of interviews to explore participant understanding of the activity and perceptions of experience.

RESULTS

The crowd-voting activity had 38 participants, focus groups 39, and the Delphi survey 74. Women outnumbered men in the focus groups and Delphi survey; otherwise, demographics among groups were similar, with participants being predominantly white, non-Hispanic, and college educated. Activities generated similar lists of research priorities, including causes of LBP, improving physician-patient communication, and self-care strategies. The evaluation survey was completed by 123 participants. Of these, 31 across all activities were interviewed about motivations to participate, understanding of activity goals, logistics, clarity of instructions, and the role of patients in research. Focus group participants rated their experience highest, in both the evaluation and interviews.

CONCLUSION

Common methods for research prioritization yielded similar priorities but differing perceptions of experience. Such comparative studies are rare but important in understanding methods to involve patients and the public in research. Preferences for different methods may vary across stakeholder groups; this warrants future study.

TRIAL REGISTRATION

NICHSR, HSRP20152274. Registered 19 February 2015.

Mapping of Crowdsourcing in Health: Systematic Review

Background

Crowdsourcing involves obtaining ideas, needed services, or content by soliciting Web-based contributions from a crowd. The 4 types of crowdsourced tasks (problem solving, data processing, surveillance or monitoring, and surveying) can be applied in the 3 categories of health (promotion, research, and care).

Objective

This study aimed to map the different applications of crowdsourcing in health to assess the fields of health that are using crowdsourcing and the crowdsourced tasks used. We also describe the logistics of crowdsourcing and the characteristics of crowd workers.

Methods

MEDLINE, EMBASE, and ClinicalTrials.gov were searched for available reports from inception to March 30, 2016, with no restriction on language or publication status.

Results

We identified 202 relevant studies that used crowdsourcing, including 9 randomized controlled trials, of which only one had posted results at ClinicalTrials.gov. Crowdsourcing was used in health promotion (91/202, 45.0%), research (73/202, 36.1%), and care (38/202, 18.8%). The 4 most frequent areas of application were public health (67/202, 33.2%), psychiatry (32/202, 15.8%), surgery (22/202, 10.9%), and oncology (14/202, 6.9%). Half of the reports (99/202, 49.0%) referred to data processing, 34.6% (70/202) referred to surveying, 10.4% (21/202) referred to surveillance or monitoring, and 5.9% (12/202) referred to problem-solving. Labor market platforms (eg, Amazon Mechanical Turk) were used in most studies (190/202, 94%). The crowd workers’ characteristics were poorly reported, and crowdsourcing logistics were missing from two-thirds of the reports. When reported, the median size of the crowd was 424 (first and third quartiles: 167-802); crowd workers’ median age was 34 years (32-36). Crowd workers were mainly recruited nationally, particularly in the United States. For many studies (58.9%, 119/202), previous experience in crowdsourcing was required, and passing a qualification test or training was seldom needed (11.9% of studies; 24/202). For half of the studies, monetary incentives were mentioned, with mainly less than US $1 to perform the task. The time needed to perform the task was mostly less than 10 min (58.9% of studies; 119/202). Data quality validation was used in 54/202 studies (26.7%), mainly by attention check questions or by replicating the task with several crowd workers.

Conclusions

The use of crowdsourcing, which allows access to a large pool of participants as well as saving time in data collection, lowering costs, and speeding up innovations, is increasing in health promotion, research, and care. However, the description of crowdsourcing logistics and crowd workers’ characteristics is frequently missing in study reports and needs to be precisely reported to better interpret the study findings and replicate them.

The Patient Perspective on the Impact of Tenosynovial Giant Cell Tumors on Daily Living: Crowdsourcing Study on Physical Function and Quality of Life

Background

Tenosynovial giant cell tumor (TGCT) is a rare, benign lesion affecting the synovial lining of joints, bursae, and tendon sheaths. It is generally characterized as a locally aggressive and often recurring tumor. A distinction is made between localized- and diffuse-type. The impact of TGCT on daily living is currently ill-described.

Objective

The aim of this crowdsourcing study was to evaluate the impact of TGCT on physical function, daily activities, societal participation (work, sports, and hobbies), and overall quality of life from a patient perspective. The secondary aim was to define risk factors for deteriorated outcome in TGCT.

Methods

Members of the largest known TGCT Facebook community, PVNS is Pants!!, were invited to an e-survey, partially consisting of validated questionnaires, for 6 months. To confirm disease presence and TGCT-type, patients were requested to share histological or radiological proof of TGCT. Unpaired t tests and chi-square tests were used to compare groups with and without proof and to define risk factors for deteriorated outcome.

Results

Three hundred thirty-seven questionnaires, originating from 30 countries, were completed. Median age at diagnosis was 33 (interquartile range [IQR]=25-42) years, majority was female (79.8% [269/337]), diffuse TGCT (70.3% [237/337]), and affected lower extremities (knee 70.9% [239/337] and hip 9.5% [32/337]). In 299 lower-extremity TGCT patients (32.4% [97/299]) with disease confirmation, recurrence rate was 36% and 69.5% in localized and diffuse type, respectively. For both types, pain and swelling decreased after treatment; in contrast, stiffness and range of motion worsened. Patients were limited in their employment (localized 13% [8/61]; diffuse 11.0% [21/191]) and sport-activities (localized 58% [40/69]; diffuse 63.9% [147/230]). Compared with general US population, all patients showed lower Patient-Reported Outcomes Measurements Information System-Physical Function (PROMIS-PF), Short Form-12 (SF-12), and EuroQoL 5 Dimensions 5 Levels (EQ5D-5L) scores, considered clinically relevant, according to estimated minimal important difference (MID). Diffuse versus localized type scored almost 0.5 standard deviation lower for PROMIS-PF (P<.001) and demonstrated a utility score of 5% lower for EQ-5D-5L (P=.03). In localized TGCT, recurrent disease and ≥2 surgeries negatively influenced scores of Visual Analog Scale (VAS)-pain/stiffness, SF-12, and EQ-5D-5L (P<.05). In diffuse type, recurrence resulted in lower score for VAS, PROMIS-PF, SF-12, and EQ-5D-5L (P<.05). In both types, patients with treatment ≤1year had significantly lower SF-12.

Conclusions

TGCT has a major impact on daily living in a relatively young and working population. Patients with diffuse type, recurrent disease, and ≥2 surgeries represent lowest functional and quality of life outcomes. Physicians should be aware that TGCT patients frequently continue to experience declined health-related quality of life and physical function and often remain limited in daily life, even after treatment(s).