Comparing output from two methods of participatory design for developing implementation strategies: traditional contextual inquiry vs. rapid crowd sourcing

Background

Participatory design methods are a key component of designing tailored implementation strategies. These methods vary in the resources required to execute and analyze their outputs. No work to date has examined the extent to which the output obtained from different approaches to participatory design varies.

Methods

We concurrently used two separate participatory design methods: (1) field observations and qualitative interviews (i.e., traditional contextual inquiry) and (2) rapid crowd sourcing (an innovation tournament). Our goal was to generate and compare information to tailor implementation strategies to increase the use of evidence-based data collection practices among one-to-one aides working with children with autism. Each method was executed and analyzed by study team members blinded to the output of the other method. We estimated the personnel time and monetary costs associated with each method to further facilitate comparison.

Results

Observations and interviews generated nearly double the number of implementation strategies (n = 26) than did the innovation tournament (n = 14). When strategies were classified into implementation strategies from the Expert Recommendations for Implementing Change (ERIC) taxonomy, there was considerable overlap in the content of identified strategies. However, strategies derived from observations and interviews were more specific than those from the innovation tournament. Five strategies (13%) reflected content unique to observations and interviews and 3 (8%) strategies were unique to the innovation tournament. Only observations and interviews identified implementation strategies related to adapting and tailoring to context; only the innovation tournament identified implementation strategies that used incentives. Observations and interviews required more than three times the personnel hours than the innovation tournament, but the innovation tournament was more costly overall due to the technological platform used.

Conclusions

There was substantial overlap in content derived from observations and interviews and the innovation tournament, although there was greater specificity in the findings from observations and interviews. However, the innovation tournament yielded unique information. To select the best participatory design approach to inform implementation strategy design for a particular context, researchers should carefully consider unique advantages of each method and weigh the resources available to invest in the process.

A pilot study of participatory and rapid implementation approaches to increase depression screening in primary care

BACKGROUND

Most individuals with depression go unidentified and untreated. In 2016 the US Preventive Services Task Force released guidelines recommending universal screening in primary care to identify patients with depression and to link them to treatment. Feasible, acceptable, and effective strategies to implement these guidelines are needed.

METHODS

This three-phased study employed rapid participatory methods to design and test strategies to increase depression screening at Penn Medicine, a large health system with 90 primary care practices. First, researchers solicited ideas and barriers from stakeholders to increase screening using an innovation tournament-a crowdsourcing method that invites stakeholders to submit ideas to address a workplace challenge. Second, a panel of stakeholders and scientists deliberated over and ranked the tournament ideas. An instant runoff election was held to select the winning idea. Third, the research team piloted the winning idea in a primary care practice using rapid prototyping, an approach that quickly refines and iterates strategy designs.

RESULTS

The innovation tournament yielded 31 ideas and 32 barriers from diverse stakeholders (12 primary care physicians, 10 medical assistants, 4 nurse practitioners, 2 practice managers, and 4 patient support assistants). A panel of 6 stakeholders and scientists deliberated on the ideas and voted for patient self-report (i.e., through tablet computers, text message, or an online patient portal) as the winning idea. The research team rapid prototyped tablets in one primary care practice with one physician over 5 five-hour shifts to examine the feasibility, acceptability, and effectiveness of the strategy. Most patients, the physician, and medical assistants found the tablets acceptable and feasible. However, patient support assistants struggled to incorporate them in their workflow and expressed concerns about scaling up the process. Depression screening rates were higher using tablets compared to usual care; follow-up was comparable between tablets and usual care.

CONCLUSIONS

Rapid participatory methods engaged and amplified the voices of diverse stakeholders in primary care. These methods helped design an acceptable and feasible implementation strategy that showed promise for increasing depression screening in a primary care setting. The next step is to evaluate the strategy in a randomized controlled trial across primary care practices.