A systematic review of the use of unannounced standardized patients (USPs) in clinical settings: A call for more detailed quality and fidelity descriptions and expansion to new areas

BACKGROUND

Unannounced standardized patients (USPs) have long been used to measure clinical performance in situ. These incognito actors capture data on clinician skills during an encounter, as well as patient experience more broadly. A robust USP program requires extensive preparation and standardization efforts. Given the widespread expansion of USPs for education, research, and improvement efforts, we conducted a systematic review with the goal of capturing the breadth of uses of USPs across settings, along with the standardization measures employed across studies.

METHODS

In collaboration with a medical librarian, we conducted systematic searches across six databases. Two independent researchers screened each report for inclusion. Three coders extracted and reviewed study characteristics and data from the studies deemed eligible for inclusion. We extracted data on: target population, setting, and assessed skills. We also captured the reliability and fidelity measures described in each study, including USP detection, USP training methods, and assessment measures.

RESULTS

128 articles were included. Individual clinicians were the most frequently targeted population (n = 114, 89 %). Common clinician roles included physicians (n = 92, 72 %) and pharmacists (n = 12, 9 %). The collective care team was the target in two studies (2 %), and systems and larger healthcare facilities were targeted in only 1 (1 %) and 13 (10 %) studies, respectively. Studies were primarily conducted in ambulatory settings (n = 118, 92 %). History gathering (n = 76, 59 %), communication (n = 55, 43 %), counseling (n = 51, 40 %), and patient education (n = 49, 38 %) were commonly assessed, as were correct diagnosis (n = 34, 27 %), appropriate ordering of labs/tests (n = 30, 23 %), referrals (n = 35, 27 %), and prescriptions (n = 36, 28 %). USP detection reporting was variable across studies; however, no detection information was provided for 48 studies. 62 % of articles reported incorporating a measure of reliability or fidelity into their study, while the remainder either failed to provide adequate information on use of these measures.

CONCLUSIONS

We explored USP use across settings and describe the scope and limitations of the literature. USPs capture a range of data domains but a lack uniform report of reliability measures can potentially undermine findings. Future studies should incorporate and uniformly report out on detection, training, and assessment.

A systematic review of the use of simulated patient methodology in pharmacy practice research from 2006 to 2016

OBJECTIVES

Simulated patient (SP) methodology (mystery shopping) is used increasingly to assess quality of pharmacy services, and evaluate impact of interventions. Our objective was to review papers reporting on the use of SP methodology in pharmacy practice research 2006-2016 in community pharmacies worldwide.

METHODS

We searched EMBASE and MEDLINE for papers reporting on the use of mystery shopping in pharmacy settings, using a wide range of terms for SPs, based on previous review. We removed irrelevant papers, duplicates, papers not written in English, and review papers and reviewed remaining papers. Two reviewers carried out data abstraction, using the same tool as the previous review and inserting data into Excel, focusing on how the SP methodology is used.

KEY FINDINGS

A total of 148 papers from 52 countries from all regions of the world were included in the review. A wide range of terms described the method, and simulated patient was the most common (49 papers). Most studies were cross-sectional (124), and most investigated only community pharmacies (115). The most common aim was to evaluate some aspect of pharmacists' or other staff's advice and counselling (94). Number of visits is 2-7785. Many papers did not cover details, such as number of visits planned, and carried out, scenario used, training and background of SPs, and ethical approval for the study.

CONCLUSIONS

The use of SP methodology has increased substantially in the field of pharmacy over the past decade. This is a useful method in a wide range of countries and settings. Greater detail is required in reporting.